Added support for Gmail Client Side Encryption

Build universal2 with macos-12, doesn't run with macos-14

.github/ISSUE_TEMPLATE.txt

@@ -0,0 +1,14 @@
+The issue tracker is for reporting product deficiencies. "How do I?" questions should be posted to the discussion forum at https://groups.google.com/group/google-apps-manager. When in doubt, start at the discussion forum and return here only when instructed to do so.
+
+Please confirm the following:
+* I have upgraded to the latest GAM release from https://github.com/GAM-team/GAM/releases and I still have this issue.
+* I am typing the command as described in the GAM Wiki at https://github.com/GAM-team/GAM/wiki
+
+Full steps to reproduce the issue:
+1.
+2.
+3.
+
+Expected outcome (what are you trying to do?):
+
+Actual outcome (what errors or bad behavior do you see instead?):

.github/ISSUE_TEMPLATE/aa-question.md

@@ -0,0 +1,14 @@
+---
+name: Question about using GAM
+about: Help with using GAM or running it for the first time
+title: Please use the GAM discussion group
+labels: invalid
+assignees: ''
+
+---
+
+If you need help with GAM, please do not file an issue here, it will be closed and ignored.
+
+Please post your question to the GAM discussion group where other admins are ready and willing to help:
+
+https://groups.google.com/g/google-apps-manager

.github/ISSUE_TEMPLATE/za-bug-report.md

@@ -0,0 +1,23 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: jay0lee
+
+---
+
+The issue tracker is for reporting product deficiencies. "How do I?" questions should be posted to the discussion forum at https://groups.google.com/group/google-apps-manager. When in doubt, start at the discussion forum and return here only when instructed to do so.
+
+Please confirm the following:
+* I have upgraded to the latest GAM release from https://github.com/GAM-team/GAM/releases and I still have this issue.
+* I am typing the command as described in the GAM Wiki at https://github.com/jay0lee/gam/wiki
+
+Full steps to reproduce the issue:
+1.
+2.
+3.
+
+Expected outcome (what are you trying to do?):
+
+Actual outcome (what errors or bad behavior do you see instead?):

.github/ISSUE_TEMPLATE/zz-feature-request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for GAM
+title: ''
+labels: enhancement
+assignees: jay0lee
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/actions/decrypt.sh

@@ -0,0 +1,32 @@
+#!/bin/sh
+credspath="$3"
+if [ ! -d "$credspath" ]; then
+    echo "creating ${credspath}"
+    mkdir -p "$credspath"
+fi
+gpgfile="$1"
+if [ -f "$gpgfile" ]; then
+    echo "source file is ${gpgfile}"
+else
+    echo "ERROR: ${gpgfile} does not exist"
+    exit 1
+fi
+credsfile="$2"
+echo "target file is ${credsfile}"
+if [ -z ${PASSCODE+x} ]; then
+  echo "ERROR: PASSCODE is unset";
+  exit 2
+else
+  echo "PASSCODE is set";
+fi
+
+gpg --batch \
+    --yes \
+    --decrypt \
+    --passphrase="${PASSCODE}" \
+    --output "${credsfile}" \
+    "${gpgfile}"
+
+tar xvvf "${credsfile}" --directory "${credspath}"
+rm -rvf "${gpgfile}"
+rm -rvf "${credsfile}"

.github/actions/package_exclusions.txt

@@ -0,0 +1,7 @@
+oauth2.txt
+nobrowser.txt
+enabledasa.txt
+lastupdatecheck.txt
+*.json
+*.lck
+*.csv

.github/stale.yml

@@ -0,0 +1,61 @@
+# Configuration for probot-stale - https://github.com/probot/stale
+
+# Number of days of inactivity before an Issue or Pull Request becomes stale
+daysUntilStale: 90
+
+# Number of days of inactivity before an Issue or Pull Request with the stale label is closed.
+# Set to false to disable. If disabled, issues still need to be closed manually, but will remain marked as stale.
+daysUntilClose: 7
+
+# Only issues or pull requests with all of these labels are check if stale. Defaults to `[]` (disabled)
+onlyLabels: []
+
+# Issues or Pull Requests with these labels will never be considered stale. Set to `[]` to disable
+exemptLabels:
+  - pinned
+  - enhancement
+  - help wanted
+  - security
+
+# Set to true to ignore issues in a project (defaults to false)
+exemptProjects: false
+
+# Set to true to ignore issues in a milestone (defaults to false)
+exemptMilestones: false
+
+# Set to true to ignore issues with an assignee (defaults to false)
+exemptAssignees: false
+
+# Label to use when marking as stale
+staleLabel: wontfix
+
+# Comment to post when marking as stale. Set to `false` to disable
+markComment: >
+  This issue has been automatically marked as stale because it has not had
+  recent activity. It will be closed if no further activity occurs.
+
+# Comment to post when removing the stale label.
+# unmarkComment: >
+#   Your comment here.
+
+# Comment to post when closing a stale Issue or Pull Request.
+# closeComment: >
+#   Your comment here.
+
+# Limit the number of actions per hour, from 1-30. Default is 30
+limitPerRun: 30
+
+# Limit to only `issues` or `pulls`
+# only: issues
+
+# Optionally, specify configuration settings that are specific to just 'issues' or 'pulls':
+# pulls:
+#   daysUntilStale: 30
+#   markComment: >
+#     This pull request has been automatically marked as stale because it has not had
+#     recent activity. It will be closed if no further activity occurs. Thank you
+#     for your contributions.
+
+# issues:
+#   exemptLabels:
+#     - confirmed

.github/workflows/build.yml

@@ -0,0 +1,925 @@
+name: Build and test GAM
+
+on:
+  push:
+  pull_request:
+  schedule:
+    - cron: '37 22 * * *'
+
+permissions:
+  contents: read
+  id-token: write
+
+defaults:
+  run:
+    shell: bash
+    working-directory: src
+
+env:
+  OPENSSL_CONFIG_OPTS: no-fips --api=3.0.0
+  OPENSSL_INSTALL_PATH: ${{ github.workspace }}/bin/ssl
+  OPENSSL_SOURCE_PATH: ${{ github.workspace }}/src/openssl
+  PYTHON_INSTALL_PATH: ${{ github.workspace }}/bin/python
+  PYTHON_SOURCE_PATH: ${{ github.workspace }}/src/cpython
+
+jobs:
+  build:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - os: ubuntu-20.04
+            jid: 1
+            goal: build
+            arch: x86_64
+            openssl_archs: linux-x86_64
+            fullGamTest: yes
+          - os: [self-hosted, linux, arm64]
+            jid: 2
+            goal: build
+            arch: aarch64
+            openssl_archs: linux-aarch64
+            fullGamTest: yes
+          - os: ubuntu-20.04
+            jid: 3
+            goal: build
+            arch: x86_64
+            openssl_archs: linux-x86_64
+            staticx: yes
+          - os: [self-hosted, linux, arm64]
+            jid: 4
+            goal: build
+            arch: aarch64
+            openssl_archs: linux-aarch64
+            staticx: yes
+          - os: macos-12
+            jid: 5
+            goal: build
+            arch: x86_64
+            openssl_archs: darwin64-x86_64
+            fullGamTest: yes
+          - os: macos-14
+            jid: 6
+            goal: build
+            arch: aarch64
+            openssl_archs: darwin64-arm64
+            fullGamTest: yes
+          - os: macos-14
+            jid: 7
+            goal: build
+            arch: universal2
+            openssl_archs: darwin64-arm64 darwin64-x86_64
+          - os: windows-2022
+            jid: 8
+            goal: build
+            arch: Win64
+            openssl_archs: VC-WIN64A
+            fullGamTest: yes
+          - os: ubuntu-22.04
+            goal: test
+            python: "3.8"
+            jid: 9
+            arch: x86_64
+          - os: ubuntu-22.04
+            goal: test
+            python: "3.9"
+            jid: 10
+            arch: x86_64
+          - os: ubuntu-22.04
+            goal: test
+            python: "3.10"
+            jid: 11
+            arch: x86_64
+          - os: ubuntu-22.04
+            goal: test
+            python: "3.11"
+            jid: 12
+            arch: x86_64
+
+    steps:
+
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+          fetch-depth: 0
+
+      - id: auth
+        name: Authenticate to Google Cloud
+        uses: google-github-actions/auth@v2
+        with:
+          workload_identity_provider: projects/297925809119/locations/global/workloadIdentityPools/gha-pool/providers/gha-provider
+          service_account: github-actions-testing-for-gam@gam-project-wyo-lub-ivl.iam.gserviceaccount.com
+
+      - name: Cache multiple paths
+        if: matrix.goal == 'build'
+        uses: actions/cache@v4
+        id: cache-python-ssl
+        with:
+          path: |
+            cache.tar.xz
+          key: gam-${{ matrix.jid }}-20240210
+
+      - name: Untar Cache archive
+        if: matrix.goal == 'build' && steps.cache-python-ssl.outputs.cache-hit == 'true'
+        working-directory: ${{ github.workspace }}
+        run: |
+          tar xvvf cache.tar.xz
+
+      - name: Use pre-compiled Python for testing
+        if: matrix.python != ''
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python }}
+          allow-prereleases: true
+
+      - name: common variables for all runs
+        env:
+          arch: ${{ matrix.arch }}
+          JID: ${{ matrix.jid }}
+          ACTIONS_CACHE: ${{ steps.cache-python-ssl.outputs.cache-hit }}
+          ACTIONS_GOAL: ${{ matrix.goal }}
+        run: |
+          echo "arch=${arch}" >> $GITHUB_ENV
+          echo "JID=${JID}" >> $GITHUB_ENV
+          echo "ACTIONS_CACHE=${ACTIONS_CACHE}" >> $GITHUB_ENV
+          echo "ACTIONS_GOAL=${ACTIONS_GOAL}" >> $GITHUB_ENV
+          curl_version=$(curl --version | head -n 1 | awk '{ print $2 }')
+          echo "cURL is ${curl_version}"
+          if [ "$curl_version" == "7.68.0" ]; then
+            export curl_retry="--retry 5 --retry-connrefused"
+          else
+            export curl_retry="--retry 5 --retry-all-errors"
+          fi
+          echo "curl_retry=${curl_retry}" >> $GITHUB_ENV
+          # GAMCFGDIR should be recreated on every run
+          GAMCFGDIR="${RUNNER_TEMP}/.gam"
+          if [ "$arch" == "Win64" ]; then
+            GAMCFGDIR=$(cygpath -u "$GAMCFGDIR")
+          fi
+          echo "GAMCFGDIR=${GAMCFGDIR}" >> $GITHUB_ENV
+          echo "GAMCFGDIR is: ${GAMCFGDIR}"
+          if [[ "${RUNNER_OS}" == "macOS" ]]; then
+            GAMOS="macos"
+          elif [[ "${RUNNER_OS}" == "Linux" ]]; then
+            GAMOS="linux"
+          elif [[ "${RUNNER_OS}" == "Windows" ]]; then
+            GAMOS="windows"
+          else
+           GAMOS='unknown'
+          fi
+          echo "GAMOS=${GAMOS}" >> $GITHUB_ENV
+          echo "GAMOS is: ${GAMOS}"
+
+      - name: Set env variables for test
+        if: matrix.goal == 'test'
+        run: |
+          export PYTHON=$(which python3)
+          export PIP=$(which pip3)
+          export gam="${PYTHON} -m gam"
+          export gampath="$(readlink -e .)"
+          echo -e "PYTHON: ${PYTHON}\nPIP: ${PIP}\gam: ${gam}\ngampath: ${gampath}"
+          echo "PYTHON=${PYTHON}" >> $GITHUB_ENV
+          echo "PIP=${PIP}" >> $GITHUB_ENV
+          echo "gam=${gam}" >> $GITHUB_ENV
+          echo "gampath=${gampath}" >> $GITHUB_ENV
+
+      - name: Install necessary Github-hosted Linux packages
+        if: runner.os == 'Linux' && runner.arch == 'X64'
+        run: |
+          echo "RUNNING: apt update..."
+          sudo apt-get -qq --yes update
+          sudo apt-get -qq --yes install swig libpcsclite-dev libxslt1-dev
+
+      - name: MacOS install tools
+        if: runner.os == 'macOS'
+        run: |
+          # Install latest Rust
+          curl $curl_retry -fsS -o rust.sh https://sh.rustup.rs
+          bash ./rust.sh -y
+          source $HOME/.cargo/env
+          # Install needed packages
+          brew update
+          brew install gpg swig
+
+      - name: Windows Configure VCode
+        uses: ilammy/msvc-dev-cmd@v1
+        if: runner.os == 'Windows' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        with:
+          arch: ${{ matrix.arch }}
+
+      - name: Set Env Variables for build
+        if: matrix.goal == 'build'
+        env:
+          openssl_archs: ${{ matrix.openssl_archs }}
+          staticx: ${{ matrix.staticx }}
+        run: |
+          echo "We are running on ${RUNNER_OS}"
+          LD_LIBRARY_PATH="${OPENSSL_INSTALL_PATH}/lib:${PYTHON_INSTALL_PATH}/lib:/usr/local/lib"
+          if [[ "${arch}" == "Win64" ]]; then
+            PYEXTERNALS_PATH="amd64"
+            PYBUILDRELEASE_ARCH="x64"
+            GAM_ARCHIVE_ARCH="x86_64"
+            WIX_ARCH="x64"
+            CHOC_OPS=""
+          elif [[ "${arch}" == "Win32" ]]; then
+            PYEXTERNALS_PATH="win32"
+            PYBUILDRELEASE_ARCH="Win32"
+            GAM_ARCHIVE_ARCH="x86"
+            WIX_ARCH="x86"
+            CHOC_OPS="--forcex86"
+          fi
+          if [[ "${RUNNER_OS}" == "macOS" ]]; then
+            MAKE=make
+            MAKEOPT="-j$(sysctl -n hw.logicalcpu)"
+            PERL=perl
+            echo "MACOSX_DEPLOYMENT_TARGET=10.15" >> $GITHUB_ENV
+            echo "PYTHON=${PYTHON_INSTALL_PATH}/bin/python3" >> $GITHUB_ENV
+          elif [[ "${RUNNER_OS}" == "Linux" ]]; then
+            MAKE=make
+            MAKEOPT="-j$(nproc)"
+            PERL=perl
+            echo "PYTHON=${PYTHON_INSTALL_PATH}/bin/python3" >> $GITHUB_ENV
+          elif [[ "${RUNNER_OS}" == "Windows" ]]; then
+            MAKE=nmake
+            MAKEOPT=""
+            PERL="c:\strawberry\perl\bin\perl.exe"
+            LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${PYTHON_SOURCE_PATH}/PCbuild/${PYEXTERNALS_PATH}"
+            echo "PYTHON=${PYTHON_SOURCE_PATH}/PCbuild/${PYEXTERNALS_PATH}/python.exe" >> $GITHUB_ENV
+            echo "GAM_ARCHIVE_ARCH=${GAM_ARCHIVE_ARCH}" >> $GITHUB_ENV
+            echo "WIX_ARCH=${WIX_ARCH}" >> $GITHUB_ENV
+          fi
+          echo "We'll run make with: ${MAKEOPT}"
+          echo "staticx=${staticx}" >> $GITHUB_ENV
+          echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH}" >> $GITHUB_ENV
+          echo "MAKE=${MAKE}" >> $GITHUB_ENV
+          echo "MAKEOPT=${MAKEOPT}" >> $GITHUB_ENV
+          echo "PERL=${PERL}" >> $GITHUB_ENV
+          echo "PYEXTERNALS_PATH=${PYEXTERNALS_PATH}" >> $GITHUB_ENV
+          echo "PYBUILDRELEASE_ARCH=${PYBUILDRELEASE_ARCH}" >> $GITHUB_ENV
+          echo "openssl_archs=${openssl_archs}" >> $GITHUB_ENV
+
+      - name: Get latest stable OpenSSL source
+        if: matrix.goal == 'build' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        run: |
+          mkdir -vp "${GITHUB_WORKSPACE}/src"
+          cd "${GITHUB_WORKSPACE}/src"
+          git clone https://github.com/openssl/openssl.git
+          cd "${OPENSSL_SOURCE_PATH}"
+          export LATEST_STABLE_TAG=$(git tag --list openssl-* | grep -v alpha | grep -v beta | sort -Vr | head -n1)
+          echo "Checking out version ${LATEST_STABLE_TAG}"
+          git checkout "${LATEST_STABLE_TAG}"
+          export COMPILED_OPENSSL_VERSION=${LATEST_STABLE_TAG:8} # Trim the openssl- prefix
+          echo "COMPILED_OPENSSL_VERSION=${COMPILED_OPENSSL_VERSION}" >> $GITHUB_ENV
+          if ([ "${RUNNER_OS}" == "macOS" ] && [ "$arch" == "universal2" ]); then
+            for openssl_arch in $openssl_archs; do
+              ssldir="${OPENSSL_SOURCE_PATH}-${openssl_arch}"
+              mkdir -v "${ssldir}"
+              cp -vrf ${OPENSSL_SOURCE_PATH}/* "${ssldir}/"
+            done
+            rm -vrf "${OPENSSL_SOURCE_PATH}"
+          else
+            mv -v "${OPENSSL_SOURCE_PATH}" "${OPENSSL_SOURCE_PATH}-${openssl_archs}"
+          fi
+
+      - name: Windows NASM Install
+        uses: ilammy/setup-nasm@v1
+        if: matrix.goal == 'build' && runner.os == 'Windows' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+
+      - name: Config OpenSSL
+        if: matrix.goal == 'build' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        run: |
+          for openssl_arch in $openssl_archs; do
+            cd "${GITHUB_WORKSPACE}/src/openssl-${openssl_arch}"
+            # --libdir=lib is needed so Python can find OpenSSL libraries
+            "${PERL}" ./Configure "${openssl_arch}" --libdir=lib --prefix="${OPENSSL_INSTALL_PATH}" $OPENSSL_CONFIG_OPTS
+          done
+
+      - name: Rename GNU link on Windows
+        if: matrix.goal == 'build' && runner.os == 'Windows' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        shell: bash
+        run: mv /usr/bin/link /usr/bin/gnulink
+
+      - name: Make OpenSSL
+        if: matrix.goal == 'build' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        run: |
+          for openssl_arch in $openssl_archs; do
+            cd "${GITHUB_WORKSPACE}/src/openssl-${openssl_arch}"
+            $MAKE "${MAKEOPT}"
+          done
+
+      - name: Install OpenSSL
+        if: matrix.goal == 'build' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        run: |
+          if ([ "${RUNNER_OS}" == "macOS" ] && [ "$arch" == "universal2" ]); then
+            for openssl_arch in $openssl_archs; do
+              cd "${GITHUB_WORKSPACE}/src/openssl-${openssl_arch}"
+              # install_sw saves us ages processing man pages :-)
+              $MAKE install_sw
+              mv "${OPENSSL_INSTALL_PATH}" "${GITHUB_WORKSPACE}/bin/ssl-${openssl_arch}"
+            done
+            mkdir -vp "${OPENSSL_INSTALL_PATH}/lib"
+            mkdir -vp "${OPENSSL_INSTALL_PATH}/bin"
+            for archlib in libcrypto.3.dylib libssl.3.dylib libcrypto.a libssl.a; do
+              lipo -create "${GITHUB_WORKSPACE}/bin/ssl-darwin64-x86_64/lib/${archlib}" \
+                           "${GITHUB_WORKSPACE}/bin/ssl-darwin64-arm64/lib/${archlib}" \
+                   -output "${GITHUB_WORKSPACE}/bin/ssl/lib/${archlib}"
+            done
+            mv ${GITHUB_WORKSPACE}/bin/ssl-darwin64-x86_64/include ${GITHUB_WORKSPACE}/bin/ssl/
+            lipo -create "${GITHUB_WORKSPACE}/bin/ssl-darwin64-x86_64/bin/openssl" \
+                         "${GITHUB_WORKSPACE}/bin/ssl-darwin64-arm64/bin/openssl" \
+                 -output "${GITHUB_WORKSPACE}/bin/ssl/bin/openssl"
+            rm -rf ${GITHUB_WORKSPACE}/bin/ssl-darwin64-x86_64
+            rm -rf ${GITHUB_WORKSPACE}/bin/ssl-darwin64-arm64
+            echo "LDFLAGS=-L${OPENSSL_INSTALL_PATH}/lib" >> $GITHUB_ENV
+            echo "CRYPTOGRAPHY_SUPPRESS_LINK_FLAGS=1" >> $GITHUB_ENV
+            echo "CFLAGS=-I${OPENSSL_INSTALL_PATH}/include -arch arm64 -arch x86_64 ${CFLAGS}" >> $GITHUB_ENV
+            echo "ARCHFLAGS=-arch x86_64 -arch arm64" >> $GITHUB_ENV
+          else
+            cd "${GITHUB_WORKSPACE}/src/openssl-${openssl_archs}"
+            # install_sw saves us ages processing man pages :-)
+            $MAKE install_sw
+          fi
+
+      - name: Run OpenSSL
+        if: matrix.goal == 'build'
+        run: |
+          "${OPENSSL_INSTALL_PATH}/bin/openssl" version
+          "${OPENSSL_INSTALL_PATH}/bin/openssl" version -f
+          file "${OPENSSL_INSTALL_PATH}/bin/openssl"
+
+      - name: Get latest stable Python source
+        if: matrix.goal == 'build' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        run: |
+          cd "${GITHUB_WORKSPACE}/src"
+          git clone https://github.com/python/cpython.git
+          cd "${PYTHON_SOURCE_PATH}"
+          # Pin Windows to 3.11.6 for the moment
+          # if [[ "${RUNNER_OS}" == "Windows" ]]; then
+          #   export LATEST_STABLE_TAG="v3.11.6"
+          # else
+          export LATEST_STABLE_TAG=$(git tag --list | grep -v a | grep -v rc | grep -v b | sort -Vr | head -n1)
+          # fi
+          git checkout "${LATEST_STABLE_TAG}"
+          export COMPILED_PYTHON_VERSION=${LATEST_STABLE_TAG:1} # Trim the "v" prefix
+          echo "COMPILED_PYTHON_VERSION=${COMPILED_PYTHON_VERSION}" >> $GITHUB_ENV
+
+      - name: Mac/Linux Configure Python
+        if: matrix.goal == 'build' && runner.os != 'Windows' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        run: |
+          cd "${PYTHON_SOURCE_PATH}"
+          if ([ "${RUNNER_OS}" == "macOS" ] && [ "$arch" == "universal2" ]); then
+            extra_args=( "--enable-universalsdk" "--with-universal-archs=universal2" )
+          else
+            extra_args=( )
+          fi
+          ./configure --with-openssl="${OPENSSL_INSTALL_PATH}" \
+                      --prefix="${PYTHON_INSTALL_PATH}" \
+                      --enable-shared \
+                      --with-ensurepip=upgrade \
+                      --enable-optimizations \
+                      --with-lto \
+                      "${extra_args[@]}"
+
+      - name: Windows Get External Python deps
+        if: matrix.goal == 'build' && runner.os == 'Windows' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        shell: powershell
+        run: |
+          cd "${env:PYTHON_SOURCE_PATH}"
+          PCBuild\get_externals.bat
+
+      - name: Windows overwrite external OpenSSL with local
+        if: matrix.goal == 'build' && runner.os == 'Windows' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        shell: powershell
+        run: |
+          cd "${env:PYTHON_SOURCE_PATH}"
+          $env:OPENSSL_EXT_PATH = "$(Get-Item externals\openssl-bin-* | Select -exp FullName)\"
+          echo "External OpenSSL was downloaded to ${env:OPENSSL_EXT_PATH}"
+          Remove-Item -recurse -force "${env:OPENSSL_EXT_PATH}*"
+          # Emulate what this script does:
+          # https://github.com/python/cpython/blob/main/PCbuild/openssl.vcxproj
+          $env:OPENSSL_EXT_TARGET_PATH = "${env:OPENSSL_EXT_PATH}${env:PYEXTERNALS_PATH}"
+          echo "Copying our OpenSSL to ${env:OPENSSL_EXT_TARGET_PATH}"
+          mkdir "${env:OPENSSL_EXT_TARGET_PATH}\include\openssl\"
+          Copy-Item -Path "${env:GITHUB_WORKSPACE}/src/openssl-${env:openssl_archs}\LICENSE.txt" -Destination "${env:OPENSSL_EXT_TARGET_PATH}\LICENSE" -Verbose
+          cp -v "$env:OPENSSL_INSTALL_PATH\lib\*" "${env:OPENSSL_EXT_TARGET_PATH}"
+          cp -v "$env:OPENSSL_INSTALL_PATH\bin\*" "${env:OPENSSL_EXT_TARGET_PATH}"
+          cp -v "$env:OPENSSL_INSTALL_PATH\include\openssl\*" "${env:OPENSSL_EXT_TARGET_PATH}\include\openssl\"
+          cp -v "$env:OPENSSL_INSTALL_PATH\include\openssl\applink.c" "${env:OPENSSL_EXT_TARGET_PATH}\include\"
+
+      - name: Windows Install sphinx-build
+        if: matrix.goal == 'build' && runner.os == 'Windows' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        shell: powershell
+        run: |
+          pip install --upgrade pip
+          pip install --upgrade sphinx
+          sphinx-build --version
+
+      - name: Windows Config/Build Python
+        if: matrix.goal == 'build' && runner.os == 'Windows' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        shell: powershell
+        run: |
+          cd "${env:PYTHON_SOURCE_PATH}"
+          # We need out custom openssl.props which uses OpenSSL 3 DLL names
+          Copy-Item -Path "${env:GITHUB_WORKSPACE}\src\tools\openssl.props" -Destination PCBuild\ -Verbose
+          echo "Building for ${env:PYBUILDRELEASE_ARCH}..."
+          PCBuild\build.bat -m --pgo -c Release -p "${env:PYBUILDRELEASE_ARCH}"
+
+      - name: Mac/Linux Build Python
+        if: matrix.goal == 'build' && runner.os != 'Windows' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        run: |
+          cd "${PYTHON_SOURCE_PATH}"
+          echo "Running: ${MAKE} ${MAKEOPT}"
+          $MAKE $MAKEOPT
+
+      - name: Mac/Linux Install Python
+        if: matrix.goal == 'build' && runner.os != 'Windows' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        run: |
+          cd "${PYTHON_SOURCE_PATH}"
+          $MAKE altinstall
+          $MAKE bininstall
+          export PATH="${PATH}:${PYTHON_INSTALL_PATH}/bin"
+          echo "PATH=${PATH}" >> $GITHUB_ENV
+          echo "PATH: ${PATH}"
+
+      - name: Run Python
+        run: |
+          "${PYTHON}" -V
+
+      - name: Upgrade pip, wheel, etc
+        run: |
+          curl $curl_retry -O https://bootstrap.pypa.io/get-pip.py
+          "${PYTHON}" get-pip.py
+          "${PYTHON}" -m pip install --upgrade pip
+          "${PYTHON}" -m pip install --upgrade wheel
+          "${PYTHON}" -m pip install --upgrade setuptools
+
+      - name: Install pip requirements
+        run: |
+          echo "before anything..."
+          "${PYTHON}" -m pip list
+          if ([ "${RUNNER_OS}" == "macOS" ] && [ "$arch" == "universal2" ]); then
+            # cffi is a dep of cryptography and doesn't ship
+            # a universal2 wheel so we must build one ourself :-/
+            export CFLAGS="-arch x86_64 -arch arm64"
+            export ARCHFLAGS="-arch x86_64 -arch arm64"
+            "${PYTHON}" -m pip install --upgrade --force-reinstall --no-binary :all: \
+                              --no-cache-dir --no-deps --use-pep517 \
+                              --use-feature=no-binary-enable-wheel-cache \
+                              cffi 
+            echo "before cryptography..."
+            "${PYTHON}" -m pip list
+            # cryptography has a universal2 wheel but getting it installed
+            # on x86-64 MacOS is a royal pain in the keester.
+            "${PYTHON}" -m pip download --only-binary :all: \
+                                          --dest . \
+                                          --no-cache \
+                                          --no-deps \
+                                          --platform macosx_10_15_universal2 \
+                                          cryptography
+            "${PYTHON}" -m pip install --force-reinstall --no-deps cryptography*.whl
+            echo "after cryptography..."
+            "${PYTHON}" -m pip list
+            "${PYTHON}" -m pip install --upgrade --no-binary :all: -r requirements.txt
+          else
+            "${PYTHON}" -m pip install --upgrade -r requirements.txt
+            echo "after requirements..."
+            "${PYTHON}" -m pip list
+            "${PYTHON}" -m pip install --force-reinstall --no-deps --upgrade cryptography
+          fi
+          echo "after everything..."
+          "${PYTHON}" -m pip list
+
+      - name: Install PyInstaller
+        if: matrix.goal == 'build'
+        run: |
+          git clone https://github.com/pyinstaller/pyinstaller.git
+          cd pyinstaller
+          export latest_release=$(git tag --list | grep -v dev | grep -v rc | sort -Vr | head -n1)
+          #V6.0.0 causes errors on staticx
+          if [[ "${staticx}" == "yes" ]]; then
+            git checkout "v5.13.2"
+          elif [[ "${RUNNER_OS}" == "Windows" ]]; then
+            git checkout "v5.13.2"
+          elif [[ "${RUNNER_OS}" == "macOS" ]]; then
+            git checkout "v5.13.2"
+          else
+            git checkout "${latest_release}"
+          fi
+          # remove pre-compiled bootloaders so we fail if bootloader compile fails
+          rm -rvf PyInstaller/bootloader/*-*/*
+          cd bootloader
+          case "${arch}" in
+            "Win32")
+              export PYINSTALLER_BUILD_ARGS="--target-arch=32bit"
+              ;;
+            "Win64")
+              export PYINSTALLER_BUILD_ARGS="--target-arch=64bit"
+              ;;
+          esac
+          echo "PyInstaller build arguments: ${PYINSTALLER_BUILD_ARGS}"
+          "${PYTHON}" ./waf all $PYINSTALLER_BUILD_ARGS
+          cd ..
+          echo "---- Installing PyInstaller ----"
+          "${PYTHON}" -m pip install .
+
+      - name: Build GAM with PyInstaller
+        if: matrix.goal != 'test'
+        run: |
+          if [[ "${staticx}" == "yes" ]]; then
+            export distpath="./dist/gam"
+            export gampath="${distpath}"
+          else
+            export distpath="./dist"
+            export gampath="${distpath}/gam"
+          fi
+          mkdir -p -v "${gampath}"
+          if [[ "${RUNNER_OS}" == "macOS" ]]; then
+            export gampath=$($PYTHON -c "import os; print(os.path.realpath('$gampath'))")
+          elif [[ "${RUNNER_OS}" == "Windows" ]]; then
+            # Work around issue where PyInstaller picks up python3.dll from other Python versions
+            # https://github.com/pyinstaller/pyinstaller/issues/7102
+            export PATH="/usr/bin"
+          else
+            export gampath=$(realpath "${gampath}")
+          fi
+          export gam="${gampath}/gam"
+          echo "gampath=${gampath}" >> $GITHUB_ENV
+          echo "gam=${gam}" >> $GITHUB_ENV
+          echo -e "GAM: ${gam}\nGAMPATH: ${gampath}"
+          # TEMP force everything back to one file.
+          export PYINSTALLER_BUILD_ONEFILE="yes"
+          export distpath="./dist/gam"
+          export gampath="${distpath}"
+          "${PYTHON}" -m PyInstaller --clean --noconfirm --distpath="${distpath}" gam.spec
+          cat build/gam/warn-gam.txt
+
+      - name: Copy extra package files
+        if: matrix.goal == 'build'
+        run: |
+          cp -v cacerts.pem $gampath
+          cp -v LICENSE $gampath
+          cp -v GamCommands.txt $gampath
+          cp -v GamUpdate.txt $gampath
+          if [[ "${RUNNER_OS}" == "Windows" ]]; then
+              cp -v gam-setup.bat $gampath
+          fi
+
+      - name: Install StaticX
+        if: matrix.staticx == 'yes'
+        run: |
+          "${PYTHON}" -m pip install --upgrade patchelf-wrapper
+          "${PYTHON}" -m pip install --upgrade staticx
+
+      - name: Make StaticX
+        if: matrix.staticx == 'yes'
+        run: |
+          case $RUNNER_ARCH in
+            X64)
+              ldlib=/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
+              ;;
+            ARM64)
+              ldlib=/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1
+              ;;
+          esac
+          echo "ldlib=${ldlib}"
+          $PYTHON -m staticx -l "${ldlib}" "${gam}" "${gam}-staticx"
+          rm -v "${gam}"
+          mv -v "${gam}-staticx" "${gam}"
+
+      - name: Basic Tests all jobs
+        id: basictests
+        run: |
+          $PYTHON -m unittest discover --start-directory ./ --pattern "*_test.py" --buffer || if [ $? != 5 ]; then exit $?; fi # exit 5 is no tests
+          $gam version extended nooffseterror
+          export GAMVERSION=$($gam version simple)
+          echo "GAM Version ${GAMVERSION}"
+          echo "GAMVERSION=${GAMVERSION}" >> $GITHUB_ENV
+
+      - name: Linux/MacOS package
+        if: runner.os != 'Windows' && matrix.goal == 'build'
+        run: |
+          if [[ "${RUNNER_OS}" == "macOS" ]]; then
+              GAM_ARCHIVE="gam-${GAMVERSION}-macos-${arch}.tar.xz"
+          elif [[ "${RUNNER_OS}" == "Linux" ]]; then
+              if [[ "${staticx}" == "yes" ]]; then
+                libver="legacy"
+              else
+                libver="glibc$(ldd --version | awk '/ldd/{print $NF}')"
+              fi
+              GAM_ARCHIVE="gam-${GAMVERSION}-linux-$(arch)-${libver}.tar.xz"
+          fi
+          tar -C dist/ --create --verbose --exclude-from "${GITHUB_WORKSPACE}/.github/actions/package_exclusions.txt" --file $GAM_ARCHIVE --xz gam
+
+      - name: Windows package
+        if: runner.os == 'Windows' && matrix.goal != 'test'
+        run: |
+          cd dist/
+          GAM_ARCHIVE="../gam-${GAMVERSION}-windows-${GAM_ARCHIVE_ARCH}.zip"
+          /c/Program\ Files/7-Zip/7z.exe a -tzip $GAM_ARCHIVE gam "-xr@${GITHUB_WORKSPACE}/.github/actions/package_exclusions.txt" -bb3
+          cd ..
+          /c/Program\ Files\ \(x86\)/WiX\ Toolset\ v3.11/bin/candle.exe -arch "${WIX_ARCH}" gam.wxs
+          /c/Program\ Files\ \(x86\)/WiX\ Toolset\ v3.11/bin/light.exe -ext /c/Program\ Files\ \(x86\)/WiX\ Toolset\ v3.11/bin/WixUIExtension.dll gam.wixobj -o "gam-${GAMVERSION}-windows-${GAM_ARCHIVE_ARCH}.msi" || true;
+          rm -v -f *.wixpdb
+
+      - name: Basic Tests build jobs only
+        if: matrix.goal != 'test' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        run: |
+          export voutput=$($gam version extended nooffseterror)
+          export python_line=$(echo -e "${voutput}" | grep "Python ")
+          export python_arr=($python_line)
+          export this_python=${python_arr[1]}
+          if [[ "${this_python}" != "${COMPILED_PYTHON_VERSION}" ]]; then
+            echo "ERROR: Tried to compile Python ${COMPILED_PYTHON_VERSION} but ended up with ${this_python}"
+            exit 1
+          fi
+          export openssl_line=$(echo -e "${voutput}" | grep "OpenSSL ")
+          export openssl_arr=($openssl_line)
+          export this_openssl="${openssl_arr[1]}"
+          if [[ "${this_openssl}" != "${COMPILED_OPENSSL_VERSION}" ]]; then
+            echo "ERROR: Tried to compile OpenSSL ${COMPILED_OPENSSL_VERSION} but ended up with ${this_openssl}"
+            exit 1
+          fi
+          echo "We successfully compiled Python ${this_python} and OpenSSL ${this_openssl}"
+
+      - name: Live API tests push only
+        if: (github.event_name == 'push' || github.event_name == 'schedule') && matrix.fullGamTest == 'yes'
+        env:
+          PASSCODE: ${{ secrets.PASSCODE }}
+        run: |
+          source ../.github/actions/decrypt.sh ../.github/actions/creds.tar.xz.gpg creds.tar.xz "${GAMCFGDIR}"
+          mv -v "${GAMCFGDIR}/oauth2.txt-gam-gha-${JID}" "${GAMCFGDIR}/oauth2.txt"
+          rm -v $GAMCFGDIR/oauth2.txt-gam*
+          export gam_user="gam-gha-${JID}@pdl.jaylee.us"
+          echo "gam_user=${gam_user}" >> $GITHUB_ENV
+          $gam config customer_id "C03uzfv2s" save
+          $gam config domain "pdl.jaylee.us" save
+          $gam config admin_email "${gam_user}" save
+          $gam config enable_dasa false save
+          $gam oauth info
+          $gam oauth refresh
+          $gam config enable_dasa true save 
+          $gam create signjwtserviceaccount
+          $gam checkconn
+          $gam user "$gam_user" check serviceaccount
+          $gam info domain
+          $gam info user
+          export tstamp=$($PYTHON -c "import time; print(time.time_ns())")
+          export newbase="gha_test_${JID}_${tstamp}"
+          export newuser="${newbase}@pdl.jaylee.us"
+          export newgroup="${newbase}-group@pdl.jaylee.us"
+          export newalias="${newbase}-alias@pdl.jaylee.us"
+          export newbuilding="${newbase}-building"
+          export newresource="${newbase}-resource"
+          export newou="aaaGithub Actions/${newbase}"
+          
+          # cleanup old runs
+          $gam config enable_dasa false save
+          $gam config csv_output_row_filter "name:regex:gha_test_${JID}_" print vaultholds || if [ $? != 55 ]; then exit $?; fi | $gam csv - gam delete vaulthold "id:~~holdId~~" matter "id:~~matterId~~"
+          $gam config enable_dasa true save
+          $gam config csv_output_row_filter "name:regex:gha_test_${JID}_" print features | $gam csv - gam delete feature ~name
+          $gam config csv_output_row_filter "name:regex:^gha_test_${JID}_" user $gam_user print shareddrives asadmin | $gam csv - gam user $gam_user delete shareddrive ~id nukefromorbit
+          $gam print users query "gha.jid=$JID" | $gam csv - gam delete user ~primaryEmail
+          $gam config csv_output_row_filter "name:regex:^gha_test_${JID}_" print ous fromparent "aaaGithub Actions" | $gam csv - gam delete ou ~orgUnitId
+          $gam config csv_output_row_filter "email:regex:^gha_test_${JID}_" print cigroups | $gam csv - gam delete cigroup ~email
+          $gam config csv_output_row_filter "resourceId:regex:^gha_test_${JID}_" print resources | $gam csv - gam delete resource ~resourceId
+          $gam config csv_output_row_filter "buildingId:regex:^gha_test_${JID}_" print buildings | $gam csv - gam delete building ~buildingId
+          $gam config csv_output_row_filter "Emails.1.address:regex:^gha_test-${JID}_" print contacts | $gam csv - gam delete contact ~ContactID
+
+          echo "Creating OrgUnit ${newou}"
+          $gam create ou "${newou}"
+          export GAM_THREADS=5
+          echo email > sample.csv;
+          for i in {1..10}; do
+            echo "${newbase}-bulkuser-$i" >> sample.csv;
+          done
+          driveid=$($gam user $gam_user add shareddrive "${newbase}" returnidonly)
+          echo "Created shared drive ${driveid}"
+          $gam create user $newuser firstname GHA lastname $JID displayname "Github Actions ${JID}" password random ou "${newou}" recoveryphone 12125121110 recoveryemail jay0lee@gmail.com gha.jid $JID languages en+,en-GB-
+          $gam user $newuser update photo https://dummyimage.com/400x600/000/fff
+          $gam user $newuser get photo
+          $gam user $newuser delete photo
+          $gam create alias $newalias user $newuser
+          $gam create group $newgroup name "GHA $JID group" description "This is a description" isarchived true
+          $gam user $gam_user sendemail recipient $newuser subject "test message $newbase" message "GHA test message"
+          $gam user $gam_user sendemail recipient exchange@pdl.jaylee.us subject "test ${tstamp}" message "test message"
+          $gam config enable_dasa false save
+          $gam create contact firstname GHA lastname "$JID" email work "${newbase}@example.com" primary
+          $gam print contacts
+          $gam user $newuser add license workspaceenterpriseplus
+          $gam print privileges
+          $gam config enable_dasa true save
+          $gam update cigroup $newgroup security memberrestriction 'member.type == 1 || member.customer_id == groupCustomerId()'
+          $gam info cigroup $newgroup
+          $gam update group $newgroup add owner $gam_user
+          $gam update group $newgroup add member $newuser
+          $gam config enable_dasa false save
+          $gam create admin $newuser _GROUPS_EDITOR_ROLE CUSTOMER # condition nonsecuritygroup
+          $gam create admin $newgroup _HELP_DESK_ADMIN_ROLE org_unit "${newou}"
+          $gam config csv_output_row_filter "assignedToUser:regex:${newuser}" print admins | $gam csv - gam delete admin "~roleAssignmentId"
+          $gam config csv_output_row_filter "assignedToGroup:regex:${newgroup}" print admins | $gam csv - gam delete admin "~roleAssignmentId"
+          $gam config enable_dasa false save
+          $gam csv sample.csv gam create user ~~email~~ firstname "GHA Bulk" lastname ~~email~~ gha.jid $JID ou "${newou}"
+          $gam csv sample.csv gam update user ~~email~~ recoveryphone 12125121110 recoveryemail jay0lee@gmail.com password random displayname "GitHub Actions Bulk ${JID}"
+          $gam csv sample.csv gam update user ~~email~~ recoveryphone "" recoveryemail ""
+          $gam config enable_dasa false save
+          $gam csv sample.csv gam user ~email add license workspaceenterpriseplus
+          $gam config enable_dasa true save
+          $gam csv sample.csv gam user $gam_user sendemail recipient ~~email~~@pdl.jaylee.us subject "test message $newbase" message "GHA test message"
+          $gam csv sample.csv gam update group $newgroup add member ~email
+          $gam info group $newgroup
+          $gam info cigroup $newgroup membertree
+          # confirm mailbox is provisoned before continuing
+          $gam user $newuser waitformailbox
+          $gam user $newuser imap on
+          $gam user $newuser show imap
+          $gam user $newuser show delegates
+          #$gam user $newuser add contactdelegate "${newbase}-bulkuser-1"
+          #$gam user $newuser print contactdelegates
+          export biohazard=$(echo -e '\xe2\x98\xa3')
+          $gam user $newuser label "$biohazard unicode biohazard $biohazard"
+          $gam user $newuser show labels
+          $gam user $newuser show labels > labels.txt
+          $gam user $gam_user importemail subject "GHA import $newbase" message "This is a test import" labels IMPORTANT,UNREAD,INBOX,STARRED
+          $gam user $gam_user insertemail subject "GHA insert $newbase" file gam.py labels INBOX,UNREAD # yep body is gam code
+          $gam user $gam_user sendemail subject "GHA send $gam_user $newbase" file gam.py recipient admin@pdl.jaylee.us
+          $gam user $gam_user draftemail subject "GHA draft $newbase" message "Draft message test"
+          $gam csvfile sample.csv:email waitformailbox
+          $gam user $newuser delegate to "${newbase}-bulkuser-1" || if [ $? != 50 ]; then exit $?; fi # expect a 50 return code (delegation failed)
+          $gam users "$gam_user $newbase-bulkuser-1 $newbase-bulkuser-2 $newbase-bulkuser-3" delete messages query in:anywhere maxtodelete 99999 doit || if [ $? != 60 ]; then exit $?; fi # expect a 60 return code (no messages)
+          $gam users "$newbase-bulkuser-4 $newbase-bulkuser-5 $newbase-bulkuser-6" trash messages query in:anywhere maxtotrash 99999 doit || if [ $? != 60 ]; then exit $?; fi # expect a 60 return code (no messages)
+          $gam users "$newbase-bulkuser-7 $newbase-bulkuser-8 $newbase-bulkuser-9" modify messages query in:anywhere maxtomodify 99999 addlabel IMPORTANT addlabel STARRED doit || if [ $? != 60 ]; then exit $?; fi # expect a 60 return code (no messages)
+          $gam user $newuser delete label --ALL_LABELS--
+          $gam config csv_output_row_filter "name:regex:gha-test-${JID}" print features | $gam csv - gam delete feature ~name
+          $gam create feature name VC-$newbase
+          $gam create feature name Whiteboard-$newbase
+          $gam create building "My Building - $newbase" id $newbuilding floors 1,2,3,4,5,6,7,8,9,10,11,12,14,15 description "No 13th floor here..."
+          $gam create resource $newresource "Resource Calendar $tstamp" capacity 25 features Whiteboard-$newbase,VC-$newbase building $newbuilding floor 15 type Room
+          $gam info resource $newresource
+          $gam user $newuser add drivefile drivefilename "TPS Reports" mimetype gfolder
+          $gam user $newuser show filelist
+          $gam calendar $gam_user printacl | $gam csv - gam calendar $gam_user delete ~id # clear ACLs
+          $gam calendar $gam_user add read domain
+          $gam calendar $gam_user add freebusy default
+          $gam calendar $gam_user add editor $newuser
+          $gam calendar $gam_user showacl
+          $gam calendar $gam_user printacl | $gam csv - gam calendar $gam_user delete ~id
+          $gam calendar $gam_user addevent summary "GHA test event" start +1h end +2h attendee $newgroup hangoutsmeet guestscanmodify true sendupdates all
+          $gam calendar $gam_user printevents after -0d
+          $gam config enable_dasa false save
+          matterid=uid:$($gam create vaultmatter name "GHA matter $newbase" description "test matter" collaborators $newuser returnidonly)
+          $gam create vaulthold matter $matterid name "GHA hold $newbase" corpus mail accounts $newuser
+          $gam print vaultmatters matterstate open
+          $gam print vaultholds matter $matterid
+          $gam print vaultcount matter $matterid corpus mail everyone todrive tdnobrowser
+          $gam create vaultexport matter $matterid name "GHA export $newbase" corpus mail accounts $newuser
+          $gam print exports matter $matterid | $gam csv - gam info export $matterid id:~~id~~
+          $gam config enable_dasa true save
+          $gam csv sample.csv gam user ~email add calendar id:$newresource
+          $gam delete resource $newresource
+          $gam delete feature Whiteboard-$newbase
+          $gam delete feature VC-$newbase
+          $gam delete building $newbuilding
+          $gam delete group $newgroup
+          $gam config enable_dasa false save
+          echo start
+          $gam user $newuser delete license workspaceenterpriseplus
+          echo finish
+          $gam config enable_dasa true save
+          $gam whatis $newuser || if [ $? != 20 ]; then exit $?; fi # expect a 20 return code (is a user)
+          $gam user $gam_user show tokens
+          $gam config enable_dasa false save
+          download_dir="${RUNNER_TEMP}/TEMP_DELETE_ME"
+          mkdir -v "$download_dir"
+          $gam print exports matter $matterid | $gam csv - gam download export $matterid id:~~id~~ targetfolder "$download_dir"
+          rm -rvf "$download_dir"
+          $gam delete hold "GHA hold $newbase" matter $matterid
+          $gam update matter $matterid action close
+          $gam update matter $matterid action delete
+          # shakes off vault hold on user so we can delete
+          $gam print users query "email:${newuser}" orgunitpath | $gam csv - gam update user ~primaryEmail ou ~orgUnitPath
+          $gam user $newuser show holds || if [ $? != 55 ]; then exit $?; fi # expect a 55 return code
+          export sn="$JID$JID$JID$JID-$(openssl rand -base64 32 | sed 's/[^a-zA-Z0-9]//g')"
+          $gam create device serialnumber $sn devicetype android
+          $gam config enable_dasa true save
+          $gam print users query "gha.jid=$JID" | $gam csv - gam delete user ~primaryEmail  || if [ $? != 50 ]; then exit $?; fi # expect a 50 return code (vault hold on user)
+          $gam delete contacts emailmatchpattern "^${newbase}@example.com$"
+          $gam print mobile
+          $gam print devices
+          $gam print browsers
+          $gam print cros allfields orderby serialnumber
+          $gam show crostelemetry storagepercentonly
+          $gam report usageparameters customer
+          $gam report usage customer parameters gmail:num_emails_sent,accounts:num_1day_logins
+          $gam report customer todrive tdnobrowser
+          #$gam report users fields accounts:is_less_secure_apps_access_allowed,gmail:last_imap_time,gmail:last_pop_time filters "accounts:last_login_time>2019-01-01T00:00:00.000Z" todrive tdnobrowser
+          $gam report users todrive tdnobrowser
+          $gam report admin start -3d todrive tdnobrowser
+          $gam print devices nopersonaldevices nodeviceusers filter "serial:$JID$JID$JID$JID-" | $gam csv - gam delete device id ~name
+          $gam config enable_dasa false save
+          $gam print userinvitations
+          $gam print userinvitations | $gam csv - gam send userinvitation ~name
+          $gam config enable_dasa false save
+          $gam create caalevel "zzz_${newbase}" basic condition ipsubnetworks 1.1.1.1/32,2.2.2.2/32 endcondition
+          $gam print caalevels
+          $gam delete caalevel "zzz_${newbase}"
+          $gam user $gam_user add drivefile localfile gam.py parentid "${driveid}"
+          $gam user $gam_user update shareddrive "${driveid}" ou "${newou}"
+          $gam user $gam_user show shareddrives asadmin
+          $gam user $gam_user update shareddrive "${driveid}" ou "aaaGithub Actions" # so we can delete our OU...
+          $gam user $gam_user delete shareddrive "${driveid}" nukefromorbit
+          echo "printer model count:"
+          ssoprofile=$($gam create inboundssoprofile name "El Goog ${newbase}" loginurl https://www.google.com logouturl https://www.google.com changepasswordurl https://www.google.com entityid ElGoog return_name_only)
+          $gam create inboundssocredential profile "id:${ssoprofile}" generate_key
+          #$gam create inboundssoassignment profile "id:${ssoprofile}" orgunit "${newou}" mode SAML_SSO
+          #$gam delete inboundssoassignment "orgunit:${newou}"
+          $gam delete inboundssoprofile "id:${ssoprofile}"
+          $gam print printermodels | wc -l
+          $gam print printers
+          printerid=$($gam create printer displayname "${newbase}" uri ipp://localhost:631 driverless description "made by $(gam_user)" ou "${newou}" nodetails | awk '{print substr($2, 1, length($2)-1)}')
+          $gam info printer "$printerid"
+          $gam delete printer "$printerid"
+          $gam delete ou "${newou}"
+
+      - name: Tar Cache archive
+        if: matrix.goal == 'build' && steps.cache-python-ssl.outputs.cache-hit != 'true'
+        working-directory: ${{ github.workspace }}
+        run: |
+          if [[ "${RUNNER_OS}" == "Windows" ]]; then
+            tar_folders="src/cpython/ bin/ssl"
+          else
+            tar_folders="bin/"
+          fi
+          tar cJvvf cache.tar.xz $tar_folders
+
+      - name: Archive production artifacts
+        uses: actions/upload-artifact@v4
+        if: (github.event_name == 'push' || github.event_name == 'schedule') && matrix.goal != 'test'
+        with:
+          name: gam-binaries-${{ env.GAMOS }}-${{ env.arch }}-${{ matrix.jid }}
+          path: |
+            src/*.tar.xz
+            src/*.zip
+            src/*.msi
+
+  merge:
+    if: (github.event_name == 'push' || github.event_name == 'schedule')
+    runs-on: ubuntu-latest
+    needs: build
+    permissions:
+      contents: write
+      packages: write
+    steps:
+      - name: Merge Artifacts
+        uses: actions/upload-artifact/merge@v4
+        with:
+          name: gam-binaries
+          pattern: gam-binaries-*
+
+#      - name: Delete Artifacts
+#        uses: geekyeggo/delete-artifact@v4
+#        with:
+#          name: gam-binaries-*
+
+  publish:
+    if: github.event_name == 'push'
+    runs-on: ubuntu-latest
+    needs: merge
+    permissions:
+      contents: write
+      packages: write
+      pull-requests: read
+
+    steps:
+
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+          fetch-depth: 0
+
+      - name: Download artifacts
+        uses: actions/download-artifact@v4
+
+      - name: VirusTotal Scan
+        uses: crazy-max/ghaction-virustotal@v4
+        with:
+          vt_api_key: ${{ secrets.VT_API_KEY }}
+          files: |
+            gam-binaries/*
+
+      - name: Set datetime version string
+        id: dateversion
+        run: |
+          export dateversion="$(date  +'%Y%m%d.%H%M%S')"
+          echo "Date version: ${dateversion}"
+          echo "dateversion=${dateversion}" >> $GITHUB_OUTPUT
+
+      - uses: "marvinpinto/action-automatic-releases@latest"
+        name: Publish draft release
+        with:
+          repo_token: "${{ secrets.GITHUB_TOKEN }}"
+          automatic_release_tag: "${{ steps.dateversion.outputs.dateversion }}"
+          prerelease: false
+          draft: true
+          files: |
+            gam-binaries/*

.github/workflows/codeql-analysis.yml

@@ -0,0 +1,70 @@
+# For most projects, this workflow file will not need changing; you simply need
+# to commit it to your repository.
+#
+# You may wish to alter this file to override the set of languages analyzed,
+# or to provide custom queries or build logic.
+#
+# ******** NOTE ********
+# We have attempted to detect the languages in your repository. Please check
+# the `language` matrix defined below to confirm you have the correct set of
+# supported CodeQL languages.
+#
+name: "CodeQL"
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    # The branches below must be a subset of the branches above
+    branches: [ main ]
+  schedule:
+    - cron: '25 10 * * 1'
+
+jobs:
+  analyze:
+    name: Analyze
+    runs-on: ubuntu-latest
+    permissions:
+      actions: read
+      contents: read
+      security-events: write
+
+    strategy:
+      fail-fast: false
+      matrix:
+        language: [ 'python' ]
+        # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
+        # Learn more about CodeQL language support at https://git.io/codeql-language-support
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v3
+
+    # Initializes the CodeQL tools for scanning.
+    - name: Initialize CodeQL
+      uses: github/codeql-action/init@v2
+      with:
+        languages: ${{ matrix.language }}
+        # If you wish to specify custom queries, you can do so here or in a config file.
+        # By default, queries listed here will override any specified in a config file.
+        # Prefix the list here with "+" to use these queries and those in the config file.
+        # queries: ./path/to/local/query, your-org/your-repo/queries@main
+
+    # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
+    # If this step fails, then you should remove it and run the build manually (see below)
+    - name: Autobuild
+      uses: github/codeql-action/autobuild@v2
+
+    # ℹ️ Command-line programs to run using the OS shell.
+    # 📚 https://git.io/JvXDl
+
+    # ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
+    #    and modify them (or add more) to build your code if your project
+    #    uses a compiled language
+
+    #- run: |
+    #   make bootstrap
+    #   make release
+
+    - name: Perform CodeQL Analysis
+      uses: github/codeql-action/analyze@v2

.github/workflows/get-cacerts.yml

@@ -0,0 +1,36 @@
+name: Check for Google Root CA Updates
+
+on:
+  push:
+  pull_request:
+  schedule:
+    - cron: '23 23 * * *'
+
+defaults:
+  run:
+    shell: bash
+    working-directory: src
+
+jobs:
+  check-apis:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@master
+        with:
+          persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal token
+          fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
+
+      - name: Check for updates
+        run: curl -o ./cacerts.pem -vvvv https://pki.goog/roots.pem
+
+      - name: Commit file
+        run: |
+          git config --local user.email "action@github.com"
+          git config --local user.name "GitHub Action"
+          git add cacerts.pem
+          git diff --quiet && git diff --staged --quiet || git commit -am '[ci skip] Updated cacerts.pem'
+
+      - name: Push changes
+        uses: ad-m/github-push-action@master
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }} 

.pre-commit-config.yaml

@@ -0,0 +1,32 @@
+# See https://pre-commit.com for more information
+
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v2.5.0
+    hooks:
+    -   id: trailing-whitespace
+    -   id: end-of-file-fixer
+    -   id: double-quote-string-fixer
+    -   id: check-yaml
+    -   id: check-docstring-first
+    -   id: name-tests-test
+    -   id: requirements-txt-fixer
+    -   id: check-merge-conflict
+
+-   repo: https://github.com/pre-commit/mirrors-yapf
+    rev: v0.30.0
+    hooks:
+    -   id: yapf
+        args: [--style=google, --in-place]
+
+-   repo: https://github.com/PyCQA/pylint
+    rev: pylint-2.5.0
+    hooks:
+    -   id: pylint
+        args: [--output-format=colorized]
+
+-   repo: https://github.com/asottile/pyupgrade
+    rev: v2.31.0
+    hooks:
+    -   id: pyupgrade
+        args: [--py37-plus]

LICENSE

@@ -199,349 +199,3 @@
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.
-
-
-
-APACHE HTTP SERVER SUBCOMPONENTS: 
-
-The Apache HTTP Server includes a number of subcomponents with
-separate copyright notices and license terms. Your use of the source
-code for the these subcomponents is subject to the terms and
-conditions of the following licenses. 
-
-For the mod_mime_magic component:
-
-/*
- * mod_mime_magic: MIME type lookup via file magic numbers
- * Copyright (c) 1996-1997 Cisco Systems, Inc.
- *
- * This software was submitted by Cisco Systems to the Apache Group in July
- * 1997.  Future revisions and derivatives of this source code must
- * acknowledge Cisco Systems as the original contributor of this module.
- * All other licensing and usage conditions are those of the Apache Group.
- *
- * Some of this code is derived from the free version of the file command
- * originally posted to comp.sources.unix.  Copyright info for that program
- * is included below as required.
- * ---------------------------------------------------------------------------
- * - Copyright (c) Ian F. Darwin, 1987. Written by Ian F. Darwin.
- *
- * This software is not subject to any license of the American Telephone and
- * Telegraph Company or of the Regents of the University of California.
- *
- * Permission is granted to anyone to use this software for any purpose on any
- * computer system, and to alter it and redistribute it freely, subject to
- * the following restrictions:
- *
- * 1. The author is not responsible for the consequences of use of this
- * software, no matter how awful, even if they arise from flaws in it.
- *
- * 2. The origin of this software must not be misrepresented, either by
- * explicit claim or by omission.  Since few users ever read sources, credits
- * must appear in the documentation.
- *
- * 3. Altered versions must be plainly marked as such, and must not be
- * misrepresented as being the original software.  Since few users ever read
- * sources, credits must appear in the documentation.
- *
- * 4. This notice may not be removed or altered.
- * -------------------------------------------------------------------------
- *
- */
-
-
-For the  modules\mappers\mod_imagemap.c component:
-
-  "macmartinized" polygon code copyright 1992 by Eric Haines, erich@eye.com
-
-For the  server\util_md5.c component:
-
-/************************************************************************
- * NCSA HTTPd Server
- * Software Development Group
- * National Center for Supercomputing Applications
- * University of Illinois at Urbana-Champaign
- * 605 E. Springfield, Champaign, IL 61820
- * httpd@ncsa.uiuc.edu
- *
- * Copyright  (C)  1995, Board of Trustees of the University of Illinois
- *
- ************************************************************************
- *
- * md5.c: NCSA HTTPd code which uses the md5c.c RSA Code
- *
- *  Original Code Copyright (C) 1994, Jeff Hostetler, Spyglass, Inc.
- *  Portions of Content-MD5 code Copyright (C) 1993, 1994 by Carnegie Mellon
- *     University (see Copyright below).
- *  Portions of Content-MD5 code Copyright (C) 1991 Bell Communications 
- *     Research, Inc. (Bellcore) (see Copyright below).
- *  Portions extracted from mpack, John G. Myers - jgm+@cmu.edu
- *  Content-MD5 Code contributed by Martin Hamilton (martin@net.lut.ac.uk)
- *
- */
-
-
-/* these portions extracted from mpack, John G. Myers - jgm+@cmu.edu */
-/* (C) Copyright 1993,1994 by Carnegie Mellon University
- * All Rights Reserved.
- *
- * Permission to use, copy, modify, distribute, and sell this software
- * and its documentation for any purpose is hereby granted without
- * fee, provided that the above copyright notice appear in all copies
- * and that both that copyright notice and this permission notice
- * appear in supporting documentation, and that the name of Carnegie
- * Mellon University not be used in advertising or publicity
- * pertaining to distribution of the software without specific,
- * written prior permission.  Carnegie Mellon University makes no
- * representations about the suitability of this software for any
- * purpose.  It is provided "as is" without express or implied
- * warranty.
- *
- * CARNEGIE MELLON UNIVERSITY DISCLAIMS ALL WARRANTIES WITH REGARD TO
- * THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
- * AND FITNESS, IN NO EVENT SHALL CARNEGIE MELLON UNIVERSITY BE LIABLE
- * FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
- * AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING
- * OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS
- * SOFTWARE.
- */
-
-/*
- * Copyright (c) 1991 Bell Communications Research, Inc. (Bellcore)
- *
- * Permission to use, copy, modify, and distribute this material
- * for any purpose and without fee is hereby granted, provided
- * that the above copyright notice and this permission notice
- * appear in all copies, and that the name of Bellcore not be
- * used in advertising or publicity pertaining to this
- * material without the specific, prior written permission
- * of an authorized representative of Bellcore.  BELLCORE
- * MAKES NO REPRESENTATIONS ABOUT THE ACCURACY OR SUITABILITY
- * OF THIS MATERIAL FOR ANY PURPOSE.  IT IS PROVIDED "AS IS",
- * WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES.  
- */
-
-For the  srclib\apr\include\apr_md5.h component: 
-/*
- * This is work is derived from material Copyright RSA Data Security, Inc.
- *
- * The RSA copyright statement and Licence for that original material is
- * included below. This is followed by the Apache copyright statement and
- * licence for the modifications made to that material.
- */
-
-/* Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
-   rights reserved.
-
-   License to copy and use this software is granted provided that it
-   is identified as the "RSA Data Security, Inc. MD5 Message-Digest
-   Algorithm" in all material mentioning or referencing this software
-   or this function.
-
-   License is also granted to make and use derivative works provided
-   that such works are identified as "derived from the RSA Data
-   Security, Inc. MD5 Message-Digest Algorithm" in all material
-   mentioning or referencing the derived work.
-
-   RSA Data Security, Inc. makes no representations concerning either
-   the merchantability of this software or the suitability of this
-   software for any particular purpose. It is provided "as is"
-   without express or implied warranty of any kind.
-
-   These notices must be retained in any copies of any part of this
-   documentation and/or software.
- */
-
-For the  srclib\apr\passwd\apr_md5.c component:
-
-/*
- * This is work is derived from material Copyright RSA Data Security, Inc.
- *
- * The RSA copyright statement and Licence for that original material is
- * included below. This is followed by the Apache copyright statement and
- * licence for the modifications made to that material.
- */
-
-/* MD5C.C - RSA Data Security, Inc., MD5 message-digest algorithm
- */
-
-/* Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
-   rights reserved.
-
-   License to copy and use this software is granted provided that it
-   is identified as the "RSA Data Security, Inc. MD5 Message-Digest
-   Algorithm" in all material mentioning or referencing this software
-   or this function.
-
-   License is also granted to make and use derivative works provided
-   that such works are identified as "derived from the RSA Data
-   Security, Inc. MD5 Message-Digest Algorithm" in all material
-   mentioning or referencing the derived work.
-
-   RSA Data Security, Inc. makes no representations concerning either
-   the merchantability of this software or the suitability of this
-   software for any particular purpose. It is provided "as is"
-   without express or implied warranty of any kind.
-
-   These notices must be retained in any copies of any part of this
-   documentation and/or software.
- */
-/*
- * The apr_md5_encode() routine uses much code obtained from the FreeBSD 3.0
- * MD5 crypt() function, which is licenced as follows:
- * ----------------------------------------------------------------------------
- * "THE BEER-WARE LICENSE" (Revision 42):
- * <phk@login.dknet.dk> wrote this file.  As long as you retain this notice you
- * can do whatever you want with this stuff. If we meet some day, and you think
- * this stuff is worth it, you can buy me a beer in return.  Poul-Henning Kamp
- * ----------------------------------------------------------------------------
- */
-
-For the srclib\apr-util\crypto\apr_md4.c component:
-
- * This is derived from material copyright RSA Data Security, Inc.
- * Their notice is reproduced below in its entirety.
- *
- * Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
- * rights reserved.
- *
- * License to copy and use this software is granted provided that it
- * is identified as the "RSA Data Security, Inc. MD4 Message-Digest
- * Algorithm" in all material mentioning or referencing this software
- * or this function.
- *
- * License is also granted to make and use derivative works provided
- * that such works are identified as "derived from the RSA Data
- * Security, Inc. MD4 Message-Digest Algorithm" in all material
- * mentioning or referencing the derived work.
- *
- * RSA Data Security, Inc. makes no representations concerning either
- * the merchantability of this software or the suitability of this
- * software for any particular purpose. It is provided "as is"
- * without express or implied warranty of any kind.
- *
- * These notices must be retained in any copies of any part of this
- * documentation and/or software.
- */
-
-For the srclib\apr-util\include\apr_md4.h component:
-
- *
- * This is derived from material copyright RSA Data Security, Inc.
- * Their notice is reproduced below in its entirety.
- *
- * Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
- * rights reserved.
- *
- * License to copy and use this software is granted provided that it
- * is identified as the "RSA Data Security, Inc. MD4 Message-Digest
- * Algorithm" in all material mentioning or referencing this software
- * or this function.
- *
- * License is also granted to make and use derivative works provided
- * that such works are identified as "derived from the RSA Data
- * Security, Inc. MD4 Message-Digest Algorithm" in all material
- * mentioning or referencing the derived work.
- *
- * RSA Data Security, Inc. makes no representations concerning either
- * the merchantability of this software or the suitability of this
- * software for any particular purpose. It is provided "as is"
- * without express or implied warranty of any kind.
- *
- * These notices must be retained in any copies of any part of this
- * documentation and/or software.
- */
-
-
-For the srclib\apr-util\test\testmd4.c component:
-
- *
- * This is derived from material copyright RSA Data Security, Inc.
- * Their notice is reproduced below in its entirety.
- *
- * Copyright (C) 1990-2, RSA Data Security, Inc. Created 1990. All
- * rights reserved.
- *
- * RSA Data Security, Inc. makes no representations concerning either
- * the merchantability of this software or the suitability of this
- * software for any particular purpose. It is provided "as is"
- * without express or implied warranty of any kind.
- *
- * These notices must be retained in any copies of any part of this
- * documentation and/or software.
- */
-
-For the srclib\apr-util\xml\expat\conftools\install-sh component:
-
-#
-# install - install a program, script, or datafile
-# This comes from X11R5 (mit/util/scripts/install.sh).
-#
-# Copyright 1991 by the Massachusetts Institute of Technology
-#
-# Permission to use, copy, modify, distribute, and sell this software and its
-# documentation for any purpose is hereby granted without fee, provided that
-# the above copyright notice appear in all copies and that both that
-# copyright notice and this permission notice appear in supporting
-# documentation, and that the name of M.I.T. not be used in advertising or
-# publicity pertaining to distribution of the software without specific,
-# written prior permission.  M.I.T. makes no representations about the
-# suitability of this software for any purpose.  It is provided "as is"
-# without express or implied warranty.
-#
-
-For the test\zb.c component:
-
-/*                          ZeusBench V1.01
-			    ===============
-
-This program is Copyright (C) Zeus Technology Limited 1996.
-
-This program may be used and copied freely providing this copyright notice
-is not removed.
-
-This software is provided "as is" and any express or implied waranties, 
-including but not limited to, the implied warranties of merchantability and
-fitness for a particular purpose are disclaimed.  In no event shall 
-Zeus Technology Ltd. be liable for any direct, indirect, incidental, special, 
-exemplary, or consequential damaged (including, but not limited to, 
-procurement of substitute good or services; loss of use, data, or profits;
-or business interruption) however caused and on theory of liability.  Whether
-in contract, strict liability or tort (including negligence or otherwise) 
-arising in any way out of the use of this software, even if advised of the
-possibility of such damage.
-
-     Written by Adam Twiss (adam@zeus.co.uk).  March 1996
-
-Thanks to the following people for their input:
-  Mike Belshe (mbelshe@netscape.com) 
-  Michael Campanella (campanella@stevms.enet.dec.com)
-
-*/
-
-For the expat xml parser component:
-
-Copyright (c) 1998, 1999, 2000 Thai Open Source Software Center Ltd
-                               and Clark Cooper
-
-Permission is hereby granted, free of charge, to any person obtaining
-a copy of this software and associated documentation files (the
-"Software"), to deal in the Software without restriction, including
-without limitation the rights to use, copy, modify, merge, publish,
-distribute, sublicense, and/or sell copies of the Software, and to
-permit persons to whom the Software is furnished to do so, subject to
-the following conditions:
-	
-The above copyright notice and this permission notice shall be included
-in all copies or substantial portions of the Software.
-	
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
-IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
-CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
-TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
-SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-
-====================================================================

README.md

@@ -1,85 +1,41 @@
-GAM, the Google Apps Manager
-============================
+GAM is a command line tool for Google Workspace admins to manage domain and user settings quickly and easily.
 
-Dito GAM is a free, open source command line tool for
-Google Apps Administrators to efficiently manage
-domain and user settings quickly and easily. GAM has support
-for many features, such as
+![Build Status](https://github.com/GAM-team/GAM/workflows/Build%20and%20test%20GAM/badge.svg)
 
-* creating, deleting, and updating users, aliases, groups, 
-  organizations, and resource calendars
-* modifying user email settings such as IMAP, signatures,
-  vacation messages, profile sharing, email forwarding,
-  send as address, labels, and features.
-* delegating mailboxes and calendars to other users
-* modifying calendar access rights for users and resource calendars.
-* auditing user accounts and mailboes
-* monitoring incoming and outgoing email
-* generating detailed reports for users, groups, resources,
-  account activity, email clients, and quotas.
+# Quick Start
 
+## Linux / MacOS
 
-Resources
-========
+Open a terminal and run:
 
-There are a number of GAM resources available via several different
-websites.
+```sh
+bash <(curl -s -S -L https://gam-shortn.appspot.com/gam-install)
+```
 
-Source Repository
------------------
+this will download GAM, install it and start setup.
 
-The official GAM source repository is on [Github][github].
+## Windows
 
+Download the MSI Installer from the [GitHub Releases] page. Install the MSI and you'll be prompted to setup GAM.
 
-Downloads
----------
+# Documentation
 
-You can download current and pre-released versions of GAM from 
-the [Github Releases][github releases] page.  Final releases
-are also available on [Google Drive][google drive]
+The GAM documentation is hosted in the [GitHub Wiki]
 
+# Mailing List / Discussion group
 
-Wiki Documentation
-----
+The GAM mailing list / discussion group is hosted on [Google Groups].  You can join the list and interact via email, or just post from the web itself.
 
-The GAM documentation is currently hosted on Google Code
-and can be found at the [Gam Getting Started Wiki][gam wiki]
+# Chat Room
 
-Mailing List / Discussion group
--------------------------------
+There is a public chat room hosted in Google Chat. [Instructions to join](https://github.com/GAM-team/GAM/wiki/GAM-Public-Chat-Room).
 
-The GAM mailing list / discussion group is hosted
-on [Google Groups].  You can join the list and interact
-via email, or just post from the web itself.
+# Author
 
+GAM is maintained by [Jay Lee](mailto:jay0lee@gmail.com). Please direct "how do I?" questions to [Google Groups].
 
-Author
-======
-
-GAM is maintained by <a href="mailto:jay0lee@gmail.com">Jay Lee</a>.
-
-THANKS TO
-=========
-
-GAM is made possible and maintained by the work of Dito.
-Who is Dito?
-
-Dito is solely focused on moving organizations to Google's
-cloud. After hundreds of successful deployments over the
-last 5 years, we have gained notoriety for our complete
-understanding of the platform, our change management &
-training ability, and our rock-star deployment engineers.
-We are known worldwide as the Google Apps Experts.
-
-Need a Google Apps Expert? 
-[Contact Dito](http://ditoweb.com/contact), which offers
-[free premium GAM support](http://www.ditoweb.com/dito-gam)
-for domains that sign up through Dito.
-
-
-[github releases]: https://github.com/jay0lee/GAM/releases
-[github]: https://github.com/jay0lee/GAM/
-[google code downloads]: https://code.google.com/p/google-apps-manager/wiki/Downloads
-[google drive]: https://googledrive.com/host/0B0YvUuHHn3MnbFl6N0k1UXcwdVk/
-[gam wiki]: https://code.google.com/p/google-apps-manager/wiki/GAM3GettingStarted
-[google groups]: http://groups.google.com/group/google-apps-manager
+[GAM release]: https://github.com/GAM-team/GAM/releases
+[GitHub Releases]: https://github.com/GAM-team/GAM/releases
+[GitHub]: https://github.com/GAM-team/GAM/tree/master
+[GitHub Wiki]: https://github.com/GAM-team/GAM/wiki/
+[Google Groups]: http://groups.google.com/group/google-apps-manager

admin-settings-v1.json

@@ -1,642 +0,0 @@
-{
- "kind": "discovery#restDescription",
- "discoveryVersion": "v1",
- "id": "admin-settings:v1",
- "name": "admin-settings",
- "version": "v1",
- "revision": "20130823",
- "title": "Admin Settings API (read-only calls)",
- "description": "Lets you access Google Apps Admin Settings",
- "ownerDomain": "google.com",
- "ownerName": "Google",
- "icons": {
-  "x16": "http://www.google.com/images/icons/product/search-16.gif",
-  "x32": "http://www.google.com/images/icons/product/search-32.gif"
- },
- "documentationLink": "https://developers.google.com/admin-sdk/admin-settings",
- "protocol": "rest",
- "baseUrl": "https://apps-apis.google.com/",
- "basePath": "/a/feeds/domain/2.0/",
- "rootUrl": "https://apps-apis.google.com/",
- "servicePath": "/a/feeds/domain/2.0/",
- "parameters": {
-  "v": {
-   "type": "string",
-   "description": "GData Version",
-   "default": "2.0",
-   "enum": [
-    "2.0"
-   ],
-   "enumDescriptions": [
-    "GData 2.0"
-   ],
-   "location": "query"
-  },
-  "alt": {
-   "type": "string",
-   "description": "Data format for the response.",
-   "default": "json",
-   "enum": [
-    "json"
-   ],
-   "enumDescriptions": [
-    "Responses with Content-Type of application/json"
-   ],
-   "location": "query"
-  },
-  "quotaUser": {
-   "type": "string",
-   "description": "Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters. Overrides userIp if both are provided.",
-   "location": "query"
-  },
-  "prettyPrint": {
-   "type": "boolean",
-   "description": "Returns response with indentations and line breaks.",
-   "default": "true",
-   "location": "query"
-  }
- },
- "auth": {
-  "oauth2": {
-   "scopes": {
-    "https://apps-apis.google.com/a/feeds/domain/": {
-     "description": "Manage domain settings"
-    }
-   }
-  }
- },
- "schemas": {
-  "DefaultLanguage": {
-   "id": "DefaultLanguage",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "default language value"
-      }
-     }
-    }
-   }
-  },
-  "OrganizationName": {
-   "id": "OrganizationName",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "organization name value"
-      }
-     }
-    }
-   }
-  },
-  "MaximumNumberOfUsers": {
-   "id": "MaximumNumberOfUsers",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "maximum number of users value"
-      }
-     }
-    }
-   }
-  },
-  "CurrentNumberOfUsers": {
-   "id": "CurrentNumbersOfUsers",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "current number of users value"
-      }
-     }
-    }
-   }
-  },
-  "IsVerified": {
-   "id": "IsVerified",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "boolean",
-        "description": "current verification status value"
-      }
-     }
-    }
-   }
-  },
-  "Edition": {
-   "id": "Edition",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "domain edition value"
-      }
-     }
-    }
-   }
-  },
-  "CustomerPIN": {
-   "id": "CustomerPIN",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "customer pin value"
-      }
-     }
-    }
-   }
-  },
-  "CreationTime": {
-   "id": "CreationTime",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "domain creation time value"
-      }
-     }
-    }
-   }
-  },
-  "CountryCode": {
-   "id": "CountryCode",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "domain country code value"
-      }
-     }
-    }
-   }
-  },
-  "AdminSecondaryEmail": {
-   "id": "AdminSecondaryEmail",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "admin secondary email value"
-      }
-     }
-    }
-   }
-  },
-  "MXVerification": {
-   "id": "MXVerification",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "domain mx verification value"
-      }
-     }
-    }
-   }
-  },
-  "SSOGeneral": {
-   "id": "SSOGeneral",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "domain sso general value"
-      }
-     }
-    }
-   }
-  },
-  "SSOSigningKey": {
-   "id": "SSOSigningKey",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "domain sso signing key value"
-      }
-     }
-    }
-   }
-  },
-  "UserEmailMigrationEnabled": {
-   "id": "UserEmailMigrationEnabled",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "user email migration enabled value"
-      }
-     }
-    }
-   }
-  },
-  "OutboundGateway": {
-   "id": "OutboundGateway",
-   "type": "object",
-   "properties": {
-    "apps$property": {
-     "type": "object",
-     "properties": {
-      "name": {
-       "type": "string",
-       "description": "property name"
-      },
-      "value": {
-        "type": "string",
-        "description": "domain outbound gateway value"
-      }
-     }
-    }
-   }
-  }
- },
- "resources": {
-  "defaultLanguage": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.defaultLanguage.get",
-     "path": "{domainName}/general/defaultLanguage",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "DefaultLanguage"
-     }
-    }
-   }
-  },
-  "organizationName": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.organizationName.get",
-     "path": "{domainName}/general/organizationName",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "OrganizationName"
-     }
-    }
-   }
-  },
-  "maximumNumberOfUsers": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.maximumNumberOfUsers.get",
-     "path": "{domainName}/general/maximumNumberOfUsers",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "MaximumNumberOfUsers"
-     }
-    }
-   }
-  },
-  "currentNumberOfUsers": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.currentNumberOfUsers.get",
-     "path": "{domainName}/general/currentNumberOfUsers",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "CurrentNumberOfUsers"
-     }
-    }
-   }
-  },
-  "isVerified": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.isVerified.get",
-     "path": "{domainName}/accountInformation/isVerified",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "IsVerified"
-     }
-    }
-   }
-  },
-  "edition": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.edition.get",
-     "path": "{domainName}/accountInformation/edition",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "Edition"
-     }
-    }
-   }
-  },
-  "customerPIN": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.customerPIN.get",
-     "path": "{domainName}/accountInformation/customerPIN",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "CustomerPIN"
-     }
-    }
-   }
-  },
-  "creationTime": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.creationTime.get",
-     "path": "{domainName}/accountInformation/creationTime",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "CreationTime"
-     }
-    }
-   }
-  },
-  "countryCode": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.countryCode.get",
-     "path": "{domainName}/accountInformation/countryCode",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "CountryCode"
-     }
-    }
-   }
-  },
-  "adminSecondaryEmail": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.adminSecondaryEmail.get",
-     "path": "{domainName}/accountInformation/adminSecondaryEmail",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "AdminSecondaryEmail"
-     }
-    }
-   }
-  },
-  "mxVerification": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.mxVerification.get",
-     "path": "{domainName}/verification/mx",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "MXVerification"
-     }
-    }
-   }
-  },
-  "ssoGeneral": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.ssoGeneral.get",
-     "path": "{domainName}/sso/general",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "SSOGeneral"
-     }
-    }
-   }
-  },
-  "ssoSigningKey": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.ssoSigningKey.get",
-     "path": "{domainName}/sso/signingkey",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "SSOSigningKey"
-     }
-    }
-   }
-  },
-  "userEmailMigrationEnabled": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.userEmailMigrationEnabled.get",
-     "path": "{domainName}/email/migration",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "UserEmailMigrationEnabled"
-     }
-    }
-   }
-  },
-  "outboundGateway": {
-   "methods": {
-    "get": {
-     "id": "admin-settings.outboundGateway.get",
-     "path": "{domainName}/email/gateway",
-     "httpMethod": "GET",
-     "parameters": {
-      "domainName": {
-       "type": "string",
-       "required": "true",
-       "location": "path"
-      }
-     },
-     "response": {
-      "$ref": "OutboundGateway"
-     }
-    }
-   }
-  }
- }
-}

apiclient/channel.py

@@ -1,285 +0,0 @@
-"""Channel notifications support.
-
-Classes and functions to support channel subscriptions and notifications
-on those channels.
-
-Notes:
-  - This code is based on experimental APIs and is subject to change.
-  - Notification does not do deduplication of notification ids, that's up to
-    the receiver.
-  - Storing the Channel between calls is up to the caller.
-
-
-Example setting up a channel:
-
-  # Create a new channel that gets notifications via webhook.
-  channel = new_webhook_channel("https://example.com/my_web_hook")
-
-  # Store the channel, keyed by 'channel.id'. Store it before calling the
-  # watch method because notifications may start arriving before the watch
-  # method returns.
-  ...
-
-  resp = service.objects().watchAll(
-    bucket="some_bucket_id", body=channel.body()).execute()
-  channel.update(resp)
-
-  # Store the channel, keyed by 'channel.id'. Store it after being updated
-  # since the resource_id value will now be correct, and that's needed to
-  # stop a subscription.
-  ...
-
-
-An example Webhook implementation using webapp2. Note that webapp2 puts
-headers in a case insensitive dictionary, as headers aren't guaranteed to
-always be upper case.
-
-  id = self.request.headers[X_GOOG_CHANNEL_ID]
-
-  # Retrieve the channel by id.
-  channel = ...
-
-  # Parse notification from the headers, including validating the id.
-  n = notification_from_headers(channel, self.request.headers)
-
-  # Do app specific stuff with the notification here.
-  if n.resource_state == 'sync':
-    # Code to handle sync state.
-  elif n.resource_state == 'exists':
-    # Code to handle the exists state.
-  elif n.resource_state == 'not_exists':
-    # Code to handle the not exists state.
-
-
-Example of unsubscribing.
-
-  service.channels().stop(channel.body())
-"""
-
-import datetime
-import uuid
-
-from apiclient import errors
-from oauth2client import util
-
-
-# The unix time epoch starts at midnight 1970.
-EPOCH = datetime.datetime.utcfromtimestamp(0)
-
-# Map the names of the parameters in the JSON channel description to
-# the parameter names we use in the Channel class.
-CHANNEL_PARAMS = {
-    'address': 'address',
-    'id': 'id',
-    'expiration': 'expiration',
-    'params': 'params',
-    'resourceId': 'resource_id',
-    'resourceUri': 'resource_uri',
-    'type': 'type',
-    'token': 'token',
-    }
-
-X_GOOG_CHANNEL_ID     = 'X-GOOG-CHANNEL-ID'
-X_GOOG_MESSAGE_NUMBER = 'X-GOOG-MESSAGE-NUMBER'
-X_GOOG_RESOURCE_STATE = 'X-GOOG-RESOURCE-STATE'
-X_GOOG_RESOURCE_URI   = 'X-GOOG-RESOURCE-URI'
-X_GOOG_RESOURCE_ID    = 'X-GOOG-RESOURCE-ID'
-
-
-def _upper_header_keys(headers):
-  new_headers = {}
-  for k, v in headers.iteritems():
-    new_headers[k.upper()] = v
-  return new_headers
-
-
-class Notification(object):
-  """A Notification from a Channel.
-
-  Notifications are not usually constructed directly, but are returned
-  from functions like notification_from_headers().
-
-  Attributes:
-    message_number: int, The unique id number of this notification.
-    state: str, The state of the resource being monitored.
-    uri: str, The address of the resource being monitored.
-    resource_id: str, The unique identifier of the version of the resource at
-      this event.
-  """
-  @util.positional(5)
-  def __init__(self, message_number, state, resource_uri, resource_id):
-    """Notification constructor.
-
-    Args:
-      message_number: int, The unique id number of this notification.
-      state: str, The state of the resource being monitored. Can be one
-        of "exists", "not_exists", or "sync".
-      resource_uri: str, The address of the resource being monitored.
-      resource_id: str, The identifier of the watched resource.
-    """
-    self.message_number = message_number
-    self.state = state
-    self.resource_uri = resource_uri
-    self.resource_id = resource_id
-
-
-class Channel(object):
-  """A Channel for notifications.
-
-  Usually not constructed directly, instead it is returned from helper
-  functions like new_webhook_channel().
-
-  Attributes:
-    type: str, The type of delivery mechanism used by this channel. For
-      example, 'web_hook'.
-    id: str, A UUID for the channel.
-    token: str, An arbitrary string associated with the channel that
-      is delivered to the target address with each event delivered
-      over this channel.
-    address: str, The address of the receiving entity where events are
-      delivered. Specific to the channel type.
-    expiration: int, The time, in milliseconds from the epoch, when this
-      channel will expire.
-    params: dict, A dictionary of string to string, with additional parameters
-      controlling delivery channel behavior.
-    resource_id: str, An opaque id that identifies the resource that is
-      being watched. Stable across different API versions.
-    resource_uri: str, The canonicalized ID of the watched resource.
-  """
-
-  @util.positional(5)
-  def __init__(self, type, id, token, address, expiration=None,
-               params=None, resource_id="", resource_uri=""):
-    """Create a new Channel.
-
-    In user code, this Channel constructor will not typically be called
-    manually since there are functions for creating channels for each specific
-    type with a more customized set of arguments to pass.
-
-    Args:
-      type: str, The type of delivery mechanism used by this channel. For
-        example, 'web_hook'.
-      id: str, A UUID for the channel.
-      token: str, An arbitrary string associated with the channel that
-        is delivered to the target address with each event delivered
-        over this channel.
-      address: str,  The address of the receiving entity where events are
-        delivered. Specific to the channel type.
-      expiration: int, The time, in milliseconds from the epoch, when this
-        channel will expire.
-      params: dict, A dictionary of string to string, with additional parameters
-        controlling delivery channel behavior.
-      resource_id: str, An opaque id that identifies the resource that is
-        being watched. Stable across different API versions.
-      resource_uri: str, The canonicalized ID of the watched resource.
-    """
-    self.type = type
-    self.id = id
-    self.token = token
-    self.address = address
-    self.expiration = expiration
-    self.params = params
-    self.resource_id = resource_id
-    self.resource_uri = resource_uri
-
-  def body(self):
-    """Build a body from the Channel.
-
-    Constructs a dictionary that's appropriate for passing into watch()
-    methods as the value of body argument.
-
-    Returns:
-      A dictionary representation of the channel.
-    """
-    result = {
-        'id': self.id,
-        'token': self.token,
-        'type': self.type,
-        'address': self.address
-        }
-    if self.params:
-      result['params'] = self.params
-    if self.resource_id:
-      result['resourceId'] = self.resource_id
-    if self.resource_uri:
-      result['resourceUri'] = self.resource_uri
-    if self.expiration:
-      result['expiration'] = self.expiration
-
-    return result
-
-  def update(self, resp):
-    """Update a channel with information from the response of watch().
-
-    When a request is sent to watch() a resource, the response returned
-    from the watch() request is a dictionary with updated channel information,
-    such as the resource_id, which is needed when stopping a subscription.
-
-    Args:
-      resp: dict, The response from a watch() method.
-    """
-    for json_name, param_name in CHANNEL_PARAMS.iteritems():
-      value = resp.get(json_name)
-      if value is not None:
-        setattr(self, param_name, value)
-
-
-def notification_from_headers(channel, headers):
-  """Parse a notification from the webhook request headers, validate
-    the notification, and return a Notification object.
-
-  Args:
-    channel: Channel, The channel that the notification is associated with.
-    headers: dict, A dictionary like object that contains the request headers
-      from the webhook HTTP request.
-
-  Returns:
-    A Notification object.
-
-  Raises:
-    errors.InvalidNotificationError if the notification is invalid.
-    ValueError if the X-GOOG-MESSAGE-NUMBER can't be converted to an int.
-  """
-  headers = _upper_header_keys(headers)
-  channel_id = headers[X_GOOG_CHANNEL_ID]
-  if channel.id != channel_id:
-    raise errors.InvalidNotificationError(
-        'Channel id mismatch: %s != %s' % (channel.id, channel_id))
-  else:
-    message_number = int(headers[X_GOOG_MESSAGE_NUMBER])
-    state = headers[X_GOOG_RESOURCE_STATE]
-    resource_uri = headers[X_GOOG_RESOURCE_URI]
-    resource_id = headers[X_GOOG_RESOURCE_ID]
-    return Notification(message_number, state, resource_uri, resource_id)
-
-
-@util.positional(2)
-def new_webhook_channel(url, token=None, expiration=None, params=None):
-    """Create a new webhook Channel.
-
-    Args:
-      url: str, URL to post notifications to.
-      token: str, An arbitrary string associated with the channel that
-        is delivered to the target address with each notification delivered
-        over this channel.
-      expiration: datetime.datetime, A time in the future when the channel
-        should expire. Can also be None if the subscription should use the
-        default expiration. Note that different services may have different
-        limits on how long a subscription lasts. Check the response from the
-        watch() method to see the value the service has set for an expiration
-        time.
-      params: dict, Extra parameters to pass on channel creation. Currently
-        not used for webhook channels.
-    """
-    expiration_ms = 0
-    if expiration:
-      delta = expiration - EPOCH
-      expiration_ms = delta.microseconds/1000 + (
-          delta.seconds + delta.days*24*3600)*1000
-      if expiration_ms < 0:
-        expiration_ms = 0
-
-    return Channel('web_hook', str(uuid.uuid4()),
-                   token, url, expiration=expiration_ms,
-                   params=params)
-

apiclient/discovery.py

@@ -1,963 +0,0 @@
-# Copyright (C) 2010 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Client for discovery based APIs.
-
-A client library for Google's discovery based APIs.
-"""
-
-__author__ = 'jcgregorio@google.com (Joe Gregorio)'
-__all__ = [
-    'build',
-    'build_from_document',
-    'fix_method_name',
-    'key2param',
-    ]
-
-
-# Standard library imports
-import copy
-from email.mime.multipart import MIMEMultipart
-from email.mime.nonmultipart import MIMENonMultipart
-import keyword
-import logging
-import mimetypes
-import os
-import re
-import urllib
-import urlparse
-
-try:
-  from urlparse import parse_qsl
-except ImportError:
-  from cgi import parse_qsl
-
-# Third-party imports
-import httplib2
-import mimeparse
-import uritemplate
-
-# Local imports
-from apiclient.errors import HttpError
-from apiclient.errors import InvalidJsonError
-from apiclient.errors import MediaUploadSizeError
-from apiclient.errors import UnacceptableMimeTypeError
-from apiclient.errors import UnknownApiNameOrVersion
-from apiclient.errors import UnknownFileType
-from apiclient.http import HttpRequest
-from apiclient.http import MediaFileUpload
-from apiclient.http import MediaUpload
-from apiclient.model import JsonModel
-from apiclient.model import MediaModel
-from apiclient.model import RawModel
-from apiclient.schema import Schemas
-from oauth2client.anyjson import simplejson
-from oauth2client.util import _add_query_parameter
-from oauth2client.util import positional
-
-
-# The client library requires a version of httplib2 that supports RETRIES.
-httplib2.RETRIES = 1
-
-logger = logging.getLogger(__name__)
-
-URITEMPLATE = re.compile('{[^}]*}')
-VARNAME = re.compile('[a-zA-Z0-9_-]+')
-if httplib2.debuglevel > 0:
-  prettyPrint = 'true'
-else:
-  prettyPrint = 'false'
-DISCOVERY_URI = ('https://www.googleapis.com/discovery/v1/apis/'
-                 '{api}/{apiVersion}/rest?prettyPrint=%s' % prettyPrint)
-DEFAULT_METHOD_DOC = 'A description of how to use this function'
-HTTP_PAYLOAD_METHODS = frozenset(['PUT', 'POST', 'PATCH'])
-_MEDIA_SIZE_BIT_SHIFTS = {'KB': 10, 'MB': 20, 'GB': 30, 'TB': 40}
-BODY_PARAMETER_DEFAULT_VALUE = {
-    'description': 'The request body.',
-    'type': 'object',
-    'required': True,
-}
-MEDIA_BODY_PARAMETER_DEFAULT_VALUE = {
-    'description': ('The filename of the media request body, or an instance '
-                    'of a MediaUpload object.'),
-    'type': 'string',
-    'required': False,
-}
-
-# Parameters accepted by the stack, but not visible via discovery.
-# TODO(dhermes): Remove 'userip' in 'v2'.
-STACK_QUERY_PARAMETERS = frozenset(['trace', 'pp', 'userip', 'strict'])
-STACK_QUERY_PARAMETER_DEFAULT_VALUE = {'type': 'string', 'location': 'query'}
-
-# Library-specific reserved words beyond Python keywords.
-RESERVED_WORDS = frozenset(['body'])
-
-
-def fix_method_name(name):
-  """Fix method names to avoid reserved word conflicts.
-
-  Args:
-    name: string, method name.
-
-  Returns:
-    The name with a '_' prefixed if the name is a reserved word.
-  """
-  if keyword.iskeyword(name) or name in RESERVED_WORDS:
-    return name + '_'
-  else:
-    return name
-
-
-def key2param(key):
-  """Converts key names into parameter names.
-
-  For example, converting "max-results" -> "max_results"
-
-  Args:
-    key: string, the method key name.
-
-  Returns:
-    A safe method name based on the key name.
-  """
-  result = []
-  key = list(key)
-  if not key[0].isalpha():
-    result.append('x')
-  for c in key:
-    if c.isalnum():
-      result.append(c)
-    else:
-      result.append('_')
-
-  return ''.join(result)
-
-
-@positional(2)
-def build(serviceName,
-          version,
-          http=None,
-          discoveryServiceUrl=DISCOVERY_URI,
-          developerKey=None,
-          model=None,
-          requestBuilder=HttpRequest):
-  """Construct a Resource for interacting with an API.
-
-  Construct a Resource object for interacting with an API. The serviceName and
-  version are the names from the Discovery service.
-
-  Args:
-    serviceName: string, name of the service.
-    version: string, the version of the service.
-    http: httplib2.Http, An instance of httplib2.Http or something that acts
-      like it that HTTP requests will be made through.
-    discoveryServiceUrl: string, a URI Template that points to the location of
-      the discovery service. It should have two parameters {api} and
-      {apiVersion} that when filled in produce an absolute URI to the discovery
-      document for that service.
-    developerKey: string, key obtained from
-      https://code.google.com/apis/console.
-    model: apiclient.Model, converts to and from the wire format.
-    requestBuilder: apiclient.http.HttpRequest, encapsulator for an HTTP
-      request.
-
-  Returns:
-    A Resource object with methods for interacting with the service.
-  """
-  params = {
-      'api': serviceName,
-      'apiVersion': version
-      }
-
-  if http is None:
-    http = httplib2.Http()
-
-  requested_url = uritemplate.expand(discoveryServiceUrl, params)
-
-  # REMOTE_ADDR is defined by the CGI spec [RFC3875] as the environment
-  # variable that contains the network address of the client sending the
-  # request. If it exists then add that to the request for the discovery
-  # document to avoid exceeding the quota on discovery requests.
-  if 'REMOTE_ADDR' in os.environ:
-    requested_url = _add_query_parameter(requested_url, 'userIp',
-                                         os.environ['REMOTE_ADDR'])
-  logger.info('URL being requested: %s' % requested_url)
-
-  resp, content = http.request(requested_url)
-
-  if resp.status == 404:
-    raise UnknownApiNameOrVersion("name: %s  version: %s" % (serviceName,
-                                                            version))
-  if resp.status >= 400:
-    raise HttpError(resp, content, uri=requested_url)
-
-  try:
-    service = simplejson.loads(content)
-  except ValueError, e:
-    logger.error('Failed to parse as JSON: ' + content)
-    raise InvalidJsonError()
-
-  return build_from_document(content, base=discoveryServiceUrl, http=http,
-      developerKey=developerKey, model=model, requestBuilder=requestBuilder)
-
-
-@positional(1)
-def build_from_document(
-    service,
-    base=None,
-    future=None,
-    http=None,
-    developerKey=None,
-    model=None,
-    requestBuilder=HttpRequest):
-  """Create a Resource for interacting with an API.
-
-  Same as `build()`, but constructs the Resource object from a discovery
-  document that is it given, as opposed to retrieving one over HTTP.
-
-  Args:
-    service: string or object, the JSON discovery document describing the API.
-      The value passed in may either be the JSON string or the deserialized
-      JSON.
-    base: string, base URI for all HTTP requests, usually the discovery URI.
-      This parameter is no longer used as rootUrl and servicePath are included
-      within the discovery document. (deprecated)
-    future: string, discovery document with future capabilities (deprecated).
-    http: httplib2.Http, An instance of httplib2.Http or something that acts
-      like it that HTTP requests will be made through.
-    developerKey: string, Key for controlling API usage, generated
-      from the API Console.
-    model: Model class instance that serializes and de-serializes requests and
-      responses.
-    requestBuilder: Takes an http request and packages it up to be executed.
-
-  Returns:
-    A Resource object with methods for interacting with the service.
-  """
-
-  # future is no longer used.
-  future = {}
-
-  if isinstance(service, basestring):
-    service = simplejson.loads(service)
-  base = urlparse.urljoin(service['rootUrl'], service['servicePath'])
-  schema = Schemas(service)
-
-  if model is None:
-    features = service.get('features', [])
-    model = JsonModel('dataWrapper' in features)
-  return Resource(http=http, baseUrl=base, model=model,
-                  developerKey=developerKey, requestBuilder=requestBuilder,
-                  resourceDesc=service, rootDesc=service, schema=schema)
-
-
-def _cast(value, schema_type):
-  """Convert value to a string based on JSON Schema type.
-
-  See http://tools.ietf.org/html/draft-zyp-json-schema-03 for more details on
-  JSON Schema.
-
-  Args:
-    value: any, the value to convert
-    schema_type: string, the type that value should be interpreted as
-
-  Returns:
-    A string representation of 'value' based on the schema_type.
-  """
-  if schema_type == 'string':
-    if type(value) == type('') or type(value) == type(u''):
-      return value
-    else:
-      return str(value)
-  elif schema_type == 'integer':
-    return str(int(value))
-  elif schema_type == 'number':
-    return str(float(value))
-  elif schema_type == 'boolean':
-    return str(bool(value)).lower()
-  else:
-    if type(value) == type('') or type(value) == type(u''):
-      return value
-    else:
-      return str(value)
-
-
-def _media_size_to_long(maxSize):
-  """Convert a string media size, such as 10GB or 3TB into an integer.
-
-  Args:
-    maxSize: string, size as a string, such as 2MB or 7GB.
-
-  Returns:
-    The size as an integer value.
-  """
-  if len(maxSize) < 2:
-    return 0L
-  units = maxSize[-2:].upper()
-  bit_shift = _MEDIA_SIZE_BIT_SHIFTS.get(units)
-  if bit_shift is not None:
-    return long(maxSize[:-2]) << bit_shift
-  else:
-    return long(maxSize)
-
-
-def _media_path_url_from_info(root_desc, path_url):
-  """Creates an absolute media path URL.
-
-  Constructed using the API root URI and service path from the discovery
-  document and the relative path for the API method.
-
-  Args:
-    root_desc: Dictionary; the entire original deserialized discovery document.
-    path_url: String; the relative URL for the API method. Relative to the API
-        root, which is specified in the discovery document.
-
-  Returns:
-    String; the absolute URI for media upload for the API method.
-  """
-  return '%(root)supload/%(service_path)s%(path)s' % {
-      'root': root_desc['rootUrl'],
-      'service_path': root_desc['servicePath'],
-      'path': path_url,
-  }
-
-
-def _fix_up_parameters(method_desc, root_desc, http_method):
-  """Updates parameters of an API method with values specific to this library.
-
-  Specifically, adds whatever global parameters are specified by the API to the
-  parameters for the individual method. Also adds parameters which don't
-  appear in the discovery document, but are available to all discovery based
-  APIs (these are listed in STACK_QUERY_PARAMETERS).
-
-  SIDE EFFECTS: This updates the parameters dictionary object in the method
-  description.
-
-  Args:
-    method_desc: Dictionary with metadata describing an API method. Value comes
-        from the dictionary of methods stored in the 'methods' key in the
-        deserialized discovery document.
-    root_desc: Dictionary; the entire original deserialized discovery document.
-    http_method: String; the HTTP method used to call the API method described
-        in method_desc.
-
-  Returns:
-    The updated Dictionary stored in the 'parameters' key of the method
-        description dictionary.
-  """
-  parameters = method_desc.setdefault('parameters', {})
-
-  # Add in the parameters common to all methods.
-  for name, description in root_desc.get('parameters', {}).iteritems():
-    parameters[name] = description
-
-  # Add in undocumented query parameters.
-  for name in STACK_QUERY_PARAMETERS:
-    parameters[name] = STACK_QUERY_PARAMETER_DEFAULT_VALUE.copy()
-
-  # Add 'body' (our own reserved word) to parameters if the method supports
-  # a request payload.
-  if http_method in HTTP_PAYLOAD_METHODS and 'request' in method_desc:
-    body = BODY_PARAMETER_DEFAULT_VALUE.copy()
-    body.update(method_desc['request'])
-    parameters['body'] = body
-
-  return parameters
-
-
-def _fix_up_media_upload(method_desc, root_desc, path_url, parameters):
-  """Updates parameters of API by adding 'media_body' if supported by method.
-
-  SIDE EFFECTS: If the method supports media upload and has a required body,
-  sets body to be optional (required=False) instead. Also, if there is a
-  'mediaUpload' in the method description, adds 'media_upload' key to
-  parameters.
-
-  Args:
-    method_desc: Dictionary with metadata describing an API method. Value comes
-        from the dictionary of methods stored in the 'methods' key in the
-        deserialized discovery document.
-    root_desc: Dictionary; the entire original deserialized discovery document.
-    path_url: String; the relative URL for the API method. Relative to the API
-        root, which is specified in the discovery document.
-    parameters: A dictionary describing method parameters for method described
-        in method_desc.
-
-  Returns:
-    Triple (accept, max_size, media_path_url) where:
-      - accept is a list of strings representing what content types are
-        accepted for media upload. Defaults to empty list if not in the
-        discovery document.
-      - max_size is a long representing the max size in bytes allowed for a
-        media upload. Defaults to 0L if not in the discovery document.
-      - media_path_url is a String; the absolute URI for media upload for the
-        API method. Constructed using the API root URI and service path from
-        the discovery document and the relative path for the API method. If
-        media upload is not supported, this is None.
-  """
-  media_upload = method_desc.get('mediaUpload', {})
-  accept = media_upload.get('accept', [])
-  max_size = _media_size_to_long(media_upload.get('maxSize', ''))
-  media_path_url = None
-
-  if media_upload:
-    media_path_url = _media_path_url_from_info(root_desc, path_url)
-    parameters['media_body'] = MEDIA_BODY_PARAMETER_DEFAULT_VALUE.copy()
-    if 'body' in parameters:
-      parameters['body']['required'] = False
-
-  return accept, max_size, media_path_url
-
-
-def _fix_up_method_description(method_desc, root_desc):
-  """Updates a method description in a discovery document.
-
-  SIDE EFFECTS: Changes the parameters dictionary in the method description with
-  extra parameters which are used locally.
-
-  Args:
-    method_desc: Dictionary with metadata describing an API method. Value comes
-        from the dictionary of methods stored in the 'methods' key in the
-        deserialized discovery document.
-    root_desc: Dictionary; the entire original deserialized discovery document.
-
-  Returns:
-    Tuple (path_url, http_method, method_id, accept, max_size, media_path_url)
-    where:
-      - path_url is a String; the relative URL for the API method. Relative to
-        the API root, which is specified in the discovery document.
-      - http_method is a String; the HTTP method used to call the API method
-        described in the method description.
-      - method_id is a String; the name of the RPC method associated with the
-        API method, and is in the method description in the 'id' key.
-      - accept is a list of strings representing what content types are
-        accepted for media upload. Defaults to empty list if not in the
-        discovery document.
-      - max_size is a long representing the max size in bytes allowed for a
-        media upload. Defaults to 0L if not in the discovery document.
-      - media_path_url is a String; the absolute URI for media upload for the
-        API method. Constructed using the API root URI and service path from
-        the discovery document and the relative path for the API method. If
-        media upload is not supported, this is None.
-  """
-  path_url = method_desc['path']
-  http_method = method_desc['httpMethod']
-  method_id = method_desc['id']
-
-  parameters = _fix_up_parameters(method_desc, root_desc, http_method)
-  # Order is important. `_fix_up_media_upload` needs `method_desc` to have a
-  # 'parameters' key and needs to know if there is a 'body' parameter because it
-  # also sets a 'media_body' parameter.
-  accept, max_size, media_path_url = _fix_up_media_upload(
-      method_desc, root_desc, path_url, parameters)
-
-  return path_url, http_method, method_id, accept, max_size, media_path_url
-
-
-# TODO(dhermes): Convert this class to ResourceMethod and make it callable
-class ResourceMethodParameters(object):
-  """Represents the parameters associated with a method.
-
-  Attributes:
-    argmap: Map from method parameter name (string) to query parameter name
-        (string).
-    required_params: List of required parameters (represented by parameter
-        name as string).
-    repeated_params: List of repeated parameters (represented by parameter
-        name as string).
-    pattern_params: Map from method parameter name (string) to regular
-        expression (as a string). If the pattern is set for a parameter, the
-        value for that parameter must match the regular expression.
-    query_params: List of parameters (represented by parameter name as string)
-        that will be used in the query string.
-    path_params: Set of parameters (represented by parameter name as string)
-        that will be used in the base URL path.
-    param_types: Map from method parameter name (string) to parameter type. Type
-        can be any valid JSON schema type; valid values are 'any', 'array',
-        'boolean', 'integer', 'number', 'object', or 'string'. Reference:
-        http://tools.ietf.org/html/draft-zyp-json-schema-03#section-5.1
-    enum_params: Map from method parameter name (string) to list of strings,
-       where each list of strings is the list of acceptable enum values.
-  """
-
-  def __init__(self, method_desc):
-    """Constructor for ResourceMethodParameters.
-
-    Sets default values and defers to set_parameters to populate.
-
-    Args:
-      method_desc: Dictionary with metadata describing an API method. Value
-          comes from the dictionary of methods stored in the 'methods' key in
-          the deserialized discovery document.
-    """
-    self.argmap = {}
-    self.required_params = []
-    self.repeated_params = []
-    self.pattern_params = {}
-    self.query_params = []
-    # TODO(dhermes): Change path_params to a list if the extra URITEMPLATE
-    #                parsing is gotten rid of.
-    self.path_params = set()
-    self.param_types = {}
-    self.enum_params = {}
-
-    self.set_parameters(method_desc)
-
-  def set_parameters(self, method_desc):
-    """Populates maps and lists based on method description.
-
-    Iterates through each parameter for the method and parses the values from
-    the parameter dictionary.
-
-    Args:
-      method_desc: Dictionary with metadata describing an API method. Value
-          comes from the dictionary of methods stored in the 'methods' key in
-          the deserialized discovery document.
-    """
-    for arg, desc in method_desc.get('parameters', {}).iteritems():
-      param = key2param(arg)
-      self.argmap[param] = arg
-
-      if desc.get('pattern'):
-        self.pattern_params[param] = desc['pattern']
-      if desc.get('enum'):
-        self.enum_params[param] = desc['enum']
-      if desc.get('required'):
-        self.required_params.append(param)
-      if desc.get('repeated'):
-        self.repeated_params.append(param)
-      if desc.get('location') == 'query':
-        self.query_params.append(param)
-      if desc.get('location') == 'path':
-        self.path_params.add(param)
-      self.param_types[param] = desc.get('type', 'string')
-
-    # TODO(dhermes): Determine if this is still necessary. Discovery based APIs
-    #                should have all path parameters already marked with
-    #                'location: path'.
-    for match in URITEMPLATE.finditer(method_desc['path']):
-      for namematch in VARNAME.finditer(match.group(0)):
-        name = key2param(namematch.group(0))
-        self.path_params.add(name)
-        if name in self.query_params:
-          self.query_params.remove(name)
-
-
-def createMethod(methodName, methodDesc, rootDesc, schema):
-  """Creates a method for attaching to a Resource.
-
-  Args:
-    methodName: string, name of the method to use.
-    methodDesc: object, fragment of deserialized discovery document that
-      describes the method.
-    rootDesc: object, the entire deserialized discovery document.
-    schema: object, mapping of schema names to schema descriptions.
-  """
-  methodName = fix_method_name(methodName)
-  (pathUrl, httpMethod, methodId, accept,
-   maxSize, mediaPathUrl) = _fix_up_method_description(methodDesc, rootDesc)
-
-  parameters = ResourceMethodParameters(methodDesc)
-
-  def method(self, **kwargs):
-    # Don't bother with doc string, it will be over-written by createMethod.
-
-    for name in kwargs.iterkeys():
-      if name not in parameters.argmap:
-        raise TypeError('Got an unexpected keyword argument "%s"' % name)
-
-    # Remove args that have a value of None.
-    keys = kwargs.keys()
-    for name in keys:
-      if kwargs[name] is None:
-        del kwargs[name]
-
-    for name in parameters.required_params:
-      if name not in kwargs:
-        raise TypeError('Missing required parameter "%s"' % name)
-
-    for name, regex in parameters.pattern_params.iteritems():
-      if name in kwargs:
-        if isinstance(kwargs[name], basestring):
-          pvalues = [kwargs[name]]
-        else:
-          pvalues = kwargs[name]
-        for pvalue in pvalues:
-          if re.match(regex, pvalue) is None:
-            raise TypeError(
-                'Parameter "%s" value "%s" does not match the pattern "%s"' %
-                (name, pvalue, regex))
-
-    for name, enums in parameters.enum_params.iteritems():
-      if name in kwargs:
-        # We need to handle the case of a repeated enum
-        # name differently, since we want to handle both
-        # arg='value' and arg=['value1', 'value2']
-        if (name in parameters.repeated_params and
-            not isinstance(kwargs[name], basestring)):
-          values = kwargs[name]
-        else:
-          values = [kwargs[name]]
-        for value in values:
-          if value not in enums:
-            raise TypeError(
-                'Parameter "%s" value "%s" is not an allowed value in "%s"' %
-                (name, value, str(enums)))
-
-    actual_query_params = {}
-    actual_path_params = {}
-    for key, value in kwargs.iteritems():
-      to_type = parameters.param_types.get(key, 'string')
-      # For repeated parameters we cast each member of the list.
-      if key in parameters.repeated_params and type(value) == type([]):
-        cast_value = [_cast(x, to_type) for x in value]
-      else:
-        cast_value = _cast(value, to_type)
-      if key in parameters.query_params:
-        actual_query_params[parameters.argmap[key]] = cast_value
-      if key in parameters.path_params:
-        actual_path_params[parameters.argmap[key]] = cast_value
-    body_value = kwargs.get('body', None)
-    media_filename = kwargs.get('media_body', None)
-
-    if self._developerKey:
-      actual_query_params['key'] = self._developerKey
-
-    model = self._model
-    if methodName.endswith('_media'):
-      model = MediaModel()
-    elif 'response' not in methodDesc:
-      model = RawModel()
-
-    headers = {}
-    headers, params, query, body = model.request(headers,
-        actual_path_params, actual_query_params, body_value)
-
-    expanded_url = uritemplate.expand(pathUrl, params)
-    url = urlparse.urljoin(self._baseUrl, expanded_url + query)
-
-    resumable = None
-    multipart_boundary = ''
-
-    if media_filename:
-      # Ensure we end up with a valid MediaUpload object.
-      if isinstance(media_filename, basestring):
-        (media_mime_type, encoding) = mimetypes.guess_type(media_filename)
-        if media_mime_type is None:
-          raise UnknownFileType(media_filename)
-        if not mimeparse.best_match([media_mime_type], ','.join(accept)):
-          raise UnacceptableMimeTypeError(media_mime_type)
-        media_upload = MediaFileUpload(media_filename,
-                                       mimetype=media_mime_type)
-      elif isinstance(media_filename, MediaUpload):
-        media_upload = media_filename
-      else:
-        raise TypeError('media_filename must be str or MediaUpload.')
-
-      # Check the maxSize
-      if maxSize > 0 and media_upload.size() > maxSize:
-        raise MediaUploadSizeError("Media larger than: %s" % maxSize)
-
-      # Use the media path uri for media uploads
-      expanded_url = uritemplate.expand(mediaPathUrl, params)
-      url = urlparse.urljoin(self._baseUrl, expanded_url + query)
-      if media_upload.resumable():
-        url = _add_query_parameter(url, 'uploadType', 'resumable')
-
-      if media_upload.resumable():
-        # This is all we need to do for resumable, if the body exists it gets
-        # sent in the first request, otherwise an empty body is sent.
-        resumable = media_upload
-      else:
-        # A non-resumable upload
-        if body is None:
-          # This is a simple media upload
-          headers['content-type'] = media_upload.mimetype()
-          body = media_upload.getbytes(0, media_upload.size())
-          url = _add_query_parameter(url, 'uploadType', 'media')
-        else:
-          # This is a multipart/related upload.
-          msgRoot = MIMEMultipart('related')
-          # msgRoot should not write out it's own headers
-          setattr(msgRoot, '_write_headers', lambda self: None)
-
-          # attach the body as one part
-          msg = MIMENonMultipart(*headers['content-type'].split('/'))
-          msg.set_payload(body)
-          msgRoot.attach(msg)
-
-          # attach the media as the second part
-          msg = MIMENonMultipart(*media_upload.mimetype().split('/'))
-          msg['Content-Transfer-Encoding'] = 'binary'
-
-          payload = media_upload.getbytes(0, media_upload.size())
-          msg.set_payload(payload)
-          msgRoot.attach(msg)
-          body = msgRoot.as_string()
-
-          multipart_boundary = msgRoot.get_boundary()
-          headers['content-type'] = ('multipart/related; '
-                                     'boundary="%s"') % multipart_boundary
-          url = _add_query_parameter(url, 'uploadType', 'multipart')
-
-    logger.info('URL being requested: %s' % url)
-    return self._requestBuilder(self._http,
-                                model.response,
-                                url,
-                                method=httpMethod,
-                                body=body,
-                                headers=headers,
-                                methodId=methodId,
-                                resumable=resumable)
-
-  docs = [methodDesc.get('description', DEFAULT_METHOD_DOC), '\n\n']
-  if len(parameters.argmap) > 0:
-    docs.append('Args:\n')
-
-  # Skip undocumented params and params common to all methods.
-  skip_parameters = rootDesc.get('parameters', {}).keys()
-  skip_parameters.extend(STACK_QUERY_PARAMETERS)
-
-  all_args = parameters.argmap.keys()
-  args_ordered = [key2param(s) for s in methodDesc.get('parameterOrder', [])]
-
-  # Move body to the front of the line.
-  if 'body' in all_args:
-    args_ordered.append('body')
-
-  for name in all_args:
-    if name not in args_ordered:
-      args_ordered.append(name)
-
-  for arg in args_ordered:
-    if arg in skip_parameters:
-      continue
-
-    repeated = ''
-    if arg in parameters.repeated_params:
-      repeated = ' (repeated)'
-    required = ''
-    if arg in parameters.required_params:
-      required = ' (required)'
-    paramdesc = methodDesc['parameters'][parameters.argmap[arg]]
-    paramdoc = paramdesc.get('description', 'A parameter')
-    if '$ref' in paramdesc:
-      docs.append(
-          ('  %s: object, %s%s%s\n    The object takes the'
-          ' form of:\n\n%s\n\n') % (arg, paramdoc, required, repeated,
-            schema.prettyPrintByName(paramdesc['$ref'])))
-    else:
-      paramtype = paramdesc.get('type', 'string')
-      docs.append('  %s: %s, %s%s%s\n' % (arg, paramtype, paramdoc, required,
-                                          repeated))
-    enum = paramdesc.get('enum', [])
-    enumDesc = paramdesc.get('enumDescriptions', [])
-    if enum and enumDesc:
-      docs.append('    Allowed values\n')
-      for (name, desc) in zip(enum, enumDesc):
-        docs.append('      %s - %s\n' % (name, desc))
-  if 'response' in methodDesc:
-    if methodName.endswith('_media'):
-      docs.append('\nReturns:\n  The media object as a string.\n\n    ')
-    else:
-      docs.append('\nReturns:\n  An object of the form:\n\n    ')
-      docs.append(schema.prettyPrintSchema(methodDesc['response']))
-
-  setattr(method, '__doc__', ''.join(docs))
-  return (methodName, method)
-
-
-def createNextMethod(methodName):
-  """Creates any _next methods for attaching to a Resource.
-
-  The _next methods allow for easy iteration through list() responses.
-
-  Args:
-    methodName: string, name of the method to use.
-  """
-  methodName = fix_method_name(methodName)
-
-  def methodNext(self, previous_request, previous_response):
-    """Retrieves the next page of results.
-
-Args:
-  previous_request: The request for the previous page. (required)
-  previous_response: The response from the request for the previous page. (required)
-
-Returns:
-  A request object that you can call 'execute()' on to request the next
-  page. Returns None if there are no more items in the collection.
-    """
-    # Retrieve nextPageToken from previous_response
-    # Use as pageToken in previous_request to create new request.
-
-    if 'nextPageToken' not in previous_response:
-      return None
-
-    request = copy.copy(previous_request)
-
-    pageToken = previous_response['nextPageToken']
-    parsed = list(urlparse.urlparse(request.uri))
-    q = parse_qsl(parsed[4])
-
-    # Find and remove old 'pageToken' value from URI
-    newq = [(key, value) for (key, value) in q if key != 'pageToken']
-    newq.append(('pageToken', pageToken))
-    parsed[4] = urllib.urlencode(newq)
-    uri = urlparse.urlunparse(parsed)
-
-    request.uri = uri
-
-    logger.info('URL being requested: %s' % uri)
-
-    return request
-
-  return (methodName, methodNext)
-
-
-class Resource(object):
-  """A class for interacting with a resource."""
-
-  def __init__(self, http, baseUrl, model, requestBuilder, developerKey,
-               resourceDesc, rootDesc, schema):
-    """Build a Resource from the API description.
-
-    Args:
-      http: httplib2.Http, Object to make http requests with.
-      baseUrl: string, base URL for the API. All requests are relative to this
-          URI.
-      model: apiclient.Model, converts to and from the wire format.
-      requestBuilder: class or callable that instantiates an
-          apiclient.HttpRequest object.
-      developerKey: string, key obtained from
-          https://code.google.com/apis/console
-      resourceDesc: object, section of deserialized discovery document that
-          describes a resource. Note that the top level discovery document
-          is considered a resource.
-      rootDesc: object, the entire deserialized discovery document.
-      schema: object, mapping of schema names to schema descriptions.
-    """
-    self._dynamic_attrs = []
-
-    self._http = http
-    self._baseUrl = baseUrl
-    self._model = model
-    self._developerKey = developerKey
-    self._requestBuilder = requestBuilder
-    self._resourceDesc = resourceDesc
-    self._rootDesc = rootDesc
-    self._schema = schema
-
-    self._set_service_methods()
-
-  def _set_dynamic_attr(self, attr_name, value):
-    """Sets an instance attribute and tracks it in a list of dynamic attributes.
-
-    Args:
-      attr_name: string; The name of the attribute to be set
-      value: The value being set on the object and tracked in the dynamic cache.
-    """
-    self._dynamic_attrs.append(attr_name)
-    self.__dict__[attr_name] = value
-
-  def __getstate__(self):
-    """Trim the state down to something that can be pickled.
-
-    Uses the fact that the instance variable _dynamic_attrs holds attrs that
-    will be wiped and restored on pickle serialization.
-    """
-    state_dict = copy.copy(self.__dict__)
-    for dynamic_attr in self._dynamic_attrs:
-      del state_dict[dynamic_attr]
-    del state_dict['_dynamic_attrs']
-    return state_dict
-
-  def __setstate__(self, state):
-    """Reconstitute the state of the object from being pickled.
-
-    Uses the fact that the instance variable _dynamic_attrs holds attrs that
-    will be wiped and restored on pickle serialization.
-    """
-    self.__dict__.update(state)
-    self._dynamic_attrs = []
-    self._set_service_methods()
-
-  def _set_service_methods(self):
-    self._add_basic_methods(self._resourceDesc, self._rootDesc, self._schema)
-    self._add_nested_resources(self._resourceDesc, self._rootDesc, self._schema)
-    self._add_next_methods(self._resourceDesc, self._schema)
-
-  def _add_basic_methods(self, resourceDesc, rootDesc, schema):
-    # Add basic methods to Resource
-    if 'methods' in resourceDesc:
-      for methodName, methodDesc in resourceDesc['methods'].iteritems():
-        fixedMethodName, method = createMethod(
-            methodName, methodDesc, rootDesc, schema)
-        self._set_dynamic_attr(fixedMethodName,
-                               method.__get__(self, self.__class__))
-        # Add in _media methods. The functionality of the attached method will
-        # change when it sees that the method name ends in _media.
-        if methodDesc.get('supportsMediaDownload', False):
-          fixedMethodName, method = createMethod(
-              methodName + '_media', methodDesc, rootDesc, schema)
-          self._set_dynamic_attr(fixedMethodName,
-                                 method.__get__(self, self.__class__))
-
-  def _add_nested_resources(self, resourceDesc, rootDesc, schema):
-    # Add in nested resources
-    if 'resources' in resourceDesc:
-
-      def createResourceMethod(methodName, methodDesc):
-        """Create a method on the Resource to access a nested Resource.
-
-        Args:
-          methodName: string, name of the method to use.
-          methodDesc: object, fragment of deserialized discovery document that
-            describes the method.
-        """
-        methodName = fix_method_name(methodName)
-
-        def methodResource(self):
-          return Resource(http=self._http, baseUrl=self._baseUrl,
-                          model=self._model, developerKey=self._developerKey,
-                          requestBuilder=self._requestBuilder,
-                          resourceDesc=methodDesc, rootDesc=rootDesc,
-                          schema=schema)
-
-        setattr(methodResource, '__doc__', 'A collection resource.')
-        setattr(methodResource, '__is_resource__', True)
-
-        return (methodName, methodResource)
-
-      for methodName, methodDesc in resourceDesc['resources'].iteritems():
-        fixedMethodName, method = createResourceMethod(methodName, methodDesc)
-        self._set_dynamic_attr(fixedMethodName,
-                               method.__get__(self, self.__class__))
-
-  def _add_next_methods(self, resourceDesc, schema):
-    # Add _next() methods
-    # Look for response bodies in schema that contain nextPageToken, and methods
-    # that take a pageToken parameter.
-    if 'methods' in resourceDesc:
-      for methodName, methodDesc in resourceDesc['methods'].iteritems():
-        if 'response' in methodDesc:
-          responseSchema = methodDesc['response']
-          if '$ref' in responseSchema:
-            responseSchema = schema.get(responseSchema['$ref'])
-          hasNextPageToken = 'nextPageToken' in responseSchema.get('properties',
-                                                                   {})
-          hasPageToken = 'pageToken' in methodDesc.get('parameters', {})
-          if hasNextPageToken and hasPageToken:
-            fixedMethodName, method = createNextMethod(methodName + '_next')
-            self._set_dynamic_attr(fixedMethodName,
-                                   method.__get__(self, self.__class__))

apiclient/errors.py

@@ -1,140 +0,0 @@
-#!/usr/bin/python2.4
-#
-# Copyright (C) 2010 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Errors for the library.
-
-All exceptions defined by the library
-should be defined in this file.
-"""
-
-__author__ = 'jcgregorio@google.com (Joe Gregorio)'
-
-
-from oauth2client import util
-from oauth2client.anyjson import simplejson
-
-
-class Error(Exception):
-  """Base error for this module."""
-  pass
-
-
-class HttpError(Error):
-  """HTTP data was invalid or unexpected."""
-
-  @util.positional(3)
-  def __init__(self, resp, content, uri=None):
-    self.resp = resp
-    self.content = content
-    self.uri = uri
-
-  def _get_reason(self):
-    """Calculate the reason for the error from the response content."""
-    reason = self.resp.reason
-    try:
-      data = simplejson.loads(self.content)
-      reason = data['error']['message']
-    except (ValueError, KeyError):
-      pass
-    if reason is None:
-      reason = ''
-    return reason
-
-  def __repr__(self):
-    if self.uri:
-      return '<HttpError %s when requesting %s returned "%s">' % (
-          self.resp.status, self.uri, self._get_reason().strip())
-    else:
-      return '<HttpError %s "%s">' % (self.resp.status, self._get_reason())
-
-  __str__ = __repr__
-
-
-class InvalidJsonError(Error):
-  """The JSON returned could not be parsed."""
-  pass
-
-
-class UnknownFileType(Error):
-  """File type unknown or unexpected."""
-  pass
-
-
-class UnknownLinkType(Error):
-  """Link type unknown or unexpected."""
-  pass
-
-
-class UnknownApiNameOrVersion(Error):
-  """No API with that name and version exists."""
-  pass
-
-
-class UnacceptableMimeTypeError(Error):
-  """That is an unacceptable mimetype for this operation."""
-  pass
-
-
-class MediaUploadSizeError(Error):
-  """Media is larger than the method can accept."""
-  pass
-
-
-class ResumableUploadError(HttpError):
-  """Error occured during resumable upload."""
-  pass
-
-
-class InvalidChunkSizeError(Error):
-  """The given chunksize is not valid."""
-  pass
-
-class InvalidNotificationError(Error):
-  """The channel Notification is invalid."""
-  pass
-
-class BatchError(HttpError):
-  """Error occured during batch operations."""
-
-  @util.positional(2)
-  def __init__(self, reason, resp=None, content=None):
-    self.resp = resp
-    self.content = content
-    self.reason = reason
-
-  def __repr__(self):
-      return '<BatchError %s "%s">' % (self.resp.status, self.reason)
-
-  __str__ = __repr__
-
-
-class UnexpectedMethodError(Error):
-  """Exception raised by RequestMockBuilder on unexpected calls."""
-
-  @util.positional(1)
-  def __init__(self, methodId=None):
-    """Constructor for an UnexpectedMethodError."""
-    super(UnexpectedMethodError, self).__init__(
-        'Received unexpected call %s' % methodId)
-
-
-class UnexpectedBodyError(Error):
-  """Exception raised by RequestMockBuilder on unexpected bodies."""
-
-  def __init__(self, expected, provided):
-    """Constructor for an UnexpectedMethodError."""
-    super(UnexpectedBodyError, self).__init__(
-        'Expected: [%s] - Provided: [%s]' % (expected, provided))

apiclient/model.py

@@ -1,383 +0,0 @@
-#!/usr/bin/python2.4
-#
-# Copyright (C) 2010 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Model objects for requests and responses.
-
-Each API may support one or more serializations, such
-as JSON, Atom, etc. The model classes are responsible
-for converting between the wire format and the Python
-object representation.
-"""
-
-__author__ = 'jcgregorio@google.com (Joe Gregorio)'
-
-import logging
-import urllib
-
-from apiclient import __version__
-from errors import HttpError
-from oauth2client.anyjson import simplejson
-
-
-dump_request_response = False
-
-
-def _abstract():
-  raise NotImplementedError('You need to override this function')
-
-
-class Model(object):
-  """Model base class.
-
-  All Model classes should implement this interface.
-  The Model serializes and de-serializes between a wire
-  format such as JSON and a Python object representation.
-  """
-
-  def request(self, headers, path_params, query_params, body_value):
-    """Updates outgoing requests with a serialized body.
-
-    Args:
-      headers: dict, request headers
-      path_params: dict, parameters that appear in the request path
-      query_params: dict, parameters that appear in the query
-      body_value: object, the request body as a Python object, which must be
-                  serializable.
-    Returns:
-      A tuple of (headers, path_params, query, body)
-
-      headers: dict, request headers
-      path_params: dict, parameters that appear in the request path
-      query: string, query part of the request URI
-      body: string, the body serialized in the desired wire format.
-    """
-    _abstract()
-
-  def response(self, resp, content):
-    """Convert the response wire format into a Python object.
-
-    Args:
-      resp: httplib2.Response, the HTTP response headers and status
-      content: string, the body of the HTTP response
-
-    Returns:
-      The body de-serialized as a Python object.
-
-    Raises:
-      apiclient.errors.HttpError if a non 2xx response is received.
-    """
-    _abstract()
-
-
-class BaseModel(Model):
-  """Base model class.
-
-  Subclasses should provide implementations for the "serialize" and
-  "deserialize" methods, as well as values for the following class attributes.
-
-  Attributes:
-    accept: The value to use for the HTTP Accept header.
-    content_type: The value to use for the HTTP Content-type header.
-    no_content_response: The value to return when deserializing a 204 "No
-        Content" response.
-    alt_param: The value to supply as the "alt" query parameter for requests.
-  """
-
-  accept = None
-  content_type = None
-  no_content_response = None
-  alt_param = None
-
-  def _log_request(self, headers, path_params, query, body):
-    """Logs debugging information about the request if requested."""
-    if dump_request_response:
-      logging.info('--request-start--')
-      logging.info('-headers-start-')
-      for h, v in headers.iteritems():
-        logging.info('%s: %s', h, v)
-      logging.info('-headers-end-')
-      logging.info('-path-parameters-start-')
-      for h, v in path_params.iteritems():
-        logging.info('%s: %s', h, v)
-      logging.info('-path-parameters-end-')
-      logging.info('body: %s', body)
-      logging.info('query: %s', query)
-      logging.info('--request-end--')
-
-  def request(self, headers, path_params, query_params, body_value):
-    """Updates outgoing requests with a serialized body.
-
-    Args:
-      headers: dict, request headers
-      path_params: dict, parameters that appear in the request path
-      query_params: dict, parameters that appear in the query
-      body_value: object, the request body as a Python object, which must be
-                  serializable by simplejson.
-    Returns:
-      A tuple of (headers, path_params, query, body)
-
-      headers: dict, request headers
-      path_params: dict, parameters that appear in the request path
-      query: string, query part of the request URI
-      body: string, the body serialized as JSON
-    """
-    query = self._build_query(query_params)
-    headers['accept'] = self.accept
-    headers['accept-encoding'] = 'gzip, deflate'
-    if 'user-agent' in headers:
-      headers['user-agent'] += ' '
-    else:
-      headers['user-agent'] = ''
-    headers['user-agent'] += 'google-api-python-client/%s (gzip)' % __version__
-
-    if body_value is not None:
-      headers['content-type'] = self.content_type
-      body_value = self.serialize(body_value)
-    self._log_request(headers, path_params, query, body_value)
-    return (headers, path_params, query, body_value)
-
-  def _build_query(self, params):
-    """Builds a query string.
-
-    Args:
-      params: dict, the query parameters
-
-    Returns:
-      The query parameters properly encoded into an HTTP URI query string.
-    """
-    if self.alt_param is not None:
-      params.update({'alt': self.alt_param})
-    astuples = []
-    for key, value in params.iteritems():
-      if type(value) == type([]):
-        for x in value:
-          x = x.encode('utf-8')
-          astuples.append((key, x))
-      else:
-        if getattr(value, 'encode', False) and callable(value.encode):
-          value = value.encode('utf-8')
-        astuples.append((key, value))
-    return '?' + urllib.urlencode(astuples)
-
-  def _log_response(self, resp, content):
-    """Logs debugging information about the response if requested."""
-    if dump_request_response:
-      logging.info('--response-start--')
-      for h, v in resp.iteritems():
-        logging.info('%s: %s', h, v)
-      if content:
-        logging.info(content)
-      logging.info('--response-end--')
-
-  def response(self, resp, content):
-    """Convert the response wire format into a Python object.
-
-    Args:
-      resp: httplib2.Response, the HTTP response headers and status
-      content: string, the body of the HTTP response
-
-    Returns:
-      The body de-serialized as a Python object.
-
-    Raises:
-      apiclient.errors.HttpError if a non 2xx response is received.
-    """
-    self._log_response(resp, content)
-    # Error handling is TBD, for example, do we retry
-    # for some operation/error combinations?
-    if resp.status < 300:
-      if resp.status == 204:
-        # A 204: No Content response should be treated differently
-        # to all the other success states
-        return self.no_content_response
-      return self.deserialize(content)
-    else:
-      logging.debug('Content from bad request was: %s' % content)
-      raise HttpError(resp, content)
-
-  def serialize(self, body_value):
-    """Perform the actual Python object serialization.
-
-    Args:
-      body_value: object, the request body as a Python object.
-
-    Returns:
-      string, the body in serialized form.
-    """
-    _abstract()
-
-  def deserialize(self, content):
-    """Perform the actual deserialization from response string to Python
-    object.
-
-    Args:
-      content: string, the body of the HTTP response
-
-    Returns:
-      The body de-serialized as a Python object.
-    """
-    _abstract()
-
-
-class JsonModel(BaseModel):
-  """Model class for JSON.
-
-  Serializes and de-serializes between JSON and the Python
-  object representation of HTTP request and response bodies.
-  """
-  accept = 'application/json'
-  content_type = 'application/json'
-  alt_param = 'json'
-
-  def __init__(self, data_wrapper=False):
-    """Construct a JsonModel.
-
-    Args:
-      data_wrapper: boolean, wrap requests and responses in a data wrapper
-    """
-    self._data_wrapper = data_wrapper
-
-  def serialize(self, body_value):
-    if (isinstance(body_value, dict) and 'data' not in body_value and
-        self._data_wrapper):
-      body_value = {'data': body_value}
-    return simplejson.dumps(body_value)
-
-  def deserialize(self, content):
-    content = content.decode('utf-8')
-    body = simplejson.loads(content)
-    if self._data_wrapper and isinstance(body, dict) and 'data' in body:
-      body = body['data']
-    return body
-
-  @property
-  def no_content_response(self):
-    return {}
-
-
-class RawModel(JsonModel):
-  """Model class for requests that don't return JSON.
-
-  Serializes and de-serializes between JSON and the Python
-  object representation of HTTP request, and returns the raw bytes
-  of the response body.
-  """
-  accept = '*/*'
-  content_type = 'application/json'
-  alt_param = None
-
-  def deserialize(self, content):
-    return content
-
-  @property
-  def no_content_response(self):
-    return ''
-
-
-class MediaModel(JsonModel):
-  """Model class for requests that return Media.
-
-  Serializes and de-serializes between JSON and the Python
-  object representation of HTTP request, and returns the raw bytes
-  of the response body.
-  """
-  accept = '*/*'
-  content_type = 'application/json'
-  alt_param = 'media'
-
-  def deserialize(self, content):
-    return content
-
-  @property
-  def no_content_response(self):
-    return ''
-
-
-class ProtocolBufferModel(BaseModel):
-  """Model class for protocol buffers.
-
-  Serializes and de-serializes the binary protocol buffer sent in the HTTP
-  request and response bodies.
-  """
-  accept = 'application/x-protobuf'
-  content_type = 'application/x-protobuf'
-  alt_param = 'proto'
-
-  def __init__(self, protocol_buffer):
-    """Constructs a ProtocolBufferModel.
-
-    The serialzed protocol buffer returned in an HTTP response will be
-    de-serialized using the given protocol buffer class.
-
-    Args:
-      protocol_buffer: The protocol buffer class used to de-serialize a
-      response from the API.
-    """
-    self._protocol_buffer = protocol_buffer
-
-  def serialize(self, body_value):
-    return body_value.SerializeToString()
-
-  def deserialize(self, content):
-    return self._protocol_buffer.FromString(content)
-
-  @property
-  def no_content_response(self):
-    return self._protocol_buffer()
-
-
-def makepatch(original, modified):
-  """Create a patch object.
-
-  Some methods support PATCH, an efficient way to send updates to a resource.
-  This method allows the easy construction of patch bodies by looking at the
-  differences between a resource before and after it was modified.
-
-  Args:
-    original: object, the original deserialized resource
-    modified: object, the modified deserialized resource
-  Returns:
-    An object that contains only the changes from original to modified, in a
-    form suitable to pass to a PATCH method.
-
-  Example usage:
-    item = service.activities().get(postid=postid, userid=userid).execute()
-    original = copy.deepcopy(item)
-    item['object']['content'] = 'This is updated.'
-    service.activities.patch(postid=postid, userid=userid,
-      body=makepatch(original, item)).execute()
-  """
-  patch = {}
-  for key, original_value in original.iteritems():
-    modified_value = modified.get(key, None)
-    if modified_value is None:
-      # Use None to signal that the element is deleted
-      patch[key] = None
-    elif original_value != modified_value:
-      if type(original_value) == type({}):
-        # Recursively descend objects
-        patch[key] = makepatch(original_value, modified_value)
-      else:
-        # In the case of simple types or arrays we just replace
-        patch[key] = modified_value
-    else:
-      # Don't add anything to patch if there's no change
-      pass
-  for key in modified:
-    if key not in original:
-      patch[key] = modified[key]
-
-  return patch

apiclient/push.py

@@ -1,274 +0,0 @@
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Push notifications support.
-
-This code is based on experimental APIs and is subject to change.
-"""
-
-__author__ = 'afshar@google.com (Ali Afshar)'
-
-import binascii
-import collections
-import os
-import urllib
-
-SUBSCRIBE = 'X-GOOG-SUBSCRIBE'
-SUBSCRIPTION_ID = 'X-GOOG-SUBSCRIPTION-ID'
-TOPIC_ID = 'X-GOOG-TOPIC-ID'
-TOPIC_URI = 'X-GOOG-TOPIC-URI'
-CLIENT_TOKEN = 'X-GOOG-CLIENT-TOKEN'
-EVENT_TYPE = 'X-GOOG-EVENT-TYPE'
-UNSUBSCRIBE = 'X-GOOG-UNSUBSCRIBE'
-
-
-class InvalidSubscriptionRequestError(ValueError):
-  """The request cannot be subscribed."""
-
-
-def new_token():
-  """Gets a random token for use as a client_token in push notifications.
-
-  Returns:
-    str, a new random token.
-  """
-  return binascii.hexlify(os.urandom(32))
-
-
-class Channel(object):
-  """Base class for channel types."""
-
-  def __init__(self, channel_type, channel_args):
-    """Create a new Channel.
-
-    You probably won't need to create this channel manually, since there are
-    subclassed Channel for each specific type with a more customized set of
-    arguments to pass. However, you may wish to just create it manually here.
-
-    Args:
-      channel_type: str, the type of channel.
-      channel_args: dict, arguments to pass to the channel.
-    """
-    self.channel_type = channel_type
-    self.channel_args = channel_args
-
-  def as_header_value(self):
-    """Create the appropriate header for this channel.
-
-    Returns:
-      str encoded channel description suitable for use as a header.
-    """
-    return '%s?%s' % (self.channel_type, urllib.urlencode(self.channel_args))
-
-  def write_header(self, headers):
-    """Write the appropriate subscribe header to a headers dict.
-
-    Args:
-      headers: dict, headers to add subscribe header to.
-    """
-    headers[SUBSCRIBE] = self.as_header_value()
-
-
-class WebhookChannel(Channel):
-  """Channel for registering web hook notifications."""
-
-  def __init__(self, url, app_engine=False):
-    """Create a new WebhookChannel
-
-    Args:
-      url: str, URL to post notifications to.
-      app_engine: bool, default=False, whether the destination for the
-      notifications is an App Engine application.
-    """
-    super(WebhookChannel, self).__init__(
-        channel_type='web_hook',
-        channel_args={
-            'url': url,
-            'app_engine': app_engine and 'true' or 'false',
-        }
-    )
-
-
-class Headers(collections.defaultdict):
-  """Headers for managing subscriptions."""
-
-
-  ALL_HEADERS = set([SUBSCRIBE, SUBSCRIPTION_ID, TOPIC_ID, TOPIC_URI,
-                     CLIENT_TOKEN, EVENT_TYPE, UNSUBSCRIBE])
-
-  def __init__(self):
-    """Create a new subscription configuration instance."""
-    collections.defaultdict.__init__(self, str)
-
-  def __setitem__(self, key, value):
-    """Set a header value, ensuring the key is an allowed value.
-
-    Args:
-      key: str, the header key.
-      value: str, the header value.
-    Raises:
-      ValueError if key is not one of the accepted headers.
-    """
-    normal_key = self._normalize_key(key)
-    if normal_key not in self.ALL_HEADERS:
-      raise ValueError('Header name must be one of %s.' % self.ALL_HEADERS)
-    else:
-      return collections.defaultdict.__setitem__(self, normal_key, value)
-
-  def __getitem__(self, key):
-    """Get a header value, normalizing the key case.
-
-    Args:
-      key: str, the header key.
-    Returns:
-      String header value.
-    Raises:
-      KeyError if the key is not one of the accepted headers.
-    """
-    normal_key = self._normalize_key(key)
-    if normal_key not in self.ALL_HEADERS:
-      raise ValueError('Header name must be one of %s.' % self.ALL_HEADERS)
-    else:
-      return collections.defaultdict.__getitem__(self, normal_key)
-
-  def _normalize_key(self, key):
-    """Normalize a header name for use as a key."""
-    return key.upper()
-
-  def items(self):
-    """Generator for each header."""
-    for header in self.ALL_HEADERS:
-      value = self[header]
-      if value:
-        yield header, value
-
-  def write(self, headers):
-    """Applies the subscription headers.
-
-    Args:
-      headers: dict of headers to insert values into.
-    """
-    for header, value in self.items():
-      headers[header.lower()] = value
-
-  def read(self, headers):
-    """Read from headers.
-
-    Args:
-      headers: dict of headers to read from.
-    """
-    for header in self.ALL_HEADERS:
-      if header.lower() in headers:
-        self[header] = headers[header.lower()]
-
-
-class Subscription(object):
-  """Information about a subscription."""
-
-  def __init__(self):
-    """Create a new Subscription."""
-    self.headers = Headers()
-
-  @classmethod
-  def for_request(cls, request, channel, client_token=None):
-    """Creates a subscription and attaches it to a request.
-
-    Args:
-      request: An http.HttpRequest to modify for making a subscription.
-      channel: A apiclient.push.Channel describing the subscription to
-               create.
-      client_token: (optional) client token to verify the notification.
-
-    Returns:
-      New subscription object.
-    """
-    subscription = cls.for_channel(channel=channel, client_token=client_token)
-    subscription.headers.write(request.headers)
-    if request.method != 'GET':
-      raise InvalidSubscriptionRequestError(
-          'Can only subscribe to requests which are GET.')
-    request.method = 'POST'
-
-    def _on_response(response, subscription=subscription):
-      """Called with the response headers. Reads the subscription headers."""
-      subscription.headers.read(response)
-
-    request.add_response_callback(_on_response)
-    return subscription
-
-  @classmethod
-  def for_channel(cls, channel, client_token=None):
-    """Alternate constructor to create a subscription from a channel.
-
-    Args:
-      channel: A apiclient.push.Channel describing the subscription to
-               create.
-      client_token: (optional) client token to verify the notification.
-
-    Returns:
-      New subscription object.
-    """
-    subscription = cls()
-    channel.write_header(subscription.headers)
-    if client_token is None:
-      client_token = new_token()
-    subscription.headers[SUBSCRIPTION_ID] = new_token()
-    subscription.headers[CLIENT_TOKEN] = client_token
-    return subscription
-
-  def verify(self, headers):
-    """Verifies that a webhook notification has the correct client_token.
-
-    Args:
-      headers: dict of request headers for a push notification.
-
-    Returns:
-      Boolean value indicating whether the notification is verified.
-    """
-    new_subscription = Subscription()
-    new_subscription.headers.read(headers)
-    return new_subscription.client_token == self.client_token
-
-  @property
-  def subscribe(self):
-    """Subscribe header value."""
-    return self.headers[SUBSCRIBE]
-
-  @property
-  def subscription_id(self):
-    """Subscription ID header value."""
-    return self.headers[SUBSCRIPTION_ID]
-
-  @property
-  def topic_id(self):
-    """Topic ID header value."""
-    return self.headers[TOPIC_ID]
-
-  @property
-  def topic_uri(self):
-    """Topic URI header value."""
-    return self.headers[TOPIC_URI]
-
-  @property
-  def client_token(self):
-    """Client Token header value."""
-    return self.headers[CLIENT_TOKEN]
-
-  @property
-  def event_type(self):
-    """Event Type header value."""
-    return self.headers[EVENT_TYPE]
-
-  @property
-  def unsubscribe(self):
-    """Unsuscribe header value."""
-    return self.headers[UNSUBSCRIBE]

apiclient/sample_tools.py

@@ -1,93 +0,0 @@
-# Copyright (C) 2013 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Utilities for making samples.
-
-Consolidates a lot of code commonly repeated in sample applications.
-"""
-
-__author__ = 'jcgregorio@google.com (Joe Gregorio)'
-__all__ = ['init']
-
-
-import argparse
-import httplib2
-import os
-
-from apiclient import discovery
-from oauth2client import client
-from oauth2client import file
-from oauth2client import tools
-
-
-def init(argv, name, version, doc, filename, scope=None, parents=[]):
-  """A common initialization routine for samples.
-
-  Many of the sample applications do the same initialization, which has now
-  been consolidated into this function. This function uses common idioms found
-  in almost all the samples, i.e. for an API with name 'apiname', the
-  credentials are stored in a file named apiname.dat, and the
-  client_secrets.json file is stored in the same directory as the application
-  main file.
-
-  Args:
-    argv: list of string, the command-line parameters of the application.
-    name: string, name of the API.
-    version: string, version of the API.
-    doc: string, description of the application. Usually set to __doc__.
-    file: string, filename of the application. Usually set to __file__.
-    parents: list of argparse.ArgumentParser, additional command-line flags.
-    scope: string, The OAuth scope used.
-
-  Returns:
-    A tuple of (service, flags), where service is the service object and flags
-    is the parsed command-line flags.
-  """
-  if scope is None:
-    scope = 'https://www.googleapis.com/auth/' + name
-
-  # Parser command-line arguments.
-  parent_parsers = [tools.argparser]
-  parent_parsers.extend(parents)
-  parser = argparse.ArgumentParser(
-      description=doc,
-      formatter_class=argparse.RawDescriptionHelpFormatter,
-      parents=parent_parsers)
-  flags = parser.parse_args(argv[1:])
-
-  # Name of a file containing the OAuth 2.0 information for this
-  # application, including client_id and client_secret, which are found
-  # on the API Access tab on the Google APIs
-  # Console <http://code.google.com/apis/console>.
-  client_secrets = os.path.join(os.path.dirname(filename),
-                                'client_secrets.json')
-
-  # Set up a Flow object to be used if we need to authenticate.
-  flow = client.flow_from_clientsecrets(client_secrets,
-      scope=scope,
-      message=tools.message_if_missing(client_secrets))
-
-  # Prepare credentials, and authorize HTTP object with them.
-  # If the credentials don't exist or are invalid run through the native client
-  # flow. The Storage object will ensure that if successful the good
-  # credentials will get written back to a file.
-  storage = file.Storage(name + '.dat')
-  credentials = storage.get()
-  if credentials is None or credentials.invalid:
-    credentials = tools.run_flow(flow, storage, flags)
-  http = credentials.authorize(http = httplib2.Http())
-
-  # Construct a service object via the discovery service.
-  service = discovery.build(name, version, http=http)
-  return (service, flags)

apiclient/schema.py

@@ -1,312 +0,0 @@
-# Copyright (C) 2010 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Schema processing for discovery based APIs
-
-Schemas holds an APIs discovery schemas. It can return those schema as
-deserialized JSON objects, or pretty print them as prototype objects that
-conform to the schema.
-
-For example, given the schema:
-
- schema = \"\"\"{
-   "Foo": {
-    "type": "object",
-    "properties": {
-     "etag": {
-      "type": "string",
-      "description": "ETag of the collection."
-     },
-     "kind": {
-      "type": "string",
-      "description": "Type of the collection ('calendar#acl').",
-      "default": "calendar#acl"
-     },
-     "nextPageToken": {
-      "type": "string",
-      "description": "Token used to access the next
-         page of this result. Omitted if no further results are available."
-     }
-    }
-   }
- }\"\"\"
-
- s = Schemas(schema)
- print s.prettyPrintByName('Foo')
-
- Produces the following output:
-
-  {
-   "nextPageToken": "A String", # Token used to access the
-       # next page of this result. Omitted if no further results are available.
-   "kind": "A String", # Type of the collection ('calendar#acl').
-   "etag": "A String", # ETag of the collection.
-  },
-
-The constructor takes a discovery document in which to look up named schema.
-"""
-
-# TODO(jcgregorio) support format, enum, minimum, maximum
-
-__author__ = 'jcgregorio@google.com (Joe Gregorio)'
-
-import copy
-
-from oauth2client import util
-from oauth2client.anyjson import simplejson
-
-
-class Schemas(object):
-  """Schemas for an API."""
-
-  def __init__(self, discovery):
-    """Constructor.
-
-    Args:
-      discovery: object, Deserialized discovery document from which we pull
-        out the named schema.
-    """
-    self.schemas = discovery.get('schemas', {})
-
-    # Cache of pretty printed schemas.
-    self.pretty = {}
-
-  @util.positional(2)
-  def _prettyPrintByName(self, name, seen=None, dent=0):
-    """Get pretty printed object prototype from the schema name.
-
-    Args:
-      name: string, Name of schema in the discovery document.
-      seen: list of string, Names of schema already seen. Used to handle
-        recursive definitions.
-
-    Returns:
-      string, A string that contains a prototype object with
-        comments that conforms to the given schema.
-    """
-    if seen is None:
-      seen = []
-
-    if name in seen:
-      # Do not fall into an infinite loop over recursive definitions.
-      return '# Object with schema name: %s' % name
-    seen.append(name)
-
-    if name not in self.pretty:
-      self.pretty[name] = _SchemaToStruct(self.schemas[name],
-          seen, dent=dent).to_str(self._prettyPrintByName)
-
-    seen.pop()
-
-    return self.pretty[name]
-
-  def prettyPrintByName(self, name):
-    """Get pretty printed object prototype from the schema name.
-
-    Args:
-      name: string, Name of schema in the discovery document.
-
-    Returns:
-      string, A string that contains a prototype object with
-        comments that conforms to the given schema.
-    """
-    # Return with trailing comma and newline removed.
-    return self._prettyPrintByName(name, seen=[], dent=1)[:-2]
-
-  @util.positional(2)
-  def _prettyPrintSchema(self, schema, seen=None, dent=0):
-    """Get pretty printed object prototype of schema.
-
-    Args:
-      schema: object, Parsed JSON schema.
-      seen: list of string, Names of schema already seen. Used to handle
-        recursive definitions.
-
-    Returns:
-      string, A string that contains a prototype object with
-        comments that conforms to the given schema.
-    """
-    if seen is None:
-      seen = []
-
-    return _SchemaToStruct(schema, seen, dent=dent).to_str(self._prettyPrintByName)
-
-  def prettyPrintSchema(self, schema):
-    """Get pretty printed object prototype of schema.
-
-    Args:
-      schema: object, Parsed JSON schema.
-
-    Returns:
-      string, A string that contains a prototype object with
-        comments that conforms to the given schema.
-    """
-    # Return with trailing comma and newline removed.
-    return self._prettyPrintSchema(schema, dent=1)[:-2]
-
-  def get(self, name):
-    """Get deserialized JSON schema from the schema name.
-
-    Args:
-      name: string, Schema name.
-    """
-    return self.schemas[name]
-
-
-class _SchemaToStruct(object):
-  """Convert schema to a prototype object."""
-
-  @util.positional(3)
-  def __init__(self, schema, seen, dent=0):
-    """Constructor.
-
-    Args:
-      schema: object, Parsed JSON schema.
-      seen: list, List of names of schema already seen while parsing. Used to
-        handle recursive definitions.
-      dent: int, Initial indentation depth.
-    """
-    # The result of this parsing kept as list of strings.
-    self.value = []
-
-    # The final value of the parsing.
-    self.string = None
-
-    # The parsed JSON schema.
-    self.schema = schema
-
-    # Indentation level.
-    self.dent = dent
-
-    # Method that when called returns a prototype object for the schema with
-    # the given name.
-    self.from_cache = None
-
-    # List of names of schema already seen while parsing.
-    self.seen = seen
-
-  def emit(self, text):
-    """Add text as a line to the output.
-
-    Args:
-      text: string, Text to output.
-    """
-    self.value.extend(["  " * self.dent, text, '\n'])
-
-  def emitBegin(self, text):
-    """Add text to the output, but with no line terminator.
-
-    Args:
-      text: string, Text to output.
-      """
-    self.value.extend(["  " * self.dent, text])
-
-  def emitEnd(self, text, comment):
-    """Add text and comment to the output with line terminator.
-
-    Args:
-      text: string, Text to output.
-      comment: string, Python comment.
-    """
-    if comment:
-      divider = '\n' + '  ' * (self.dent + 2) + '# '
-      lines = comment.splitlines()
-      lines = [x.rstrip() for x in lines]
-      comment = divider.join(lines)
-      self.value.extend([text, ' # ', comment, '\n'])
-    else:
-      self.value.extend([text, '\n'])
-
-  def indent(self):
-    """Increase indentation level."""
-    self.dent += 1
-
-  def undent(self):
-    """Decrease indentation level."""
-    self.dent -= 1
-
-  def _to_str_impl(self, schema):
-    """Prototype object based on the schema, in Python code with comments.
-
-    Args:
-      schema: object, Parsed JSON schema file.
-
-    Returns:
-      Prototype object based on the schema, in Python code with comments.
-    """
-    stype = schema.get('type')
-    if stype == 'object':
-      self.emitEnd('{', schema.get('description', ''))
-      self.indent()
-      if 'properties' in schema:
-        for pname, pschema in schema.get('properties', {}).iteritems():
-          self.emitBegin('"%s": ' % pname)
-          self._to_str_impl(pschema)
-      elif 'additionalProperties' in schema:
-        self.emitBegin('"a_key": ')
-        self._to_str_impl(schema['additionalProperties'])
-      self.undent()
-      self.emit('},')
-    elif '$ref' in schema:
-      schemaName = schema['$ref']
-      description = schema.get('description', '')
-      s = self.from_cache(schemaName, seen=self.seen)
-      parts = s.splitlines()
-      self.emitEnd(parts[0], description)
-      for line in parts[1:]:
-        self.emit(line.rstrip())
-    elif stype == 'boolean':
-      value = schema.get('default', 'True or False')
-      self.emitEnd('%s,' % str(value), schema.get('description', ''))
-    elif stype == 'string':
-      value = schema.get('default', 'A String')
-      self.emitEnd('"%s",' % str(value), schema.get('description', ''))
-    elif stype == 'integer':
-      value = schema.get('default', '42')
-      self.emitEnd('%s,' % str(value), schema.get('description', ''))
-    elif stype == 'number':
-      value = schema.get('default', '3.14')
-      self.emitEnd('%s,' % str(value), schema.get('description', ''))
-    elif stype == 'null':
-      self.emitEnd('None,', schema.get('description', ''))
-    elif stype == 'any':
-      self.emitEnd('"",', schema.get('description', ''))
-    elif stype == 'array':
-      self.emitEnd('[', schema.get('description'))
-      self.indent()
-      self.emitBegin('')
-      self._to_str_impl(schema['items'])
-      self.undent()
-      self.emit('],')
-    else:
-      self.emit('Unknown type! %s' % stype)
-      self.emitEnd('', '')
-
-    self.string = ''.join(self.value)
-    return self.string
-
-  def to_str(self, from_cache):
-    """Prototype object based on the schema, in Python code with comments.
-
-    Args:
-      from_cache: callable(name, seen), Callable that retrieves an object
-         prototype for a schema with the given name. Seen is a list of schema
-         names already seen as we recursively descend the schema definition.
-
-    Returns:
-      Prototype object based on the schema, in Python code with comments.
-      The lines of the code will all be properly indented.
-    """
-    self.from_cache = from_cache
-    return self._to_str_impl(self.schema)

atom/client.py

@@ -1,221 +0,0 @@
-#!/usr/bin/env python
-#
-# Copyright (C) 2009 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-"""AtomPubClient provides CRUD ops. in line with the Atom Publishing Protocol.
-
-"""
-
-__author__ = 'j.s@google.com (Jeff Scudder)'
-
-
-import atom.http_core
-
-
-class Error(Exception):
-  pass
-
-
-class MissingHost(Error):
-  pass
-
-
-class AtomPubClient(object):
-  host = None
-  auth_token = None
-  ssl = False # Whether to force all requests over https
-  xoauth_requestor_id = None
-
-  def __init__(self, http_client=None, host=None, auth_token=None, source=None,
-               xoauth_requestor_id=None, **kwargs):
-    """Creates a new AtomPubClient instance.
-
-    Args:
-      source: The name of your application.
-      http_client: An object capable of performing HTTP requests through a
-                   request method. This object is used to perform the request
-                   when the AtomPubClient's request method is called. Used to
-                   allow HTTP requests to be directed to a mock server, or use
-                   an alternate library instead of the default of httplib to
-                   make HTTP requests.
-      host: str The default host name to use if a host is not specified in the
-            requested URI.
-      auth_token: An object which sets the HTTP Authorization header when its
-                  modify_request method is called.
-    """
-    self.http_client = http_client or atom.http_core.ProxiedHttpClient()
-    if host is not None:
-      self.host = host
-    if auth_token is not None:
-      self.auth_token = auth_token
-    self.xoauth_requestor_id = xoauth_requestor_id
-    self.source = source
-
-  def request(self, method=None, uri=None, auth_token=None,
-              http_request=None, **kwargs):
-    """Performs an HTTP request to the server indicated.
-
-    Uses the http_client instance to make the request.
-
-    Args:
-      method: The HTTP method as a string, usually one of 'GET', 'POST',
-              'PUT', or 'DELETE'
-      uri: The URI desired as a string or atom.http_core.Uri.
-      http_request:
-      auth_token: An authorization token object whose modify_request method
-                  sets the HTTP Authorization header.
-
-    Returns:
-      The results of calling self.http_client.request. With the default
-      http_client, this is an HTTP response object.
-    """
-    # Modify the request based on the AtomPubClient settings and parameters
-    # passed in to the request.
-    http_request = self.modify_request(http_request)
-    if isinstance(uri, (str, unicode)):
-      uri = atom.http_core.Uri.parse_uri(uri)
-    if uri is not None:
-      uri.modify_request(http_request)
-    if isinstance(method, (str, unicode)):
-      http_request.method = method
-    # Any unrecognized arguments are assumed to be capable of modifying the
-    # HTTP request.
-    for name, value in kwargs.iteritems():
-      if value is not None:
-        value.modify_request(http_request)
-    # Default to an http request if the protocol scheme is not set.
-    if http_request.uri.scheme is None:
-      http_request.uri.scheme = 'http'
-    # Override scheme. Force requests over https.
-    if self.ssl:
-      http_request.uri.scheme = 'https'
-    if http_request.uri.path is None:
-      http_request.uri.path = '/'
-    # Add the Authorization header at the very end. The Authorization header
-    # value may need to be calculated using information in the request.
-    if auth_token:
-      auth_token.modify_request(http_request)
-    elif self.auth_token:
-      self.auth_token.modify_request(http_request)
-    # Check to make sure there is a host in the http_request.
-    if http_request.uri.host is None:
-      raise MissingHost('No host provided in request %s %s' % (
-          http_request.method, str(http_request.uri)))
-    # Perform the fully specified request using the http_client instance.
-    # Sends the request to the server and returns the server's response.
-    return self.http_client.request(http_request)
-
-  Request = request
-
-  def get(self, uri=None, auth_token=None, http_request=None, **kwargs):
-    """Performs a request using the GET method, returns an HTTP response."""
-    return self.request(method='GET', uri=uri, auth_token=auth_token,
-                        http_request=http_request, **kwargs)
-
-  Get = get
-
-  def post(self, uri=None, data=None, auth_token=None, http_request=None,
-           **kwargs):
-    """Sends data using the POST method, returns an HTTP response."""
-    return self.request(method='POST', uri=uri, auth_token=auth_token,
-                        http_request=http_request, data=data, **kwargs)
-
-  Post = post
-
-  def put(self, uri=None, data=None, auth_token=None, http_request=None,
-          **kwargs):
-    """Sends data using the PUT method, returns an HTTP response."""
-    return self.request(method='PUT', uri=uri, auth_token=auth_token,
-                        http_request=http_request, data=data, **kwargs)
-
-  Put = put
-
-  def delete(self, uri=None, auth_token=None, http_request=None, **kwargs):
-    """Performs a request using the DELETE method, returns an HTTP response."""
-    return self.request(method='DELETE', uri=uri, auth_token=auth_token,
-                        http_request=http_request, **kwargs)
-
-  Delete = delete
-
-  def modify_request(self, http_request):
-    """Changes the HTTP request before sending it to the server.
-
-    Sets the User-Agent HTTP header and fills in the HTTP host portion
-    of the URL if one was not included in the request (for this it uses
-    the self.host member if one is set). This method is called in
-    self.request.
-
-    Args:
-      http_request: An atom.http_core.HttpRequest() (optional) If one is
-                    not provided, a new HttpRequest is instantiated.
-
-    Returns:
-      An atom.http_core.HttpRequest() with the User-Agent header set and
-      if this client has a value in its host member, the host in the request
-      URL is set.
-    """
-    if http_request is None:
-      http_request = atom.http_core.HttpRequest()
-
-    if self.host is not None and http_request.uri.host is None:
-      http_request.uri.host = self.host
-
-    if self.xoauth_requestor_id is not None:
-      http_request.uri.query['xoauth_requestor_id'] = self.xoauth_requestor_id
-
-    # Set the user agent header for logging purposes.
-    if self.source:
-      http_request.headers['User-Agent'] = '%s gdata-py/2.0.14' % self.source
-    else:
-      http_request.headers['User-Agent'] = 'gdata-py/2.0.14'
-
-    return http_request
-
-  ModifyRequest = modify_request
-
-
-class CustomHeaders(object):
-  """Add custom headers to an http_request.
-
-  Usage:
-    >>> custom_headers = atom.client.CustomHeaders(header1='value1',
-            header2='value2')
-    >>> client.get(uri, custom_headers=custom_headers)
-  """
-
-  def __init__(self, **kwargs):
-    """Creates a CustomHeaders instance.
-
-    Initialize the headers dictionary with the arguments list.
-    """
-    self.headers = kwargs
-
-  def modify_request(self, http_request):
-    """Changes the HTTP request before sending it to the server.
-
-    Adds the custom headers to the HTTP request.
-
-    Args:
-      http_request: An atom.http_core.HttpRequest().
-
-    Returns:
-      An atom.http_core.HttpRequest() with the added custom headers.
-    """
-
-    for name, value in self.headers.iteritems():
-      if value is not None:
-        http_request.headers[name] = value
-    return http_request

atom/core.py

@@ -1,550 +0,0 @@
-#!/usr/bin/env python
-#
-#    Copyright (C) 2008 Google Inc.
-#
-#   Licensed under the Apache License, Version 2.0 (the "License");
-#   you may not use this file except in compliance with the License.
-#   You may obtain a copy of the License at
-#
-#       http://www.apache.org/licenses/LICENSE-2.0
-#
-#   Unless required by applicable law or agreed to in writing, software
-#   distributed under the License is distributed on an "AS IS" BASIS,
-#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-#   See the License for the specific language governing permissions and
-#   limitations under the License.
-
-
-# This module is used for version 2 of the Google Data APIs.
-
-
-__author__ = 'j.s@google.com (Jeff Scudder)'
-
-
-import inspect
-try:
-  from xml.etree import cElementTree as ElementTree
-except ImportError:
-  try:
-    import cElementTree as ElementTree
-  except ImportError:
-    try:
-      from xml.etree import ElementTree
-    except ImportError:
-      from elementtree import ElementTree
-
-
-try:
-    from xml.dom.minidom import parseString as xmlString
-except ImportError:
-    xmlString = None
-
-STRING_ENCODING = 'utf-8'
-
-
-class XmlElement(object):
-  """Represents an element node in an XML document.
-
-  The text member is a UTF-8 encoded str or unicode.
-  """
-  _qname = None
-  _other_elements = None
-  _other_attributes = None
-  # The rule set contains mappings for XML qnames to child members and the
-  # appropriate member classes.
-  _rule_set = None
-  _members = None
-  text = None
-
-  def __init__(self, text=None, *args, **kwargs):
-    if ('_members' not in self.__class__.__dict__
-        or self.__class__._members is None):
-      self.__class__._members = tuple(self.__class__._list_xml_members())
-    for member_name, member_type in self.__class__._members:
-      if member_name in kwargs:
-        setattr(self, member_name, kwargs[member_name])
-      else:
-        if isinstance(member_type, list):
-          setattr(self, member_name, [])
-        else:
-          setattr(self, member_name, None)
-    self._other_elements = []
-    self._other_attributes = {}
-    if text is not None:
-      self.text = text
-
-  def _list_xml_members(cls):
-    """Generator listing all members which are XML elements or attributes.
-
-    The following members would be considered XML members:
-    foo = 'abc' - indicates an XML attribute with the qname abc
-    foo = SomeElement - indicates an XML child element
-    foo = [AnElement] - indicates a repeating XML child element, each instance
-        will be stored in a list in this member
-    foo = ('att1', '{http://example.com/namespace}att2') - indicates an XML
-        attribute which has different parsing rules in different versions of
-        the protocol. Version 1 of the XML parsing rules will look for an
-        attribute with the qname 'att1' but verion 2 of the parsing rules will
-        look for a namespaced attribute with the local name of 'att2' and an
-        XML namespace of 'http://example.com/namespace'.
-    """
-    members = []
-    for pair in inspect.getmembers(cls):
-      if not pair[0].startswith('_') and pair[0] != 'text':
-        member_type = pair[1]
-        if (isinstance(member_type, tuple) or isinstance(member_type, list)
-            or isinstance(member_type, (str, unicode))
-            or (inspect.isclass(member_type)
-                and issubclass(member_type, XmlElement))):
-          members.append(pair)
-    return members
-
-  _list_xml_members = classmethod(_list_xml_members)
-
-  def _get_rules(cls, version):
-    """Initializes the _rule_set for the class which is used when parsing XML.
-
-    This method is used internally for parsing and generating XML for an
-    XmlElement. It is not recommended that you call this method directly.
-
-    Returns:
-      A tuple containing the XML parsing rules for the appropriate version.
-
-      The tuple looks like:
-      (qname, {sub_element_qname: (member_name, member_class, repeating), ..},
-       {attribute_qname: member_name})
-
-      To give a couple of concrete example, the atom.data.Control _get_rules
-      with version of 2 will return:
-      ('{http://www.w3.org/2007/app}control',
-       {'{http://www.w3.org/2007/app}draft': ('draft',
-                                              <class 'atom.data.Draft'>,
-                                              False)},
-       {})
-      Calling _get_rules with version 1 on gdata.data.FeedLink will produce:
-      ('{http://schemas.google.com/g/2005}feedLink',
-       {'{http://www.w3.org/2005/Atom}feed': ('feed',
-                                              <class 'gdata.data.GDFeed'>,
-                                              False)},
-       {'href': 'href', 'readOnly': 'read_only', 'countHint': 'count_hint',
-        'rel': 'rel'})
-    """
-    # Initialize the _rule_set to make sure there is a slot available to store
-    # the parsing rules for this version of the XML schema.
-    # Look for rule set in the class __dict__ proxy so that only the
-    # _rule_set for this class will be found. By using the dict proxy
-    # we avoid finding rule_sets defined in superclasses.
-    # The four lines below provide support for any number of versions, but it
-    # runs a bit slower then hard coding slots for two versions, so I'm using
-    # the below two lines.
-    #if '_rule_set' not in cls.__dict__ or cls._rule_set is None:
-    #  cls._rule_set = []
-    #while len(cls.__dict__['_rule_set']) < version:
-    #  cls._rule_set.append(None)
-    # If there is no rule set cache in the class, provide slots for two XML
-    # versions. If and when there is a version 3, this list will need to be
-    # expanded.
-    if '_rule_set' not in cls.__dict__ or cls._rule_set is None:
-      cls._rule_set = [None, None]
-    # If a version higher than 2 is requested, fall back to version 2 because
-    # 2 is currently the highest supported version.
-    if version > 2:
-      return cls._get_rules(2)
-    # Check the dict proxy for the rule set to avoid finding any rule sets
-    # which belong to the superclass. We only want rule sets for this class.
-    if cls._rule_set[version-1] is None:
-      # The rule set for each version consists of the qname for this element
-      # ('{namespace}tag'), a dictionary (elements) for looking up the
-      # corresponding class member when given a child element's qname, and a
-      # dictionary (attributes) for looking up the corresponding class member
-      # when given an XML attribute's qname.
-      elements = {}
-      attributes = {}
-      if ('_members' not in cls.__dict__ or cls._members is None):
-        cls._members = tuple(cls._list_xml_members())
-      for member_name, target in cls._members:
-        if isinstance(target, list):
-          # This member points to a repeating element.
-          elements[_get_qname(target[0], version)] = (member_name, target[0],
-              True)
-        elif isinstance(target, tuple):
-          # This member points to a versioned XML attribute.
-          if version <= len(target):
-            attributes[target[version-1]] = member_name
-          else:
-            attributes[target[-1]] = member_name
-        elif isinstance(target, (str, unicode)):
-          # This member points to an XML attribute.
-          attributes[target] = member_name
-        elif issubclass(target, XmlElement):
-          # This member points to a single occurance element.
-          elements[_get_qname(target, version)] = (member_name, target, False)
-      version_rules = (_get_qname(cls, version), elements, attributes)
-      cls._rule_set[version-1] = version_rules
-      return version_rules
-    else:
-      return cls._rule_set[version-1]
-
-  _get_rules = classmethod(_get_rules)
-
-  def get_elements(self, tag=None, namespace=None, version=1):
-    """Find all sub elements which match the tag and namespace.
-
-    To find all elements in this object, call get_elements with the tag and
-    namespace both set to None (the default). This method searches through
-    the object's members and the elements stored in _other_elements which
-    did not match any of the XML parsing rules for this class.
-
-    Args:
-      tag: str
-      namespace: str
-      version: int Specifies the version of the XML rules to be used when
-               searching for matching elements.
-
-    Returns:
-      A list of the matching XmlElements.
-    """
-    matches = []
-    ignored1, elements, ignored2 = self.__class__._get_rules(version)
-    if elements:
-      for qname, element_def in elements.iteritems():
-        member = getattr(self, element_def[0])
-        if member:
-          if _qname_matches(tag, namespace, qname):
-            if element_def[2]:
-              # If this is a repeating element, copy all instances into the
-              # result list.
-              matches.extend(member)
-            else:
-              matches.append(member)
-    for element in self._other_elements:
-      if _qname_matches(tag, namespace, element._qname):
-        matches.append(element)
-    return matches
-
-  GetElements = get_elements
-  # FindExtensions and FindChildren are provided for backwards compatibility
-  # to the atom.AtomBase class.
-  # However, FindExtensions may return more results than the v1 atom.AtomBase
-  # method does, because get_elements searches both the expected children
-  # and the unexpected "other elements". The old AtomBase.FindExtensions
-  # method searched only "other elements" AKA extension_elements.
-  FindExtensions = get_elements
-  FindChildren = get_elements
-
-  def get_attributes(self, tag=None, namespace=None, version=1):
-    """Find all attributes which match the tag and namespace.
-
-    To find all attributes in this object, call get_attributes with the tag
-    and namespace both set to None (the default). This method searches
-    through the object's members and the attributes stored in
-    _other_attributes which did not fit any of the XML parsing rules for this
-    class.
-
-    Args:
-      tag: str
-      namespace: str
-      version: int Specifies the version of the XML rules to be used when
-               searching for matching attributes.
-
-    Returns:
-      A list of XmlAttribute objects for the matching attributes.
-    """
-    matches = []
-    ignored1, ignored2, attributes = self.__class__._get_rules(version)
-    if attributes:
-      for qname, attribute_def in attributes.iteritems():
-        if isinstance(attribute_def, (list, tuple)):
-          attribute_def = attribute_def[0]
-        member = getattr(self, attribute_def)
-        # TODO: ensure this hasn't broken existing behavior.
-        #member = getattr(self, attribute_def[0])
-        if member:
-          if _qname_matches(tag, namespace, qname):
-            matches.append(XmlAttribute(qname, member))
-    for qname, value in self._other_attributes.iteritems():
-      if _qname_matches(tag, namespace, qname):
-        matches.append(XmlAttribute(qname, value))
-    return matches
-
-  GetAttributes = get_attributes
-
-  def _harvest_tree(self, tree, version=1):
-    """Populates object members from the data in the tree Element."""
-    qname, elements, attributes = self.__class__._get_rules(version)
-    for element in tree:
-      if elements and element.tag in elements:
-        definition = elements[element.tag]
-        # If this is a repeating element, make sure the member is set to a
-        # list.
-        if definition[2]:
-          if getattr(self, definition[0]) is None:
-            setattr(self, definition[0], [])
-          getattr(self, definition[0]).append(_xml_element_from_tree(element,
-              definition[1], version))
-        else:
-          setattr(self, definition[0], _xml_element_from_tree(element,
-              definition[1], version))
-      else:
-        self._other_elements.append(_xml_element_from_tree(element, XmlElement,
-                                                           version))
-    for attrib, value in tree.attrib.iteritems():
-      if attributes and attrib in attributes:
-        setattr(self, attributes[attrib], value)
-      else:
-        self._other_attributes[attrib] = value
-    if tree.text:
-      self.text = tree.text
-
-  def _to_tree(self, version=1, encoding=None):
-    new_tree = ElementTree.Element(_get_qname(self, version))
-    self._attach_members(new_tree, version, encoding)
-    return new_tree
-
-  def _attach_members(self, tree, version=1, encoding=None):
-    """Convert members to XML elements/attributes and add them to the tree.
-
-    Args:
-      tree: An ElementTree.Element which will be modified. The members of
-            this object will be added as child elements or attributes
-            according to the rules described in _expected_elements and
-            _expected_attributes. The elements and attributes stored in
-            other_attributes and other_elements are also added a children
-            of this tree.
-      version: int Ingnored in this method but used by VersionedElement.
-      encoding: str (optional)
-    """
-    qname, elements, attributes = self.__class__._get_rules(version)
-    encoding = encoding or STRING_ENCODING
-    # Add the expected elements and attributes to the tree.
-    if elements:
-      for tag, element_def in elements.iteritems():
-        member = getattr(self, element_def[0])
-        # If this is a repeating element and there are members in the list.
-        if member and element_def[2]:
-          for instance in member:
-            instance._become_child(tree, version)
-        elif member:
-          member._become_child(tree, version)
-    if attributes:
-      for attribute_tag, member_name in attributes.iteritems():
-        value = getattr(self, member_name)
-        if value:
-          tree.attrib[attribute_tag] = value
-    # Add the unexpected (other) elements and attributes to the tree.
-    for element in self._other_elements:
-      element._become_child(tree, version)
-    for key, value in self._other_attributes.iteritems():
-      # I'm not sure if unicode can be used in the attribute name, so for now
-      # we assume the encoding is correct for the attribute name.
-      if not isinstance(value, unicode):
-        value = value.decode(encoding)
-      tree.attrib[key] = value
-    if self.text:
-      if isinstance(self.text, unicode):
-        tree.text = self.text
-      else:
-        tree.text = self.text.decode(encoding)
-
-  def to_string(self, version=1, encoding=None, pretty_print=None):
-    """Converts this object to XML."""
-
-    tree_string = ElementTree.tostring(self._to_tree(version, encoding))
-
-    if pretty_print and xmlString is not None:
-        return xmlString(tree_string).toprettyxml()
- 
-    return tree_string
- 
-  ToString = to_string
-
-  def __str__(self):
-    return self.to_string()
-
-  def _become_child(self, tree, version=1):
-    """Adds a child element to tree with the XML data in self."""
-    new_child = ElementTree.Element('')
-    tree.append(new_child)
-    new_child.tag = _get_qname(self, version)
-    self._attach_members(new_child, version)
-
-  def __get_extension_elements(self):
-    return self._other_elements
-
-  def __set_extension_elements(self, elements):
-    self._other_elements = elements
-
-  extension_elements = property(__get_extension_elements,
-      __set_extension_elements,
-      """Provides backwards compatibility for v1 atom.AtomBase classes.""")
-
-  def __get_extension_attributes(self):
-    return self._other_attributes
-
-  def __set_extension_attributes(self, attributes):
-    self._other_attributes = attributes
-
-  extension_attributes = property(__get_extension_attributes,
-      __set_extension_attributes,
-      """Provides backwards compatibility for v1 atom.AtomBase classes.""")
-
-  def _get_tag(self, version=1):
-    qname = _get_qname(self, version)
-    if qname:
-      return qname[qname.find('}')+1:]
-    return None
-
-  def _get_namespace(self, version=1):
-    qname = _get_qname(self, version)
-    if qname.startswith('{'):
-      return qname[1:qname.find('}')]
-    else:
-      return None
-
-  def _set_tag(self, tag):
-    if isinstance(self._qname, tuple):
-      self._qname = self._qname.copy()
-      if self._qname[0].startswith('{'):
-        self._qname[0] = '{%s}%s' % (self._get_namespace(1), tag)
-      else:
-        self._qname[0] = tag
-    else:
-      if self._qname is not None and self._qname.startswith('{'):
-        self._qname = '{%s}%s' % (self._get_namespace(), tag)
-      else:
-        self._qname = tag
-
-  def _set_namespace(self, namespace):
-    tag = self._get_tag(1)
-    if tag is None:
-      tag = ''
-    if isinstance(self._qname, tuple):
-      self._qname = self._qname.copy()
-      if namespace:
-         self._qname[0] = '{%s}%s' % (namespace, tag)
-      else:
-         self._qname[0] = tag
-    else:
-      if namespace:
-         self._qname = '{%s}%s' % (namespace, tag)
-      else:
-         self._qname = tag
-
-  tag = property(_get_tag, _set_tag,
-      """Provides backwards compatibility for v1 atom.AtomBase classes.""")
-
-  namespace = property(_get_namespace, _set_namespace,
-      """Provides backwards compatibility for v1 atom.AtomBase classes.""")
-
-  # Provided for backwards compatibility to atom.ExtensionElement
-  children = extension_elements
-  attributes = extension_attributes
-
-
-def _get_qname(element, version):
-  if isinstance(element._qname, tuple):
-    if version <= len(element._qname):
-      return element._qname[version-1]
-    else:
-      return element._qname[-1]
-  else:
-    return element._qname
-
-
-def _qname_matches(tag, namespace, qname):
-  """Logic determines if a QName matches the desired local tag and namespace.
-
-  This is used in XmlElement.get_elements and XmlElement.get_attributes to
-  find matches in the element's members (among all expected-and-unexpected
-  elements-and-attributes).
-
-  Args:
-    expected_tag: string
-    expected_namespace: string
-    qname: string in the form '{xml_namespace}localtag' or 'tag' if there is
-           no namespace.
-
-  Returns:
-    boolean True if the member's tag and namespace fit the expected tag and
-    namespace.
-  """
-  # If there is no expected namespace or tag, then everything will match.
-  if qname is None:
-    member_tag = None
-    member_namespace = None
-  else:
-    if qname.startswith('{'):
-      member_namespace = qname[1:qname.index('}')]
-      member_tag = qname[qname.index('}') + 1:]
-    else:
-      member_namespace = None
-      member_tag = qname
-  return ((tag is None and namespace is None)
-      # If there is a tag, but no namespace, see if the local tag matches.
-      or (namespace is None and member_tag == tag)
-      # There was no tag, but there was a namespace so see if the namespaces
-      # match.
-      or (tag is None and member_namespace == namespace)
-      # There was no tag, and the desired elements have no namespace, so check
-      # to see that the member's namespace is None.
-      or (tag is None and namespace == ''
-          and member_namespace is None)
-      # The tag and the namespace both match.
-      or (tag == member_tag
-          and namespace == member_namespace)
-      # The tag matches, and the expected namespace is the empty namespace,
-      # check to make sure the member's namespace is None.
-      or (tag == member_tag and namespace == ''
-          and member_namespace is None))
-
-
-def parse(xml_string, target_class=None, version=1, encoding=None):
-  """Parses the XML string according to the rules for the target_class.
-
-  Args:
-    xml_string: str or unicode
-    target_class: XmlElement or a subclass. If None is specified, the
-        XmlElement class is used.
-    version: int (optional) The version of the schema which should be used when
-        converting the XML into an object. The default is 1.
-    encoding: str (optional) The character encoding of the bytes in the
-        xml_string. Default is 'UTF-8'.
-  """
-  if target_class is None:
-    target_class = XmlElement
-  if isinstance(xml_string, unicode):
-    if encoding is None:
-      xml_string = xml_string.encode(STRING_ENCODING)
-    else:
-      xml_string = xml_string.encode(encoding)
-  tree = ElementTree.fromstring(xml_string)
-  return _xml_element_from_tree(tree, target_class, version)
-
-
-Parse = parse
-xml_element_from_string = parse
-XmlElementFromString = xml_element_from_string
-
-
-def _xml_element_from_tree(tree, target_class, version=1):
-  if target_class._qname is None:
-    instance = target_class()
-    instance._qname = tree.tag
-    instance._harvest_tree(tree, version)
-    return instance
-  # TODO handle the namespace-only case
-  # Namespace only will be used with Google Spreadsheets rows and
-  # Google Base item attributes.
-  elif tree.tag == _get_qname(target_class, version):
-    instance = target_class()
-    instance._harvest_tree(tree, version)
-    return instance
-  return None
-
-
-class XmlAttribute(object):
-
-  def __init__(self, qname, value):
-    self._qname = qname
-    self.value = value
-

atom/data.py

@@ -1,338 +0,0 @@
-#!/usr/bin/env python
-#
-# Copyright (C) 2009 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-# This module is used for version 2 of the Google Data APIs.
-
-
-__author__ = 'j.s@google.com (Jeff Scudder)'
-
-
-import atom.core
-
-
-XML_TEMPLATE = '{http://www.w3.org/XML/1998/namespace}%s'
-ATOM_TEMPLATE = '{http://www.w3.org/2005/Atom}%s'
-APP_TEMPLATE_V1 = '{http://purl.org/atom/app#}%s'
-APP_TEMPLATE_V2 = '{http://www.w3.org/2007/app}%s'
-
-
-class Name(atom.core.XmlElement):
-  """The atom:name element."""
-  _qname = ATOM_TEMPLATE % 'name'
-
-
-class Email(atom.core.XmlElement):
-  """The atom:email element."""
-  _qname = ATOM_TEMPLATE % 'email'
-
-
-class Uri(atom.core.XmlElement):
-  """The atom:uri element."""
-  _qname = ATOM_TEMPLATE % 'uri'
-
-
-class Person(atom.core.XmlElement):
-  """A foundation class which atom:author and atom:contributor extend.
-
-  A person contains information like name, email address, and web page URI for
-  an author or contributor to an Atom feed.
-  """
-  name = Name
-  email = Email
-  uri = Uri
-
-
-class Author(Person):
-  """The atom:author element.
-
-  An author is a required element in Feed unless each Entry contains an Author.
-  """
-  _qname = ATOM_TEMPLATE % 'author'
-
-
-class Contributor(Person):
-  """The atom:contributor element."""
-  _qname = ATOM_TEMPLATE % 'contributor'
-
-
-class Link(atom.core.XmlElement):
-  """The atom:link element."""
-  _qname = ATOM_TEMPLATE % 'link'
-  href = 'href'
-  rel = 'rel'
-  type = 'type'
-  hreflang = 'hreflang'
-  title = 'title'
-  length = 'length'
-
-
-class Generator(atom.core.XmlElement):
-  """The atom:generator element."""
-  _qname = ATOM_TEMPLATE % 'generator'
-  uri = 'uri'
-  version = 'version'
-
-
-class Text(atom.core.XmlElement):
-  """A foundation class from which atom:title, summary, etc. extend.
-
-  This class should never be instantiated.
-  """
-  type = 'type'
-
-
-class Title(Text):
-  """The atom:title element."""
-  _qname = ATOM_TEMPLATE % 'title'
-
-
-class Subtitle(Text):
-  """The atom:subtitle element."""
-  _qname = ATOM_TEMPLATE % 'subtitle'
-
-
-class Rights(Text):
-  """The atom:rights element."""
-  _qname = ATOM_TEMPLATE % 'rights'
-
-
-class Summary(Text):
-  """The atom:summary element."""
-  _qname = ATOM_TEMPLATE % 'summary'
-
-
-class Content(Text):
-  """The atom:content element."""
-  _qname = ATOM_TEMPLATE % 'content'
-  src = 'src'
-
-
-class Category(atom.core.XmlElement):
-  """The atom:category element."""
-  _qname = ATOM_TEMPLATE % 'category'
-  term = 'term'
-  scheme = 'scheme'
-  label = 'label'
-
-
-class Id(atom.core.XmlElement):
-  """The atom:id element."""
-  _qname = ATOM_TEMPLATE % 'id'
-
-
-class Icon(atom.core.XmlElement):
-  """The atom:icon element."""
-  _qname = ATOM_TEMPLATE % 'icon'
-
-
-class Logo(atom.core.XmlElement):
-  """The atom:logo element."""
-  _qname = ATOM_TEMPLATE % 'logo'
-
-
-class Draft(atom.core.XmlElement):
-  """The app:draft element which indicates if this entry should be public."""
-  _qname = (APP_TEMPLATE_V1 % 'draft', APP_TEMPLATE_V2 % 'draft')
-
-
-class Control(atom.core.XmlElement):
-  """The app:control element indicating restrictions on publication.
-
-  The APP control element may contain a draft element indicating whether or
-  not this entry should be publicly available.
-  """
-  _qname = (APP_TEMPLATE_V1 % 'control', APP_TEMPLATE_V2 % 'control')
-  draft = Draft
-
-
-class Date(atom.core.XmlElement):
-  """A parent class for atom:updated, published, etc."""
-
-
-class Updated(Date):
-  """The atom:updated element."""
-  _qname = ATOM_TEMPLATE % 'updated'
-
-
-class Published(Date):
-  """The atom:published element."""
-  _qname = ATOM_TEMPLATE % 'published'
-
-
-class LinkFinder(object):
-  """An "interface" providing methods to find link elements
-
-  Entry elements often contain multiple links which differ in the rel
-  attribute or content type. Often, developers are interested in a specific
-  type of link so this class provides methods to find specific classes of
-  links.
-
-  This class is used as a mixin in Atom entries and feeds.
-  """
-
-  def find_url(self, rel):
-    """Returns the URL (as a string) in a link with the desired rel value."""
-    for link in self.link:
-      if link.rel == rel and link.href:
-        return link.href
-    return None
-
-  FindUrl = find_url
-
-  def get_link(self, rel):
-    """Returns a link object which has the desired rel value.
-
-    If you are interested in the URL instead of the link object,
-    consider using find_url instead.
-    """
-    for link in self.link:
-      if link.rel == rel and link.href:
-        return link
-    return None
-
-  GetLink = get_link
-
-  def find_self_link(self):
-    """Find the first link with rel set to 'self'
-
-    Returns:
-      A str containing the link's href or None if none of the links had rel
-      equal to 'self'
-    """
-    return self.find_url('self')
-
-  FindSelfLink = find_self_link
-
-  def get_self_link(self):
-    return self.get_link('self')
-
-  GetSelfLink = get_self_link
-
-  def find_edit_link(self):
-    return self.find_url('edit')
-
-  FindEditLink = find_edit_link
-
-  def get_edit_link(self):
-    return self.get_link('edit')
-
-  GetEditLink = get_edit_link
-
-  def find_edit_media_link(self):
-    link = self.find_url('edit-media')
-    # Search for media-edit as well since Picasa API used media-edit instead.
-    if link is None:
-      return self.find_url('media-edit')
-    return link
-
-  FindEditMediaLink = find_edit_media_link
-
-  def get_edit_media_link(self):
-    link = self.get_link('edit-media')
-    if link is None:
-      return self.get_link('media-edit')
-    return link
-
-  GetEditMediaLink = get_edit_media_link
-
-  def find_next_link(self):
-    return self.find_url('next')
-
-  FindNextLink = find_next_link
-
-  def get_next_link(self):
-    return self.get_link('next')
-
-  GetNextLink = get_next_link
-
-  def find_license_link(self):
-    return self.find_url('license')
-
-  FindLicenseLink = find_license_link
-
-  def get_license_link(self):
-    return self.get_link('license')
-
-  GetLicenseLink = get_license_link
-
-  def find_alternate_link(self):
-    return self.find_url('alternate')
-
-  FindAlternateLink = find_alternate_link
-
-  def get_alternate_link(self):
-    return self.get_link('alternate')
-
-  GetAlternateLink = get_alternate_link
-
-
-class FeedEntryParent(atom.core.XmlElement, LinkFinder):
-  """A super class for atom:feed and entry, contains shared attributes"""
-  author = [Author]
-  category = [Category]
-  contributor = [Contributor]
-  id = Id
-  link = [Link]
-  rights = Rights
-  title = Title
-  updated = Updated
-
-  def __init__(self, atom_id=None, text=None, *args, **kwargs):
-    if atom_id is not None:
-      self.id = atom_id
-    atom.core.XmlElement.__init__(self, text=text, *args, **kwargs)
-
-
-class Source(FeedEntryParent):
-  """The atom:source element."""
-  _qname = ATOM_TEMPLATE % 'source'
-  generator = Generator
-  icon = Icon
-  logo = Logo
-  subtitle = Subtitle
-
-
-class Entry(FeedEntryParent):
-  """The atom:entry element."""
-  _qname = ATOM_TEMPLATE % 'entry'
-  content = Content
-  published = Published
-  source = Source
-  summary = Summary
-  control = Control
-
-
-class Feed(Source):
-  """The atom:feed element which contains entries."""
-  _qname = ATOM_TEMPLATE % 'feed'
-  entry = [Entry]
-
-
-class ExtensionElement(atom.core.XmlElement):
-  """Provided for backwards compatibility to the v1 atom.ExtensionElement."""
-
-  def __init__(self, tag=None, namespace=None, attributes=None,
-      children=None, text=None, *args, **kwargs):
-    if namespace:
-      self._qname = '{%s}%s' % (namespace, tag)
-    else:
-      self._qname = tag
-    self.children = children or []
-    self.attributes = attributes or {}
-    self.text = text
-
-  _BecomeChildElement = atom.core.XmlElement._become_child

atom/http.py

@@ -1,364 +0,0 @@
-#!/usr/bin/python
-#
-# Copyright (C) 2008 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-"""HttpClients in this module use httplib to make HTTP requests.
-
-This module make HTTP requests based on httplib, but there are environments
-in which an httplib based approach will not work (if running in Google App
-Engine for example). In those cases, higher level classes (like AtomService
-and GDataService) can swap out the HttpClient to transparently use a 
-different mechanism for making HTTP requests.
-
-  HttpClient: Contains a request method which performs an HTTP call to the 
-      server.
-      
-  ProxiedHttpClient: Contains a request method which connects to a proxy using
-      settings stored in operating system environment variables then 
-      performs an HTTP call to the endpoint server.
-"""
-
-
-__author__ = 'api.jscudder (Jeff Scudder)'
-
-
-import types
-import os
-import httplib
-import atom.url
-import atom.http_interface
-import socket
-import base64
-import atom.http_core
-ssl_imported = False
-ssl = None
-try:
-  import ssl
-  ssl_imported = True
-except ImportError:
-  pass
-  
-
-
-class ProxyError(atom.http_interface.Error):
-  pass
-
-
-class TestConfigurationError(Exception):
-  pass
-
-
-DEFAULT_CONTENT_TYPE = 'application/atom+xml; charset=UTF-8'
-
-
-class HttpClient(atom.http_interface.GenericHttpClient):
-  # Added to allow old v1 HttpClient objects to use the new 
-  # http_code.HttpClient. Used in unit tests to inject a mock client.
-  v2_http_client = None
-
-  def __init__(self, headers=None):
-    self.debug = False
-    self.headers = headers or {}
-
-  def request(self, operation, url, data=None, headers=None):
-    """Performs an HTTP call to the server, supports GET, POST, PUT, and 
-    DELETE.
-
-    Usage example, perform and HTTP GET on http://www.google.com/:
-      import atom.http
-      client = atom.http.HttpClient()
-      http_response = client.request('GET', 'http://www.google.com/')
-
-    Args:
-      operation: str The HTTP operation to be performed. This is usually one
-          of 'GET', 'POST', 'PUT', or 'DELETE'
-      data: filestream, list of parts, or other object which can be converted
-          to a string. Should be set to None when performing a GET or DELETE.
-          If data is a file-like object which can be read, this method will 
-          read a chunk of 100K bytes at a time and send them. 
-          If the data is a list of parts to be sent, each part will be 
-          evaluated and sent.
-      url: The full URL to which the request should be sent. Can be a string
-          or atom.url.Url.
-      headers: dict of strings. HTTP headers which should be sent
-          in the request. 
-    """
-    all_headers = self.headers.copy()
-    if headers:
-      all_headers.update(headers)
-
-    # If the list of headers does not include a Content-Length, attempt to
-    # calculate it based on the data object.
-    if data and 'Content-Length' not in all_headers:
-      if isinstance(data, types.StringTypes):
-        all_headers['Content-Length'] = str(len(data))
-      else:
-        raise atom.http_interface.ContentLengthRequired('Unable to calculate '
-            'the length of the data parameter. Specify a value for '
-            'Content-Length')
-
-    # Set the content type to the default value if none was set.
-    if 'Content-Type' not in all_headers:
-      all_headers['Content-Type'] = DEFAULT_CONTENT_TYPE
-
-    if self.v2_http_client is not None:
-      http_request = atom.http_core.HttpRequest(method=operation)
-      atom.http_core.Uri.parse_uri(str(url)).modify_request(http_request)
-      http_request.headers = all_headers
-      if data:
-        http_request._body_parts.append(data)
-      return self.v2_http_client.request(http_request=http_request)
-
-    if not isinstance(url, atom.url.Url):
-      if isinstance(url, types.StringTypes):
-        url = atom.url.parse_url(url)
-      else:
-        raise atom.http_interface.UnparsableUrlObject('Unable to parse url '
-            'parameter because it was not a string or atom.url.Url')
-    
-    connection = self._prepare_connection(url, all_headers)
-
-    if self.debug:
-      connection.debuglevel = 1
-
-    connection.putrequest(operation, self._get_access_url(url), 
-        skip_host=True)
-    if url.port is not None:
-      connection.putheader('Host', '%s:%s' % (url.host, url.port))
-    else:
-      connection.putheader('Host', url.host)
-
-    # Overcome a bug in Python 2.4 and 2.5
-    # httplib.HTTPConnection.putrequest adding
-    # HTTP request header 'Host: www.google.com:443' instead of
-    # 'Host: www.google.com', and thus resulting the error message
-    # 'Token invalid - AuthSub token has wrong scope' in the HTTP response.
-    if (url.protocol == 'https' and int(url.port or 443) == 443 and
-        hasattr(connection, '_buffer') and
-        isinstance(connection._buffer, list)):
-      header_line = 'Host: %s:443' % url.host
-      replacement_header_line = 'Host: %s' % url.host
-      try:
-        connection._buffer[connection._buffer.index(header_line)] = (
-            replacement_header_line)
-      except ValueError:  # header_line missing from connection._buffer
-        pass
-
-    # Send the HTTP headers.
-    for header_name in all_headers:
-      connection.putheader(header_name, all_headers[header_name])
-    connection.endheaders()
-
-    # If there is data, send it in the request.
-    if data:
-      if isinstance(data, list):
-        for data_part in data:
-          _send_data_part(data_part, connection)
-      else:
-        _send_data_part(data, connection)
-
-    # Return the HTTP Response from the server.
-    return connection.getresponse()
-    
-  def _prepare_connection(self, url, headers):
-    if not isinstance(url, atom.url.Url):
-      if isinstance(url, types.StringTypes):
-        url = atom.url.parse_url(url)
-      else:
-        raise atom.http_interface.UnparsableUrlObject('Unable to parse url '
-            'parameter because it was not a string or atom.url.Url')
-    if url.protocol == 'https':
-      if not url.port:
-        return httplib.HTTPSConnection(url.host)
-      return httplib.HTTPSConnection(url.host, int(url.port))
-    else:
-      if not url.port:
-        return httplib.HTTPConnection(url.host)
-      return httplib.HTTPConnection(url.host, int(url.port))
-
-  def _get_access_url(self, url):
-    return url.to_string()
-
-
-class ProxiedHttpClient(HttpClient):
-  """Performs an HTTP request through a proxy.
-  
-  The proxy settings are obtained from enviroment variables. The URL of the 
-  proxy server is assumed to be stored in the environment variables 
-  'https_proxy' and 'http_proxy' respectively. If the proxy server requires
-  a Basic Auth authorization header, the username and password are expected to 
-  be in the 'proxy-username' or 'proxy_username' variable and the 
-  'proxy-password' or 'proxy_password' variable, or in 'http_proxy' or
-  'https_proxy' as "protocol://[username:password@]host:port".
-  
-  After connecting to the proxy server, the request is completed as in 
-  HttpClient.request.
-  """
-  def _prepare_connection(self, url, headers):
-    proxy_settings = os.environ.get('%s_proxy' % url.protocol)
-    if not proxy_settings:
-      # The request was HTTP or HTTPS, but there was no appropriate proxy set.
-      return HttpClient._prepare_connection(self, url, headers)
-    else:
-      #print '!!!!%s' % proxy_settings
-      proxy_auth = _get_proxy_auth(proxy_settings)
-      proxy_netloc = _get_proxy_net_location(proxy_settings)
-      #print '!!!!%s' % proxy_auth
-      #print '!!!!%s' % proxy_netloc
-      if url.protocol == 'https':
-        # Set any proxy auth headers 
-        if proxy_auth:
-          proxy_auth = 'Proxy-authorization: %s' % proxy_auth
-          
-        # Construct the proxy connect command.
-        port = url.port
-        if not port:
-          port = '443'
-        proxy_connect = 'CONNECT %s:%s HTTP/1.0\r\n' % (url.host, port)
-        
-        # Set the user agent to send to the proxy
-        if headers and 'User-Agent' in headers:
-          user_agent = 'User-Agent: %s\r\n' % (headers['User-Agent'])
-        else:
-          user_agent = 'User-Agent: python\r\n'
-        
-        proxy_pieces = '%s%s%s\r\n' % (proxy_connect, proxy_auth, user_agent)
-        
-        # Find the proxy host and port.
-        proxy_url = atom.url.parse_url(proxy_netloc)
-        if not proxy_url.port:
-          proxy_url.port = '80'
-        
-        # Connect to the proxy server, very simple recv and error checking
-        p_sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
-        p_sock.connect((proxy_url.host, int(proxy_url.port)))
-        p_sock.sendall(proxy_pieces)
-        response = ''
-
-        # Wait for the full response.
-        while response.find("\r\n\r\n") == -1:
-          response += p_sock.recv(8192)
-       
-        p_status = response.split()[1]
-        if p_status != str(200):
-          raise ProxyError('Error status=%s' % str(p_status))
-
-        # Trivial setup for ssl socket.
-        sslobj = None
-        if ssl_imported:
-          sslobj = ssl.wrap_socket(p_sock, None, None)
-        else:
-          sock_ssl = socket.ssl(p_sock, None, None)
-          sslobj = httplib.FakeSocket(p_sock, sock_ssl)
- 
-        # Initalize httplib and replace with the proxy socket.
-        connection = httplib.HTTPConnection(proxy_url.host)
-        connection.sock = sslobj
-        return connection
-      else:
-        # If protocol was not https.
-        # Find the proxy host and port.
-        proxy_url = atom.url.parse_url(proxy_netloc)
-        if not proxy_url.port:
-          proxy_url.port = '80'
-        
-        if proxy_auth:
-          headers['Proxy-Authorization'] = proxy_auth.strip()
-
-        return httplib.HTTPConnection(proxy_url.host, int(proxy_url.port))
-
-  def _get_access_url(self, url):
-    return url.to_string()
-
-
-def _get_proxy_auth(proxy_settings):
-  """Returns proxy authentication string for header.
-
-  Will check environment variables for proxy authentication info, starting with
-  proxy(_/-)username and proxy(_/-)password before checking the given
-  proxy_settings for a [protocol://]username:password@host[:port] string.
-
-  Args:
-    proxy_settings: String from http_proxy or https_proxy environment variable.
-
-  Returns:
-    Authentication string for proxy, or empty string if no proxy username was
-    found.
-  """
-  proxy_username = None
-  proxy_password = None
-
-  proxy_username = os.environ.get('proxy-username')
-  if not proxy_username:
-    proxy_username = os.environ.get('proxy_username')
-  proxy_password = os.environ.get('proxy-password')
-  if not proxy_password:
-    proxy_password = os.environ.get('proxy_password')
-
-  if not proxy_username:
-    if '@' in proxy_settings:
-      protocol_and_proxy_auth = proxy_settings.split('@')[0].split(':')
-      if len(protocol_and_proxy_auth) == 3:
-        # 3 elements means we have [<protocol>, //<user>, <password>]
-        proxy_username = protocol_and_proxy_auth[1].lstrip('/')
-        proxy_password = protocol_and_proxy_auth[2]
-      elif len(protocol_and_proxy_auth) == 2:
-        # 2 elements means we have [<user>, <password>]
-        proxy_username = protocol_and_proxy_auth[0]
-        proxy_password = protocol_and_proxy_auth[1]
-  if proxy_username:
-    user_auth = base64.encodestring('%s:%s' % (proxy_username,
-                                               proxy_password))
-    return 'Basic %s\r\n' % (user_auth.strip())
-  else:
-    return ''
-
-
-def _get_proxy_net_location(proxy_settings):
-  """Returns proxy host and port.
-
-  Args:
-    proxy_settings: String from http_proxy or https_proxy environment variable.
-        Must be in the form of protocol://[username:password@]host:port
-
-  Returns:
-    String in the form of protocol://host:port
-  """
-  if '@' in proxy_settings:
-    protocol = proxy_settings.split(':')[0]
-    netloc = proxy_settings.split('@')[1]
-    return '%s://%s' % (protocol, netloc)
-  else:
-    return proxy_settings
-
-
-def _send_data_part(data, connection):
-  if isinstance(data, types.StringTypes):
-    connection.send(data)
-    return
-  # Check to see if data is a file-like object that has a read method.
-  elif hasattr(data, 'read'):
-    # Read the file and send it a chunk at a time.
-    while 1:
-      binarydata = data.read(100000)
-      if binarydata == '': break
-      connection.send(binarydata)
-    return
-  else:
-    # The data object was not a file.
-    # Try to convert to a string and send the data.
-    connection.send(str(data))
-    return

atom/http_core.py

@@ -1,597 +0,0 @@
-#!/usr/bin/env python
-#
-#    Copyright (C) 2009 Google Inc.
-#
-#   Licensed under the Apache License, Version 2.0 (the "License");
-#   you may not use this file except in compliance with the License.
-#   You may obtain a copy of the License at
-#
-#       http://www.apache.org/licenses/LICENSE-2.0
-#
-#   Unless required by applicable law or agreed to in writing, software
-#   distributed under the License is distributed on an "AS IS" BASIS,
-#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-#   See the License for the specific language governing permissions and
-#   limitations under the License.
-
-
-# This module is used for version 2 of the Google Data APIs.
-# TODO: add proxy handling.
-
-
-__author__ = 'j.s@google.com (Jeff Scudder)'
-
-
-import os
-import StringIO
-import urlparse
-import urllib
-import httplib
-ssl = None
-try:
-  import ssl
-except ImportError:
-  pass
-
-
-
-class Error(Exception):
-  pass
-
-
-class UnknownSize(Error):
-  pass
-
-
-class ProxyError(Error):
-  pass
-
-
-MIME_BOUNDARY = 'END_OF_PART'
-
-
-def get_headers(http_response):
-  """Retrieves all HTTP headers from an HTTP response from the server.
-  
-  This method is provided for backwards compatibility for Python2.2 and 2.3.
-  The httplib.HTTPResponse object in 2.2 and 2.3 does not have a getheaders
-  method so this function will use getheaders if available, but if not it
-  will retrieve a few using getheader.
-  """
-  if hasattr(http_response, 'getheaders'):
-    return http_response.getheaders()
-  else:
-    headers = []
-    for header in (
-        'location', 'content-type', 'content-length', 'age', 'allow',
-        'cache-control', 'content-location', 'content-encoding', 'date',
-        'etag', 'expires', 'last-modified', 'pragma', 'server',
-        'set-cookie', 'transfer-encoding', 'vary', 'via', 'warning',
-        'www-authenticate', 'gdata-version'):
-      value = http_response.getheader(header, None)
-      if value is not None:
-        headers.append((header, value))
-    return headers
-
-
-class HttpRequest(object):
-  """Contains all of the parameters for an HTTP 1.1 request.
-
-  The HTTP headers are represented by a dictionary, and it is the
-  responsibility of the user to ensure that duplicate field names are combined
-  into one header value according to the rules in section 4.2 of RFC 2616.
-  """
-  method = None
-  uri = None
-
-  def __init__(self, uri=None, method=None, headers=None):
-    """Construct an HTTP request.
-
-    Args:
-      uri: The full path or partial path as a Uri object or a string.
-      method: The HTTP method for the request, examples include 'GET', 'POST',
-              etc.
-      headers: dict of strings The HTTP headers to include in the request.
-    """
-    self.headers = headers or {}
-    self._body_parts = []
-    if method is not None:
-      self.method = method
-    if isinstance(uri, (str, unicode)):
-      uri = Uri.parse_uri(uri)
-    self.uri = uri or Uri()
-
-
-  def add_body_part(self, data, mime_type, size=None):
-    """Adds data to the HTTP request body.
-
-    If more than one part is added, this is assumed to be a mime-multipart
-    request. This method is designed to create MIME 1.0 requests as specified
-    in RFC 1341.
-
-    Args:
-      data: str or a file-like object containing a part of the request body.
-      mime_type: str The MIME type describing the data
-      size: int Required if the data is a file like object. If the data is a
-            string, the size is calculated so this parameter is ignored.
-    """
-    if isinstance(data, str):
-      size = len(data)
-    if size is None:
-      # TODO: support chunked transfer if some of the body is of unknown size.
-      raise UnknownSize('Each part of the body must have a known size.')
-    if 'Content-Length' in self.headers:
-      content_length = int(self.headers['Content-Length'])
-    else:
-      content_length = 0
-    # If this is the first part added to the body, then this is not a multipart
-    # request.
-    if len(self._body_parts) == 0:
-      self.headers['Content-Type'] = mime_type
-      content_length = size
-      self._body_parts.append(data)
-    elif len(self._body_parts) == 1:
-      # This is the first member in a mime-multipart request, so change the
-      # _body_parts list to indicate a multipart payload.
-      self._body_parts.insert(0, 'Media multipart posting')
-      boundary_string = '\r\n--%s\r\n' % (MIME_BOUNDARY,)
-      content_length += len(boundary_string) + size
-      self._body_parts.insert(1, boundary_string)
-      content_length += len('Media multipart posting')
-      # Put the content type of the first part of the body into the multipart
-      # payload.
-      original_type_string = 'Content-Type: %s\r\n\r\n' % (
-          self.headers['Content-Type'],)
-      self._body_parts.insert(2, original_type_string)
-      content_length += len(original_type_string)
-      boundary_string = '\r\n--%s\r\n' % (MIME_BOUNDARY,)
-      self._body_parts.append(boundary_string)
-      content_length += len(boundary_string)
-      # Change the headers to indicate this is now a mime multipart request.
-      self.headers['Content-Type'] = 'multipart/related; boundary="%s"' % (
-          MIME_BOUNDARY,)
-      self.headers['MIME-version'] = '1.0'
-      # Include the mime type of this part.
-      type_string = 'Content-Type: %s\r\n\r\n' % (mime_type)
-      self._body_parts.append(type_string)
-      content_length += len(type_string)
-      self._body_parts.append(data)
-      ending_boundary_string = '\r\n--%s--' % (MIME_BOUNDARY,)
-      self._body_parts.append(ending_boundary_string)
-      content_length += len(ending_boundary_string)
-    else:
-      # This is a mime multipart request.
-      boundary_string = '\r\n--%s\r\n' % (MIME_BOUNDARY,)
-      self._body_parts.insert(-1, boundary_string)
-      content_length += len(boundary_string) + size
-      # Include the mime type of this part.
-      type_string = 'Content-Type: %s\r\n\r\n' % (mime_type)
-      self._body_parts.insert(-1, type_string)
-      content_length += len(type_string)
-      self._body_parts.insert(-1, data)
-    self.headers['Content-Length'] = str(content_length)
-  # I could add an "append_to_body_part" method as well.
-
-  AddBodyPart = add_body_part
-
-  def add_form_inputs(self, form_data,
-                      mime_type='application/x-www-form-urlencoded'):
-    """Form-encodes and adds data to the request body.
-
-    Args:
-      form_data: dict or sequnce or two member tuples which contains the
-                 form keys and values.
-      mime_type: str The MIME type of the form data being sent. Defaults
-                 to 'application/x-www-form-urlencoded'.
-    """
-    body = urllib.urlencode(form_data)
-    self.add_body_part(body, mime_type)
-
-  AddFormInputs = add_form_inputs
-
-  def _copy(self):
-    """Creates a deep copy of this request."""
-    copied_uri = Uri(self.uri.scheme, self.uri.host, self.uri.port,
-                     self.uri.path, self.uri.query.copy())
-    new_request = HttpRequest(uri=copied_uri, method=self.method,
-                              headers=self.headers.copy())
-    new_request._body_parts = self._body_parts[:]
-    return new_request
-
-  def _dump(self):
-    """Converts to a printable string for debugging purposes.
-    
-    In order to preserve the request, it does not read from file-like objects
-    in the body.
-    """
-    output =  'HTTP Request\n  method: %s\n  url: %s\n  headers:\n' % (
-        self.method, str(self.uri))
-    for header, value in self.headers.iteritems():
-      output += '    %s: %s\n' % (header, value)
-    output += '  body sections:\n'
-    i = 0
-    for part in self._body_parts:
-      if isinstance(part, (str, unicode)):
-        output += '    %s: %s\n' % (i, part)
-      else:
-        output += '    %s: <file like object>\n' % i
-      i += 1
-    return output
-
-
-def _apply_defaults(http_request):
-  if http_request.uri.scheme is None:
-    if http_request.uri.port == 443:
-      http_request.uri.scheme = 'https'
-    else:
-      http_request.uri.scheme = 'http'
-
-
-class Uri(object):
-  """A URI as used in HTTP 1.1"""
-  scheme = None
-  host = None
-  port = None
-  path = None
-
-  def __init__(self, scheme=None, host=None, port=None, path=None, query=None):
-    """Constructor for a URI.
-
-    Args:
-      scheme: str This is usually 'http' or 'https'.
-      host: str The host name or IP address of the desired server.
-      post: int The server's port number.
-      path: str The path of the resource following the host. This begins with
-            a /, example: '/calendar/feeds/default/allcalendars/full'
-      query: dict of strings The URL query parameters. The keys and values are
-             both escaped so this dict should contain the unescaped values.
-             For example {'my key': 'val', 'second': '!!!'} will become
-             '?my+key=val&second=%21%21%21' which is appended to the path.
-    """
-    self.query = query or {}
-    if scheme is not None:
-      self.scheme = scheme
-    if host is not None:
-      self.host = host
-    if port is not None:
-      self.port = port
-    if path:
-      self.path = path
-
-  def _get_query_string(self):
-    param_pairs = []
-    for key, value in self.query.iteritems():
-      param_pairs.append('='.join((urllib.quote_plus(key),
-          urllib.quote_plus(str(value)))))
-    return '&'.join(param_pairs)
-
-  def _get_relative_path(self):
-    """Returns the path with the query parameters escaped and appended."""
-    param_string = self._get_query_string()
-    if self.path is None:
-      path = '/'
-    else:
-      path = self.path
-    if param_string:
-      return '?'.join([path, param_string])
-    else:
-      return path
-
-  def _to_string(self):
-    if self.scheme is None and self.port == 443:
-      scheme = 'https'
-    elif self.scheme is None:
-      scheme = 'http'
-    else:
-      scheme = self.scheme
-    if self.path is None:
-      path = '/'
-    else:
-      path = self.path
-    if self.port is None:
-      return '%s://%s%s' % (scheme, self.host, self._get_relative_path())
-    else:
-      return '%s://%s:%s%s' % (scheme, self.host, str(self.port),
-                               self._get_relative_path())
-
-  def __str__(self):
-    return self._to_string()
-
-  def modify_request(self, http_request=None):
-    """Sets HTTP request components based on the URI."""
-    if http_request is None:
-      http_request = HttpRequest()
-    if http_request.uri is None:
-      http_request.uri = Uri()
-    # Determine the correct scheme.
-    if self.scheme:
-      http_request.uri.scheme = self.scheme
-    if self.port:
-      http_request.uri.port = self.port
-    if self.host:
-      http_request.uri.host = self.host
-    # Set the relative uri path
-    if self.path:
-      http_request.uri.path = self.path
-    if self.query:
-      http_request.uri.query = self.query.copy()
-    return http_request
-
-  ModifyRequest = modify_request
-
-  def parse_uri(uri_string):
-    """Creates a Uri object which corresponds to the URI string.
-
-    This method can accept partial URIs, but it will leave missing
-    members of the Uri unset.
-    """
-    parts = urlparse.urlparse(uri_string)
-    uri = Uri()
-    if parts[0]:
-      uri.scheme = parts[0]
-    if parts[1]:
-      host_parts = parts[1].split(':')
-      if host_parts[0]:
-        uri.host = host_parts[0]
-      if len(host_parts) > 1:
-        uri.port = int(host_parts[1])
-    if parts[2]:
-      uri.path = parts[2]
-    if parts[4]:
-      param_pairs = parts[4].split('&')
-      for pair in param_pairs:
-        pair_parts = pair.split('=')
-        if len(pair_parts) > 1:
-          uri.query[urllib.unquote_plus(pair_parts[0])] = (
-              urllib.unquote_plus(pair_parts[1]))
-        elif len(pair_parts) == 1:
-          uri.query[urllib.unquote_plus(pair_parts[0])] = None
-    return uri
-
-  parse_uri = staticmethod(parse_uri)
-
-  ParseUri = parse_uri
-
-
-parse_uri = Uri.parse_uri
-
-
-ParseUri = Uri.parse_uri
-
-
-class HttpResponse(object):
-  status = None
-  reason = None
-  _body = None
-
-  def __init__(self, status=None, reason=None, headers=None, body=None):
-    self._headers = headers or {}
-    if status is not None:
-      self.status = status
-    if reason is not None:
-      self.reason = reason
-    if body is not None:
-      if hasattr(body, 'read'):
-        self._body = body
-      else:
-        self._body = StringIO.StringIO(body)
-
-  def getheader(self, name, default=None):
-    if name in self._headers:
-      return self._headers[name]
-    else:
-      return default
-
-  def getheaders(self):
-    return self._headers
-
-  def read(self, amt=None):
-    if self._body is None:
-      return None
-    if not amt:
-      return self._body.read()
-    else:
-      return self._body.read(amt)
-
-
-def _dump_response(http_response):
-  """Converts to a string for printing debug messages.
-  
-  Does not read the body since that may consume the content.
-  """
-  output = 'HttpResponse\n  status: %s\n  reason: %s\n  headers:' % (
-      http_response.status, http_response.reason)
-  headers = get_headers(http_response)
-  if isinstance(headers, dict):
-    for header, value in headers.iteritems():
-      output += '    %s: %s\n' % (header, value)
-  else:
-    for pair in headers:
-      output += '    %s: %s\n' % (pair[0], pair[1])
-  return output
-
-
-class HttpClient(object):
-  """Performs HTTP requests using httplib."""
-  debug = None
-
-  def request(self, http_request):
-    return self._http_request(http_request.method, http_request.uri,
-                              http_request.headers, http_request._body_parts)
-
-  Request = request
-
-  def _get_connection(self, uri, headers=None):
-    """Opens a socket connection to the server to set up an HTTP request.
-
-    Args:
-      uri: The full URL for the request as a Uri object.
-      headers: A dict of string pairs containing the HTTP headers for the
-          request.
-    """
-    connection = None
-    if uri.scheme == 'https':
-      if not uri.port:
-        connection = httplib.HTTPSConnection(uri.host)
-      else:
-        connection = httplib.HTTPSConnection(uri.host, int(uri.port))
-    else:
-      if not uri.port:
-        connection = httplib.HTTPConnection(uri.host)
-      else:
-        connection = httplib.HTTPConnection(uri.host, int(uri.port))
-    return connection
-
-  def _http_request(self, method, uri, headers=None, body_parts=None):
-    """Makes an HTTP request using httplib.
-
-    Args:
-      method: str example: 'GET', 'POST', 'PUT', 'DELETE', etc.
-      uri: str or atom.http_core.Uri
-      headers: dict of strings mapping to strings which will be sent as HTTP
-               headers in the request.
-      body_parts: list of strings, objects with a read method, or objects
-                  which can be converted to strings using str. Each of these
-                  will be sent in order as the body of the HTTP request.
-    """
-    if isinstance(uri, (str, unicode)):
-      uri = Uri.parse_uri(uri)
-
-    connection = self._get_connection(uri, headers=headers)
-
-    if self.debug:
-      connection.debuglevel = 1
-
-    if connection.host != uri.host:
-      connection.putrequest(method, str(uri))
-    else:
-      connection.putrequest(method, uri._get_relative_path())
-
-    # Overcome a bug in Python 2.4 and 2.5
-    # httplib.HTTPConnection.putrequest adding
-    # HTTP request header 'Host: www.google.com:443' instead of
-    # 'Host: www.google.com', and thus resulting the error message
-    # 'Token invalid - AuthSub token has wrong scope' in the HTTP response.
-    if (uri.scheme == 'https' and int(uri.port or 443) == 443 and
-        hasattr(connection, '_buffer') and
-        isinstance(connection._buffer, list)):
-      header_line = 'Host: %s:443' % uri.host
-      replacement_header_line = 'Host: %s' % uri.host
-      try:
-        connection._buffer[connection._buffer.index(header_line)] = (
-            replacement_header_line)
-      except ValueError:  # header_line missing from connection._buffer
-        pass
-
-    # Send the HTTP headers.
-    for header_name, value in headers.iteritems():
-      connection.putheader(header_name, value)
-    connection.endheaders()
-
-    # If there is data, send it in the request.
-    if body_parts and filter(lambda x: x != '', body_parts):
-      for part in body_parts:
-        _send_data_part(part, connection)
-
-    # Return the HTTP Response from the server.
-    return connection.getresponse()
-
-
-def _send_data_part(data, connection):
-  if isinstance(data, (str, unicode)):
-    # I might want to just allow str, not unicode.
-    connection.send(data)
-    return
-  # Check to see if data is a file-like object that has a read method.
-  elif hasattr(data, 'read'):
-    # Read the file and send it a chunk at a time.
-    while 1:
-      binarydata = data.read(100000)
-      if binarydata == '': break
-      connection.send(binarydata)
-    return
-  else:
-    # The data object was not a file.
-    # Try to convert to a string and send the data.
-    connection.send(str(data))
-    return
-
-
-class ProxiedHttpClient(HttpClient):
-
-  def _get_connection(self, uri, headers=None):
-    # Check to see if there are proxy settings required for this request.
-    proxy = None
-    if uri.scheme == 'https':
-      proxy = os.environ.get('https_proxy')
-    elif uri.scheme == 'http':
-      proxy = os.environ.get('http_proxy')
-    if not proxy:
-      return HttpClient._get_connection(self, uri, headers=headers)
-    # Now we have the URL of the appropriate proxy server.
-    # Get a username and password for the proxy if required.
-    proxy_auth = _get_proxy_auth()
-    if uri.scheme == 'https':
-      import socket
-      if proxy_auth:
-        proxy_auth = 'Proxy-authorization: %s' % proxy_auth
-      # Construct the proxy connect command.
-      port = uri.port
-      if not port:
-        port = 443
-      proxy_connect = 'CONNECT %s:%s HTTP/1.0\r\n' % (uri.host, port)
-      # Set the user agent to send to the proxy
-      user_agent = ''
-      if headers and 'User-Agent' in headers:
-        user_agent = 'User-Agent: %s\r\n' % (headers['User-Agent'])
-      proxy_pieces = '%s%s%s\r\n' % (proxy_connect, proxy_auth, user_agent)
-      # Find the proxy host and port.
-      proxy_uri = Uri.parse_uri(proxy)
-      if not proxy_uri.port:
-        proxy_uri.port = '80'
-      # Connect to the proxy server, very simple recv and error checking
-      p_sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
-      p_sock.connect((proxy_uri.host, int(proxy_uri.port)))
-      p_sock.sendall(proxy_pieces)
-      response = ''
-      # Wait for the full response.
-      while response.find("\r\n\r\n") == -1:
-        response += p_sock.recv(8192)
-      p_status = response.split()[1]
-      if p_status != str(200):
-        raise ProxyError('Error status=%s' % str(p_status))
-      # Trivial setup for ssl socket.
-      sslobj = None
-      if ssl is not None:
-        sslobj = ssl.wrap_socket(p_sock, None, None)
-      else:
-        sock_ssl = socket.ssl(p_sock, None, Nonesock_)
-        sslobj = httplib.FakeSocket(p_sock, sock_ssl)
-      # Initalize httplib and replace with the proxy socket.
-      connection = httplib.HTTPConnection(proxy_uri.host)
-      connection.sock = sslobj
-      return connection
-    elif uri.scheme == 'http':
-      proxy_uri = Uri.parse_uri(proxy)
-      if not proxy_uri.port:
-        proxy_uri.port = '80'
-      if proxy_auth:
-        headers['Proxy-Authorization'] = proxy_auth.strip()
-      return httplib.HTTPConnection(proxy_uri.host, int(proxy_uri.port))
-    return None
-
-
-def _get_proxy_auth():
-  import base64
-  proxy_username = os.environ.get('proxy-username')
-  if not proxy_username:
-    proxy_username = os.environ.get('proxy_username')
-  proxy_password = os.environ.get('proxy-password')
-  if not proxy_password:
-    proxy_password = os.environ.get('proxy_password')
-  if proxy_username:
-    user_auth = base64.b64encode('%s:%s' % (proxy_username,
-                                            proxy_password))
-    return 'Basic %s\r\n' % (user_auth.strip())
-  else:
-    return ''

atom/http_interface.py

@@ -1,156 +0,0 @@
-#!/usr/bin/python
-#
-# Copyright (C) 2008 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""This module provides a common interface for all HTTP requests.
-
-  HttpResponse: Represents the server's response to an HTTP request. Provides
-      an interface identical to httplib.HTTPResponse which is the response
-      expected from higher level classes which use HttpClient.request.
-
-  GenericHttpClient: Provides an interface (superclass) for an object 
-      responsible for making HTTP requests. Subclasses of this object are
-      used in AtomService and GDataService to make requests to the server. By
-      changing the http_client member object, the AtomService is able to make
-      HTTP requests using different logic (for example, when running on 
-      Google App Engine, the http_client makes requests using the App Engine
-      urlfetch API). 
-"""
-
-
-__author__ = 'api.jscudder (Jeff Scudder)'
-
-
-import StringIO
-
-
-USER_AGENT = '%s GData-Python 2.0.14+20110902+custom_mods'
-
-
-class Error(Exception):
-  pass
-
-
-class UnparsableUrlObject(Error):
-  pass
-
-
-class ContentLengthRequired(Error):
-  pass
-  
-
-class HttpResponse(object):
-  def __init__(self, body=None, status=None, reason=None, headers=None):
-    """Constructor for an HttpResponse object. 
-
-    HttpResponse represents the server's response to an HTTP request from
-    the client. The HttpClient.request method returns a httplib.HTTPResponse
-    object and this HttpResponse class is designed to mirror the interface
-    exposed by httplib.HTTPResponse.
-
-    Args:
-      body: A file like object, with a read() method. The body could also
-          be a string, and the constructor will wrap it so that 
-          HttpResponse.read(self) will return the full string.
-      status: The HTTP status code as an int. Example: 200, 201, 404.
-      reason: The HTTP status message which follows the code. Example: 
-          OK, Created, Not Found
-      headers: A dictionary containing the HTTP headers in the server's 
-          response. A common header in the response is Content-Length.
-    """
-    if body:
-      if hasattr(body, 'read'):
-        self._body = body
-      else:
-        self._body = StringIO.StringIO(body)
-    else:
-      self._body = None
-    if status is not None:
-      self.status = int(status)
-    else:
-      self.status = None
-    self.reason = reason
-    self._headers = headers or {}
-
-  def getheader(self, name, default=None):
-    if name in self._headers:
-      return self._headers[name]
-    else:
-      return default
-    
-  def read(self, amt=None):
-    if not amt:
-      return self._body.read()
-    else:
-      return self._body.read(amt)
-
-
-class GenericHttpClient(object):
-  debug = False
-
-  def __init__(self, http_client, headers=None):
-    """
-    
-    Args:
-      http_client: An object which provides a request method to make an HTTP 
-          request. The request method in GenericHttpClient performs a 
-          call-through to the contained HTTP client object.
-      headers: A dictionary containing HTTP headers which should be included
-          in every HTTP request. Common persistent headers include 
-          'User-Agent'.
-    """
-    self.http_client = http_client
-    self.headers = headers or {}
-
-  def request(self, operation, url, data=None, headers=None):
-    all_headers = self.headers.copy()
-    if headers:
-      all_headers.update(headers)
-    return self.http_client.request(operation, url, data=data, 
-        headers=all_headers)
-
-  def get(self, url, headers=None):
-    return self.request('GET', url, headers=headers)
-
-  def post(self, url, data, headers=None):
-    return self.request('POST', url, data=data, headers=headers)
-
-  def put(self, url, data, headers=None):
-    return self.request('PUT', url, data=data, headers=headers)
-
-  def delete(self, url, headers=None):
-    return self.request('DELETE', url, headers=headers)
-
-
-class GenericToken(object):
-  """Represents an Authorization token to be added to HTTP requests.
-  
-  Some Authorization headers included calculated fields (digital
-  signatures for example) which are based on the parameters of the HTTP
-  request. Therefore the token is responsible for signing the request
-  and adding the Authorization header. 
-  """
-  def perform_request(self, http_client, operation, url, data=None, 
-                      headers=None):
-    """For the GenericToken, no Authorization token is set."""
-    return http_client.request(operation, url, data=data, headers=headers)
-
-  def valid_for_scope(self, url):
-    """Tells the caller if the token authorizes access to the desired URL.
-    
-    Since the generic token doesn't add an auth header, it is not valid for
-    any scope.
-    """
-    return False

atom/mock_http.py

@@ -1,132 +0,0 @@
-#!/usr/bin/python
-#
-# Copyright (C) 2008 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-__author__ = 'api.jscudder (Jeff Scudder)'
-
-
-import atom.http_interface
-import atom.url
-
-
-class Error(Exception):
-  pass
-
-
-class NoRecordingFound(Error):
-  pass
-
-
-class MockRequest(object):
-  """Holds parameters of an HTTP request for matching against future requests.
-  """
-  def __init__(self, operation, url, data=None, headers=None):
-    self.operation = operation
-    if isinstance(url, (str, unicode)):
-      url = atom.url.parse_url(url)
-    self.url = url
-    self.data = data
-    self.headers = headers
-
-
-class MockResponse(atom.http_interface.HttpResponse):
-  """Simulates an httplib.HTTPResponse object."""
-  def __init__(self, body=None, status=None, reason=None, headers=None):
-    if body and hasattr(body, 'read'):
-      self.body = body.read()
-    else:
-      self.body = body
-    if status is not None:
-      self.status = int(status)
-    else:
-      self.status = None
-    self.reason = reason
-    self._headers = headers or {}
-
-  def read(self):
-    return self.body
-
-
-class MockHttpClient(atom.http_interface.GenericHttpClient):
-  def __init__(self, headers=None, recordings=None, real_client=None):
-    """An HttpClient which responds to request with stored data.
-
-    The request-response pairs are stored as tuples in a member list named
-    recordings.
-
-    The MockHttpClient can be switched from replay mode to record mode by
-    setting the real_client member to an instance of an HttpClient which will
-    make real HTTP requests and store the server's response in list of 
-    recordings.
-    
-    Args:
-      headers: dict containing HTTP headers which should be included in all
-          HTTP requests.
-      recordings: The initial recordings to be used for responses. This list
-          contains tuples in the form: (MockRequest, MockResponse)
-      real_client: An HttpClient which will make a real HTTP request. The 
-          response will be converted into a MockResponse and stored in 
-          recordings.
-    """
-    self.recordings = recordings or []
-    self.real_client = real_client
-    self.headers = headers or {}
-
-  def add_response(self, response, operation, url, data=None, headers=None):
-    """Adds a request-response pair to the recordings list.
-    
-    After the recording is added, future matching requests will receive the
-    response.
-    
-    Args:
-      response: MockResponse
-      operation: str
-      url: str
-      data: str, Currently the data is ignored when looking for matching
-          requests.
-      headers: dict of strings: Currently the headers are ignored when
-          looking for matching requests.
-    """
-    request = MockRequest(operation, url, data=data, headers=headers)
-    self.recordings.append((request, response))
-
-  def request(self, operation, url, data=None, headers=None):
-    """Returns a matching MockResponse from the recordings.
-    
-    If the real_client is set, the request will be passed along and the 
-    server's response will be added to the recordings and also returned. 
-
-    If there is no match, a NoRecordingFound error will be raised.
-    """
-    if self.real_client is None:
-      if isinstance(url, (str, unicode)):
-        url = atom.url.parse_url(url)
-      for recording in self.recordings:
-        if recording[0].operation == operation and recording[0].url == url:
-          return recording[1]
-      raise NoRecordingFound('No recodings found for %s %s' % (
-          operation, url))
-    else:
-      # There is a real HTTP client, so make the request, and record the 
-      # response.
-      response = self.real_client.request(operation, url, data=data, 
-          headers=headers)
-      # TODO: copy the headers
-      stored_response = MockResponse(body=response, status=response.status,
-          reason=response.reason)
-      self.add_response(stored_response, operation, url, data=data, 
-          headers=headers)
-      return stored_response

atom/mock_http_core.py

@@ -1,323 +0,0 @@
-#!/usr/bin/env python
-#
-# Copyright (C) 2009 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-# This module is used for version 2 of the Google Data APIs.
-
-
-__author__ = 'j.s@google.com (Jeff Scudder)'
-
-
-import StringIO
-import pickle
-import os.path
-import tempfile
-import atom.http_core
-
-
-class Error(Exception):
-  pass
-
-
-class NoRecordingFound(Error):
-  pass
-
-
-class MockHttpClient(object):
-  debug = None
-  real_client = None
-  last_request_was_live = False
-
-  # The following members are used to construct the session cache temp file
-  # name.
-  # These are combined to form the file name
-  # /tmp/cache_prefix.cache_case_name.cache_test_name
-  cache_name_prefix = 'gdata_live_test'
-  cache_case_name = ''
-  cache_test_name = ''
-
-  def __init__(self, recordings=None, real_client=None):
-    self._recordings = recordings or []
-    if real_client is not None:
-      self.real_client = real_client
-
-  def add_response(self, http_request, status, reason, headers=None,
-      body=None):
-    response = MockHttpResponse(status, reason, headers, body)
-    # TODO Scrub the request and the response.
-    self._recordings.append((http_request._copy(), response))
-
-  AddResponse = add_response
-
-  def request(self, http_request):
-    """Provide a recorded response, or record a response for replay.
-
-    If the real_client is set, the request will be made using the
-    real_client, and the response from the server will be recorded.
-    If the real_client is None (the default), this method will examine
-    the recordings and find the first which matches.
-    """
-    request = http_request._copy()
-    _scrub_request(request)
-    if self.real_client is None:
-      self.last_request_was_live = False
-      for recording in self._recordings:
-        if _match_request(recording[0], request):
-          return recording[1]
-    else:
-      # Pass along the debug settings to the real client.
-      self.real_client.debug = self.debug
-      # Make an actual request since we can use the real HTTP client.
-      self.last_request_was_live = True
-      response = self.real_client.request(http_request)
-      scrubbed_response = _scrub_response(response)
-      self.add_response(request, scrubbed_response.status,
-                        scrubbed_response.reason,
-                        dict(atom.http_core.get_headers(scrubbed_response)),
-                        scrubbed_response.read())
-      # Return the recording which we just added.
-      return self._recordings[-1][1]
-    raise NoRecordingFound('No recoding was found for request: %s %s' % (
-        request.method, str(request.uri)))
-
-  Request = request
-
-  def _save_recordings(self, filename):
-    recording_file = open(os.path.join(tempfile.gettempdir(), filename),
-                          'wb')
-    pickle.dump(self._recordings, recording_file)
-    recording_file.close()
-
-  def _load_recordings(self, filename):
-    recording_file = open(os.path.join(tempfile.gettempdir(), filename),
-                          'rb')
-    self._recordings = pickle.load(recording_file)
-    recording_file.close()
-
-  def _delete_recordings(self, filename):
-    full_path = os.path.join(tempfile.gettempdir(), filename)
-    if os.path.exists(full_path):
-      os.remove(full_path)
-
-  def _load_or_use_client(self, filename, http_client):
-    if os.path.exists(os.path.join(tempfile.gettempdir(), filename)):
-      self._load_recordings(filename)
-    else:
-      self.real_client = http_client
-
-  def use_cached_session(self, name=None, real_http_client=None):
-    """Attempts to load recordings from a previous live request.
-
-    If a temp file with the recordings exists, then it is used to fulfill
-    requests. If the file does not exist, then a real client is used to
-    actually make the desired HTTP requests. Requests and responses are
-    recorded and will be written to the desired temprary cache file when
-    close_session is called.
-
-    Args:
-      name: str (optional) The file name of session file to be used. The file
-            is loaded from the temporary directory of this machine. If no name
-            is passed in, a default name will be constructed using the
-            cache_name_prefix, cache_case_name, and cache_test_name of this
-            object.
-      real_http_client: atom.http_core.HttpClient the real client to be used
-                        if the cached recordings are not found. If the default
-                        value is used, this will be an
-                        atom.http_core.HttpClient.
-    """
-    if real_http_client is None:
-      real_http_client = atom.http_core.HttpClient()
-    if name is None:
-      self._recordings_cache_name = self.get_cache_file_name()
-    else:
-      self._recordings_cache_name = name
-    self._load_or_use_client(self._recordings_cache_name, real_http_client)
-
-  def close_session(self):
-    """Saves recordings in the temporary file named in use_cached_session."""
-    if self.real_client is not None:
-      self._save_recordings(self._recordings_cache_name)
-
-  def delete_session(self, name=None):
-    """Removes recordings from a previous live request."""
-    if name is None:
-      self._delete_recordings(self._recordings_cache_name)
-    else:
-      self._delete_recordings(name)
-
-  def get_cache_file_name(self):
-    return '%s.%s.%s' % (self.cache_name_prefix, self.cache_case_name,
-                         self.cache_test_name)
-
-  def _dump(self):
-    """Provides debug information in a string."""
-    output = 'MockHttpClient\n  real_client: %s\n  cache file name: %s\n' % (
-        self.real_client, self.get_cache_file_name())
-    output += '  recordings:\n'
-    i = 0
-    for recording in self._recordings:
-      output += '    recording %i is for: %s %s\n' % (
-          i, recording[0].method, str(recording[0].uri))
-      i += 1
-    return output
-
-
-def _match_request(http_request, stored_request):
-  """Determines whether a request is similar enough to a stored request
-     to cause the stored response to be returned."""
-  # Check to see if the host names match.
-  if (http_request.uri.host is not None
-      and http_request.uri.host != stored_request.uri.host):
-    return False
-  # Check the request path in the URL (/feeds/private/full/x)
-  elif http_request.uri.path != stored_request.uri.path:
-    return False
-  # Check the method used in the request (GET, POST, etc.)
-  elif http_request.method != stored_request.method:
-    return False
-  # If there is a gsession ID in either request, make sure that it is matched
-  # exactly.
-  elif ('gsessionid' in http_request.uri.query
-        or 'gsessionid' in stored_request.uri.query):
-    if 'gsessionid' not in stored_request.uri.query:
-      return False
-    elif 'gsessionid' not in http_request.uri.query:
-      return False
-    elif (http_request.uri.query['gsessionid']
-          != stored_request.uri.query['gsessionid']):
-      return False
-  # Ignores differences in the query params (?start-index=5&max-results=20),
-  # the body of the request, the port number, HTTP headers, just to name a
-  # few.
-  return True
-
-
-def _scrub_request(http_request):
-  """ Removes email address and password from a client login request.
-
-  Since the mock server saves the request and response in plantext, sensitive
-  information like the password should be removed before saving the
-  recordings. At the moment only requests sent to a ClientLogin url are
-  scrubbed.
-  """
-  if (http_request and http_request.uri and http_request.uri.path and
-      http_request.uri.path.endswith('ClientLogin')):
-    # Remove the email and password from a ClientLogin request.
-    http_request._body_parts = []
-    http_request.add_form_inputs(
-        {'form_data': 'client login request has been scrubbed'})
-  else:
-    # We can remove the body of the post from the recorded request, since
-    # the request body is not used when finding a matching recording.
-    http_request._body_parts = []
-  return http_request
-
-
-def _scrub_response(http_response):
-  return http_response
-
-
-class EchoHttpClient(object):
-  """Sends the request data back in the response.
-
-  Used to check the formatting of the request as it was sent. Always responds
-  with a 200 OK, and some information from the HTTP request is returned in
-  special Echo-X headers in the response. The following headers are added
-  in the response:
-  'Echo-Host': The host name and port number to which the HTTP connection is
-               made. If no port was passed in, the header will contain
-               host:None.
-  'Echo-Uri': The path portion of the URL being requested. /example?x=1&y=2
-  'Echo-Scheme': The beginning of the URL, usually 'http' or 'https'
-  'Echo-Method': The HTTP method being used, 'GET', 'POST', 'PUT', etc.
-  """
-
-  def request(self, http_request):
-    return self._http_request(http_request.uri, http_request.method,
-                              http_request.headers, http_request._body_parts)
-
-  def _http_request(self, uri, method, headers=None, body_parts=None):
-    body = StringIO.StringIO()
-    response = atom.http_core.HttpResponse(status=200, reason='OK', body=body)
-    if headers is None:
-      response._headers = {}
-    else:
-      # Copy headers from the request to the response but convert values to
-      # strings. Server response headers always come in as strings, so an int
-      # should be converted to a corresponding string when echoing.
-      for header, value in headers.iteritems():
-        response._headers[header] = str(value)
-    response._headers['Echo-Host'] = '%s:%s' % (uri.host, str(uri.port))
-    response._headers['Echo-Uri'] = uri._get_relative_path()
-    response._headers['Echo-Scheme'] = uri.scheme
-    response._headers['Echo-Method'] = method
-    for part in body_parts:
-      if isinstance(part, str):
-        body.write(part)
-      elif hasattr(part, 'read'):
-        body.write(part.read())
-    body.seek(0)
-    return response
-
-
-class SettableHttpClient(object):
-  """An HTTP Client which responds with the data given in set_response."""
-
-  def __init__(self, status, reason, body, headers):
-    """Configures the response for the server.
-
-    See set_response for details on the arguments to the constructor.
-    """
-    self.set_response(status, reason, body, headers)
-    self.last_request = None
-
-  def set_response(self, status, reason, body, headers):
-    """Determines the response which will be sent for each request.
-
-    Args:
-      status: An int for the HTTP status code, example: 200, 404, etc.
-      reason: String for the HTTP reason, example: OK, NOT FOUND, etc.
-      body: The body of the HTTP response as a string or a file-like
-            object (something with a read method).
-      headers: dict of strings containing the HTTP headers in the response.
-    """
-    self.response = atom.http_core.HttpResponse(status=status, reason=reason,
-        body=body)
-    self.response._headers = headers.copy()
-
-  def request(self, http_request):
-    self.last_request = http_request
-    return self.response
-
-
-class MockHttpResponse(atom.http_core.HttpResponse):
-
-  def __init__(self, status=None, reason=None, headers=None, body=None):
-    self._headers = headers or {}
-    if status is not None:
-      self.status = status
-    if reason is not None:
-      self.reason = reason
-    if body is not None:
-      # Instead of using a file-like object for the body, store as a string
-      # so that reads can be repeated.
-      if hasattr(body, 'read'):
-        self._body = body.read()
-      else:
-        self._body = body
-
-  def read(self):
-    return self._body

atom/mock_service.py

@@ -1,243 +0,0 @@
-#!/usr/bin/python
-#
-# Copyright (C) 2008 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-"""MockService provides CRUD ops. for mocking calls to AtomPub services.
-
-  MockService: Exposes the publicly used methods of AtomService to provide
-      a mock interface which can be used in unit tests.
-"""
-
-import atom.service
-import pickle
-
-
-__author__ = 'api.jscudder (Jeffrey Scudder)'
-
-
-# Recordings contains pairings of HTTP MockRequest objects with MockHttpResponse objects.
-recordings = []
-# If set, the mock service HttpRequest are actually made through this object.
-real_request_handler = None
-
-def ConcealValueWithSha(source):
-  import sha
-  return sha.new(source[:-5]).hexdigest()
-
-def DumpRecordings(conceal_func=ConcealValueWithSha):
-  if conceal_func:
-    for recording_pair in recordings:
-      recording_pair[0].ConcealSecrets(conceal_func)
-  return pickle.dumps(recordings)
-
-def LoadRecordings(recordings_file_or_string):
-  if isinstance(recordings_file_or_string, str):
-    atom.mock_service.recordings =  pickle.loads(recordings_file_or_string)
-  elif hasattr(recordings_file_or_string, 'read'):
-    atom.mock_service.recordings = pickle.loads(
-      recordings_file_or_string.read())
-
-def HttpRequest(service, operation, data, uri, extra_headers=None,
-    url_params=None, escape_params=True, content_type='application/atom+xml'):
-  """Simulates an HTTP call to the server, makes an actual HTTP request if 
-  real_request_handler is set.
-
-  This function operates in two different modes depending on if 
-  real_request_handler is set or not. If real_request_handler is not set,
-  HttpRequest will look in this module's recordings list to find a response
-  which matches the parameters in the function call. If real_request_handler
-  is set, this function will call real_request_handler.HttpRequest, add the
-  response to the recordings list, and respond with the actual response.
-
-  Args:
-    service: atom.AtomService object which contains some of the parameters
-        needed to make the request. The following members are used to
-        construct the HTTP call: server (str), additional_headers (dict),
-        port (int), and ssl (bool).
-    operation: str The HTTP operation to be performed. This is usually one of
-        'GET', 'POST', 'PUT', or 'DELETE'
-    data: ElementTree, filestream, list of parts, or other object which can be
-        converted to a string.
-        Should be set to None when performing a GET or PUT.
-        If data is a file-like object which can be read, this method will read
-        a chunk of 100K bytes at a time and send them.
-        If the data is a list of parts to be sent, each part will be evaluated
-        and sent.
-    uri: The beginning of the URL to which the request should be sent.
-        Examples: '/', '/base/feeds/snippets',
-        '/m8/feeds/contacts/default/base'
-    extra_headers: dict of strings. HTTP headers which should be sent
-        in the request. These headers are in addition to those stored in
-        service.additional_headers.
-    url_params: dict of strings. Key value pairs to be added to the URL as
-        URL parameters. For example {'foo':'bar', 'test':'param'} will
-        become ?foo=bar&test=param.
-    escape_params: bool default True. If true, the keys and values in
-        url_params will be URL escaped when the form is constructed
-        (Special characters converted to %XX form.)
-    content_type: str The MIME type for the data being sent. Defaults to
-        'application/atom+xml', this is only used if data is set.
-  """
-  full_uri = atom.service.BuildUri(uri, url_params, escape_params)
-  (server, port, ssl, uri) = atom.service.ProcessUrl(service, uri)
-  current_request = MockRequest(operation, full_uri, host=server, ssl=ssl, 
-      data=data, extra_headers=extra_headers, url_params=url_params, 
-      escape_params=escape_params, content_type=content_type)
-  # If the request handler is set, we should actually make the request using 
-  # the request handler and record the response to replay later.
-  if real_request_handler:
-    response = real_request_handler.HttpRequest(service, operation, data, uri,
-        extra_headers=extra_headers, url_params=url_params, 
-        escape_params=escape_params, content_type=content_type)
-    # TODO: need to copy the HTTP headers from the real response into the
-    # recorded_response.
-    recorded_response = MockHttpResponse(body=response.read(), 
-        status=response.status, reason=response.reason)
-    # Insert a tuple which maps the request to the response object returned
-    # when making an HTTP call using the real_request_handler.
-    recordings.append((current_request, recorded_response))
-    return recorded_response
-  else:
-    # Look through available recordings to see if one matches the current 
-    # request.
-    for request_response_pair in recordings:
-      if request_response_pair[0].IsMatch(current_request):
-        return request_response_pair[1]
-  return None
-
-
-class MockRequest(object):
-  """Represents a request made to an AtomPub server.
-  
-  These objects are used to determine if a client request matches a recorded
-  HTTP request to determine what the mock server's response will be. 
-  """
-
-  def __init__(self, operation, uri, host=None, ssl=False, port=None, 
-      data=None, extra_headers=None, url_params=None, escape_params=True,
-      content_type='application/atom+xml'):
-    """Constructor for a MockRequest
-    
-    Args:
-      operation: str One of 'GET', 'POST', 'PUT', or 'DELETE' this is the
-          HTTP operation requested on the resource.
-      uri: str The URL describing the resource to be modified or feed to be
-          retrieved. This should include the protocol (http/https) and the host
-          (aka domain). For example, these are some valud full_uris:
-          'http://example.com', 'https://www.google.com/accounts/ClientLogin'
-      host: str (optional) The server name which will be placed at the 
-          beginning of the URL if the uri parameter does not begin with 'http'.
-          Examples include 'example.com', 'www.google.com', 'www.blogger.com'.
-      ssl: boolean (optional) If true, the request URL will begin with https 
-          instead of http.
-      data: ElementTree, filestream, list of parts, or other object which can be
-          converted to a string. (optional)
-          Should be set to None when performing a GET or PUT.
-          If data is a file-like object which can be read, the constructor 
-          will read the entire file into memory. If the data is a list of 
-          parts to be sent, each part will be evaluated and stored.
-      extra_headers: dict (optional) HTTP headers included in the request.
-      url_params: dict (optional) Key value pairs which should be added to 
-          the URL as URL parameters in the request. For example uri='/', 
-          url_parameters={'foo':'1','bar':'2'} could become '/?foo=1&bar=2'.
-      escape_params: boolean (optional) Perform URL escaping on the keys and 
-          values specified in url_params. Defaults to True.
-      content_type: str (optional) Provides the MIME type of the data being 
-          sent.
-    """
-    self.operation = operation
-    self.uri = _ConstructFullUrlBase(uri, host=host, ssl=ssl)
-    self.data = data
-    self.extra_headers = extra_headers
-    self.url_params = url_params or {}
-    self.escape_params = escape_params
-    self.content_type = content_type
-
-  def ConcealSecrets(self, conceal_func):
-    """Conceal secret data in this request."""
-    if self.extra_headers.has_key('Authorization'):
-      self.extra_headers['Authorization'] = conceal_func(
-        self.extra_headers['Authorization'])
-
-  def IsMatch(self, other_request):
-    """Check to see if the other_request is equivalent to this request.
-    
-    Used to determine if a recording matches an incoming request so that a
-    recorded response should be sent to the client.
-
-    The matching is not exact, only the operation and URL are examined 
-    currently.
-
-    Args:
-      other_request: MockRequest The request which we want to check this
-          (self) MockRequest against to see if they are equivalent.
-    """
-    # More accurate matching logic will likely be required.
-    return (self.operation == other_request.operation and self.uri == 
-        other_request.uri)
-
-
-def _ConstructFullUrlBase(uri, host=None, ssl=False):
-  """Puts URL components into the form http(s)://full.host.strinf/uri/path
-  
-  Used to construct a roughly canonical URL so that URLs which begin with 
-  'http://example.com/' can be compared to a uri of '/' when the host is 
-  set to 'example.com'
-
-  If the uri contains 'http://host' already, the host and ssl parameters
-  are ignored.
-
-  Args:
-    uri: str The path component of the URL, examples include '/'
-    host: str (optional) The host name which should prepend the URL. Example:
-        'example.com'
-    ssl: boolean (optional) If true, the returned URL will begin with https
-        instead of http.
-
-  Returns:
-    String which has the form http(s)://example.com/uri/string/contents
-  """
-  if uri.startswith('http'):
-    return uri
-  if ssl:
-    return 'https://%s%s' % (host, uri)
-  else:
-    return 'http://%s%s' % (host, uri)
-
-
-class MockHttpResponse(object):
-  """Returned from MockService crud methods as the server's response."""
-
-  def __init__(self, body=None, status=None, reason=None, headers=None):
-    """Construct a mock HTTPResponse and set members.
-
-    Args:
-      body: str (optional) The HTTP body of the server's response. 
-      status: int (optional) 
-      reason: str (optional)
-      headers: dict (optional)
-    """
-    self.body = body
-    self.status = status
-    self.reason = reason
-    self.headers = headers or {}
-
-  def read(self):
-    return self.body
-
-  def getheader(self, header_name):
-    return self.headers[header_name]
-

atom/service.py

@@ -1,745 +0,0 @@
-#!/usr/bin/python
-#
-# Copyright (C) 2006, 2007, 2008 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-"""AtomService provides CRUD ops. in line with the Atom Publishing Protocol.
-
-  AtomService: Encapsulates the ability to perform insert, update and delete
-               operations with the Atom Publishing Protocol on which GData is
-               based. An instance can perform query, insertion, deletion, and
-               update.
-
-  HttpRequest: Function that performs a GET, POST, PUT, or DELETE HTTP request
-       to the specified end point. An AtomService object or a subclass can be
-       used to specify information about the request.
-"""
-
-__author__ = 'api.jscudder (Jeff Scudder)'
-
-
-import atom.http_interface
-import atom.url
-import atom.http
-import atom.token_store
-
-import os
-import types
-import httplib
-import urllib
-import re
-import base64
-import socket
-import warnings
-try:
-  from xml.etree import cElementTree as ElementTree
-except ImportError:
-  try:
-    import cElementTree as ElementTree
-  except ImportError:
-    try:
-      from xml.etree import ElementTree
-    except ImportError:
-      from elementtree import ElementTree
-import atom
-
-
-class AtomService(object):
-  """Performs Atom Publishing Protocol CRUD operations.
-  
-  The AtomService contains methods to perform HTTP CRUD operations. 
-  """
-
-  # Default values for members
-  port = 80
-  ssl = False 
-  # Set the current_token to force the AtomService to use this token
-  # instead of searching for an appropriate token in the token_store.
-  current_token = None
-  auto_store_tokens = True
-  auto_set_current_token = True
-
-  def _get_override_token(self):
-    return self.current_token
-
-  def _set_override_token(self, token):
-    self.current_token = token
-
-  override_token = property(_get_override_token, _set_override_token)
-
-  #@atom.v1_deprecated('Please use atom.client.AtomPubClient instead.')
-  def __init__(self, server=None, additional_headers=None, 
-      application_name='', http_client=None, token_store=None):
-    """Creates a new AtomService client.
-    
-    Args:
-      server: string (optional) The start of a URL for the server
-              to which all operations should be directed. Example: 
-              'www.google.com'
-      additional_headers: dict (optional) Any additional HTTP headers which
-                          should be included with CRUD operations.
-      http_client: An object responsible for making HTTP requests using a
-                   request method. If none is provided, a new instance of
-                   atom.http.ProxiedHttpClient will be used.
-      token_store: Keeps a collection of authorization tokens which can be
-                   applied to requests for a specific URLs. Critical methods are
-                   find_token based on a URL (atom.url.Url or a string), add_token,
-                   and remove_token.
-    """
-    self.http_client = http_client or atom.http.ProxiedHttpClient()
-    self.token_store = token_store or atom.token_store.TokenStore()
-    self.server = server
-    self.additional_headers = additional_headers or {}
-    self.additional_headers['User-Agent'] = atom.http_interface.USER_AGENT % (
-        application_name,)
-    # If debug is True, the HTTPConnection will display debug information
-    self._set_debug(False)
-
-  __init__ = atom.v1_deprecated(
-      'Please use atom.client.AtomPubClient instead.')(
-          __init__)
-
-  def _get_debug(self):
-    return self.http_client.debug
-
-  def _set_debug(self, value):
-    self.http_client.debug = value
-
-  debug = property(_get_debug, _set_debug, 
-      doc='If True, HTTP debug information is printed.')
-
-  def use_basic_auth(self, username, password, scopes=None):
-    if username is not None and password is not None:
-      if scopes is None:
-        scopes = [atom.token_store.SCOPE_ALL]
-      base_64_string = base64.encodestring('%s:%s' % (username, password))
-      token = BasicAuthToken('Basic %s' % base_64_string.strip(), 
-          scopes=[atom.token_store.SCOPE_ALL])
-      if self.auto_set_current_token:
-        self.current_token = token
-      if self.auto_store_tokens:
-        return self.token_store.add_token(token)
-      return True
-    return False
-
-  def UseBasicAuth(self, username, password, for_proxy=False):
-    """Sets an Authenticaiton: Basic HTTP header containing plaintext.
-
-    Deprecated, use use_basic_auth instead.
-    
-    The username and password are base64 encoded and added to an HTTP header
-    which will be included in each request. Note that your username and 
-    password are sent in plaintext.
-
-    Args:
-      username: str
-      password: str
-    """
-    self.use_basic_auth(username, password)
-
-  #@atom.v1_deprecated('Please use atom.client.AtomPubClient for requests.')
-  def request(self, operation, url, data=None, headers=None, 
-      url_params=None):
-    if isinstance(url, (str, unicode)):
-      if url.startswith('http:') and self.ssl:
-        # Force all requests to be https if self.ssl is True.
-        url = atom.url.parse_url('https:' + url[5:])
-      elif not url.startswith('http') and self.ssl: 
-        url = atom.url.parse_url('https://%s%s' % (self.server, url))
-      elif not url.startswith('http'):
-        url = atom.url.parse_url('http://%s%s' % (self.server, url))
-      else:
-        url = atom.url.parse_url(url)
-
-    if url_params:
-      for name, value in url_params.iteritems():
-        url.params[name] = value
-
-    all_headers = self.additional_headers.copy()
-    if headers:
-      all_headers.update(headers)
-
-    if isinstance(data, types.StringTypes):
-      data = data.encode('utf-8')
-    
-    # If the list of headers does not include a Content-Length, attempt to
-    # calculate it based on the data object.
-    if data and 'Content-Length' not in all_headers:
-      content_length = CalculateDataLength(data)
-      if content_length:
-        all_headers['Content-Length'] = str(content_length)
-
-    all_headers['GData-Version'] = '2.0'
-    # Find an Authorization token for this URL if one is available.
-    if self.override_token:
-      auth_token = self.override_token
-    else:
-      auth_token = self.token_store.find_token(url)
-    return auth_token.perform_request(self.http_client, operation, url, 
-        data=data, headers=all_headers)
-
-  request = atom.v1_deprecated(
-      'Please use atom.client.AtomPubClient for requests.')(
-          request)
-
-  # CRUD operations
-  def Get(self, uri, extra_headers=None, url_params=None, escape_params=True):
-    """Query the APP server with the given URI
-
-    The uri is the portion of the URI after the server value 
-    (server example: 'www.google.com').
-
-    Example use:
-    To perform a query against Google Base, set the server to 
-    'base.google.com' and set the uri to '/base/feeds/...', where ... is 
-    your query. For example, to find snippets for all digital cameras uri 
-    should be set to: '/base/feeds/snippets?bq=digital+camera'
-
-    Args:
-      uri: string The query in the form of a URI. Example:
-           '/base/feeds/snippets?bq=digital+camera'.
-      extra_headers: dicty (optional) Extra HTTP headers to be included
-                     in the GET request. These headers are in addition to 
-                     those stored in the client's additional_headers property.
-                     The client automatically sets the Content-Type and 
-                     Authorization headers.
-      url_params: dict (optional) Additional URL parameters to be included
-                  in the query. These are translated into query arguments
-                  in the form '&dict_key=value&...'.
-                  Example: {'max-results': '250'} becomes &max-results=250
-      escape_params: boolean (optional) If false, the calling code has already
-                     ensured that the query will form a valid URL (all
-                     reserved characters have been escaped). If true, this
-                     method will escape the query and any URL parameters
-                     provided.
-
-    Returns:
-      httplib.HTTPResponse The server's response to the GET request.
-    """
-    return self.request('GET', uri, data=None, headers=extra_headers, 
-                        url_params=url_params)
-
-  def Post(self, data, uri, extra_headers=None, url_params=None, 
-           escape_params=True, content_type='application/atom+xml; charset=UTF-8'):
-    """Insert data into an APP server at the given URI.
-
-    Args:
-      data: string, ElementTree._Element, or something with a __str__ method 
-            The XML to be sent to the uri. 
-      uri: string The location (feed) to which the data should be inserted. 
-           Example: '/base/feeds/items'. 
-      extra_headers: dict (optional) HTTP headers which are to be included. 
-                     The client automatically sets the Content-Type,
-                     Authorization, and Content-Length headers.
-      url_params: dict (optional) Additional URL parameters to be included
-                  in the URI. These are translated into query arguments
-                  in the form '&dict_key=value&...'.
-                  Example: {'max-results': '250'} becomes &max-results=250
-      escape_params: boolean (optional) If false, the calling code has already
-                     ensured that the query will form a valid URL (all
-                     reserved characters have been escaped). If true, this
-                     method will escape the query and any URL parameters
-                     provided.
-
-    Returns:
-      httplib.HTTPResponse Server's response to the POST request.
-    """
-    if extra_headers is None:
-      extra_headers = {}
-    if content_type:
-      extra_headers['Content-Type'] = content_type
-    return self.request('POST', uri, data=data, headers=extra_headers, 
-                        url_params=url_params)
-
-  def Put(self, data, uri, extra_headers=None, url_params=None, 
-           escape_params=True, content_type='application/atom+xml; charset=UTF-8'):
-    """Updates an entry at the given URI.
-     
-    Args:
-      data: string, ElementTree._Element, or xml_wrapper.ElementWrapper The 
-            XML containing the updated data.
-      uri: string A URI indicating entry to which the update will be applied.
-           Example: '/base/feeds/items/ITEM-ID'
-      extra_headers: dict (optional) HTTP headers which are to be included.
-                     The client automatically sets the Content-Type,
-                     Authorization, and Content-Length headers.
-      url_params: dict (optional) Additional URL parameters to be included
-                  in the URI. These are translated into query arguments
-                  in the form '&dict_key=value&...'.
-                  Example: {'max-results': '250'} becomes &max-results=250
-      escape_params: boolean (optional) If false, the calling code has already
-                     ensured that the query will form a valid URL (all
-                     reserved characters have been escaped). If true, this
-                     method will escape the query and any URL parameters
-                     provided.
-  
-    Returns:
-      httplib.HTTPResponse Server's response to the PUT request.
-    """
-    if extra_headers is None:
-      extra_headers = {}
-    if content_type:
-      extra_headers['Content-Type'] = content_type
-    return self.request('PUT', uri, data=data, headers=extra_headers, 
-                        url_params=url_params)
-
-  def Delete(self, uri, extra_headers=None, url_params=None, 
-             escape_params=True):
-    """Deletes the entry at the given URI.
-
-    Args:
-      uri: string The URI of the entry to be deleted. Example: 
-           '/base/feeds/items/ITEM-ID'
-      extra_headers: dict (optional) HTTP headers which are to be included.
-                     The client automatically sets the Content-Type and
-                     Authorization headers.
-      url_params: dict (optional) Additional URL parameters to be included
-                  in the URI. These are translated into query arguments
-                  in the form '&dict_key=value&...'.
-                  Example: {'max-results': '250'} becomes &max-results=250
-      escape_params: boolean (optional) If false, the calling code has already
-                     ensured that the query will form a valid URL (all
-                     reserved characters have been escaped). If true, this
-                     method will escape the query and any URL parameters
-                     provided.
-
-    Returns:
-      httplib.HTTPResponse Server's response to the DELETE request.
-    """
-    return self.request('DELETE', uri, data=None, headers=extra_headers, 
-                        url_params=url_params)
-
-
-class BasicAuthToken(atom.http_interface.GenericToken):
-  def __init__(self, auth_header, scopes=None):
-    """Creates a token used to add Basic Auth headers to HTTP requests.
-
-    Args:
-      auth_header: str The value for the Authorization header.
-      scopes: list of str or atom.url.Url specifying the beginnings of URLs
-          for which this token can be used. For example, if scopes contains
-          'http://example.com/foo', then this token can be used for a request to
-          'http://example.com/foo/bar' but it cannot be used for a request to
-          'http://example.com/baz'
-    """
-    self.auth_header = auth_header
-    self.scopes = scopes or []
-
-  def perform_request(self, http_client, operation, url, data=None,
-                      headers=None):
-    """Sets the Authorization header to the basic auth string."""
-    if headers is None:
-      headers = {'Authorization':self.auth_header}
-    else:
-      headers['Authorization'] = self.auth_header
-    return http_client.request(operation, url, data=data, headers=headers)
-
-  def __str__(self):
-    return self.auth_header
-
-  def valid_for_scope(self, url):
-    """Tells the caller if the token authorizes access to the desired URL.
-    """
-    if isinstance(url, (str, unicode)):
-      url = atom.url.parse_url(url)
-    for scope in self.scopes:
-      if scope == atom.token_store.SCOPE_ALL:
-        return True
-      if isinstance(scope, (str, unicode)):
-        scope = atom.url.parse_url(scope)
-      if scope == url:
-        return True
-      # Check the host and the path, but ignore the port and protocol.
-      elif scope.host == url.host and not scope.path:
-        return True
-      elif scope.host == url.host and scope.path and not url.path:
-        continue
-      elif scope.host == url.host and url.path.startswith(scope.path):
-        return True
-    return False
-
-
-def PrepareConnection(service, full_uri):
-  """Opens a connection to the server based on the full URI.
-
-  This method is deprecated, instead use atom.http.HttpClient.request.
-
-  Examines the target URI and the proxy settings, which are set as
-  environment variables, to open a connection with the server. This
-  connection is used to make an HTTP request.
-
-  Args:
-    service: atom.AtomService or a subclass. It must have a server string which
-      represents the server host to which the request should be made. It may also
-      have a dictionary of additional_headers to send in the HTTP request.
-    full_uri: str Which is the target relative (lacks protocol and host) or
-    absolute URL to be opened. Example:
-    'https://www.google.com/accounts/ClientLogin' or
-    'base/feeds/snippets' where the server is set to www.google.com.
-
-  Returns:
-    A tuple containing the httplib.HTTPConnection and the full_uri for the
-    request.
-  """
-  deprecation('calling deprecated function PrepareConnection')
-  (server, port, ssl, partial_uri) = ProcessUrl(service, full_uri)
-  if ssl:
-    # destination is https
-    proxy = os.environ.get('https_proxy')
-    if proxy:
-      (p_server, p_port, p_ssl, p_uri) = ProcessUrl(service, proxy, True)
-      proxy_username = os.environ.get('proxy-username')
-      if not proxy_username:
-        proxy_username = os.environ.get('proxy_username')
-      proxy_password = os.environ.get('proxy-password')
-      if not proxy_password:
-        proxy_password = os.environ.get('proxy_password')
-      if proxy_username:
-        user_auth = base64.encodestring('%s:%s' % (proxy_username,
-                                                   proxy_password))
-        proxy_authorization = ('Proxy-authorization: Basic %s\r\n' % (
-            user_auth.strip()))
-      else:
-        proxy_authorization = ''
-      proxy_connect = 'CONNECT %s:%s HTTP/1.0\r\n' % (server, port)
-      user_agent = 'User-Agent: %s\r\n' % (
-          service.additional_headers['User-Agent'])
-      proxy_pieces = (proxy_connect + proxy_authorization + user_agent
-                       + '\r\n')
-
-      #now connect, very simple recv and error checking
-      p_sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
-      p_sock.connect((p_server,p_port))
-      p_sock.sendall(proxy_pieces)
-      response = ''
-
-      # Wait for the full response.
-      while response.find("\r\n\r\n") == -1:
-        response += p_sock.recv(8192)
-       
-      p_status=response.split()[1]
-      if p_status!=str(200):
-        raise atom.http.ProxyError('Error status=%s' % p_status)
-
-      # Trivial setup for ssl socket.
-      ssl = socket.ssl(p_sock, None, None)
-      fake_sock = httplib.FakeSocket(p_sock, ssl)
-
-      # Initalize httplib and replace with the proxy socket.
-      connection = httplib.HTTPConnection(server)
-      connection.sock=fake_sock
-      full_uri = partial_uri
-
-    else:
-      connection = httplib.HTTPSConnection(server, port)
-      full_uri = partial_uri
-
-  else:
-    # destination is http
-    proxy = os.environ.get('http_proxy')
-    if proxy:
-      (p_server, p_port, p_ssl, p_uri) = ProcessUrl(service.server, proxy, True)
-      proxy_username = os.environ.get('proxy-username')
-      if not proxy_username:
-        proxy_username = os.environ.get('proxy_username')
-      proxy_password = os.environ.get('proxy-password')
-      if not proxy_password:
-        proxy_password = os.environ.get('proxy_password')
-      if proxy_username:
-        UseBasicAuth(service, proxy_username, proxy_password, True)
-      connection = httplib.HTTPConnection(p_server, p_port)
-      if not full_uri.startswith("http://"):
-        if full_uri.startswith("/"):
-          full_uri = "http://%s%s" % (service.server, full_uri)
-        else:
-          full_uri = "http://%s/%s" % (service.server, full_uri)
-    else:
-      connection = httplib.HTTPConnection(server, port)
-      full_uri = partial_uri
-
-  return (connection, full_uri)
-
-
-def UseBasicAuth(service, username, password, for_proxy=False):
-  """Sets an Authenticaiton: Basic HTTP header containing plaintext.
-
-  Deprecated, use AtomService.use_basic_auth insread.
-  
-  The username and password are base64 encoded and added to an HTTP header
-  which will be included in each request. Note that your username and 
-  password are sent in plaintext. The auth header is added to the 
-  additional_headers dictionary in the service object.
-
-  Args:
-    service: atom.AtomService or a subclass which has an 
-        additional_headers dict as a member.
-    username: str
-    password: str
-  """
-  deprecation('calling deprecated function UseBasicAuth')
-  base_64_string = base64.encodestring('%s:%s' % (username, password))
-  base_64_string = base_64_string.strip()
-  if for_proxy:
-    header_name = 'Proxy-Authorization'
-  else:
-    header_name = 'Authorization'
-  service.additional_headers[header_name] = 'Basic %s' % (base_64_string,)
-
-
-def ProcessUrl(service, url, for_proxy=False):
-  """Processes a passed URL.  If the URL does not begin with https?, then
-  the default value for server is used
-
-  This method is deprecated, use atom.url.parse_url instead.
-  """
-  if not isinstance(url, atom.url.Url):
-    url = atom.url.parse_url(url)
-
-  server = url.host
-  ssl = False
-  port = 80
-
-  if not server:
-    if hasattr(service, 'server'):
-      server = service.server
-    else:
-      server = service
-    if not url.protocol and hasattr(service, 'ssl'):
-      ssl = service.ssl
-    if hasattr(service, 'port'):
-      port = service.port
-  else:
-    if url.protocol == 'https':
-      ssl = True
-    elif url.protocol == 'http':
-      ssl = False
-    if url.port:
-      port = int(url.port)
-    elif port == 80 and ssl:
-      port = 443
-
-  return (server, port, ssl, url.get_request_uri())
-
-def DictionaryToParamList(url_parameters, escape_params=True):
-  """Convert a dictionary of URL arguments into a URL parameter string.
-
-  This function is deprcated, use atom.url.Url instead.
-
-  Args:
-    url_parameters: The dictionaty of key-value pairs which will be converted
-                    into URL parameters. For example,
-                    {'dry-run': 'true', 'foo': 'bar'}
-                    will become ['dry-run=true', 'foo=bar'].
-
-  Returns:
-    A list which contains a string for each key-value pair. The strings are
-    ready to be incorporated into a URL by using '&'.join([] + parameter_list)
-  """
-  # Choose which function to use when modifying the query and parameters.
-  # Use quote_plus when escape_params is true.
-  transform_op = [str, urllib.quote_plus][bool(escape_params)]
-  # Create a list of tuples containing the escaped version of the
-  # parameter-value pairs.
-  parameter_tuples = [(transform_op(param), transform_op(value))
-                     for param, value in (url_parameters or {}).items()]
-  # Turn parameter-value tuples into a list of strings in the form
-  # 'PARAMETER=VALUE'.
-  return ['='.join(x) for x in parameter_tuples]
-
-
-def BuildUri(uri, url_params=None, escape_params=True):
-  """Converts a uri string and a collection of parameters into a URI.
-
-  This function is deprcated, use atom.url.Url instead.
-
-  Args:
-    uri: string
-    url_params: dict (optional)
-    escape_params: boolean (optional)
-    uri: string The start of the desired URI. This string can alrady contain
-         URL parameters. Examples: '/base/feeds/snippets', 
-         '/base/feeds/snippets?bq=digital+camera'
-    url_parameters: dict (optional) Additional URL parameters to be included
-                    in the query. These are translated into query arguments
-                    in the form '&dict_key=value&...'.
-                    Example: {'max-results': '250'} becomes &max-results=250
-    escape_params: boolean (optional) If false, the calling code has already
-                   ensured that the query will form a valid URL (all
-                   reserved characters have been escaped). If true, this
-                   method will escape the query and any URL parameters
-                   provided.
-
-  Returns:
-    string The URI consisting of the escaped URL parameters appended to the
-    initial uri string.
-  """
-  # Prepare URL parameters for inclusion into the GET request.
-  parameter_list = DictionaryToParamList(url_params, escape_params)
-
-  # Append the URL parameters to the URL.
-  if parameter_list:
-    if uri.find('?') != -1:
-      # If there are already URL parameters in the uri string, add the
-      # parameters after a new & character.
-      full_uri = '&'.join([uri] + parameter_list)
-    else:
-      # The uri string did not have any URL parameters (no ? character)
-      # so put a ? between the uri and URL parameters.
-      full_uri = '%s%s' % (uri, '?%s' % ('&'.join([] + parameter_list)))  
-  else:
-    full_uri = uri
-        
-  return full_uri
-
-  
-def HttpRequest(service, operation, data, uri, extra_headers=None, 
-    url_params=None, escape_params=True, content_type='application/atom+xml; charset=UTF-8'):
-  """Performs an HTTP call to the server, supports GET, POST, PUT, and DELETE.
-  
-  This method is deprecated, use atom.http.HttpClient.request instead.
-
-  Usage example, perform and HTTP GET on http://www.google.com/:
-    import atom.service
-    client = atom.service.AtomService()
-    http_response = client.Get('http://www.google.com/')
-  or you could set the client.server to 'www.google.com' and use the 
-  following:
-    client.server = 'www.google.com'
-    http_response = client.Get('/')
-
-  Args:
-    service: atom.AtomService object which contains some of the parameters 
-        needed to make the request. The following members are used to 
-        construct the HTTP call: server (str), additional_headers (dict), 
-        port (int), and ssl (bool).
-    operation: str The HTTP operation to be performed. This is usually one of
-        'GET', 'POST', 'PUT', or 'DELETE'
-    data: ElementTree, filestream, list of parts, or other object which can be 
-        converted to a string. 
-        Should be set to None when performing a GET or PUT.
-        If data is a file-like object which can be read, this method will read
-        a chunk of 100K bytes at a time and send them. 
-        If the data is a list of parts to be sent, each part will be evaluated
-        and sent.
-    uri: The beginning of the URL to which the request should be sent. 
-        Examples: '/', '/base/feeds/snippets', 
-        '/m8/feeds/contacts/default/base'
-    extra_headers: dict of strings. HTTP headers which should be sent
-        in the request. These headers are in addition to those stored in 
-        service.additional_headers.
-    url_params: dict of strings. Key value pairs to be added to the URL as
-        URL parameters. For example {'foo':'bar', 'test':'param'} will 
-        become ?foo=bar&test=param.
-    escape_params: bool default True. If true, the keys and values in 
-        url_params will be URL escaped when the form is constructed 
-        (Special characters converted to %XX form.)
-    content_type: str The MIME type for the data being sent. Defaults to
-        'application/atom+xml', this is only used if data is set.
-  """
-  deprecation('call to deprecated function HttpRequest')
-  full_uri = BuildUri(uri, url_params, escape_params)
-  (connection, full_uri) = PrepareConnection(service, full_uri)
-
-  if extra_headers is None:
-    extra_headers = {}
-
-  # Turn on debug mode if the debug member is set.
-  if service.debug:
-    connection.debuglevel = 1
-
-  connection.putrequest(operation, full_uri)
-
-  # If the list of headers does not include a Content-Length, attempt to 
-  # calculate it based on the data object.
-  if (data and not service.additional_headers.has_key('Content-Length') and 
-      not extra_headers.has_key('Content-Length')):
-    content_length = CalculateDataLength(data)
-    if content_length:
-      extra_headers['Content-Length'] = str(content_length)
-
-  if content_type:
-    extra_headers['Content-Type'] = content_type 
-
-  # Send the HTTP headers.
-  if isinstance(service.additional_headers, dict):
-    for header in service.additional_headers:
-      connection.putheader(header, service.additional_headers[header])
-  if isinstance(extra_headers, dict):
-    for header in extra_headers:
-      connection.putheader(header, extra_headers[header])
-  connection.endheaders()
-
-  # If there is data, send it in the request.
-  if data:
-    if isinstance(data, list):
-      for data_part in data:
-        __SendDataPart(data_part, connection)
-    else:
-      __SendDataPart(data, connection)
-
-  # Return the HTTP Response from the server.
-  return connection.getresponse()
-  
-
-def __SendDataPart(data, connection):
-  """This method is deprecated, use atom.http._send_data_part"""
-  deprecated('call to deprecated function __SendDataPart')
-  if isinstance(data, str):
-    #TODO add handling for unicode.
-    connection.send(data)
-    return
-  elif ElementTree.iselement(data):
-    connection.send(ElementTree.tostring(data))
-    return
-  # Check to see if data is a file-like object that has a read method.
-  elif hasattr(data, 'read'):
-    # Read the file and send it a chunk at a time.
-    while 1:
-      binarydata = data.read(100000)
-      if binarydata == '': break
-      connection.send(binarydata)
-    return
-  else:
-    # The data object was not a file.
-    # Try to convert to a string and send the data.
-    connection.send(str(data))
-    return
-
-
-def CalculateDataLength(data):
-  """Attempts to determine the length of the data to send. 
-  
-  This method will respond with a length only if the data is a string or
-  and ElementTree element.
-
-  Args:
-    data: object If this is not a string or ElementTree element this funtion
-        will return None.
-  """
-  if isinstance(data, str):
-    return len(data)
-  elif isinstance(data, list):
-    return None
-  elif ElementTree.iselement(data):
-    return len(ElementTree.tostring(data))
-  elif hasattr(data, 'read'):
-    # If this is a file-like object, don't try to guess the length.
-    return None
-  else:
-    return len(str(data))
-    
-
-def deprecation(message):
-  warnings.warn(message, DeprecationWarning, stacklevel=2)

atom/token_store.py

@@ -1,117 +0,0 @@
-#!/usr/bin/python
-#
-# Copyright (C) 2008 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""This module provides a TokenStore class which is designed to manage
-auth tokens required for different services.
-
-Each token is valid for a set of scopes which is the start of a URL. An HTTP
-client will use a token store to find a valid Authorization header to send
-in requests to the specified URL. If the HTTP client determines that a token
-has expired or been revoked, it can remove the token from the store so that
-it will not be used in future requests.
-"""
-
-
-__author__ = 'api.jscudder (Jeff Scudder)'
-
-
-import atom.http_interface
-import atom.url
-
-
-SCOPE_ALL = 'http'
-
-
-class TokenStore(object):
-  """Manages Authorization tokens which will be sent in HTTP headers."""
-  def __init__(self, scoped_tokens=None):
-    self._tokens = scoped_tokens or {}
-
-  def add_token(self, token):
-    """Adds a new token to the store (replaces tokens with the same scope).
-
-    Args:
-      token: A subclass of http_interface.GenericToken. The token object is 
-          responsible for adding the Authorization header to the HTTP request.
-          The scopes defined in the token are used to determine if the token
-          is valid for a requested scope when find_token is called.
-
-    Returns:
-      True if the token was added, False if the token was not added becase
-      no scopes were provided.
-    """
-    if not hasattr(token, 'scopes') or not token.scopes:
-      return False
-
-    for scope in token.scopes:
-      self._tokens[str(scope)] = token
-    return True  
-
-  def find_token(self, url):
-    """Selects an Authorization header token which can be used for the URL.
-
-    Args:
-      url: str or atom.url.Url or a list containing the same.
-          The URL which is going to be requested. All
-          tokens are examined to see if any scopes begin match the beginning
-          of the URL. The first match found is returned.
-
-    Returns:
-      The token object which should execute the HTTP request. If there was
-      no token for the url (the url did not begin with any of the token
-      scopes available), then the atom.http_interface.GenericToken will be 
-      returned because the GenericToken calls through to the http client
-      without adding an Authorization header.
-    """
-    if url is None:
-      return None
-    if isinstance(url, (str, unicode)):
-      url = atom.url.parse_url(url)
-    if url in self._tokens:
-      token = self._tokens[url]
-      if token.valid_for_scope(url):
-        return token
-      else:
-        del self._tokens[url]
-    for scope, token in self._tokens.iteritems():
-      if token.valid_for_scope(url):
-        return token
-    return atom.http_interface.GenericToken()
-
-  def remove_token(self, token):
-    """Removes the token from the token_store.
-
-    This method is used when a token is determined to be invalid. If the
-    token was found by find_token, but resulted in a 401 or 403 error stating
-    that the token was invlid, then the token should be removed to prevent
-    future use.
-
-    Returns:
-      True if a token was found and then removed from the token
-      store. False if the token was not in the TokenStore.
-    """
-    token_found = False
-    scopes_to_delete = []
-    for scope, stored_token in self._tokens.iteritems():
-      if stored_token == token:
-        scopes_to_delete.append(scope)
-        token_found = True
-    for scope in scopes_to_delete:
-      del self._tokens[scope]
-    return token_found
-
-  def remove_all_tokens(self):
-    self._tokens = {} 

atom/url.py

@@ -1,139 +0,0 @@
-#!/usr/bin/python
-#
-# Copyright (C) 2008 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-__author__ = 'api.jscudder (Jeff Scudder)'
-
-
-import urlparse
-import urllib
-
-
-DEFAULT_PROTOCOL = 'http'
-DEFAULT_PORT = 80
-
-
-def parse_url(url_string):
-  """Creates a Url object which corresponds to the URL string.
-  
-  This method can accept partial URLs, but it will leave missing
-  members of the Url unset.
-  """
-  parts = urlparse.urlparse(url_string)
-  url = Url()
-  if parts[0]:
-    url.protocol = parts[0]
-  if parts[1]:
-    host_parts = parts[1].split(':')
-    if host_parts[0]:
-      url.host = host_parts[0]
-    if len(host_parts) > 1:
-      url.port = host_parts[1]
-  if parts[2]:
-    url.path = parts[2]
-  if parts[4]:
-    param_pairs = parts[4].split('&')
-    for pair in param_pairs:
-      pair_parts = pair.split('=')
-      if len(pair_parts) > 1:
-        url.params[urllib.unquote_plus(pair_parts[0])] = (
-            urllib.unquote_plus(pair_parts[1]))
-      elif len(pair_parts) == 1:
-        url.params[urllib.unquote_plus(pair_parts[0])] = None
-  return url
-   
-class Url(object):
-  """Represents a URL and implements comparison logic.
-  
-  URL strings which are not identical can still be equivalent, so this object
-  provides a better interface for comparing and manipulating URLs than 
-  strings. URL parameters are represented as a dictionary of strings, and
-  defaults are used for the protocol (http) and port (80) if not provided.
-  """
-  def __init__(self, protocol=None, host=None, port=None, path=None, 
-               params=None):
-    self.protocol = protocol
-    self.host = host
-    self.port = port
-    self.path = path
-    self.params = params or {}
-
-  def to_string(self):
-    url_parts = ['', '', '', '', '', '']
-    if self.protocol:
-      url_parts[0] = self.protocol
-    if self.host:
-      if self.port:
-        url_parts[1] = ':'.join((self.host, str(self.port)))
-      else:
-        url_parts[1] = self.host
-    if self.path:
-      url_parts[2] = self.path
-    if self.params:
-      url_parts[4] = self.get_param_string()
-    return urlparse.urlunparse(url_parts)
-
-  def get_param_string(self):
-    param_pairs = []
-    for key, value in self.params.iteritems():
-      param_pairs.append('='.join((urllib.quote_plus(key), 
-          urllib.quote_plus(str(value)))))
-    return '&'.join(param_pairs)
-
-  def get_request_uri(self):
-    """Returns the path with the parameters escaped and appended."""
-    param_string = self.get_param_string()
-    if param_string:
-      return '?'.join([self.path, param_string])
-    else:
-      return self.path
-
-  def __cmp__(self, other):
-    if not isinstance(other, Url):
-      return cmp(self.to_string(), str(other))
-    difference = 0
-    # Compare the protocol
-    if self.protocol and other.protocol:
-      difference = cmp(self.protocol, other.protocol)
-    elif self.protocol and not other.protocol:
-      difference = cmp(self.protocol, DEFAULT_PROTOCOL)
-    elif not self.protocol and other.protocol:
-      difference = cmp(DEFAULT_PROTOCOL, other.protocol)
-    if difference != 0:
-      return difference
-    # Compare the host
-    difference = cmp(self.host, other.host)
-    if difference != 0:
-      return difference
-    # Compare the port
-    if self.port and other.port:
-      difference = cmp(self.port, other.port)
-    elif self.port and not other.port:
-      difference = cmp(self.port, DEFAULT_PORT)
-    elif not self.port and other.port:
-      difference = cmp(DEFAULT_PORT, other.port)
-    if difference != 0:
-      return difference
-    # Compare the path
-    difference = cmp(self.path, other.path)
-    if difference != 0:
-      return difference
-    # Compare the parameters
-    return cmp(self.params, other.params)
-
-  def __str__(self):
-    return self.to_string()
-    

build.bat

@@ -1,23 +0,0 @@
-rmdir /q /s gam
-rmdir /q /s gam-64
-rmdir /q /s python-src-%1
-rmdir /q /s build
-rmdir /q /s dist
-del /q /f gam-%1-python-src.zip
-del /q /f gam-%1-windows.zip
-del /q /f gam-%1-windows-x64.zip
-
-\python27-32\python.exe setup.py py2exe
-xcopy LICENSE gam\
-xcopy whatsnew.txt gam\
-xcopy cacert.pem gam\
-xcopy admin-settings-v1.json gam\
-del gam\w9xpopen.exe
-"%ProgramFiles(x86)%\7-Zip\7z.exe" a -tzip gam-%1-windows.zip gam\ -xr!.svn
-
-\python27\python.exe setup-64.py py2exe
-xcopy LICENSE gam-64\
-xcopy whatsnew.txt gam-64\
-xcopy cacert.pem gam-64\
-xcopy admin-settings-v1.json gam-64\
-"%ProgramFiles(x86)%\7-Zip\7z.exe" a -tzip gam-%1-windows-x64.zip gam-64\ -xr!.svn

docs/Addresses.md

@@ -0,0 +1,30 @@
+# Addresses
+- [API documentation](#api-documentation)
+- [Display addresses](#display-addresses)
+
+## API documentation
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/domains
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/groups
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/resources.calendars
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/users
+
+## Display addresses
+Produces a three column CSV file (headers Type, Email, Target) that displays all group and user primary
+email addresses and aliases; resource calendar addresses and domain names.
+
+The types are:
+```
+DomainPrimary, DomainSecondary, DomainAlias
+Group, GroupAlias, GroupNEAlias
+Resource
+SuspendedUser, SuspendedUserAlias, SuspendedUserNEAlias
+User, UserAlias, UserNEAlias
+```
+'NE' is an abbreviation for NonEditable.
+```
+gam print addresses [todrive <ToDriveAttribute>*]
+        [domain <DomainName>]
+```
+By default, groups and users in all domains in the account are selected; this options allows selection of subsets of groups and users:
+* `domain <DomainName>` - Limit groups and users to those in `<DomainName>`
+

docs/Administrators.md

@@ -0,0 +1,905 @@
+# Administrators
+- [Administrator roles documentation](#administrator-roles-documentation)
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Display administrative privileges](#display-administrative-privileges)
+- [Manage administrative roles](#manage-administrative-roles)
+- [Display administrative roles](#display-administrative-roles)
+- [Create an administrator](#create-an-administrator)
+- [Delete an administrator](#delete-an-administrator)
+- [Display administrators](#display-administrators)
+- [Copy roles from one administrator to another](#copy-roles-from-one-administrator-to-another)
+
+## Administrator roles documentation
+* https://support.google.com/a/answer/33325?ref_topic=4514341
+
+## API documentation
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/privileges
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/roles
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/roleAssignments
+
+## Definitions
+```
+<DomainName> ::= <String>(.<String>)+
+<EmailAddress> ::= <String>@<DomainName>
+<GroupItem> ::= <EmailAddress>|<UniqueID>|<String>
+<OrgUnitID> ::= id:<String>
+<OrgUnitPath> ::= /|(/<String)+
+<OrgUnitItem> ::= <OrgUnitID>|<OrgUnitPath>
+<Privilege> ::= <String>
+<PrivilegeList> ::= "<Privilege>(,<Privilege)*"
+<RoleAssignmentID> ::= <String>
+<RoleItem> ::= id:<String>|uid:<String>|<String>
+<UniqueID> ::= id:<String>
+<UserItem> ::= <EmailAddress>|<UniqueID>|<String>
+```
+## Display administrative privileges
+```
+gam print privileges [todrive <ToDriveAttribute>*]
+gam show privileges
+```
+
+Here is the output from `gam show privileges`; use this to find `<Privilege>`.
+```
+Show 91 Privileges
+  Privilege: MANAGE_CSE_SETTINGS (1/91)
+    serviceId: 02pta16n4hxgyp2
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: MANAGE_PLAY_FOR_WORK_STORE (2/91)
+    serviceId: 00tyjcwt49hs5nq
+    serviceName: play_for_work
+    isOuScopable: False
+  Privilege: MANAGE_ENTERPRISE_PRIVATE_APPS (3/91)
+    serviceId: 00tyjcwt49hs5nq
+    serviceName: play_for_work
+    isOuScopable: False
+  Privilege: MANAGE_EXTERNALLY_HOSTED_APK_UPLOAD_IN_PLAY (4/91)
+    serviceId: 00tyjcwt49hs5nq
+    serviceName: play_for_work
+    isOuScopable: False
+  Privilege: MANAGE_PLAY_FOR_WORK_STORE (5/91)
+    serviceId: 02w5ecyt3pkeyqi
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: MANAGE_ENTERPRISE_PRIVATE_APPS (6/91)
+    serviceId: 02w5ecyt3pkeyqi
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: MANAGE_EXTERNALLY_HOSTED_APK_UPLOAD_IN_PLAY (7/91)
+    serviceId: 02w5ecyt3pkeyqi
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: APP_ADMIN (8/91)
+    serviceId: 01ci93xb43sd8me
+    serviceName: Unknown
+    isOuScopable: True
+    childPrivileges: 2
+      Privilege: DELEGATES_READ (1/2)
+        serviceId: 01ci93xb43sd8me
+        serviceName: Unknown
+        isOuScopable: True
+      Privilege: DELEGATES_WRITE (2/2)
+        serviceId: 01ci93xb43sd8me
+        serviceName: Unknown
+        isOuScopable: True
+  Privilege: APP_ADMIN (9/91)
+    serviceId: 03cqmetx3hnlpuf
+    serviceName: gplus
+    isOuScopable: False
+  Privilege: GPLUS_SQUARE_BATCH_ADD (10/91)
+    serviceId: 03cqmetx3hnlpuf
+    serviceName: gplus
+    isOuScopable: False
+  Privilege: GPLUS_CONTENT_MANAGER_PRIVILEGE (11/91)
+    serviceId: 03cqmetx3hnlpuf
+    serviceName: gplus
+    isOuScopable: False
+  Privilege: APP_ADMIN (12/91)
+    serviceId: 039kk8xu49mji9t
+    serviceName: gmail
+    isOuScopable: False
+  Privilege: ACCESS_EMAIL_LOG_SEARCH (13/91)
+    serviceId: 039kk8xu49mji9t
+    serviceName: gmail
+    isOuScopable: False
+  Privilege: ACCESS_ADMIN_QUARANTINE (14/91)
+    serviceId: 039kk8xu49mji9t
+    serviceName: gmail
+    isOuScopable: False
+  Privilege: ACCESS_RESTRICTED_QUARANTINE (15/91)
+    serviceId: 039kk8xu49mji9t
+    serviceName: gmail
+    isOuScopable: False
+  Privilege: APP_ADMIN (16/91)
+    serviceId: 01tuee744837sjz
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: MANAGE_COURSE_SETTINGS (17/91)
+    serviceId: 037m2jsg4g9nirj
+    serviceName: Unknown
+    isOuScopable: True
+  Privilege: MANAGE_LTI_CREDENTIAL_MANAGEMENT_MODE (18/91)
+    serviceId: 037m2jsg4g9nirj
+    serviceName: Unknown
+    isOuScopable: True
+  Privilege: APP_ADMIN (19/91)
+    serviceId: 01baon6m1wv6b0p
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: APP_ADMIN (20/91)
+    serviceId: 01yyy98l4k9lq4l
+    serviceName: directory
+    isOuScopable: False
+    childPrivileges: 3
+      Privilege: DIRECTORY_SETTINGS_READONLY (1/3)
+        serviceId: 01yyy98l4k9lq4l
+        serviceName: directory
+        isOuScopable: False
+        childPrivileges: 2
+          Privilege: PROFILE_EDITABILITY_READONLY (1/2)
+            serviceId: 01yyy98l4k9lq4l
+            serviceName: directory
+            isOuScopable: False
+          Privilege: CUSTOM_DIRECTORY_READONLY (2/2)
+            serviceId: 01yyy98l4k9lq4l
+            serviceName: directory
+            isOuScopable: False
+      Privilege: PROFILE_EDITABILITY_READWRITE (2/3)
+        serviceId: 01yyy98l4k9lq4l
+        serviceName: directory
+        isOuScopable: False
+      Privilege: CUSTOM_DIRECTORY_READWRITE (3/3)
+        serviceId: 01yyy98l4k9lq4l
+        serviceName: directory
+        isOuScopable: False
+  Privilege: LDAP_MANAGER (21/91)
+    serviceId: 02lwamvv18la4iw
+    serviceName: ldap
+    isOuScopable: False
+  Privilege: LDAP_PASSWORD_REBIND (22/91)
+    serviceId: 02lwamvv18la4iw
+    serviceName: ldap
+    isOuScopable: True
+    childPrivileges: 1
+      Privilege: LDAP_PASSWORD_REBIND_READONLY
+        serviceId: 02lwamvv18la4iw
+        serviceName: ldap
+        isOuScopable: True
+  Privilege: APP_ADMIN (23/91)
+    serviceId: 0319y80a15kueje
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: APP_ADMIN (24/91)
+    serviceId: 044sinio4cntx2o
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: APP_ADMIN (25/91)
+    serviceId: 01ksv4uv2d2noaq
+    serviceName: sites
+    isOuScopable: False
+  Privilege: ADMIN_DASHBOARD (26/91)
+    serviceId: 01ci93xb3tmzyin
+    serviceName: admin
+    isOuScopable: True
+  Privilege: SERVICES (27/91)
+    serviceId: 01ci93xb3tmzyin
+    serviceName: admin
+    isOuScopable: False
+  Privilege: SECURITY_SETTINGS (28/91)
+    serviceId: 01ci93xb3tmzyin
+    serviceName: admin
+    isOuScopable: False
+  Privilege: SUPPORT (29/91)
+    serviceId: 01ci93xb3tmzyin
+    serviceName: admin
+    isOuScopable: False
+  Privilege: ADMIN_DOMAIN_SETTINGS (30/91)
+    serviceId: 01ci93xb3tmzyin
+    serviceName: admin
+    isOuScopable: False
+  Privilege: REPORTS (31/91)
+    serviceId: 01ci93xb3tmzyin
+    serviceName: admin
+    isOuScopable: False
+  Privilege: ADMIN_DASHBOARD (32/91)
+    serviceId: 01ci93xb3tmzyin
+    serviceName: admin
+    isOuScopable: True
+  Privilege: SERVICES (33/91)
+    serviceId: 01ci93xb3tmzyin
+    serviceName: admin
+    isOuScopable: False
+  Privilege: SUPPORT (34/91)
+    serviceId: 01ci93xb3tmzyin
+    serviceName: admin
+    isOuScopable: False
+  Privilege: REPORTS (35/91)
+    serviceId: 01ci93xb3tmzyin
+    serviceName: admin
+    isOuScopable: False
+  Privilege: APP_ADMIN (36/91)
+    serviceId: 03fwokq01e2ht7x
+    serviceName: Unknown
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: UDM_NETWORK_ADMIN
+        serviceId: 03fwokq01e2ht7x
+        serviceName: Unknown
+        isOuScopable: True
+  Privilege: ADMIN_MATTER (37/91)
+    serviceId: 03l18frh45c63dw
+    serviceName: vault
+    isOuScopable: True
+  Privilege: REMOVE_HOLD (38/91)
+    serviceId: 03l18frh45c63dw
+    serviceName: vault
+    isOuScopable: True
+  Privilege: MANAGE_SEARCHES (39/91)
+    serviceId: 03l18frh45c63dw
+    serviceName: vault
+    isOuScopable: True
+  Privilege: MANAGE_EXPORTS (40/91)
+    serviceId: 03l18frh45c63dw
+    serviceName: vault
+    isOuScopable: True
+  Privilege: MANAGE_RETENTION_POLICY (41/91)
+    serviceId: 03l18frh45c63dw
+    serviceName: vault
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: VIEW_RETENTION_POLICY
+        serviceId: 03l18frh45c63dw
+        serviceName: vault
+        isOuScopable: False
+  Privilege: AUDIT_SYSTEM (42/91)
+    serviceId: 03l18frh45c63dw
+    serviceName: vault
+    isOuScopable: False
+  Privilege: ACCESS_ALL_MATTERS (43/91)
+    serviceId: 03l18frh45c63dw
+    serviceName: vault
+    isOuScopable: False
+  Privilege: APP_ADMIN (44/91)
+    serviceId: 02afmg282jiquyg
+    serviceName: device_management
+    isOuScopable: False
+  Privilege: APP_ADMIN (45/91)
+    serviceId: 037m2jsg3ckz96v
+    serviceName: calendar
+    isOuScopable: False
+    childPrivileges: 2
+      Privilege: CALENDAR_SETTINGS (1/2)
+        serviceId: 037m2jsg3ckz96v
+        serviceName: calendar
+        isOuScopable: False
+        childPrivileges: 1
+          Privilege: CALENDAR_SETTINGS_READ
+            serviceId: 037m2jsg3ckz96v
+            serviceName: calendar
+            isOuScopable: False
+      Privilege: CALENDAR_RESOURCE (2/2)
+        serviceId: 037m2jsg3ckz96v
+        serviceName: calendar
+        isOuScopable: False
+        childPrivileges: 2
+          Privilege: ROOM_INSIGHTS_DASHBOARD_ACCESS (1/2)
+            serviceId: 037m2jsg3ckz96v
+            serviceName: calendar
+            isOuScopable: False
+          Privilege: CALENDAR_RESOURCE_MANAGE (2/2)
+            serviceId: 037m2jsg3ckz96v
+            serviceName: calendar
+            isOuScopable: False
+            childPrivileges: 1
+              Privilege: CALENDAR_RESOURCE_READ
+                serviceId: 037m2jsg3ckz96v
+                serviceName: calendar
+                isOuScopable: False
+  Privilege: APP_ADMIN (46/91)
+    serviceId: 03dy6vkm2sk0pzo
+    serviceName: docs
+    isOuScopable: False
+    childPrivileges: 5
+      Privilege: DOCS_TEMPLATE_ADMIN (1/5)
+        serviceId: 03dy6vkm2sk0pzo
+        serviceName: docs
+        isOuScopable: False
+      Privilege: MIGRATE_TO_TEAM_DRIVE (2/5)
+        serviceId: 03dy6vkm2sk0pzo
+        serviceName: docs
+        isOuScopable: False
+      Privilege: WRITE_APPS_METADATA_SCHEMAS (3/5)
+        serviceId: 03dy6vkm2sk0pzo
+        serviceName: docs
+        isOuScopable: False
+      Privilege: VIEW_SITE_DETAILS (4/5)
+        serviceId: 03dy6vkm2sk0pzo
+        serviceName: docs
+        isOuScopable: False
+      Privilege: MANAGE_CLASSIC_GOOGLE_SITES (5/5)
+        serviceId: 03dy6vkm2sk0pzo
+        serviceName: docs
+        isOuScopable: False
+  Privilege: APP_ACCESS (47/91)
+    serviceId: 03cqmetx1vygwki
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: ORGANIZATION_UNITS_ALL (48/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: True
+    childPrivileges: 4
+      Privilege: ORGANIZATION_UNITS_CREATE (1/4)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: True
+      Privilege: ORGANIZATION_UNITS_RETRIEVE (2/4)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: True
+      Privilege: ORGANIZATION_UNITS_UPDATE (3/4)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: True
+      Privilege: ORGANIZATION_UNITS_DELETE (4/4)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: True
+  Privilege: USERS_ALL (49/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: True
+    childPrivileges: 5
+      Privilege: USERS_CREATE (1/5)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: True
+      Privilege: USERS_RETRIEVE (2/5)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: True
+      Privilege: USERS_UPDATE (3/5)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: True
+        childPrivileges: 6
+          Privilege: USERS_ALIAS (1/6)
+            serviceId: 00haapch16h1ysv
+            serviceName: admin_apis
+            isOuScopable: True
+          Privilege: USERS_MOVE (2/6)
+            serviceId: 00haapch16h1ysv
+            serviceName: admin_apis
+            isOuScopable: True
+          Privilege: USERS_RESET_PASSWORD (3/6)
+            serviceId: 00haapch16h1ysv
+            serviceName: admin_apis
+            isOuScopable: True
+          Privilege: USERS_FORCE_PASSWORD_CHANGE (4/6)
+            serviceId: 00haapch16h1ysv
+            serviceName: admin_apis
+            isOuScopable: True
+          Privilege: USERS_ADD_NICKNAME (5/6)
+            serviceId: 00haapch16h1ysv
+            serviceName: admin_apis
+            isOuScopable: True
+          Privilege: USERS_SUSPEND (6/6)
+            serviceId: 00haapch16h1ysv
+            serviceName: admin_apis
+            isOuScopable: True
+      Privilege: USERS_UPDATE_CUSTOM_ATTRIBUTES_USER_PRIVILEGE_GROUP (4/5)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: True
+      Privilege: USERS_DELETE (5/5)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: True
+  Privilege: GROUPS_ALL (50/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+    childPrivileges: 4
+      Privilege: GROUPS_CREATE (1/4)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: False
+      Privilege: GROUPS_RETRIEVE (2/4)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: False
+      Privilege: GROUPS_UPDATE (3/4)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: False
+      Privilege: GROUPS_DELETE (4/4)
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: False
+  Privilege: USER_SECURITY_ALL (51/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: True
+  Privilege: DATATRANSFER_API_PRIVILEGE_GROUP (52/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+  Privilege: DOMAIN_REGISTRATION_MANAGEMENT (53/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+  Privilege: SCHEMA_MANAGEMENT (54/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: SCHEMA_RETRIEVE
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: False
+  Privilege: LICENSING (55/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: LICENSING_READ
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: False
+  Privilege: BILLING (56/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: BILLING_READ
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: False
+  Privilege: SAML2_SERVICE_PROVIDER (57/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+  Privilege: DOMAIN_MANAGEMENT (58/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+  Privilege: UPGRADE_CONSUMER_CONVERSION (59/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+  Privilege: TRUSTED_DOMAIN_WHITELIST_WRITE (60/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: TRUSTED_DOMAIN_WHITELIST_READ
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: False
+  Privilege: FULL_MIGRATION_ACCESS (61/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: EXECUTE_MIGRATION
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: False
+        childPrivileges: 1
+          Privilege: MODIFY_MIGRATION
+            serviceId: 00haapch16h1ysv
+            serviceName: admin_apis
+            isOuScopable: False
+            childPrivileges: 1
+              Privilege: VIEW_MIGRATION
+                serviceId: 00haapch16h1ysv
+                serviceName: admin_apis
+                isOuScopable: False
+  Privilege: GROUPS_MANAGE_SECURITY_LABEL (62/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+  Privilege: GROUPS_MANAGE_LOCKED_LABEL (63/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+  Privilege: ADMIN_REPORTING_ACCESS (64/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: REPORTING_AUDIT_ACCESS
+        serviceId: 00haapch16h1ysv
+        serviceName: admin_apis
+        isOuScopable: False
+  Privilege: SUPPORT_PRIVILEGE_GROUP (65/91)
+    serviceId: 00haapch16h1ysv
+    serviceName: admin_apis
+    isOuScopable: False
+  Privilege: APPS_INCIDENTS_FULL_ACCESS (66/91)
+    serviceId: 02pta16n3efhw69
+    serviceName: Unknown
+    isOuScopable: False
+    childPrivileges: 2
+      Privilege: APPS_INCIDENTS_READONLY (1/2)
+        serviceId: 02pta16n3efhw69
+        serviceName: Unknown
+        isOuScopable: False
+      Privilege: APPS_INCIDENTS_VIEW_VIRUSTOTAL_REPORTS (2/2)
+        serviceId: 02pta16n3efhw69
+        serviceName: Unknown
+        isOuScopable: False
+  Privilege: APP_ADMIN (67/91)
+    serviceId: 019c6y1840fzfkt
+    serviceName: classroom
+    isOuScopable: True
+  Privilege: ADMIN_OVERSIGHT_MANAGE_CLASSES (68/91)
+    serviceId: 019c6y1840fzfkt
+    serviceName: classroom
+    isOuScopable: True
+  Privilege: EDU_ANALYTICS_DATA_ACCESS (69/91)
+    serviceId: 019c6y1840fzfkt
+    serviceName: classroom
+    isOuScopable: True
+  Privilege: APP_ADMIN (70/91)
+    serviceId: 037m2jsg46www3g
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: MANAGE_DYNAMITE_SETTINGS (71/91)
+    serviceId: 03whwml44f3n4vd
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: MODERATE_DYNAMITE_REPORT (72/91)
+    serviceId: 03whwml44f3n4vd
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: MANAGE_DYNAMITE_SPACES (73/91)
+    serviceId: 03whwml44f3n4vd
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: APP_ADMIN (74/91)
+    serviceId: 03hv69ve4bjwe54
+    serviceName: Unknown
+    isOuScopable: True
+    childPrivileges: 6
+      Privilege: MANAGE_CHROME_USER_SETTINGS (1/6)
+        serviceId: 03hv69ve4bjwe54
+        serviceName: Unknown
+        isOuScopable: True
+        childPrivileges: 2
+          Privilege: MANAGE_CHROME_APPLICATION_SETTINGS (1/2)
+            serviceId: 03hv69ve4bjwe54
+            serviceName: Unknown
+            isOuScopable: True
+          Privilege: MANAGE_CHROME_WEB_SETTINGS (2/2)
+            serviceId: 03hv69ve4bjwe54
+            serviceName: Unknown
+            isOuScopable: True
+      Privilege: MANAGE_CHROME_BROWSERS (2/6)
+        serviceId: 03hv69ve4bjwe54
+        serviceName: Unknown
+        isOuScopable: True
+        childPrivileges: 1
+          Privilege: MANAGED_CHROME_BROWSERS_READ_ONLY
+            serviceId: 03hv69ve4bjwe54
+            serviceName: Unknown
+            isOuScopable: True
+      Privilege: VIEW_CHROME_REPORTS (3/6)
+        serviceId: 03hv69ve4bjwe54
+        serviceName: Unknown
+        isOuScopable: True
+        childPrivileges: 4
+          Privilege: VIEW_CHROME_EXTENSIONS_REPORT (1/4)
+            serviceId: 03hv69ve4bjwe54
+            serviceName: Unknown
+            isOuScopable: True
+          Privilege: VIEW_CHROME_VERSION_REPORT (2/4)
+            serviceId: 03hv69ve4bjwe54
+            serviceName: Unknown
+            isOuScopable: True
+          Privilege: VIEW_CHROME_INSIGHTS_REPORT (3/4)
+            serviceId: 03hv69ve4bjwe54
+            serviceName: Unknown
+            isOuScopable: True
+          Privilege: VIEW_CHROME_PRINTERS_REPORT (4/4)
+            serviceId: 03hv69ve4bjwe54
+            serviceName: Unknown
+            isOuScopable: True
+      Privilege: MANAGE_PRINTERS (4/6)
+        serviceId: 03hv69ve4bjwe54
+        serviceName: Unknown
+        isOuScopable: True
+      Privilege: MANAGE_DEVICES (5/6)
+        serviceId: 03hv69ve4bjwe54
+        serviceName: Unknown
+        isOuScopable: True
+        childPrivileges: 2
+          Privilege: MANAGE_DEVICES_READ_ONLY (1/2)
+            serviceId: 03hv69ve4bjwe54
+            serviceName: Unknown
+            isOuScopable: True
+            childPrivileges: 1
+              Privilege: TELEMETRY_API
+                serviceId: 03hv69ve4bjwe54
+                serviceName: Unknown
+                isOuScopable: True
+                childPrivileges: 19
+                  Privilege: TELEMETRY_API_DEVICE (1/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_USER (2/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_AUDIO_REPORT (3/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_BUS_DEVICE_INFO (4/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_OS_REPORT (5/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_CPU_INFO (6/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_CPU_REPORT (7/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_MEMORY_INFO (8/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_MEMORY_REPORT (9/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_GRAPHICS_INFO (10/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_GRAPHICS_REPORT (11/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_BATTERY_INFO (12/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_BATTERY_REPORT (13/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_STORAGE_INFO (14/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_STORAGE_REPORT (15/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_NETWORK_INFO (16/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_NETWORK_REPORT (17/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_DEVICE_ACTIVITY_REPORT (18/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+                  Privilege: TELEMETRY_API_PERIPHERALS_REPORT (19/19)
+                    serviceId: 03hv69ve4bjwe54
+                    serviceName: Unknown
+                    isOuScopable: True
+          Privilege: DEVICE_ACTION_CRD (2/2)
+            serviceId: 03hv69ve4bjwe54
+            serviceName: Unknown
+            isOuScopable: True
+      Privilege: MANAGE_DEVICE_SETTINGS (6/6)
+        serviceId: 03hv69ve4bjwe54
+        serviceName: Unknown
+        isOuScopable: True
+  Privilege: SERVICE_DATA_DOWNLOADER (75/91)
+    serviceId: 03hv69ve4bjwe54
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: MANAGE_DIRECTORY_SYNC_SETTINGS (76/91)
+    serviceId: 0147n2zr1ynkkmf
+    serviceName: Unknown
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: READ_DIRECTORY_SYNC_SETTINGS
+        serviceId: 0147n2zr1ynkkmf
+        serviceName: Unknown
+        isOuScopable: False
+  Privilege: APP_ADMIN (77/91)
+    serviceId: 0279ka651l5iy5q
+    serviceName: Unknown
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: ADMIN_QUALITY_DASHBOARD_ACCESS
+        serviceId: 0279ka651l5iy5q
+        serviceName: Unknown
+        isOuScopable: False
+  Privilege: SECURITY_SETTINGS (78/91)
+    serviceId: 00vx122734tbite
+    serviceName: Unknown
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: INBOUND_SSO_SETTINGS
+        serviceId: 00vx122734tbite
+        serviceName: Unknown
+        isOuScopable: False
+  Privilege: VIEW_DLP_RULE (79/91)
+    serviceId: 02250f4o3hg8pg8
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: MANAGE_DLP_RULE (80/91)
+    serviceId: 02250f4o3hg8pg8
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: APP_ADMIN (81/91)
+    serviceId: 00nmf14n14wtgcf
+    serviceName: app_maker
+    isOuScopable: False
+  Privilege: VIEW_ALL_PROJECTS (82/91)
+    serviceId: 00nmf14n14wtgcf
+    serviceName: app_maker
+    isOuScopable: False
+  Privilege: APP_ADMIN (83/91)
+    serviceId: 02zbgiuw2wdxo5p
+    serviceName: youtube
+    isOuScopable: False
+  Privilege: APP_ADMIN (84/91)
+    serviceId: 03as4poj2zjehv7
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: APP_ADMIN (85/91)
+    serviceId: 02afmg283v5nmx6
+    serviceName: Unknown
+    isOuScopable: False
+    childPrivileges: 1
+      Privilege: ADMIN_QUALITY_DASHBOARD_ACCESS
+        serviceId: 02afmg283v5nmx6
+        serviceName: Unknown
+        isOuScopable: False
+  Privilege: APP_ADMIN (86/91)
+    serviceId: 00upglbi0qz687j
+    serviceName: takeout
+    isOuScopable: False
+  Privilege: CLOUD_PRINT_MANAGER (87/91)
+    serviceId: 02bn6wsx379ol8g
+    serviceName: cloud_print
+    isOuScopable: False
+  Privilege: MANAGE_AGE_BASED_ACCESS_SETTINGS_AGE_LABEL (88/91)
+    serviceId: 046r0co22dnadsi
+    serviceName: Unknown
+    isOuScopable: True
+    childPrivileges: 1
+      Privilege: AGE_BASED_ACCESS_SETTINGS_AGE_LABEL_READ
+        serviceId: 046r0co22dnadsi
+        serviceName: Unknown
+        isOuScopable: True
+  Privilege: LOGO_PRIVILEGE_GROUP (89/91)
+    serviceId: 03j2qqm31d4j55e
+    serviceName: Unknown
+    isOuScopable: False
+  Privilege: APP_ADMIN (90/91)
+    serviceId: 04f1mdlm0ki64aw
+    serviceName: cros
+    isOuScopable: True
+    childPrivileges: 7
+      Privilege: MANAGE_DEVICES (1/7)
+        serviceId: 04f1mdlm0ki64aw
+        serviceName: cros
+        isOuScopable: True
+      Privilege: MANAGE_USER_SETTINGS (2/7)
+        serviceId: 04f1mdlm0ki64aw
+        serviceName: cros
+        isOuScopable: True
+        childPrivileges: 1
+          Privilege: MANAGE_APPLICATION_SETTINGS
+            serviceId: 04f1mdlm0ki64aw
+            serviceName: cros
+            isOuScopable: True
+      Privilege: MANAGE_DEVICE_SETTINGS (3/7)
+        serviceId: 04f1mdlm0ki64aw
+        serviceName: cros
+        isOuScopable: True
+      Privilege: MANAGE_BROWSERS (4/7)
+        serviceId: 04f1mdlm0ki64aw
+        serviceName: cros
+        isOuScopable: True
+      Privilege: VIEW_EXTENSIONS_REPORT (5/7)
+        serviceId: 04f1mdlm0ki64aw
+        serviceName: cros
+        isOuScopable: True
+      Privilege: VIEW_VERSION_REPORT (6/7)
+        serviceId: 04f1mdlm0ki64aw
+        serviceName: cros
+        isOuScopable: True
+      Privilege: MANAGE_PRINTERS (7/7)
+        serviceId: 04f1mdlm0ki64aw
+        serviceName: cros
+        isOuScopable: True
+  Privilege: APP_ADMIN (91/91)
+    serviceId: 02et92p02l9sq0n
+    serviceName: Unknown
+    isOuScopable: True
+```
+
+## Manage administrative roles
+```
+gam create adminrole <String> privileges all|all_ou|<PrivilegeList> [description <String>]
+gam update adminrole <RoleItem> [name <String>] [privileges all|all_ou|<PrivilegeList>] [description <String>]
+gam delete adminrole <RoleItem>
+```
+* `privileges all` - All defined privileges
+* `privileges all_ou` - All defined privileges than can be scoped to an OU
+* `privileges <PrivilegeList>` - A specific list of privileges
+
+## Display administrative roles
+```
+gam info adminrole <RoleItem> [privileges]
+gam print adminroles|roles [todrive <ToDriveAttribute>*] [privileges]
+gam show adminroles|roles [todrive <ToDriveAttribute>*] [privileges]
+```
+* `privileges` - Display privileges associated with each role
+
+## Create an administrator
+Add an administrator role to an administrator.
+```
+gam create admin <EmailAddress>|<UniqueID> <RoleItem> customer|(org_unit <OrgUnitItem>)
+        [condition securitygroup|nonsecuritygroup]
+```
+* `customer` - The administrator can manage all organization units
+* `org_unit <OrgUnitItem>` - The administrator can manage the specified organization unit
+
+The option `condition` limits the conditions for delegate admin access. This currently only works with the _GROUPS_EDITOR_ROLE and _GROUPS_READER_ROLE roles.
+* `condition securitygroup` - limit the delegated admin to managing security groups
+* `condition nonsecuritygroup` - limit the delegated admin to managing non-security groups
+
+## Delete an administrator
+Remove an administrator role from an administrator.
+```
+gam delete admin <RoleAssignmentId>
+```
+## Display administrators
+```
+gam print admins [todrive <ToDriveAttribute>*]
+        [user|group <EmailAddress>|<UniqueID>] [role <RoleItem>] [condition] [privileges]
+gam show admins
+        [user|group <EmailAddress>|<UniqueID>] [role <RoleItem>] [condition] [privileges]
+```
+By default, all administrators and roles are displayed; choose from the following
+options to limit the display:
+* `user <UserItem>` -  Display only this administrator
+* `role <RoleItem>` - Display only administrators with this role
+
+* `condition` - Display any conditions associated with a role assignment
+* `privileges` - Display privileges associated with each role assignment
+
+In versions prior to 6.07.01, specification of both `user <UserItem>`
+and `role <RoleItem>` generated no output due to an undocumented API rule that disallows both.
+
+## Copy roles from one administrator to another
+Get roles for current admin.
+```
+gam redirect csv ./CurrentAdminRoles.csv print admins user currentadmin@domain.com
+```
+Add roles to new admin.
+```
+gam config csv_input_row_filter "scopeType:regex:CUSTOMER" redirect stdout ./UpdateNewAdminCustomerRoles.txt multiprocess redirect stderr stdout csv CurrentAdminRoles.csv gam create admin newadmin@domain.com "id:~~roleId~~" customer
+gam config csv_input_row_filter "scopeType:regex:ORG_UNIT" redirect stdout ./UpdateNewAdminOrgUnitRoles.txt multiprocess redirect stderr stdout csv CurrentAdminRoles.csv gam create admin newadmin@domain.com "id:~~roleId~~" org_unit "id:~~orgUnitId~~"
+```
+

docs/Alert-Center.md

@@ -0,0 +1,94 @@
+# Alert Center
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Introduction](#introduction)
+- [Manage alerts](#manage-alerts)
+- [Display alerts](#display-alerts)
+- [Manage alert feedback](#manage-alert-feedback)
+- [Display alert feedback](#display-alert-feedback)
+
+## API documentation
+* https://developers.google.com/admin-sdk/alertcenter/reference/rest/
+* https://developers.google.com/admin-sdk/alertcenter/guides/query-filters
+* https://developers.google.com/admin-sdk/alertcenter/reference/filter-fields
+
+## Definitions
+```
+<AlertID> ::= <String>
+<QueryAlert> ::= <String> See: https://developers.google.com/admin-sdk/alertcenter/guides/query-filters
+```
+## Introduction
+For an introduction, start here: https://support.google.com/a/answer/9105393
+
+This API is in beta, most things seem to work although the filter queries don't all work, in particular those that
+select alertId and feedbackId.
+
+To use these commands you must update your gam project and service account authorization.
+```
+gam update project
+gam user user@domain.com check serviceaccount
+```
+## Manage alerts
+```
+gam delete alert <AlertID>
+gam undelete alert <AlertID>
+```
+## Display alerts
+```
+gam info alert <AlertID> [formatjson]
+gam show alerts [filter <QueryAlert>] [orderby createtime [ascending|descending]]
+        [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print alerts [todrive <ToDriveAttributes>*] [filter <QueryAlert>] [orderby createtime [ascending|descending]]
+        [formatjson [quotechar <Character>]]
+```
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+### Eliminate unwanted fields
+You can use [CSV Print Filtering](CSV-Print-Filtering) to reduce the amount of output.
+This command will drop all of the data.messages columns.
+```
+gam config csv_output_header_drop_filter "^data.messages" redirect csv alerts.csv print alerts
+```
+
+## Manage alert feedback
+```
+gam create alertfeedback <AlertID> not_useful|somewhat_useful|very_useful
+```
+## Display alert feedback
+```
+gam show alertfeedback [alert <AlertID>] [filter <QueryAlert>] [orderby createtime [ascending|descending]]
+        [formatjson]
+```
+By default, Gam displays feedback for all alerts.
+* `alert <AlertID>` - Display feedback for the selected alert
+* `filter <QueryAlert>` - Display feebback for the filtered alerts
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print alertfeedback [todrive <ToDriveAttributes>*] [alert <AlertID>] [filter <QueryAlert>] [orderby createtime [ascending|descending]]
+        [formatjson [quotechar <Character>]]
+```
+By default, Gam displays feedback for all alerts.
+* `alert <AlertID>` - Display feedback for the selected alert
+* `filter <QueryAlert>` - Display feebback for the filtered alerts
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Aliases.md

@@ -0,0 +1,194 @@
+# Aliases
+- [API documentation](#api-documentation)
+- [Query documentation](#query-documentation)
+- [Python Regular Expressions](Python-Regular-Expressions) Match function
+- [Definitions](#definitions)
+- [Create an alias for a target](#create-an-alias-for-a-target)
+- [Update an alias to point to a new target](#update-an-alias-to-point-to-a-new-target)
+- [Delete an alias regardless of the target](#delete-an-alias-regardless-of-the-target)
+- [Remove aliases from a specified target](#remove-aliases-from-a-specified-target)
+- [Delete all of a user's aliases](#delete-all-of-a-users-aliases)
+- [Display aliases](#display-aliases)
+- [Bulk delete aliases](#bulk-delete-aliases)
+- [Bulk reassign aliases](#bulk-reassign-aliases)
+- [Determine if an address is a user, user alias, group or group alias](#determine-if-an-address-is-a-user-user-alias-group-or-group-alias)
+
+## API documentation
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/users.aliases
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/groups.aliases
+
+## Query documentation
+* https://developers.google.com/admin-sdk/directory/v1/guides/search-users
+
+## Definitions
+See [Collections of Items](Collections-of-Items)
+```
+<DomainName> ::= <String>(.<String>)+
+<DomainNameList> ::= "<DomainName>(,<DomainName>)*"
+<DomainNameEntity> ::=
+        <DomainNameList> | <FileSelector> | <CSVFileSelector>
+<EmailAddress> ::= <String>@<DomainName>
+<EmailAddressList> ::= "<EmailAddress>(,<EmailAddress>)*"
+<EmailAddressEntity> ::= <EmailAddressList> | <FileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<UniqueID> ::= id:<String>
+```
+## Create an alias for a target
+```
+gam create alias|aliases <EmailAddressEntity> user|group|target <UniqueID>|<EmailAddress>
+        [verifynotinvitable]
+```
+`<EmailAddressEntity>` are the aliases, `<EmailAddress>` is the target.
+
+The `verifynotinvitable` option causes GAM to verify that the alias email address being created is not that of an unmanaged account;
+if it is, the command is not performed.
+
+### Example
+
+To allow Robert to also receive mail as Bob:
+
+```
+gam create alias bob[@yourdomain.com] user robert[@yourdomain.com]
+```
+
+## Update an alias to point to a new target
+The existing alias is deleted and a new alias is created.
+```
+gam update alias|aliases <EmailAddressEntity> user|group|target <UniqueID>|<EmailAddress>
+        [notargetverify] [waitafterdelete <Integer>]
+```
+`<EmailAddressEntity>` are the aliases, `<EmailAddress>` is the target.
+
+By default, GAM makes additional API calls to verify that the target email address exists before updating the alias;
+if you know that the target exists, you can suppress the verification with `notargetverify.
+
+GAM updates an alias to point to a new target by deleting the alias and then recreates the alias pointing to the new target.
+Unfortunately, if these commands are executed back-to-back; Google generates the `Update Failed: Duplicate` error.
+Now, GAM waits 2 seconds between the delete and the insert which seems to eliminate the problem. If the problem persists,
+use the option `waitafterdelete <Integer>` to increase the wait time to a maximum of 10 seconds.
+
+## Delete an alias regardless of the target
+```
+gam delete alias|aliases [user|group|target] <EmailAddressEntity>
+```
+`<EmailAddressEntity>` are the aliases.
+
+## Remove aliases from a specified target
+```
+gam remove alias|aliases <EmailAddress> user|group <EmailAddressEntity>
+```
+`<EmailAddress>` is the target, `<EmailAddressEntity>` are the aliases.
+
+## Delete all of a user's aliases
+```
+gam <UserTypeEntity> delete aliases
+```
+
+## Display aliases
+Display a specific alias.
+```
+gam info alias|aliases <EmailAddressEntity>
+```
+
+Display selected aliases.
+```
+gam print aliases [todrive <ToDriveAttribute>*]
+        ([domain|domains <DomainNameEntity>] [(query <QueryUser>)|(queries <QueryUserList>)]
+         [limittoou <OrgUnitItem>])
+        [user|users <EmailAddressList>] [group|groups <EmailAddressList>]
+        [select <UserTypeEntity>]
+        [aliasmatchpattern <RegularExpression>]
+        [shownoneditable] [nogroups] [nousers]
+        [onerowpertarget] [delimiter <Character>]
+        [suppressnoaliasrows]
+        (addcsvdata <FieldName> <String>)*
+```
+By default, group and user aliases in all domains in the account are selected; these options allow selection of subsets of aliases:
+* `domain|domains <DomainNameEntity>` - Limit aliases to those in the domains specified by `<DomainNameEntity>`
+  * You can predefine this list with the `print_agu_domains` variable in `gam.cfg`.
+* `(query <QueryUser>)|(queries <QueryUserList>)` - Print aliases for users/groups that match a query; each query is run against each domain
+* `limittoou <OrgUnitItem>` - Print aliases for users in the specified `<OrgUnitItem>`
+* `user|users <EmailAddressList>` - Print aliases for users in `<EmailAddressList`
+* `select <UserTypeEntity>` - Print aliases for users in `<UserTypeEntity>`
+* `group|groups <EmailAddressList>` - Print aliases for groups in `<EmailAddressList`
+* `aliasmatchpattern <RegularExpression>` - Print aliases that match a pattern
+* `nogroups` - Print only user aliases
+* `nousers` - Print only group aliases
+
+By default, the CSV output has three columns: `Alias,Target,TargetType`; if a target
+has multiple aliases, there will be multiple rows, one per alias.
+
+Use `shownoneditable` to list non-editable alias email addresses; these are typically outside of the account's primary domain or subdomains.
+This adds the column `NonEditableAlias`.
+
+Specifying `onerowpertarget` changes the three columns to: `Target,TargetType,Aliases`; all aliases for the target are listed in the
+`Aliases` column. If `shownoneditable` is specified, there will be a fourth column `NonEditableAliases` with a list of non-editable aliases.
+
+By default, the aliases in a list are separated by the `csv_output_field_delimiter' from `gam.cfg`.
+* `delimiter <Character>` - Separate aliases in a list with `<Character>`
+
+Specifying both `onerowpertarget` and `suppressnoaliasrows` causes GAM to not display any targets that have no aliases.
+
+Add additional columns of data from the command line to the output
+* `addcsvdata <FieldName> <String>`
+
+When multiple domains are specified and a query/queries are specified, an API call is made for each domain/query combination.
+```
+$ gam print aliases domains school.org,students.school.org queries "'email:admin*','email:test*'"
+Getting all Users that match query (domain=school.org, query="email:admin*"), may take some time on a large Google Workspace Account...
+Got 3 Users: admin@school.org - admindirector@school.org
+Getting all Users that match query (domain=school.org, query="email:test*"), may take some time on a large Google Workspace Account...
+Got 20 Users: testusera@school.org - testuserx@school.org
+Getting all Users that match query (domain=students.school.org, query="email:admin*"), may take some time on a large Google Workspace Account...
+Got 1 User: admin@students.school.org - admin@students.school.org
+Getting all Users that match query (domain=students.school.org, query="email:test*"), may take some time on a large Google Workspace Account...
+Got 1 User: testuser1@students.school.org - testuser1@students.school.org
+Alias,Target,TargetType
+...
+```
+
+## Bulk delete aliases
+You can bulk delete aliases as follows; use `(query <QueryUser>)|(queries <QueryUserList>)` and
+`aliasmatchpattern <RegularExpression>` as desired.
+```
+gam redirect csv ./OldDomainAliases.csv print aliases aliasmatchpattern ".*@olddomain.com" onerowpertarget suppressnoaliasrows
+gam redirect stdout ./DeleteAliases.txt multiprocess redirect stderr stdout csv ./OldDomainAliases.csv gam remove aliases "~Target" "~TargetType" "~Aliases"
+```
+
+## Bulk reassign aliases
+You can bulk reassign aliases as follows. Make a CSV file ReassignAliases.csv with two columns: OldTarget,NewTarget.
+From this CSV file, all of the aliases for the users in the OldTarget column will be listed with an additional column showing the NewTarget.
+```
+gam redirect stdout ./GetAliases.txt multiprocess redirect stderr stdout redirect csv ./ReassignAliases.csv gam print aliases user "~OldTarget" addcsvdata NewTarget "~NewTarget"
+```
+If an OldTarget's aliases are to be reassigned to more than the one NewTarget, edit ReassignAliases.csv and make changes as required.
+```
+gam redirect stdout ./ReassignAliases.txt multiprocess redirect stderr stdout csv ReassignAliases.csv gam update alias "~Alias" user "~NewTarget"
+```
+
+## Determine if an address is a user, user alias, group or group alias
+```
+gam whatis <EmailItem> [noinfo] [noinvitablecheck]
+```
+The first line of output is: `<TypeOfEmailItem>: <EmailItem>`
+
+There is additional output based on `<TypeOfEmailItem>`:
+* User - `gam info user <EmailItem>`
+* Group - `gam info group <EmailItem>`
+* User Alias - `gam info alias <EmailItem>`
+* Group Alias - `gam info alias <EmailItem>`
+* User Invitation - `gam info userinvitation <EmailItem>`
+
+The `noinfo` argument suppresses the additional output.
+
+The `noinvitablecheck` argument suppresses the user invitation check
+to avoid exceeding quota limits when checking a large number of addresses.
+
+The return code is set based on `<TypeOfEmailItem>`:
+* User - 20
+* User Alias - 21
+* Group - 22
+* Group Alias - 23
+* User Invitation - 24
+* Unknown - 59
+

docs/BNF-Syntax.md

@@ -0,0 +1,31 @@
+# Syntax
+
+## BNF Syntax
+This Wiki describes the GAM command line syntax in modified BNF.
+* https://en.wikipedia.org/wiki/Backus-Naur_Form
+
+Skip the History section and start reading at Introduction.
+
+Items on the command line are space separated, when an actual space character is required, it will be indicated by ```<Space>```.
+If an item contains spaces, it should be surrounded by ".
+
+Metasyntactic symbols
+```
+[] optional item
+() group items
+* item may appear zero or more times
++ item may appear one or more times
+| separates alternative items
+```
+## Items
+- [Basic](Basic-Items)
+- [Lists](List-Items)
+
+## Collections
+- [ChromeOS Devices](Collections-of-ChromeOS-Devices)
+- [Users](Collections-of-Users)
+- [Items](Collections-of-Items)
+- [Verify Collections](List)
+
+## Python Regular Expressions
+- [Python Regular Expressions](Python-Regular-Expressions)

docs/Basic-Items.md

@@ -0,0 +1,611 @@
+# Basic Items
+- [Primitives](#primitives)
+- [Items built from primitives](#items-built-from-primitives)
+- [Named items](#named-items)
+- [List Items](List-Items)
+
+## Primitives
+```
+<Character> ::= a single character
+<Digit> ::= 0|1|2|3|4|5|6|7|8|9
+<Number> ::= <Digit>+
+<Float> ::= <Digit>*.<Digit>+
+<Hex> ::= <Digit>|a|b|c|d|e|f|A|B|C|D|E|F
+<Space> ::= an actual space character
+<String> ::= a string of characters, surrounded by " if it contains spaces
+<FalseValues>= false|off|no|disabled|0
+<TrueValues> ::= true|on|yes|enabled|1
+
+<BCP47LanguageCode> ::=
+        ar-sa| # Arabic Saudi Arabia
+        cs-cz| # Czech Czech Republic
+        da-dk| # Danish Denmark
+        de-de| # German Germany
+        el-gr| # Modern Greek Greece
+        en-au| # English Australia
+        en-gb| # English United Kingdom
+        en-ie| # English Ireland
+        en-us| # English United States
+        en-za| # English South Africa
+        es-es| # Spanish Spain
+        es-mx| # Spanish Mexico
+        fi-fi| # Finnish Finland
+        fr-ca| # French Canada
+        fr-fr| # French France
+        he-il| # Hebrew Israel
+        hi-in| # Hindi India
+        hu-hu| # Hungarian Hungary
+        id-id| # Indonesian Indonesia
+        it-it| # Italian Italy
+        ja-jp| # Japanese Japan
+        ko-kr| # Korean Republic of Korea
+        nl-be| # Dutch Belgium
+        nl-nl| # Dutch Netherlands
+        no-no| # Norwegian Norway
+        pl-pl| # Polish Poland
+        pt-br| # Portuguese Brazil
+        pt-pt| # Portuguese Portugal
+        ro-ro| # Romanian Romania
+        ru-ru| # Russian Russian Federation
+        sk-sk| # Slovak Slovakia
+        sv-se| # Swedish Sweden
+        th-th| # Thai Thailand
+        tr-tr| # Turkish Turkey
+        zh-cn| # Chinese China
+        zh-hk| # Chinese Hong Kong
+        zh-tw  # Chinese Taiwan
+<Charset> ::= ascii|latin1|mbcs|utf-8|utf-8-sig|utf-16|<String>
+<CalendarColorIndex> ::= <Number in range 1-24>
+<CalendarColorName> ::=
+        amethyst|avocado|banana|basil|birch|blueberry|
+        cherryblossom|citron|cobalt|cocoa|eucalyptus|flamingo|
+        grape|graphite|lavender|mango|peacock|pistachio|
+        pumpkin|radicchio|sage|tangerine|tomato|wisteria|
+<ColorHex> ::= "#<Hex><Hex><Hex><Hex><Hex><Hex>"
+<ColorNameGoogle> ::=
+        asparagus|bluevelvet|bubblegum|cardinal|chocolateicecream|denim|desertsand|
+        earthworm|macaroni|marsorange|mountaingray|mountaingrey|mouse|oldbrickred|
+        pool|purpledino|purplerain|rainysky|seafoam|slimegreen|spearmint|
+        toyeggplant|vernfern|wildstrawberries|yellowcab
+<ColorNameWeb> ::=
+        aliceblue|antiquewhite|aqua|aquamarine|azure|beige|bisque|black|blanchedalmond|
+        blue|blueviolet|brown|burlywood|cadetblue|chartreuse|chocolate|coral|
+        cornflowerblue|cornsilk|crimson|cyan|darkblue|darkcyan|darkgoldenrod|darkgray|
+        darkgrey|darkgreen|darkkhaki|darkmagenta|darkolivegreen|darkorange|darkorchid|
+        darkred|darksalmon|darkseagreen|darkslateblue|darkslategray|darkslategrey|
+        darkturquoise|darkviolet|deeppink|deepskyblue|dimgray|dimgrey|dodgerblue|
+        firebrick|floralwhite|forestgreen|fuchsia|gainsboro|ghostwhite|gold|goldenrod|
+        gray|grey|green|greenyellow|honeydew|hotpink|indianred|indigo|ivory|khaki|
+        lavender|lavenderblush|lawngreen|lemonchiffon|lightblue|lightcoral|lightcyan|
+        lightgoldenrodyellow|lightgray|lightgrey|lightgreen|lightpink|lightsalmon|
+        lightseagreen|lightskyblue|lightslategray|lightslategrey|lightsteelblue|
+        lightyellow|lime|limegreen|linen|magenta|maroon|mediumaquamarine|mediumblue|
+        mediumorchid|mediumpurple|mediumseagreen|mediumslateblue|mediumspringgreen|
+        mediumturquoise|mediumvioletred|midnightblue|mintcream|mistyrose|moccasin|
+        navajowhite|navy|oldlace|olive|olivedrab|orange|orangered|orchid|
+        palegoldenrod|palegreen|paleturquoise|palevioletred|papayawhip|peachpuff|
+        peru|pink|plum|powderblue|purple|red|rosybrown|royalblue|saddlebrown|salmon|
+        sandybrown|seagreen|seashell|sienna|silver|skyblue|slateblue|slategray|
+        slategrey|snow|springgreen|steelblue|tan|teal|thistle|tomato|turquoise|violet|
+        wheat|white|whitesmoke|yellow|yellowgreen
+<ColorName> ::= <ColorNameGoogle>|<ColorNameWeb>
+<ColorValue> ::= <ColorName>|<ColorHex>
+<DayOfWeek> ::= mon|tue|wed|thu|fri|sat|sun
+<EventColorIndex> ::= <Number in range 1-11>
+<EventColorName> ::=
+        banana|basil|blueberry|flamingo|graphite|grape|
+        lavender|peacock|sage|tangerine|tomato
+<FileFormat> ::=
+        csv|doc|dot|docx|dotx|epub|html|jpeg|jpg|mht|odp|ods|odt|
+        pdf|png|ppt|pot|potx|pptx|rtf|svg|tsv|txt|xls|xlt|xlsx|xltx|zip|
+        ms|microsoft|openoffice|
+<LabelColorHex> ::=
+        #000000|#076239|#0b804b|#149e60|#16a766|#1a764d|#1c4587|#285bac|
+        #2a9c68|#3c78d8|#3dc789|#41236d|#434343|#43d692|#44b984|#4a86e8|
+        #653e9b|#666666|#68dfa9|#6d9eeb|#822111|#83334c|#89d3b2|#8e63ce|
+        #999999|#a0eac9|#a46a21|#a479e2|#a4c2f4|#aa8831|#ac2b16|#b65775|
+        #b694e8|#b9e4d0|#c6f3de|#c9daf8|#cc3a21|#cccccc|#cf8933|#d0bcf1|
+        #d5ae49|#e07798|#e4d7f5|#e66550|#eaa041|#efa093|#efefef|#f2c960|
+        #f3f3f3|#f691b3|#f6c5be|#f7a7c0|#fad165|#fb4c2f|#fbc8d9|#fcda83|
+        #fcdee8|#fce8b3|#fef1d1|#ffad47|#ffbc6b|#ffd6a2|#ffe6c7|#ffffff
+<LabelBackgroundColorHex> ::=
+        #16a765|#2da2bb|#42d692|#4986e7|#98d7e4|#a2dcc1|
+        #b3efd3|#b6cff5|#b99aff|#c2c2c2|#cca6ac|#e3d7ff|
+        #e7e7e7|#ebdbde|#f2b2a8|#f691b2|#fb4c2f|#fbd3e0|
+        #fbe983|#fdedc1|#ff7537|#ffad46|#ffc8af|#ffdeb5
+<LabelTextColorHex> ::=
+        #04502e|#094228|#0b4f30|#0d3472|#0d3b44|#3d188e|
+        #464646|#594c05|#662e37|#684e07|#711a36|#7a2e0b|
+        #7a4706|#8a1c0a|#994a64|#ffffff
+<LanguageCode> ::=
+        ach|af|ag|ak|am|ar|az|be|bem|bg|bn|br|bs|ca|chr|ckb|co|crs|cs|cy|da|de|
+        ee|el|en|en-gb|en-us|eo|es|es-419|et|eu|fa|fi|fil|fo|fr|fr-ca|fy|
+        ga|gaa|gd|gl|gn|gu|ha|haw|he|hi|hr|ht|hu|hy|ia|id|ig|in|is|it|iw|ja|jw|
+        ka|kg|kk|km|kn|ko|kri|ku|ky|la|lg|ln|lo|loz|lt|lua|lv|
+        mfe|mg|mi|mk|ml|mn|mo|mr|ms|mt|my|ne|nl|nn|no|nso|ny|nyn|oc|om|or|
+        pa|pcm|pl|ps|pt-br|pt-pt|qu|rm|rn|ro|ru|rw|
+        sd|sh|si|sk|sl|sn|so|sq|sr|sr-me|st|su|sv|sw|
+        ta|te|tg|th|ti|tk|tl|tn|to|tr|tt|tum|tw|
+        ug|uk|ur|uz|vi|wo|xh|yi|yo|zh-cn|zh-hk|zh-tw|zu
+<Language> ::=
+        <LanguageCode>[+|-]|
+        <String>
+<Locale> ::=
+        ''|    #Not defined
+        ar-eg| #Arabic, Egypt
+        az-az| #Azerbaijani, Azerbaijan
+        be-by| #Belarusian, Belarus
+        bg-bg| #Bulgarian, Bulgaria
+        bn-in| #Bengali, India
+        ca-es| #Catalan, Spain
+        cs-cz| #Czech, Czech Republic
+        cy-gb| #Welsh, United Kingdom
+        da-dk| #Danish, Denmark
+        de-ch| #German, Switzerland
+        de-de| #German, Germany
+        el-gr| #Greek, Greece
+        en-au| #English, Australia
+        en-ca| #English, Canada
+        en-gb| #English, United Kingdom
+        en-ie| #English, Ireland
+        en-us| #English, U.S.A.
+        es-ar| #Spanish, Argentina
+        es-bo| #Spanish, Bolivia
+        es-cl| #Spanish, Chile
+        es-co| #Spanish, Colombia
+        es-ec| #Spanish, Ecuador
+        es-es| #Spanish, Spain
+        es-mx| #Spanish, Mexico
+        es-py| #Spanish, Paraguay
+        es-uy| #Spanish, Uruguay
+        es-ve| #Spanish, Venezuela
+        fi-fi| #Finnish, Finland
+        fil-ph| #Filipino, Philippines
+        fr-ca| #French, Canada
+        fr-fr| #French, France
+        gu-in| #Gujarati, India
+        hi-in| #Hindi, India
+        hr-hr| #Croatian, Croatia
+        hu-hu| #Hungarian, Hungary
+        hy-am| #Armenian, Armenia
+        in-id| #Indonesian, Indonesia
+        it-it| #Italian, Italy
+        iw-il| #Hebrew, Israel
+        ja-jp| #Japanese, Japan
+        ka-ge| #Georgian, Georgia
+        kk-kz| #Kazakh, Kazakhstan
+        kn-in| #Kannada, India
+        ko-kr| #Korean, Korea
+        lt-lt| #Lithuanian, Lithuania
+        lv-lv| #Latvian, Latvia
+        ml-in| #Malayalam, India
+        mn-mn| #Mongolian, Mongolia
+        mr-in| #Marathi, India
+        my-mn| #Burmese, Myanmar
+        nl-nl| #Dutch, Netherlands
+        nn-no| #Nynorsk, Norway
+        no-no| #Bokmal, Norway
+        pa-in| #Punjabi, India
+        pl-pl| #Polish, Poland
+        pt-br| #Portuguese, Brazil
+        pt-pt| #Portuguese, Portugal
+        ro-ro| #Romanian, Romania
+        ru-ru| #Russian, Russia
+        sk-sk| #Slovak, Slovakia
+        sl-si| #Slovenian, Slovenia
+        sr-rs| #Serbian, Serbia
+        sv-se| #Swedish, Sweden
+        ta-in| #Tamil, India
+        te-in| #Telugu, India
+        th-th| #Thai, Thailand
+        tr-tr| #Turkish, Turkey
+        uk-ua| #Ukrainian, Ukraine
+        vi-vn| #Vietnamese, Vietnam
+        zh-cn| #Simplified Chinese, China
+        zh-hk| #Traditional Chinese, Hong Kong SAR China
+        zh-tw  #Traditional Chinese, Taiwan
+<MimeTypeShortcut> ::=
+        gdoc|gdocument|
+        gdrawing|
+        gfile|
+        gfolder|gdirectory|
+        gform|
+        gfusion|
+        gjam|
+        gmap|
+        gpresentation|
+        gscript|
+        gsheet|gspreadsheet|
+        gshortcut|
+        g3pshortcut|
+        gsite|
+        shortcut
+<MimeTypeName> ::= application|audio|font|image|message|model|multipart|text|video
+<MimeType> ::= <MimeTypeShortcut>|(<MimeTypeName>/<String>)
+<ProductID> ::=
+        nv:<String> |
+        101001 |
+        101005 |
+        101031 |
+        101033 |
+        101034 |
+        101035 |
+        101036 |
+        101037 |
+        101039 |
+        101040 |
+        Google-Apps |
+        Google-Chrome-Device-Management |
+        Google-Drive-storage |
+        Google-Vault
+<SKUID> ::=
+        nv:<String>:<String> |
+        20gb | drive20gb | googledrivestorage20gb | Google-Drive-storage-20GB |
+        50gb | drive50gb | googledrivestorage50gb | Google-Drive-storage-50GB |
+        200gb | drive200gb | googledrivestorage200gb | Google-Drive-storage-200GB |
+        400gb | drive400gb | googledrivestorage400gb | Google-Drive-storage-400GB |
+        1tb | drive1tb | googledrivestorage1tb | Google-Drive-storage-1TB |
+        2tb | drive2tb | googledrivestorage2tb | Google-Drive-storage-2TB |
+        4tb | drive4tb | googledrivestorage4tb | Google-Drive-storage-4TB |
+        8tb | drive8tb | googledrivestorage8tb | Google-Drive-storage-8TB |
+        16tb | drive16tb | googledrivestorage16tb | Google-Drive-storage-16TB |
+        assuredcontrols | 1010390001 |
+        bce | beyondcorp | beyondcorpenterprise | 1010400001 |
+        cdm | chrome | googlechromedevicemanagement | Google-Chrome-Device-Management |
+        cloudidentity | identity | 1010010001 |
+        cloudidentitypremium | identitypremium | 1010050001 |
+        cloudsearch | 1010350001 |
+        gsuitebasic | gafb | gafw | basic | Google-Apps-For-Business |
+        gsuitebusiness | gau | gsb | unlimited | Google-Apps-Unlimited |
+        gsuitebusinessarchived | gsbau | businessarchived | 1010340002 |
+        gsuiteenterprisearchived | gseau | enterprisearchived | 1010340001 |
+        gsuiteenterpriseeducation | gsefe | e4e | 1010310002 |
+        gsuiteenterpriseeducationstudent | gsefes | e4es | 1010310003 |
+        gsuitegov | gafg | gsuitegovernment | Google-Apps-For-Government |
+        gsuitelite | gal | gsl | lite | Google-Apps-Lite |
+        gwep | workspaceeducationplus | 1010310008 |
+        gwepstaff | workspaceeducationplusstaff | 1010310009 |
+        gwepstudent | workspaceeducationplusstudent | 1010310010 |
+        gwes | workspaceeducationstandard | 1010310005 |
+        gwesstaff | workspaceeducationstandardstaff | 1010310006 |
+        gwesstudent | workspaceeducationstandardstudent | 1010310007 |
+        gwetlu | workspaceeducationupgrade | 1010370001 |
+        meetdialing | googlemeetglobaldialing | 1010360001 |
+        postini | gams | gsuitegams | gsuitepostini | gsuitemessagesecurity | Google-Apps-For-Postini |
+        standard | free | Google-Apps |
+        vault | googlevault | Google-Vault |
+        vfe | googlevaultformeremployee | Google-Vault-Former-Employee |
+        voicepremier | gvpremier | googlevoicepremier | 1010330002 |
+        voicestandard | gvstandard | googlevoicestandard | 1010330004 |
+        voicestarter | gvstarter | googlevoicestarter | 1010330003 |
+        wsbizplus | workspacebusinessplus | 1010020025 |
+        wsbizplusarchived | workspacebusinessplusarchived | 1010340003 |
+        wsbizstan | workspacebusinessstandard | 1010020028 |
+        wsbizstarter | workspacebusinessstarter | wsbizstart | 1010020027 |
+        wsentess | workspaceenterpriseessentials | 1010060003 |
+        wsentplus | workspaceenterpriseplus | gae | gse | enterprise | gsuiteenterprise | 1010020020 |
+        wsentstan | workspaceenterprisestandard | 1010020026 |
+        wsentstanarchived | workspaceenterprisestandardarchived | 1010340004 |
+        wsess | workspaceesentials | gsuiteessentials | essentials | d4e | driveenterprise | drive4enterprise | 1010060001 |
+        wsflw | workspacefrontline | workspacefrontlineworker | 1010020030
+```
+## Items built from primitives
+```
+<Boolean> ::= <TrueValues>|<FalseValues>
+<ByteCount> ::= <Number>[m|k|b]
+<CIDRnetmask> ::= <Number>.<Number>.<Number>.<Number>/<Number>
+<Year> ::= <Digit><Digit><Digit><Digit>
+<Month> ::= <Digit><Digit>
+<Day> ::= <Digit><Digit>
+<Hour> ::= <Digit><Digit>
+<Minute> ::= <Digit><Digit>
+<Second> ::= <Digit><Digit>
+<MilliSeconds> ::= <Digit><Digit><Digit>
+<Date> ::=
+        <Year>-<Month>-<Day> |
+        (+|-)<Number>(d|w|y) |
+        never|
+        today
+<DateTime> ::=
+        <Year>-<Month>-<Day>(<Space>|T)<Hour>:<Minute> |
+        (+|-)<Number>(m|h|d|w|y) |
+        never|
+        now|today
+<Time> ::=
+        <Year>-<Month>-<Day>(<Space>|T)<Hour>:<Minute>:<Second>[.<MilliSeconds>](Z|(+|-(<Hour>:<Minute>))) |
+        (+|-)<Number>(m|h|d|w|y) |
+        never|
+        now|today
+<RegularExpression> ::= <String>
+        See: https://docs.python.org/3/library/re.html
+<ProjectID> ::= <String>
+        Must match this Python Regular Expression: [a-z][a-z0-9-]{4,28}[a-z0-9]
+<ServiceAccountName> ::= <String>
+        Must match this Python Regular Expression: [a-z][a-z0-9-]{4,28}[a-z0-9]
+<SiteName> ::= [a-z,0-9,-]+
+<UniqueID> ::= id:<String>|uid:<String>
+```
+## Named items
+```
+<AccessToken> ::= <String>
+<AlertID> ::= <String>
+<APIScopeURL> ::= <String>
+<APPID> ::= <String>
+<ASPID> ::= <String>
+<AssetTag> ::= <String>
+<BrowserTokenPermanentID> ::= <String>
+<BuildingID> ::= <String>|id:<String>
+<CAALevelName> ::= <String>
+<CalendarACLScope> ::=
+        <EmailAddress>|user:<EmailAddress>|group:<EmailAddress>|
+        domain:<DomainName>|domain|default
+<CalendarItem> ::= <EmailAddress>
+<GIGroupAlias> ::= <EmailAddress>
+<GIGroupItem> ::= <EmailAddress>|<UniqueID>|groups/<String>
+<CIGroupType> ::= customer|group|other|serviceaccount|user
+<ChannelCustomerID> ::= <String>
+<ChatMember> ::= spaces/<String>/members/<String>
+<ChatMessage> ::= spaces/<String>/messages/<String>
+<ChatSpace> ::= spaces/<String> | <String>
+<ChatThread> ::= spaces/<String>/threads/<String>
+<ClassroomInvitationID> ::= <String>
+<ClientID> ::= <String>
+<CommandID> ::= <String>
+<ContactID> ::= <String>
+<ContactGroupID> ::= id:<String>
+<ContactGroupName> ::= <String>
+<ContactGroupItem> ::= <ContactGroupID>|<ContactGroupName>
+<CorporaAttribute> ::= alldrives|allteamdrives|domain|onlyteamdrives|user
+<CourseAlias> ::= <String>
+<CourseAnnouncementID> ::= <Number>
+<CourseAnnouncementState> ::= draft|published|deleted
+<CourseID> ::= <Number>|d:<CourseAlias>
+<CourseMaterialID> ::= <Number>
+<CourseMaterialState> ::= draft|published|deleted
+<CourseParticipantType> ::= teacher|teachers|student|students
+<CourseState> ::= active|archived|provisioned|declined|suspended
+<CourseSubmissionID> ::= <Number>
+<CourseSubmissionState> ::= new|created|turned_in|returned|reclaimed_by_student
+<CourseTopic> ::= <String>
+<CourseTopicID> ::= <Number>
+<CourseWorkID> ::= <Number>
+<CourseWorkState> ::= draft|published|deleted
+<CrOSID> ::= <String>
+<CustomerID> ::= <String>
+<DeliverySetting> ::=
+        allmail|
+        abridged|daily|
+        digest|
+        disabled|
+        none|nomail
+<DeviceID> ::= devices/<String>
+<DeviceType> ::= android|chrome_os|google_sync|ios|linux|mac_os|windows
+<DeviceUserID> ::= devices/<String>/deviceUsers/<String>
+<DomainAlias> ::= <String>
+<DomainName> ::= <String>(.<String>)+
+<DriveFileACLRole> ::=
+        commenter|
+        contentmanager|fileorganizer|
+        contributor|editor|writer|
+        manager|organizer|owner|
+        reader|viewer
+<DriveFileACLType> ::= anyone|domain|group|user
+<DriveFileID> ::= <String>
+<DriveFileURL> ::=
+        https://drive.google.com/open?id=<DriveFileID>
+        https://drive.google.com/drive/files/<DriveFileID>
+        https://drive.google.com/drive/folders/<DriveFileID>
+        https://drive.google.com/drive/folders/<DriveFileID>?resourcekey=<String>
+        https://drive.google.com/file/d/<DriveFileID>/<String>
+        https://docs.google.com/document/d/<DriveFileID>/<String>
+        https://docs.google.com/drawings/d/<DriveFileID>/<String>
+        https://docs.google.com/forms/d/<DriveFileID>/<String>
+        https://docs.google.com/presentation/d/<DriveFileID>/<String>
+        https://docs.google.com/spreadsheets/d/<DriveFileID>/<String>
+<DriveFileItem> ::= <DriveFileID>|<DriveFileURL>
+<DriveFolderID> ::= <String>
+<DriveFileName> ::= <String>
+<DriveFolderName> ::= <String>
+<DriveFolderPath> ::= <String>(/<String>)*
+<DriveFilePermission> ::=
+        anyone;<DriveFileACLRole>|
+        anyonewithlink;<DriveFileACLRole>|
+        domain:<DomainName>;<DriveFileACLRole>|
+        domainwithlink:<DomainName>;<DriveFileACLRole>|
+        group:<EmailAddress>;<DriveFileACLRole>|
+        user:<EmailAddress>;<DriveFileACLRole>
+<DriveFilePermissionID> ::= anyone|anyonewithlink|id:<String>
+<DriveFilePermissionIDorEmail> ::= <DriveFilePermissionID>|<EmailAddress>
+<DriveFileRevisionID> ::= <String>
+<DriveLabelID> ::= <String>
+<DriveLabelFieldID> ::= <String>
+<DriveLabelSelectionID> ::= <String>
+<DriveLabelName> ::= labels/<DriveLabelID>[@latest|@published|@<Number>]
+<EmailAddress> ::= <String>@<DomainName>
+<EmailItem> ::= <EmailAddress>|<UniqueID>|<String>
+<EmailReplacement> ::= <String>
+<EventID> ::= <String>
+<EventName> ::= <String>
+<ExportItem> ::= <UniqueID>|<String>
+<ExportStatus> ::= completed|failed|inprogrsss
+<FeatureName> ::= <String>
+<FieldName> ::= <String>
+<FileName> ::= <String>
+<FileNamePattern> ::= <String>
+<FilterID> ::= <String>
+<FloorName> ::= <String>
+<GroupItem> ::= <EmailAddress>|<UniqueID>|<String>
+<GroupRole> ::= owner|manager|member
+<GroupType> ::= customer|group|user
+<GuardianItem> ::= <EmailAddress>|<UniqueID>|<String>
+<GuardianInvitationID> ::= <String>
+<HoldItem> ::= <UniqueID>|<String>
+<HostName> ::= <String>
+<iCalUID> ::= <String>
+<JSONData> ::= (json [charset <Charset>] <String>) | (json file <FileName> [charset <Charset>]) |
+<Key> ::= <String>
+<LabelID> ::= Label_<String>
+<LabelName> ::= <String>
+<LabelReplacement> ::= <String>
+<LookerStudioAssetID> ::= <String>
+<LookerStudioPermission> ::=
+        user:<EmailAddress>|
+        group:<EmailAddress>|
+        domain:<DomainName>|
+        serviceAccount:<EmailAddress>
+<Marker> ::= <String>
+<MatterItem> ::= <UniqueID>|<String>
+<MatterState> ::= open|closed|deleted
+<MessageID> ::= <String>
+<Namespace> ::= <String>
+<NotesName> ::= notes/<String>
+<NotifyMessageContent> ::=
+        (message|textmessage|htmlmessage <String>)|
+        (file|textfile|htmlfile <FileName> [charset <Charset>])|
+        (gdoc|ghtml <UserGoogleDoc>)|
+        (gcsdoc|gcshtml <StorageBucketObjectName>)
+<NumberOfSeats> ::= <Number>
+<OrgUnitID> ::= id:<String>
+<OrgUnitPath> ::= /|(/<String>)+
+<OrgUnitItem> ::= <OrgUnitID>|<OrgUnitPath>
+<OtherContactsResourceName> ::= otherContacts/<String>
+<ParameterKey> ::= <String>
+<ParameterValue> ::= <String>
+<Password> ::= <String>
+<PeopleResourceName> ::= people/<String>
+<PrinterID> ::= <String>
+<ProjectID> ::= <String>
+        Must match this Python Regular Expression: [a-z][a-z0-9-]{4,28}[a-z0-9]
+<ProjectName> ::= <String>
+        Must match this Python Regular Expression: [a-zA-Z0-9 '"!-]{4,30}
+<PropertyKey> ::= <String>
+<PropertyValue> ::= <String>
+<QueryAlert> ::= <String>
+        See: https://developers.google.com/admin-sdk/alertcenter/guides/query-filters
+<QueryBrowser> ::= <String>
+        See: https://support.google.com/chrome/a/answer/9681204#retrieve_all_chrome_devices_for_an_account
+<QueryBrowserToken> ::= <String>
+        See: https://support.google.com/chrome/a/answer/9949706?ref_topic=9301744
+<QueryCalendar> ::= <String>
+<QueryCEL> ::= <String>
+        See: https://cloud.google.com/access-context-manager/docs/custom-access-level-spec
+<QueryContact> ::= <String>
+        See: https://developers.google.com/google-apps/contacts/v3/reference#contacts-query-parameters-reference
+<QueryCrOS> ::= <String>
+        See: https://support.google.com/chrome/a/answer/1698333
+<QueryDevice> ::= <String>
+        See: https://support.google.com/a/answer/7549103
+<QueryDriveFile> ::= <String>
+        See: https://developers.google.com/drive/api/v3/search-files
+<QueryDynamicGroup> ::= <String>
+        See: https://cloud.google.com/identity/docs/reference/rest/v1/groups#dynamicgroupquery
+<QueryGmail> ::= <String>
+        See: https://support.google.com/mail/answer/7190
+<QueryGroup> ::= <String>
+        See: https://developers.google.com/admin-sdk/directory/v1/guides/search-groups
+<QueryMemberRestrictions> ::= <String>
+        See: https://cloud.google.com/identity/docs/reference/rest/v1beta1/SecuritySettings#MemberRestriction
+<QueryMobile> ::= <String>
+        See: https://support.google.com/a/answer/7549103
+<QueryTeamDrive> ::= <String>
+        See: https://developers.google.com/drive/api/v3/search-parameters
+<QueryUser> ::= <String>
+        See: https://developers.google.com/admin-sdk/directory/v1/guides/search-users
+<QueryVaultCorpus> ::= <String>
+        See: https://developers.google.com/vault/reference/rest/v1/matters.holds#CorpusQuery
+<RequestID> ::= <String>
+<ResellerID> ::= <String>
+<ResourceID> ::= <String>
+<SchemaName> ::= <String>
+<Section> ::= <String>
+<SendAsContent> ::=
+        (sig|signature|htmlsig <String>)|
+        (file|htmlfile <FileName> [charset <Charset>])|
+        (gdoc|ghtml <UserGoogleDoc>)|
+        (gcsdoc|gcshtml <StorageBucketObjectName>)
+<SerialNumber> ::= <String>
+<ServiceAccountName> ::= <String>
+        Must match this Python Regular Expression: [a-z][a-z0-9-]{4,28}[a-z0-9]
+<ServiceAccountDisplayName> ::= <String>
+        Maximum of 100 characters
+<ServiceAccountDescrition> ::= <String>
+       Maximumof 256 chcracters
+<ServiceAccountEmail> ::= <ServiceAccountName>@<ProjectID>.iam.gserviceaccount.com
+<ServiceAccountUniqueID> ::= <Number>
+<ServiceAccountKey> ::= <String>
+<SheetEntity> ::= <String>|id:<Number>
+<SignatureContent> ::=
+        (<String>)|
+        (file|htmlfile <FileName> [charset <Charset>])|
+        (gdoc|ghtml <UserGoogleDoc>)|
+        (gcsdoc|gcshtml <StorageBucketObjectName>)
+<SiteACLScope> ::=
+        <EmailAddress>|user:<EmailAddress>|group:<EmailAddress>|
+        domain:<DomainName>|domain|default
+<SiteItem> ::= [<DomainName>/]<SiteName>
+<S/MIMEID> ::= <String>
+<SMTPHostName> ::= <String>
+<StudentItem> ::= <EmailAddress>|<UniqueID>|<String>
+<SharedDriveACLRole> ::=
+        commenter|
+        contentmanager|fileorganizer|
+        contributor|editor|writer|
+        manager|organizer|owner|
+        reader|viewer
+<SharedDriveID> ::= <String>
+<SharedDriveName> ::= <String>
+<StorageBucketName> ::= <String>
+<StorageObjectName> ::= <String>
+<StorageBucketObjectName> ::=
+        https://storage.cloud.google.com/<StorageBucketName>/<StorageObjectName>|
+        https://storage.googleapis.com/<StorageBucketName>/<StorageObjectName>|
+        gs://<StorageBucketName>/<StorageObjectName>|
+        <StorageBucketName>/<StorageObjectName>
+<Tag> ::= <String>
+<TakeoutBucketName> ::= takeout-export-[a-f,0-9,-]*
+<TaskID> ::= <String>
+<TaskListID> ::= <String>
+<TaskListTitle> ::= tltitle:<String>
+<TasklistIDTaskID> ::= <TasklistID>/<TaskID>
+<ThreadID> ::= <String>
+<TimeZone> ::= <String>
+        See: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
+<Title> ::= <String>
+<ToDriveAttribute> ::=
+        (tdaddsheet [<Boolean>])|
+        (tdbackupsheet (id:<Number>)|<String>)|
+        (tdcellnumberformat text|number)|
+        (tdcellwrap clip|overflow|wrap)|
+        (tdclearfilter [<Boolean>])|
+        (tdcopysheet (id:<Number>)|<String>)|
+        (tddescription <String>)|
+        (tdfileid <DriveFileID>)|
+        (tdlocalcopy [<Boolean>])|
+        (tdlocale <Locale>)|
+        (tdnobrowser [<Boolean>])|
+        (tdnoemail [<Boolean>])|
+        (tdparent (id:<DriveFolderID>)|<DriveFolderName>)|
+        (tdshare <EmailAddress> commenter|reader|writer)|
+        (tdsheet (id:<Number>)|<String>)|
+        (tdsheettimestamp [<Boolean>] [tdsheettimeformat <String>])
+        (tdsheettitle <String>)|
+        ([tdsheetdaysoffset <Number>] [tdsheethoursoffset <Number>])|
+        (tdtimestamp [<Boolean>] [tdtimeformat <String>]
+            [tddaysoffset <Number>] [tdhoursoffset <Number>])|
+        (tdtimezone <TimeZone>)|
+        (tdtitle <String>)|
+        (tdupdatesheet [<Boolean>])|
+        (tduploadnodata [<Boolean>])|
+        (tduser <EmailAddress>)
+<TransferID> ::= <String>
+<URI> ::= <String>
+<URL> ::= <String>
+<UserItem> ::= <EmailAddress>|<UniqueID>|<String>
+<UserName> ::= <String>
+<VacationMessageContent> ::=
+        (message|textmessage|htmlmessage <String>)|
+        (file|textfile|htmlfile <FileName> [charset <Charset>])|
+        (gdoc|ghtml <UserGoogleDoc>)|
+        (gcsdoc|gcshtml <StorageBucketObjectName>)
+<YouTubeChannelID> ::= <String>
+```

docs/Bulk-Processing.md

@@ -0,0 +1,153 @@
+# Bulk Processing
+- [Introduction](#introduction)
+- [Python Regular Expressions](Python-Regular-Expressions)
+- [GAM Configuration](gam.cfg)
+- [Meta Commands and File Redirection](Meta-Commands-and-File-Redirection)
+- [Definitions](#definitions)
+- [Batch files](#batch-files)
+- [CSV files](#csv-files)
+- [CSV files with redirection and select](#csv-files-with-redirection-and-select)
+- [Automatic batch processing](#automatic-batch-processing)
+
+## Introduction
+Batch and CSV file processing can improve performance by executing Gam commands in parallel.
+The variables `num_threads`, `num_tbatch_threads` and `auto_batch_min` in `gam.cfg` control parallelism.
+
+## Definitions
+* [Command data from Google Docs/Sheets/Storage](Command-Data-From-Google-Docs-Sheets-Storage)
+`gdoc <UserGoogleDoc>` and `gsheet <UserGoogleSheet>`
+
+## Batch files
+There are two types of batch processing, one that uses processes and one that uses threads. Using processes is higher performance but `gam csv` commands are not supported.
+* `gam batch` - gam commands are run as processes, gam csv commands are not allowed in the batch file
+* `gam tbatch` - gam commands are run as threads, gam csv commands are allowed in the batch file
+```
+gam batch <FileName>|-|(gdoc <UserGoogleDoc>) [charset <Charset>] [showcmds [<Boolean>]]
+gam tbatch <FileName>|-|(gdoc <UserGoogleDoc>) [charset <Charset>] [showcmds [<Boolean>]]
+```
+* `<FileName>` - A flat file containing Gam commands
+* `-` - Gam commands coming from stdin
+* `gdoc <UserGoogleDoc>` - A Google Doc containing Gam commands
+* `showcmds` - Write `timestamp,command number/number of commands,command` to stderr when each command starts; write `timestamp, command number/numberof commands,complete` to stderr when command completes
+
+Batch files can contain the following types of lines:
+* Blank lines - Ignored
+* \# Comment line - Ignored
+* gam \<GAMArgumentList\> - Execute a GAM command 
+* commit-batch
+  * GAM waits for all running GAM commands to complete
+  * GAM continues
+* commit-batch \<String\>
+  * GAM waits for all running GAM commands to complete
+  * GAM prints \<String\> and waits for the user to press any key
+  * GAM continues
+* sleep \<Integer\> - Batch processing will suspend for \<Integer\> seconds before the next command line is processed
+* print \<String\> - Print \<String\> on stderr
+* set \<KeywordString\> \<ValueString\>
+  * Subsequent lines will have %\<KeywordString\>% replaced with \<ValueString\>
+* clear \<KeywordString\>
+  * Subsequent lines will not be scanned for %\<KeywordString\>%
+
+Tbatch files can also contain the following line:
+* execute \<Program\> \<ArgumentList\> - Execute an arbitrary command; use the full path to specify \<Program\>
+
+### Example
+* You need to create accounts for your new students and assign them to groups based on their graduation year.
+* You have a CSV file NewStudents.csv with columns: Email,First,Last,GradYear,Password
+* You have a batch file NewStudents.bat containing these commands:
+```
+gam csv NewStudents.csv gam create user ~Email firstname ~First lastname ~Last org "/Students/~~GradYear~~" password ~Password
+commit-batch
+gam update group seniors sync members ou /Students/2020
+gam update group juniors sync members ou /Students/2021
+gam update group sophomores sync members ou /Students/2022
+gam update group highschool sync members ous "'/Students/2020','/Students/2021','/Students/2022'"
+```
+* Execute the batch file
+```
+gam redirect stdout ./NewStudents.out redirect stderr ./NewStudents.err tbatch NewStudents.bat showcmds
+```
+## CSV files
+```
+gam csv <FileName>|-|(gsheet <UserGoogleSheet>)|(gdoc <UserGoogleDoc>) [charset <Charset>] [warnifnodata]
+        [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>] [fields <FieldNameList>]
+        (matchfield|skipfield <FieldName> <RegularExpression>)* [showcmds [<Boolean>]]
+        [skiprows <Integer>] [maxrows <Integer>]
+        gam <GAMArgumentList>
+
+gam loop <FileName>|-|(gsheet <UserGoogleSheet>)|(gdoc <UserGoogleDoc>) [charset <Charset>] [warnifnodata]
+        [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>] [fields <FieldNameList>]
+        (matchfield|skipfield <FieldName> <RegularExpression>)* [showcmds [<Boolean>]]
+        [skiprows <Integer>] [maxrows <Integer>]
+        gam <GAMArgumentList>
+```
+* `gam csv` - Use parallel processing
+* `gam loop` - Use serial processing
+* `<FileName>` - A CSV file and the one or more columns that contain data
+* `-` - The one or more columns that contain data from stdin
+* `gsheet <UserGoogleSheet>` - A Google Sheet and the one or more columns that contain data
+* `gdoc <UserGoogleDoc>` - A Google Doc and the one or more columns that contain data
+* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
+* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
+* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
+* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings.
+* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
+* `showcmds` - Write `timestamp,command number/number of commands,command` to stderr when each command starts; write `timestamp, command number/numberof commands,complete` to stderr when command completes
+* `skiprows <Integer>` - Skip filtered rows from the CSV file/Google Sheet.
+  * `skiprows 0` - All rows are processed, this is the default
+  * `skiprows N` - The first N filtered rows are skipped
+* `maxrows <Integer>` - Limit the number of filtered rows processed from the CSV file/Google Sheet after any skipped rows.
+  * `maxrows 0` - All rows are processed, this is the default
+  * `maxrows N` - N filtered rows are processed
+
+### Use CSV file values in command line
+You can make substitutions in `<GAMArgumentList>` with values from the CSV file.
+- Reference the field xxx with `~xxx` if the argument contains no other text
+- Reference the field xxx with `~~xxx~~` if the argument contains other text
+- An argument containing exactly `~xxx` is replaced by the value of field xxx
+- An argument containing instances of `~~xxx~~` has `~~xxx~~` replaced by the value of field xxx
+- An argument containing instances of `~~xxx~!~pattern~!~replacement~~` has `~~xxx~!~pattern~!~replacement~~` replaced by re.sub(pattern, replacement, value of field xxx) See: https://docs.python.org/3/library/re.html
+
+If an argument is specifying a file path and it starts with a `~`, e.g., `targetfolder "~/Documents/GamWork"`, GAM will flag it as an error:
+```
+ERROR: Header "/Documents/GamWork/" not found in CSV headers of "Owner,id,title".
+```
+Put a space in front of the `~`: `targetfolder " ~/Documents/GamWork"` to avoid the error.
+
+### Example
+* You need to update the work addresses of a set of users
+* You want a note field that shows their email address as name AT domain.com
+* You have a CSV file Users.csv with columns: primaryEmail,Street,City,State,ZIP
+```
+gam csv Users.csv gam update user ~primaryEmail address type work unstructured "~~Street~~, ~~City~~, ~~State~~ ~~ZIP~~" primary note text_plain "~~primaryEmail~!~^(.+)@(.+)$~!~\1 AT \2~~"
+```
+* You want to do the above using a Google Sheet
+```
+gam csv gsheet <user> <fileID> "<sheetName>" gam update user "~primaryEmail" address type work unstructured "~~Street~~, ~~City~~, ~~State~~ ~~ZIP~~" primary note text_plain "~~primaryEmail~!~^(.+)@(.+)$~!~\1 AT \2~~"
+```
+
+## CSV files with redirection and select
+You should use the `multiprocess` option on any redirected files: `csv`, `stdout`, `stderr`.
+```
+gam redirect csv ./filelistperms.csv multiprocess csv Users.csv gam user ~primaryEmail print filelist fields id,title,permissions,owners.emailaddress
+```
+
+If you want to select a `gam.cfg` section for the command, you can select the section at the outer `gam` and save it
+or select the section at the inner `gam`.
+```
+gam select <Section> save redirect csv ./filelistperms.csv multiprocess csv Users.csv gam user ~primaryEmail print filelist fields id,title,permissions,owners.emailaddress
+gam redirect csv ./filelistperms.csv multiprocess csv Users.csv gam select <Section> user ~primaryEmail print filelist fields id,title,permissions,owners.emailaddress
+```
+
+## Automatic batch processing
+You can enable automatic batch (parallel) processing when issuing commands of the form `gam <UserTypeEntity> ...`.
+In the following example, if the number of users in group sales@domain.com exceeds 1, then the `print filelist` command will be processed in parallel.
+```
+gam config auto_batch_min 1 redirect csv ./filelistperms.csv multiprocess group sales@domain.com print filelist fields id,title,permissions,owners.emailaddress
+```
+With automatic batch processing, you should use the `multiprocess` option on any redirected files: `csv`, `stdout`, `stderr`.
+
+If you want to select a `gam.cfg` section for the command, you must select and save it for it to be processed correctly.
+```
+gam select <Section> save config auto_batch_min 1 redirect csv ./filelistperms.csv multiprocess group sales@domain.com print filelist fields id,title,permissions,owners.emailaddress
+```

docs/CSV-Input-Filtering.md

@@ -0,0 +1,259 @@
+# CSV Input Filtering
+- [Python Regular Expressions](Python-Regular-Expressions) Search function
+- [Definitions](#definitions)
+- [Quoting rules](#quoting-rules)
+- [Column row filtering](#column-row-filtering)
+- [Column row limiting](#column-row-limiting)
+- [Saving filters in gam.cfg](#saving-filters-in-gamcfg)
+- [Validate filters](#validate-filters)
+
+There are two values in `gam.cfg` that can be used to filter the input from `gam csv` commands.
+* `csv_input_row_filter` - A list or JSON dictionary used to include specific rows based on column values
+* `csv_input_row_drop_filter` - A list or JSON dictionary used to exclude specific rows based on column values
+
+These filters can be used alone or in conjunction with the `matchfield|skipfield <FieldName> <RegularExpression>` options.
+* https://github.com/taers232c/GAMADV-XTD3/wiki/Bulk-Processing#csv-files
+
+## Definitions
+[Data Selectors](Collections-of-items)
+```
+<DataSelector> ::=
+        <ListSelector>|
+        <FileSelector>|
+        <CSVFileSelector>
+```
+```
+<Date> ::=
+        <Year>-<Month>-<Day> |
+        (+|-)<Number>(d|w|y) |
+        never|
+        today
+<Time> ::=
+        <Year>-<Month>-<Day>T<Hour>:<Minute>:<Second>[.<MilliSeconds>](Z|(+|-(<Hour>:<Minute>))) |
+        (+|-)<Number>(m|h|d|w|y) |
+        never|
+        now|today
+<Operator> ::= <|<=|>=|>|=|!=
+<RegularExpression> ::= <String>
+        See: https://docs.python.org/3/library/re.html>
+
+<FieldNameFilter> :: = <RegularExpression>
+<RowValueFilter> ::=
+        [(any|all):]count<Operator><Number>|
+        [(any|all):]countrange=<Number>/<Number>|
+        [(any|all):]countrange!=<Number>/<Number>|
+        [(any|all):]date<Operator><Date>|
+        [(any|all):]daterange=<Date>/<Date>|
+        [(any|all):]daterange!=<Date>/<Date>|
+        [(any|all):]length<Operator><Number>|
+        [(any|all):]lengthrange=<Number>/<Number>|
+        [(any|all):]lengthrange!=<Number>/<Number>|
+        [(any|all):]text<Operator><String>|
+        [(any|all):]textrange=<String>/<String>|
+        [(any|all):]textrange!=<String>/<String>|
+        [(any|all):]time<Operator><Time>|
+        [(any|all):]timerange=<Time>/<Time>|
+        [(any|all):]timerange!=<Time>/<Time>|
+        [(any|all):]boolean:<Boolean>|
+        [(any|all):]regex:<RegularExpression>|
+        [(any|all):]regexcs:<RegularExpression>|
+        [(any|all):]notregex:<RegularExpression>|
+        [(any|all):]notregexcs:<RegularExpression>|
+        [(any|all):]data:<DataSelector>|
+        [(any|all):]notdata:<DataSelector>|
+<RowValueFilterList> ::=
+        "'<FieldNameFilter>:<RowValueFilter>'(,'<FieldNameFilter>:<RowValueFilter>')*"
+<RowValueFilterJSONList> ::=
+        '{"<FieldNameFilter>": "<RowValueFilter>"(,"<FieldNameFilter>": "<RowValueFilter>")*}' |
+        "{\"<FieldNameFilter>\": \"<RowValueFilter>\"(,\"<FieldNameFilter>\": \"<RowValueFilter>\")*}"
+```
+## Quoting rules
+Name:value form.
+```
+<RowValueFilterList> ::=
+        "'<FieldNameFilter>:<RowValueFilter>'(,'<FieldNameFilter>:<RowValueFilter>')*"
+```
+* `<RowValueFilterList>`, even if it has one element, should be enclosed in `"`.
+* Each `<FieldNameFilter>:<RowValueFilter>` pair should be enclosed in `'`.
+* If `<FieldNameFilter>` contains a `:` or a space, it should be enclosed in `\"`.
+* If `<RegularExpression>` or `<DataSelector>` in `<RowValueFilter>` contain a space, it should be enclosed in `\"`.
+
+Example:
+```
+csv_input_row_filter "'\"accounts:used_quota_in_mb\":count>15000'"
+csv_input_row_filter "'email:data:\"csvfile gsheet:email user@domain.com FileID Sheet1\"'"
+```
+JSON form.
+```
+<RowValueFilterJSONList> ::=
+        '{"<FieldNameFilter>": "<RowValueFilter>"(,"<FieldNameFilter>": "<RowValueFilter>")*}' |
+        "{\"<FieldNameFilter>\": \"<RowValueFilter>\"(,\"<FieldNameFilter>\": \"<RowValueFilter>\")*}"
+```
+* The first JSON form can be used on Linux and Mac OS; it can not be used on Windows.
+* The second JSON form can be used on Linux, Mac OS and Windows.
+* If `<FieldNameFilter>` contains a `:` or a space, no additional quoting is required
+
+Example:
+```
+csv_input_row_filter '{"accounts:used_quota_in_mb": "count>=150"}'
+csv_input_row_filter "{\"accounts:used_quota_in_mb\": \"count>=150\"}"
+```
+
+## Column row filtering
+Row filtering includes/excludes rows based on column values.
+
+### Field names
+Field names are specified by regular expressions; at its simplest, you specify a complete field name.
+Field names are matched in a case insensitive manner.
+
+If the field name doesn't contain any of the following regular expression characters `^$*+|$[{(`,
+it will be surrounded with `^$` so that it doesn't match any subfields that begin with the field name as a prefix.
+
+The following filter will match the count field and not the subfields.
+```
+config csv_input_row_filter "'externalIds:countrange=1/10'"
+
+primaryEmail,externalIds,externalIds.0.type,externalIds.0.value,externalIds.1.type,externalIds.1.value,...
+```
+
+### Inclusive filters
+You can include rows for gam csv commands based on column values. You specify a list
+of fields(headers) and the values they must have. `csv_input_row_filter` is used to specify the 
+fields and values. Each field name/expression can appear only once in the list.
+
+You specify whether all or any value filters must match for the row to be included in the input.
+
+* `csv_input_row_filter_mode allmatch` - All value filters must match for the row to be included in the input; this is the default
+* `csv_input_row_filter_mode anymatch` - Any value filter must match for the row to be included in the input
+
+```
+gam config csv_input_row_filter <RowValueFilterList> ...
+gam config csv_input_row_filter <RowValueFilterJSONList> ...
+```
+
+### Exclusive filters
+You can exclude rows for gam csv commands based on column values. You specify a list
+of fields(headers) and the values they must not have. `csv_input_row_drop_filter` is used to specify the 
+fields and values. Each field name/expression can appear only once in the list.
+
+You specify whether all or any value filters must match for the row to be excluded from the input.
+
+* `csv_input_row_filter_drop_mode allmatch` - If all value filters match, the row is excluded from the input
+* `csv_input_row_filter_drop_mode anymatch` - If any value filter matches, the row is excluded from the input; this is the default
+
+```
+gam config csv_input_row_drop_filter <RowValueFilterList> ...
+gam config csv_input_row_drop_filter <RowValueFilterJSONList> ...
+```
+
+### Matches
+A filter matches if the field has the desired value. lf you specify a regular expression for a field name that matches
+several columns, the filter matches if any of the columns has a match. In the case of `notregex|notregexcs|notdata`,
+the filter matches if none (not any) of the columns has a match.
+
+`<RowValueFilter>` allows specifying that the filter will match only if all of the columns have a match.
+In the case of `notregex|notregexcs|notdata`, the filter matches if some (not all) of the columns have a match.
+If neither `any` or `all` is explicitly specified, `any` is the default.
+
+These are the row value filter types:
+* `count<Operator><Number>` - Used on fields with numbers; a blank field will not match
+* `countrange=<Number>/<Number>` - Used on fields with numbers; a blank field will not match
+  * The field value must be `>=` the left `<Number>` and `<=` the right `<Number>`
+* `countrange!=<Number>/<Number>` - Used on fields with numbers; a blank field will not match
+  * The field value must be `<` the left `<Number>` or `>` the right `<Number>`
+* `date<Operator><Date>` - Used on fields with dates or times; only the date portion of a time field is compared; a blank field will not match
+* `daterange=<Date>/<Date>` - Used on fields with dates or times; only the date portion of a time field is compared; a blank field will not match
+  * The field value must be `>=` the left `<Date>` and `<=` the right `<Date>`
+* `daterange!=<Date>/<Date>` - Used on fields with dates or times; only the date portion of a time field is compared; a blank field will not match
+  * The field value must be `<` the left `<Date>` or `>` the right `<Date>`
+* `length<Operator><Number>` - Used on fields with strings; non string fields will not match
+* `lengthrange=<Number>/<Number>` - Used on fields with strings; non string fields will not match
+  * The field length must be `>=` the left `<Number>` and `<=` the right `<Number>`
+* `lengthrange!=<Number>/<Number>` - Used on fields with strings; non string fields will not match
+  * The field length must be `<` the left `<Number>` or `>` the right `<Number>`
+* `text<Operator><String>` - Used on fields with text
+* `textrange=<String>/<String>` - Used on fields with strings
+  * The field value must be `>=` the left `<String>` and `<=` the right `<String>`
+* `textrange!=<String>/<String>` - Used on fields with strings
+  * The field value must be `<` the left `<String>` or `>` the right `<String>`
+* `time<Operator><Time>` - Used on fields with times; a blank field will not match
+* `timerange=<Time>/<Time>` - Used on fields with times; a blank field will not match
+  * The field value must be `>=` the left `<Time>` and `<=` the right `<Time>`
+* `timerange!=<Time>/<Time>` - Used on fields with times; a blank field will not match
+  * The field value must be `<` the left `<Time>` or `>` the right `<Time>`
+* `boolean:<Boolean>` - Used on fields with Boolean values; a blank field is considered False
+* `regex:<RegularExpression>` - Used on fields with text; field value must match `<RegularExpression>`; case insensitive
+* `regexcs:<RegularExpression>` - Used on fields with text; field value must match `<RegularExpression>`; case sensitive
+* `notregex:<RegularExpression>` - Used on fields with text; field value must not match `<RegularExpression>`; case insensitive
+* `notregexcs:<RegularExpression>` - Used on fields with text; field value must not match `<RegularExpression>`; case sensitive
+* `data:<DataSelector>` - Used on fields with text; field value must match some value in `<DataSelector>`; case sensitive
+* `notdata:<DataSelector>` - Used on fields with text; field value must not match any value in `<DataSelector>`; case sensitive
+
+### **Change in behavior.**
+In versions prior to `5.12.00`, `regex:<RegularExpression>` and `notregex:<RegularExpression>` were processed in a case sensitive manner;
+in many cases this is probably not desirable; e.g., matching file names which are case insensitive.
+
+Now, `regex:<RegularExpression>` and `notregex:<RegularExpression>` are processed in a case insensitive manner.
+To get the prior case sensitive processing, use `regexcs:<RegularExpression>` and `notregexcs:<RegularExpression>`.
+
+### Examples
+You want to process groups with 100 or more direct members.
+```
+gam redirect csv GroupInfo.csv print groups fields directmemberscount
+gam config csv_input_row_filter "'directMembersCount:count>100'" csv GroupInfo.csv gam group "~email" ...
+```
+You want to process groups not created by an administrator.
+```
+gam redirect csv GroupInfo.csv print groups fields admincreated
+gam config csv_input_row_drop_filter "'adminCreated:boolean:true'" csv GroupInfo.csv gam group "~email" ...
+```
+You want to process users created in the last 30 days.
+```
+gam redirect csv UserInfo.csv print users fields creationtime
+gam config csv_input_row_filter "'creationTime:date>=-30d'" csv UserInfo.csv gam user "~primaryEmail" ...
+```
+You want to process users that are consuming more than 15GB of storage.
+Special quoting is required because the field name contains a colon.
+```
+gam redirect csv UserInfo.csv report user services accounts fields "accounts:used_quota_in_mb"
+gam config csv_input_row_filter "'\"accounts:used_quota_in_mb\":count>15000'" csv UserInfo.csv gam user "~primaryEmail" ...
+```
+## Column row limiting
+You can limit the number of rows read from a CSV file.
+
+You want to process the first 10 users that are consuming more than 15GB of storage.
+Special quoting is required because the field name contains a colon.
+```
+gam redirect csv UserInfo.csv report user services accounts fields "accounts:used_quota_in_mb"
+gam config csv_input_row_filter "'\"accounts:used_quota_in_mb\":count>15000'" csv_input_row_limit 10 csv UserInfo.csv gam user "~primaryEmail" ...
+```
+
+## Saving filters in gam.cfg
+If you define a value for `csv_input_row_filter`, `csv_input_row_drop_filter` or `csv_input_row_limit` in the `[DEFAULT]` section of `gam.cfg`,
+it will apply to every `gam csv` command which is probably not desirable. You can store them in `gam.cfg` in named sections.
+```
+[Filter510]
+csv_input_row_filter = 'phones.\\\d+.value:regex:(?:^\\\(510\\\) )|(?:^510[- ])\\\d{3}-\\\d{4}'
+```
+You want to process users with phone numbers in the area code 510; the number can be in the format `(510) ddd-dddd` or `510-ddd-dddd` or `510 ddd-dddd`.
+```
+gam redirect csv UserInfo.csv print users fields name,phones
+gam selectinputfilter Filter510 csv UserInfo.csv gam user "~primaryEmail" ...
+```
+
+## Validate filters
+Version `6.30.00` added the `gam comment <String>*` command that can be used to validate input row filters.
+```
+$ more Comment.csv 
+col1,col2
+aaa,111
+bbb,222
+ccc,333
+$ gam config csv_input_row_drop_filter "col1:regex:bbb" csv Comment.csv gam comment "Col1:~~col1~~" "Col2:~~col2~~"
+2022-12-16T12:41:50.045-08:00,0/2,Using 2 processes...
+Col1:aaa Col2:111
+Col1:ccc Col2:333
+$ gam config csv_input_row_filter "col1:regex:bbb"  csv Comment.csv gam comment "Col1:~~col1~~" "Col2:~~col2~~"
+2022-12-18T09:42:26.108-08:00,0/1,Using 1 process...
+Col1:bbb Col2:222
+```

docs/CSV-Output-Filtering.md

@@ -0,0 +1,360 @@
+# CSV Output Filtering
+- [Python Regular Expressions](Python-Regular-Expressions) Search function
+- [Definitions](#definitions)
+- [Quoting rules](#quoting-rules)
+- [Column header filtering](#column-header-filtering)
+- [Column row filtering](#column-row-filtering)
+- [Column row limiting](#column-row-limiting)
+- [Saving filters in gam.cfg](#saving-filters-in-gamcfg)
+
+There are five values in `gam.cfg` that can be used to filter the output from `gam print` commands.
+* `csv_output_header_filter` - A list of `<RegularExpressions>` used to select specific column headers to include
+* `csv_output_header_drop_filter` - A list of `<RegularExpressions>` used to select specific column headers to exclude
+* `csv_output_row_filter` - A list or JSON dictionary used to include specific rows based on column values
+* `csv_output_row_drop_filter` - A list or JSON dictionary used to exclude specific rows based on column values
+* `csv_output_row_limit` - A limit on the number of rows written
+
+The original implementation required that row filters be expressed in JSON notation; these are almost
+impossible to enter correctly in Windows; on Mac OS or Linux, it's easy. You can now enter the row filters as lists
+on all platforms.
+
+## Definitions
+[Data Selectors](Collections-of-items)
+```
+<DataSelector> ::=
+        <ListSelector>|
+        <FileSelector>|
+        <CSVFileSelector>
+```
+```
+<Date> ::=
+        <Year>-<Month>-<Day> |
+        (+|-)<Number>(d|w|y) |
+        never|
+        today
+<Time> ::=
+        <Year>-<Month>-<Day>T<Hour>:<Minute>:<Second>[.<MilliSeconds>](Z|(+|-(<Hour>:<Minute>))) |
+        (+|-)<Number>(m|h|d|w|y) |
+        never|
+        now|today
+<Operator> ::= <|<=|>=|>|=|!=
+<RegularExpression> ::= <String>
+        See: https://docs.python.org/3/library/re.html>
+
+<FieldNameFilter> :: = <RegularExpression>
+<ColumnFieldNameFilterList> ::= "<FieldNameFilter>(,<FieldNameFilter>)*"
+<RowValueFilter> ::=
+        [(any|all):]count<Operator><Number>|
+        [(any|all):]countrange=<Number>/<Number>|
+        [(any|all):]countrange!=<Number>/<Number>|
+        [(any|all):]date<Operator><Date>|
+        [(any|all):]textrange=<String>/<String>|
+        [(any|all):]textrange!=<String>/<String>|
+        [(any|all):]daterange=<Date>/<Date>|
+        [(any|all):]daterange!=<Date>/<Date>|
+        [(any|all):]length<Operator><Number>|
+        [(any|all):]lengthrange=<Number>/<Number>|
+        [(any|all):]lengthrange!=<Number>/<Number>|
+        [(any|all):]text<Operator><String>|
+        [(any|all):]time<Operator><Time>|
+        [(any|all):]timerange=<Time>/<Time>|
+        [(any|all):]timerange!=<Time>/<Time>|
+        [(any|all):]boolean:<Boolean>|
+        [(any|all):]regex:<RegularExpression>|
+        [(any|all):]regexcs:<RegularExpression>|
+        [(any|all):]notregex:<RegularExpression>|
+        [(any|all):]notregexcs:<RegularExpression>|
+        [(any|all):]data:<DataSelector>|
+        [(any|all):]notdata:<DataSelector>
+<RowValueFilterList> ::=
+        "'<FieldNameFilter>:<RowValueFilter>'(,'<FieldNameFilter>:<RowValueFilter>')*"
+<RowValueFilterJSONList> ::=
+        '{"<FieldNameFilter>": "<RowValueFilter>"(,"<FieldNameFilter>": "<RowValueFilter>")*}' |
+        "{\"<FieldNameFilter>\": \"<RowValueFilter>\"(,\"<FieldNameFilter>\": \"<RowValueFilter>\")*}"
+```
+## Quoting rules
+Name:value form.
+```
+<RowValueFilterList> ::=
+        "'<FieldNameFilter>:<RowValueFilter>'(,'<FieldNameFilter>:<RowValueFilter>')*"
+```
+* `<RowValueFilterList>`, even if it has one element, should be enclosed in `"`.
+* Each `<FieldNameFilter>:<RowValueFilter>` pair should be enclosed in `'`.
+* If `<FieldNameFilter>` contains a `:` or a space, it should be enclosed in `\"`.
+* If `<RegularExpression>` or `<DataSelector>` in `<RowValueFilter>` contain a space, it should be enclosed in `\"`.
+* If `<FieldNameFilter>` or `<RegularExpression>` in `<RowValueFilter>` contain a `\` to escape a special character
+or enter a special sequence, enter `\\\`.
+
+Example:
+```
+csv_output_row_filter "'\"accounts:used_quota_in_mb\":count>15000'"
+csv_output_row_filter "'email:data:\"csvfile gsheet:email user@domain.com FileID Sheet1\"'"
+csv_output_row_filter "'phones.\\\d+.value:regex:(?:^\\\(510\\\) )|(?:^510[- ])\\\d{3}-\\\d{4}'"
+```
+JSON form.
+```
+<RowValueFilterJSONList> ::=
+        '{"<FieldNameFilter>": "<RowValueFilter>"(,"<FieldNameFilter>": "<RowValueFilter>")*}' |
+        "{\"<FieldNameFilter>\": \"<RowValueFilter>\"(,\"<FieldNameFilter>\": \"<RowValueFilter>\")*}"
+```
+* The first form can be used on Linux and Mac OS; it can not be used on Windows.
+* The second form can be used on Linux, Mac OS and Windows.
+* If `<FieldNameFilter>` contains a `:`, no additional quoting is required
+
+Example:
+```
+csv_output_row_filter '{"accounts:used_quota_in_mb": "count>=150"}'
+csv_output_row_filter "{\"accounts:used_quota_in_mb\": \"count>=150\"}"
+```
+
+## Column header filtering
+Gam gives you the ability to select fields(column headers) in its print commands, but there may be cases
+where you get more columns than is desirable.
+* `csv_output_header_filter` - Used to select the column headers to include in the output
+* `csv_output_header_drop_filter` - Used to select the column headers to exclude from the output
+
+Typically, you would use the option that involes typing the fewest column names but both options can be used.
+When both options are used, `csv_output_header_drop_filter` is processed first, then `csv_output_header_filter`.
+
+Field names are specified by regular expressions; at its simplest, you specify a complete field name.
+Field names are matched in a case insensitive manner.
+```
+gam config csv_output_header_filter <ColumnFieldNameFilterList> ...
+gam config csv_output_header_drop_filter <ColumnFieldNameFilterList> ...
+```
+### Example
+you want a list of user email addresses and full names; you do not need the given or family names.
+
+No filtering.
+```
+gam print users name
+primaryEmail,name.givenName,name.familyName,name.fullName
+testuser1@domain.com,Test,User1,Test User1
+testuser2@domain.com,Test,User2,Test User2
+...
+```
+With inclusion filtering.
+```
+gam config csv_output_header_filter "primaryEmail,name.fullName" print users name
+primaryEmail,name.fullName
+testuser1@domain.com,Test User1
+testuser2@domain.com,Test User2
+...
+```
+With exclusion filtering.
+```
+gam config csv_output_header_drop_filter "name.givenName,name.familyName" print users name
+primaryEmail,name.fullName
+testuser1@domain.com,Test User1
+testuser2@domain.com,Test User2
+...
+```
+
+## Column row filtering
+Row filtering includes/excludes rows based on column values.
+
+### Field names
+Field names are specified by regular expressions; at its simplest, you specify a complete field name.
+Field names are matched in a case insensitive manner.
+
+If the field name doesn't contain any of the following regular expression characters `^$*+|$[{(`,
+it will be surrounded with `^$` so that it doesn't match any subfields that begin with the field name as a prefix.
+
+The following filter will match the count field and not the subfields.
+```
+config csv_output_row_filter "'externalIds:countrange=1/10'"
+
+primaryEmail,externalIds,externalIds.0.type,externalIds.0.value,externalIds.1.type,externalIds.1.value,...
+```
+
+### Inclusive filters
+You can include rows generated by gam print commands based on column values. You specify a list
+of fields (headers) and the values they must have. `csv_output_row_filter` is used to specify the 
+fields and values. Each field name/expression can appear only once in the list.
+```
+gam config csv_output_row_filter <RowValueFilterList> ...
+gam config csv_output_row_filter <RowValueFilterJSONList> ...
+```
+
+You optionally specify whether all or any value filters must match for the row to be included in the output.
+
+* `csv_output_row_filter_mode allmatch` - All value filters must match for the row to be included in the output; this is the default
+* `csv_output_row_filter_mode anymatch` - Any value filter must match for the row to be included in the output
+```
+gam config csv_output_row_filter_mode anymatch csv_output_row_filter <RowValueFilterList> ...
+gam config csv_output_row_filter_mode anymatch csv_output_row_filter <RowValueFilterJSONList> ...
+```
+
+
+### Exclusive filters
+You can exclude rows generated by gam print commands based on column values. You specify a list
+of fields (headers) and the values they must not have. `csv_output_row_drop_filter` is used to specify the 
+fields and values. Each field name/expression can appear only once in the list.
+```
+gam config csv_output_row_drop_filter <RowValueFilterList> ...
+gam config csv_output_row_drop_filter <RowValueFilterJSONList> ...
+```
+
+You optionally specify whether all or any value filters must match for the row to be excluded from the output.
+
+* `csv_output_row_drop_filter_mode allmatch` - If all value filters match, the row is excluded from the output
+* `csv_output_row_drop_filter_mode anymatch` - If any value filter matches, the row is excluded from the output; this is the default
+```
+gam config csv_output_row_drop_filter_mode allmatch csv_output_row_drop_filter <RowValueFilterList> ...
+gam config csv_output_row_drop_filter_mode allmatch csv_output_row_drop_filter <RowValueFilterJSONList> ...
+```
+
+### Matches
+A filter matches if the field has the desired value. lf you specify a regular expression for a field name that matches
+several columns, the filter matches if any of the columns has a match. In the case of `notregex|notregexcs|notdata`,
+the filter matches if none (not any) of the columns has a match.
+
+`<RowValueFilter>` allows specifying that the filter will match only if all of the columns have a match.
+In the case of `notregex|notregexcs|notdata`, the filter matches if some (not all) of the columns have a match.
+If neither `any` or `all` is explicitly specified, `any` is the default.
+
+These are the row value filter types:
+* `count<Operator><Number>` - Used on fields with numbers; a blank field will not match
+* `countrange=<Number>/<Number>` - Used on fields with numbers; a blank field will not match
+  * The field value must be `>=` the left `<Number>` and `<=` the right `<Number>`
+* `countrange!=<Number>/<Number>` - Used on fields with numbers; a blank field will not match
+  * The field value must be `<` the left `<Number>` or `>` the right `<Number>`
+* `date<Operator><Date>` - Used on fields with dates or times; only the date portion of a time field is compared; a blank field will not match
+* `daterange=<Date>/<Date>` - Used on fields with dates or times; only the date portion of a time field is compared; a blank field will not match
+  * The field value must be `>=` the left `<Date>` and `<=` the right `<Date>`
+* `daterange!=<Date>/<Date>` - Used on fields with dates or times; only the date portion of a time field is compared; a blank field will not match
+  * The field value must be `<` the left `<Date>` or `>` the right `<Date>`
+* `length<Operator><Number>` - Used on fields with strings; non string fields will not match
+* `lengthrange=<Number>/<Number>` - Used on fields with strings; non string fields will not match
+  * The field length must be `>=` the left `<Number>` and `<=` the right `<Number>`
+* `lengthrange!=<Number>/<Number>` - Used on fields with strings; non string fields will not match
+  * The field length must be `<` the left `<Number>` or `>` the right `<Number>`
+* `text<Operator><String>` - Used on fields with text
+* `textrange=<String>/<String>` - Used on fields with strings
+  * The field value must be `>=` the left `<String>` and `<=` the right `<String>`
+* `textrange!=<String>/<String>` - Used on fields with strings
+  * The field value must be `<` the left `<String>` or `>` the right `<String>`
+* `time<Operator><Time>` - Used on fields with times; a blank field will not match
+* `timerange=<Time>/<Time>` - Used on fields with times; a blank field will not match
+  * The field value must be `>=` the left `<Time>` and `<=` the right `<Time>`
+* `timerange!=<Time>/<Time>` - Used on fields with times; a blank field will not match
+  * The field value must be `<` the left `<Time>` or `>` the right `<Time>`
+* `boolean:<Boolean>` - Used on fields with Boolean values; a blank field is considered False
+* `regex:<RegularExpression>` - Used on fields with text; field value must match `<RegularExpression>`; case insensitive
+* `regexcs:<RegularExpression>` - Used on fields with text; field value must match `<RegularExpression>`; case sensitive
+* `notregex:<RegularExpression>` - Used on fields with text; field value must not match `<RegularExpression>`; case insensitive
+* `notregexcs:<RegularExpression>` - Used on fields with text; field value must not match `<RegularExpression>`; case sensitive
+* `data:<DataSelector>` - Used on fields with text; field value must match some value in `<DataSelector>`; case sensitive
+* `notdata:<DataSelector>` - Used on fields with text; field value must not match any value in `<DataSelector>`; case sensitive
+
+### **Change in behavior.**
+In versions prior to `5.12.00`, `regex:<RegularExpression>` and `notregex:<RegularExpression>` were processed in a case sensitive manner;
+in many cases this is probably not desirable; e.g., matching file names which are case insensitive.
+
+Now, `regex:<RegularExpression>` and `notregex:<RegularExpression>` are processed in a case insensitive manner.
+To get the prior case sensitive processing, use `regexcs:<RegularExpression>` and `notregexcs:<RegularExpression>`.
+
+### Examples
+You want a list of groups with 100 or more direct members.
+```
+gam config csv_output_row_filter "'directMembersCount:count>100'" print groups fields directmemberscount
+```
+You want a list of users created in the last 30 days.
+```
+gam config csv_output_row_filter "'creationTime:date>=-30d'" print users fields creationtime
+```
+You want a list of users in the OU /Test that are consuming more than 15GB of storage.
+Special quoting is required because the field name contains a colon.
+```
+gam config csv_output_row_filter "'\"accounts:used_quota_in_mb\":count>15000'" report users select ou /Test fields accounts:used_quota_in_mb
+```
+You want the names of users directly in the OU /Test, you do not want users in any sub-OUs of /Test.
+* The Google API will only supply users in an OU and sub-OUs, GAM has to filter out the users in the sub-OU.
+```
+gam config csv_output_row_filter "'orgUnitPath:regex:^/Test$'" print users query "orgUnitPath=/Test" fields name,ou
+```
+You want the names of female users directly in the OU /Test, you do not want users in any sub-OUs of /Test.
+* The Google API will only supply users in an OU and sub-OUs, GAM has to filter out the users in the sub-OU.
+```
+gam config csv_output_row_filter "'orgUnitPath:regex:^/Test$','gender:regex:female'" print users query "orgUnitPath=/Test" fields name,ou,gender
+```
+You want a list of groups not created by an administrator.
+```
+gam config csv_output_row_filter "'adminCreated:boolean:false'" print groups fields admincreated
+```
+You want a list of users with phone numbers in the area code 510; the number can be in the format `(510) ddd-dddd` or `510-ddd-dddd` or `510 ddd-dddd`.
+```
+gam config csv_output_header_filter "primaryEmail,name.fullName,phones.*value" csv_output_row_filter "'"'phones.\\\d+.value:regex:(?:^\\\(510\\\) )|(?:^510[- ])\\\d{3}-\\\d{4}'"'" print users name phones
+primaryEmail,name.fullName,phones.0.value
+testuser1@domain.com,Test User1,(510) 555-1212
+testuser2@domain.com,Test User2,510-555-1212
+testuser3@domain.com,Test User3,510 555-1212
+```
+You want a list of users not in the organization cost center "Tech Support".
+```
+gam config csv_output_header_filter "primaryEmail,name.fullName,orgUnitPath,organizations.*costCenter" csv_output_row_filter 'organizations.*costCenter:notregex:"Tech Support"' print users fields name,ou,organizations
+gam config csv_output_header_filter "primaryEmail,name.fullName,orgUnitPath,organizations.*costCenter" csv_output_row_drop_filter 'organizations.*costCenter:regex:"Tech Support"' print users fields name,ou,organizations
+primaryEmail,name.fullName,orgUnitPath,organizations.0.costCenter
+testuser1@domain.com,Test User1,/Test,Sales
+testuser2@domain.com,Test User2,/Test,Development
+```
+You want a list of recurring events with at least one external guest.
+```
+gam config csv_output_row_filter "'^attendees$:count>1','recurrence:count>=1','attendees.*email:all:notregex:(^$)|(.+@domain.com)'" csv_output_row_drop_filter "'attendees.*email:regex:.+@resource.calendar.google.com'" redirect csv ./externalrecurringEvents.csv calendar <CalendarEntity> print events
+```
+## Column row limiting
+You can limit the number of rows written to a CSV file.
+
+When single processing, the limit is on the total number of rows written to the file.
+
+When multiprocessing, the limit is on the number of rows written to the file by each subprocess.
+
+### Examples
+Display the 10 files with the largest quotaBytesUsed values for a single user.
+```
+gam config csv_output_row_limit 10 redirect csv ./BigQuotaFiles.csv user user@domain.com print filelist fields id,name,quotabytesused orderby quotabytesused descending
+```
+
+Display the 10 files with the largest quotaBytesUsed values for all users
+```
+gam config csv_output_row_limit 10 auto_batch_min 1 redirect csv ./BigQuotaFiles.csv multiprocess all users print filelist fields id,name,quotabytesused orderby quotabytesused descending
+```
+
+## Saving filters in gam.cfg
+If you define a value for `csv_output_header_filter`, `csv_output_header_drop_filter`, `csv_output_row_filter`, `csv_output_row_drop_filter` or `csv_output_row_limit` in the `[DEFAULT]` section of `gam.cfg`,
+it will apply to every `gam print` command which is probably not desirable. You can store them in `gam.cfg` in named sections.
+```
+[Filter510]
+csv_output_header_filter = primaryEmail,name.fullName,phones.*value
+csv_output_row_filter = 'phones.\\\d+.value:regex:(?:^\\\(510\\\) )|(?:^510[- ])\\\d{3}-\\\d{4}'
+
+$ gam selectfilter Filter510 print users name phone
+primaryEmail,name.fullName,phones.0.value
+testuser1@domain.com,Test User1,(510) 555-1212
+testuser2@domain.com,Test User2,510-555-1212
+testuser3@domain.com,Test User3,510 555-1212
+```
+
+If you have multiple customers or domains in separate sections of gam.cfg, you use `select` to choose the customer/domain
+and `selectfilter` to choose a filter.
+```
+[foo]
+domain = foo.com
+customer_id = C111111111
+config_dir = foo
+
+[goo]
+domain = goo.com
+customer_id = C222222222
+config_dir = goo
+
+[Filter510]
+csv_output_header_filter = primaryEmail,name.fullName,phones.*value
+csv_output_row_filter = 'phones.\\\d+.value:regex:(?:^\\\(510\\\) )|(?:^510[- ])\\\d{3}-\\\d{4}'
+
+$ gam select foo selectfilter Filter510 print users name phone
+primaryEmail,name.fullName,phones.0.value
+testuser1@foo.com,Test User1,(510) 555-1212
+testuser2@foo.com,Test User2,510-555-1212
+testuser3@foo.com,Test User2,510 555-1212
+```

docs/CSV-Special-Characters.md

@@ -0,0 +1,94 @@
+# CSV Special Characters
+- [Python CSV documentation](https://docs.python.org/3/library/csv.html#dialects-and-formatting-parameters)
+
+## Python variables that control CSV file reading/writing:
+```
+Dialect.delimiter
+    A one-character string used to separate fields.
+    It defaults to ','.
+
+Dialect.doublequote
+    Controls how instances of quotechar appearing inside a field should themselves be quoted.
+    When True, the character is doubled. When False, the escapechar is used as a prefix to the quotechar.
+    It defaults to True.
+
+Dialect.escapechar
+    A one-character string used by the writer to escape the delimiter if quoting is set to QUOTE_NONE and the quotechar if doublequote is False.
+    On reading, the escapechar removes any special meaning from the following character.
+    It defaults to None, which disables escaping.
+
+Dialect.lineterminator
+    The string used to terminate lines produced by the writer.
+    It defaults to '\r\n'.
+
+    The reader is hard-coded to recognise either '\r' or '\n' as end-of-line, and ignores lineterminator.
+
+Dialect.quotechar
+    A one-character string used to quote fields containing special characters, such as the delimiter or quotechar, or which contain new-line characters.
+    It defaults to '"'.
+
+Dialect.quoting
+    Controls when quotes should be generated by the writer and recognised by the reader. It can take on any of the QUOTE_* constants (see section Module Contents).
+    It defaults to QUOTE_MINIMAL.
+```
+
+## GAM variables that control CSV file reading/writing:
+```
+csv_input_column_delimiter = , - Dialect.delimiter
+csv_input_no_escape_char = true - Dialect.escapechar is set to None if true, '\' if false
+csv_input_quote_char = " - Dialect.quotechar
+csv_output_column_delimiter = , - Dialect.delimiter
+csv_output_no_escape_char = false - Dialect.escapechar is set to None if true, '\' if false
+csv_output_line_terminator = lf - Dialect.lineterminator
+csv_output_quote_char = " - Dialect.quotechar
+todrive_no_escape_char = true - Dialect.escapechar is set to None if true, '\' if false
+```
+
+GAM sets Dialect.doublequote to true and Dialect.quoting to QUOTE_MINIMAL; there are no variables to change these values.
+
+## Examples
+
+### Local file, default settings
+With these settings, here are examples of how field values are mapped on output to a local file:
+```
+csv_output_column_delimiter = ,
+csv_output_no_escape_char = false
+csv_output_quote_char = "
+```
+
+| Input | Output |
+|-------|--------|
+| abc def | abc def |
+| abc,def | "abc,def" |
+| abc"def | "abc""def" |
+| abc\def | abc\\\\def |
+
+### Local file, modified settings
+With these settings, here are examples of how field values are mapped on output to a local file:
+```
+csv_output_column_delimiter = ,
+csv_output_no_escape_char = true
+csv_output_quote_char = "
+```
+
+| Input | Output |
+|-------|--------|
+| abc def | abc def |
+| abc,def | "abc,def" |
+| abc"def | "abc""def" |
+| abc\def | abc\def |
+
+### todrive, default settings
+With these settings, here are examples of how field values are mapped on output to todrive
+```
+csv_output_column_delimiter = ,
+todrive_no_escape_char = true
+csv_output_quote_char = "
+```
+
+| Input | Output |
+|-------|--------|
+| abc def | abc def |
+| abc,def | "abc,def" |
+| abc"def | "abc""def" |
+| abc\def | abc\def |

docs/Calendars-Access.md

@@ -0,0 +1,86 @@
+# Calendars - Access
+- [Notes](#Notes)
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Manage calendar access](#manage-calendar-access)
+- [Display calendar access](#display-calendar-access)
+- [Old format commands](#old-format-commands)
+
+## Notes
+These commands use Client access for all commands except those that reference user's primary calendars
+where Service Account access is used. When using Client access on user's secondary calendars, some operations are restricted.
+In general, you should use the following commands to manage user's calendars access.
+* [Users - Calendars - Access](Users-Calendars-Access)
+
+Client access works when accessing Resource calendars.
+
+Calendar ACL roles (as seen in Calendar GUI):
+  * `reader` - See all event details
+  * `writer` & `editor`  Make changes to events
+  * `owner` - Make changes to events and manage sharing
+  * `freebusy` & `freebusyreader` - See only free/busy (hide details)
+
+## API documentation
+* https://developers.google.com/calendar/v3/reference/acl
+
+## Definitions
+```
+<CalendarItem> ::= <EmailAddress>
+<CalendarList> ::= "<CalendarItem>(,<CalendarItem>)*"
+<CalendarEntity> ::= <CalendarList> | <FileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+
+<CalendarACLRole> ::= editor|freebusy|freebusyreader|owner|reader|writer
+<CalendarACLScope> ::= <EmailAddress>|user:<EmailAdress>|group:<EmailAddress>|domain:<DomainName>|domain|default
+<CalendarACLScopeList> ::= "<CalendarACLScope>(,<CalendarACLScope>)*"
+<CalendarACLScopeEntity>::= <CalendarACLScopeList> | <FileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+```
+## Manage calendar access
+```
+gam calendars <CalendarEntity> add acls|calendaracls <CalendarACLRole> <CalendarACLScopeEntity> [sendnotifications <Boolean>]
+gam calendars <CalendarEntity> update acls|calendaracls <CalendarACLRole> <CalendarACLScopeEntity> [sendnotifications <Boolean>]
+gam calendars <CalendarEntity> delete acls|calendaracls [<CalendarACLRole>] <CalendarACLScopeEntity>
+```
+By default, when you add or update a calendar ACL, notification is sent to the members referenced in the `<CalendarACLScopeEntity>`.
+Use `sendnotifications false` to suppress sending the notification.
+
+## Display calendar access
+```
+gam calendars <CalendarEntity> info acls|calendaracls <CalendarACLScopeEntity> [formatjson]
+gam calendars <CalendarEntity> show acls|calendaracls
+        [noselfowner]
+        [formatjson]
+```
+Option `noselfowner` suppresses the display of ACLs that reference the calendar itself as its owner.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam calendars <CalendarEntity> print acls|calendaracls [todrive <ToDriveAttribute>*]
+        [noselfowner] (addcsvdata <FieldName> <String>)*
+        [formatjson [quotechar <Character>]]
+```
+Option `noselfowner` suppresses the display of ACLs that reference the calendar itself as its owner.
+
+Add additional columns of data from the command line to the output
+* `addcsvdata <FieldName> <String>`
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+### Old format commands
+These commands are backwards compatible with basic GAM.
+```
+gam calendar <CalendarEntity> add <CalendarACLRole> ([user] <EmailAddress>)|(group <EmailAddress>)|(domain [<DomainName>])|default [sendnotifications <Boolean>]
+gam calendar <CalendarEntity> update <CalendarACLRole> ([user] <EmailAddress>)|(group <EmailAddress>)|(domain [<DomainName>])|default [sendnotifications <Boolean>]
+gam calendar <CalendarEntity> delete [<CalendarACLRole>] ([user] <EmailAddress>)|(group <EmailAddress>)|(domain [<DomainName>])|default
+gam calendar <CalendarEntity> showacl [formatjson]
+gam calendar <CalendarEntity> printacl [todrive <ToDriveAttribute>*]
+        (addcsvdata <FieldName> <String>)*
+        [formatjson [quotechar <Character>]]
+```
+By default, when you add or update a calendar ACL, notification is sent to the members referenced in the `<CalendarACLScopeEntity>`.
+Use `sendnotifications false` to suppress sending the notification.

docs/Calendars-Events.md

@@ -0,0 +1,594 @@
+# Calendars - Events
+- [Notes](#Notes)
+- [API documentation](#api-documentation)
+- [Python Regular Expressions](Python-Regular-Expressions) Search function
+- [Collections of Users](Collections-of-Users)
+- [Definitions](#definitions)
+- [Recurrence rules](#recurrence-rules)
+- [Event colors](#event-colors)
+- [Event selection](#event-selection)
+- [Add and import calendar events](#add-and-import-calendar-events)
+  - [Add calendar attendees](#add-calendar-attendees)
+- [Update calendar events](#update-calendar-events)
+  - [Update calendar attendees](#update-calendar-attendees)
+- [Specify calendar attendees with JSON data](#specify-calendar-attendees-with-json-data)
+- [Delete selected calendar events](#delete-selected-calendar-events)
+- [Delete all calendar events](#delete-all-calendar-events)
+- [Move calendar events to another calendar](#move-calendar-events-to-another-calendar)
+- [Empty calendar trash](#empty-calendar-trash)
+- [Display calendar events](#display-calendar-events)
+- [Old format commands](#old-format-commands)
+
+## Notes
+These commands use Client access for all commands except those that reference user's primary calendars
+where Service Account access is used. When using Client access on user's secondary calendars, some operations are restricted.
+In general, you should use the following commands to manage user's calendars events.
+* [Users - Calendars - Events](Users-Calendars-Events)
+
+Client access works when accessing Resource calendars.
+
+## API documentation:
+* https://developers.google.com/calendar/v3/reference/events
+* https://developers.google.com/calendar/v3/reference/events/import
+
+## Definitions
+```
+<Year> ::= <Digit><Digit><Digit><Digit>
+<Month> ::= <Digit><Digit>
+<Day> ::= <Digit><Digit>
+<Hour> ::= <Digit><Digit>
+<Minute> ::= <Digit><Digit>
+<Second> ::= <Digit><Digit>
+<MilliSeconds> ::= <Digit><Digit><Digit>
+<Date> ::=
+        <Year>-<Month>-<Day> |
+        (+|-)<Number>(d|w|y) |
+        never|
+        today
+<DateTime> ::=
+        <Year>-<Month>-<Day>(<Space>|T)<Hour>:<Minute> |
+        (+|-)<Number>(m|h|d|w|y) |
+        never|
+        now|today
+<Time> ::=
+        <Year>-<Month>-<Day>(<Space>|T)<Hour>:<Minute>:<Second>[.<MilliSeconds>](Z|(+|-(<Hour>:<Minute>))) |
+        (+|-)<Number>(m|h|d|w|y) |
+        never|
+        now|today
+<TimeZone> ::= <String>
+        See: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
+
+<JSONData> ::= (json [charset <Charset>] <String>) | (json file <FileName> [charset <Charset>]) |
+
+<CalendarItem> ::= <EmailAddress>
+<CalendarList> ::= "<CalendarItem>(,<CalendarItem>)*"
+<CalendarEntity> ::= <CalendarList> | <FileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<DomainName> ::= <String>(.<String>)+
+<EmailAddress> ::= <String>@<DomainName>
+<EmailAddressList> ::= "<EmailAddess>(,<EmailAddress>)*"
+<EmailAddressEntity> ::= <EmailAddressList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+
+<EventAttachmentsSubfieldName> ::=
+        attachments.fileid|
+        attachments.fileurl|
+        attachments.iconlink|
+        attachments.mimetype|
+        attachments.title
+
+<EventAttendeesSubfieldName> ::=
+        attendees.additionalguests|
+        attendees.comment|
+        attendees.displayname|
+        attendees.email|
+        attendees.id|
+        attendees.optional|
+        attendees.organizer|
+        attendees.resource|
+        attendees.responseStatus|
+        attendees.self
+
+<EventConferenceDataSubfieldName> ::=
+        conferencedata.conferenceid|
+        conferencedata.conferencesolution|
+        conferencedata.createrequest|
+        conferencedata.entrypoints|
+        conferencedata.notes|
+        conferencedata.signature
+
+<EventCreatorSubfieldName> ::=
+        creator.displayname|
+        creator.email|
+        creator.id|
+        creator.self
+
+<EventFocusTimePropertiesSubfieldName> ::=
+        focustimeproperties.chatstatus|
+        focustimeproperties.declinemode|
+        focustimeproperties.declinemessage
+
+<EventOrganizerSubfieldName> ::=
+        organizer.displayname|
+        organizer.email|
+        organizer.id|
+        organizer.self
+
+<EventOutOfOfficePropertiesSubfieldName> ::=
+        outofoffice.declinemode|
+        outofoffice.declinemessage
+
+<EventWorkingLocationPropertiesSubfieldName> ::=
+        workinglocationproperties.homeoffice|
+        workinglocationproperties.customlocation|
+        workinglocationproperties.officelocation
+
+<EventFieldName> ::=
+        anyonecanaddself|
+        attachments|
+        <EventAttachmentsSubfieldName>|
+        attendees|
+        <EventAttendeesSubfieldName>|
+        attendeesomitted|
+        colorid|
+        conferencedata|
+        <EventConferenceDataSubfieldName>|
+        created|
+        creator|
+        <EventCreatorSubfieldName>|
+        description|
+        end|endtime|
+        endtimeunspecified|
+        extendedproperties|
+        eventtype|
+        <EventFocusTimePropertiesSubfieldName>
+        gadget|
+        guestscaninviteothers|
+        guestscanmodify|
+        guestscanseeotherguests|
+        hangoutlink|
+        htmllink|
+        icaluid|
+        id|
+        location|
+        locked|
+        organizer|
+        <EventOrganizerSubfieldName>|
+        originalstart|originalstarttime|
+        <EventOutOfOfficePropertiesSubfieldName>
+        privatecopy|
+        recurrence|
+        recurringeventid|
+        reminders|
+        sequence|
+        source|
+        start|starttime|
+        status|
+        summary|
+        transparency|
+        updated|
+        visibility|
+        workinglocationproperties|
+        <EventWorkingLocationPropertiesSubfieldName>
+<EventFieldNameList> ::= "<EventFieldName>(,<EventFieldName>)*"
+
+<AttendeeAttendance> ::= optional|required
+<AttendeeStatus> ::= accepted|declined|needsaction|tentative
+
+<EventType> ::=
+        default|
+        focustime|
+        outofoffice|
+        workinglocation
+<EventTypeList> ::= "<EventType>(,<EventType>)*"
+
+<EventSelectProperty> ::=
+        (after|starttime|timemin <Time>)|
+        (before|endtime|timemax <Time>)|
+        (eventtype|eventtypes <EventTypeList>)|
+        (query <QueryCalendar>)|
+        (privateextendedproperty <String>)|
+        (sharedextendedproperty <String>)|
+        showdeletedevents|
+        showhiddeninvitations|
+        singleevents|
+        (updatedmin <Time>)
+
+<EventMatchProperty> ::=
+        (matchfield attendees <EmailAddressEntity>)|
+        (matchfield attendeespattern <RegularExpression>)|
+        (matchfield attendeesstatus [<AttendeeAttendance>] [<AttendeeStatus>] <EmailAddressEntity>)|
+        (matchfield creatoremail <RegularExpression>)|
+        (matchfield creatorname <RegularExpression>)|
+        (matchfield description <RegularExpression>)|
+        (matchfield hangoutlink <RegularExpression>)|
+        (matchfield location <RegularExpression>)|
+        (matchfield organizeremail <RegularExpression>)|
+        (matchfield organizername <RegularExpression>)|
+        (matchfield organizerself <Boolean>)|
+        (matchfield status <RegularExpression>)|
+        (matchfield summary <RegularExpression>)|
+        (matchfield transparency <RegularExpression>)|
+        (matchfield visibility <RegularExpression>)
+
+<EventIDEntity> ::=
+        (id|eventid <EventId>) |
+        (event|events <EventIdList> |
+        <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVSubkeySelector> | <CSVDataSelector>)
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+
+<EventSelectEntity> ::=
+        (<EventSelectProperty>+ <EventMatchProperty>*)
+
+<EventEntity> ::=
+        <EventIDEntity> | <EventSelectEntity>
+
+<EventColorIndex> ::= <Number in range 1-11>
+<EventColorName> ::=
+        banana|basil|blueberry|flamingo|graphite|grape|
+        lavender|peacock|sage|tangerine|tomato
+<PropertyKey> ::= <String>
+<PropertyValue> ::= <String>
+
+<EventAttribute> ::=
+        (allday <Date>)|
+        (anyonecanaddself [<Boolean>])|
+        (attachment <String> <URL>)|
+        (attendee <EmailAddress>)|
+        (attendeestatus [<AttendeeAttendance>] [<AttendeeStatus>] <EmailAddress>)|
+        available|
+        (color <EventColorName>)|
+        (colorindex|colorid <EventColorIndex>)|
+        (description <String>)|
+        (end|endtime (allday <Date>)|<Time>)|
+        (guestscaninviteothers <Boolean>)|
+        guestscantinviteothers|
+        (guestscanmodify <Boolean>)|
+        (guestscanseeotherguests <Boolean>)|
+        guestscantseeotherguests|
+        hangoutsmeet|
+        <JSONData>|
+        (jsonattendees [charset <Charset>] <String>)|
+        (jsonattendees file <FileName> [charset <Charset>])|
+        (location <String>)|
+        (noreminders|(reminder email|popup <Number>))|
+        (optionalattendee <EmailAddress>)|
+        (originalstart|originalstarttime (allday <Date>)|<Time>)|
+        (privateproperty <PropertyKey> <PropertyValue>)|
+        (range <Date> <Date>)|
+        (recurrence <RRULE, EXRULE, RDATE and EXDATE line>)|
+        (reminder <Number> email|popup))|
+        (selectattendees [<AttendeeAttendance>] [<AttendeeStatus>] <UserTypeEntity>)|
+        (sequence <Integer>)|
+        (sharedproperty <PropertyKey> <PropertyValue>)|
+        (source <String> <URL>)|
+        (start|starttime (allday <Date>)|<Time>)|
+        (status confirmed|tentative|cancelled)|
+        (summary <String>)|
+        tentative|
+        (timerange <Time> <Time>)|
+        (timezone <TimeZone>)|
+        (transparency opaque|transparent)|
+        (visibility default|public|private)
+
+The following attributes are equivalent:
+        available - transparency transparent
+        guestscantinviteothers - guestscaninviteothers False
+        guestscantseeothers - guestscanseeotherguests False
+        tentative - status tentative
+
+<EventImportAttribute> ::=
+        <EventAttribute>|
+        (organizername <String>)|
+        (organizeremail <EmailAddress>)
+
+<EventUpdateAttribute> ::=
+        <EventAttribute>|
+        clearattachments|
+        clearattendees|
+        clearhangoutsmeet|
+        (clearprivateproperty <PropertyKey>)|
+        (clearsharedproperty <PropertyKey>)|
+        (removeattendee <EmailAddress>)|
+        (replacedescription <RegularExpression> <String>)|
+        (selectremoveattendees <UserTypeEntity>)
+
+<EventNotificationAttribute> ::=
+        notifyattendees|(sendnotifications <Boolean>)|(sendupdates all|enternalonly|none)
+
+The following attributes are equivalent:
+        notifyattendees - sendupdates all
+        sendnotifications false - sendupdates none
+        sendnotifications true - sendupdates all
+
+<EventDisplayProperty> ::=
+        (alwaysincludeemail)|
+        (icaluid <String>)|
+        (maxattendees <Integer>)|
+        (orderby starttime|updated)|
+        (timezone <TimeZone>)
+```
+## Recurrence rules
+Recurring events require a rule: `recurrence <RRULE, EXRULE, RDATE and EXDATE line>`
+* https://tools.ietf.org/html/rfc5545#section-3.8.5
+
+This is dense reading; a simpler approach is to define a test event in Google Calendar with
+the recurrence rule that you want, then use `gam info event` to get the recurrence rule and use it in subsequent commands.
+
+```
+RRULE:FREQ=DAILY - Daily
+RRULE:FREQ=DAILY;COUNT=30 - Daily for 30 days
+RRULE:FREQ=WEEKLY - Weekly on the same day of the week as the starting day; e.g., every Wednesday
+RRULE:FREQ=WEEKLY;COUNT=13 - Weekly on the same day of the week as the starting day; e.g., every Wednesday, for 13 weeks
+RRULE:FREQ=MONTHLY - Monthly on the same day of the month as the starting day; e.g., every 15th of the month
+RRULE:FREQ=MONTHLY;BYDAY=4TH - Monthly on the fourth instance of the starting day; e.g., every 4th Thursday
+```
+
+## Event colors
+The event color grid presented in calendar.google.com and `<EventColorIndex>` are related like this:
+```
+11:tomato 4:flamingo
+6:tangerine 5:banana
+2:sage 10:basil
+7:peacock 9:blueberry
+1:lavender 3:grape
+8:graphite
+```
+
+## Event selection
+These are the possible values for `<EventEntity>`; you either specify event IDs or properties used to select events.
+If none of the following options are selected, all events are selected.
+* `id|eventid <EventId>` - A single event ID
+* `event|events <EventIdList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVSubkeySelector> | <CSVDataSelector>)` - A collection of event IDs: [Collections of Items](Collections-of-Items)
+* `<EventSelectProperty>* <EventMatchProperty>*` - Properties used to select events
+
+The Google Calendar API processes `<EventSelectProperty>*`; you may specify none or multiple properties.
+* `after|starttime|timemin <Time>` - Lower bound (inclusive) for an event's end time to filter by. If timeMax is set, timeMin must be smaller than timeMax.
+* `before|endtime|timemax <Time>` - Upper bound (exclusive) for an event's start time to filter by. If timeMin is set, timeMax must be greater than timeMin.
+* `eventtypes <EventTypeList>` - Select events based on their type.
+* `query <QueryCalendar>` - Free text search terms to find events that match these terms in any field, except for extended properties
+* `privateextendedproperty <String>` - A required private property; `<String>` must be of the form `propertyName=value`
+* `sharedextendedproperty <String>` - A required shared property; `<String>` must be of the form `propertyName=value`
+* `showdeletedevents` - Whether to include deleted events (with status equals "cancelled") in the result
+* `showhiddeninvitations` - Whether to include hidden invitations in the result
+* `singleevents` - Whether to expand recurring events into instances and only return single one-off events and instances of recurring events, but not the underlying recurring events themselves
+* `updatedmin <Time>` - Lower bound for an event's last modification time (as a RFC3339 timestamp) to filter by. When specified, entries deleted since this time will always be included regardless of showdeletedevents
+
+GAM processes `<EventMatchProperty>*`; you may specify none or multiple properties.
+* `matchfield attendees <EmailAddressEntity>` - All of the attendees in `<EmailAddressEntity>` must be present
+* `matchfield attendeespattern <RegularExpression>` - Some attendee must match `<RegularExpression>`
+* `matchfield attendeesstatus [<AttendeeAttendance>] [<AttendeeStatus>] <EmailAddressEntity>` - All of the attendees in `<EmailAddressEntity>` must be present
+and must have the specified values.
+    * `<AttendeeAttendance>` - Default is `required`
+    * `<AttendanceStatus>` - Default is`needsaction`
+* `matchfield creatoremail <RegularExpression>` - The creator email address must match `<RegularExpression>`
+* `matchfield creatorname <RegularExpression>` - The creator name must match `<RegularExpression>`
+* `matchfield description <RegularExpression>` - The description (summary) must match `<RegularExpression>`
+* `matchfield location <RegularExpression>` - The location must match `<RegularExpression>`
+* `matchfield organizeremail <RegularExpression>` - The organizer email address must match `<RegularExpression>`
+* `matchfield organizername <RegularExpression>` - The orgainzer name must match `<RegularExpression>`
+* `matchfield status <RegularExpression>` - The summary must match `<RegularExpression>`. The API documented values are:
+    * `confirmed`
+    * `tentative`
+    * `cancelled`
+* `matchfield summary <RegularExpression>` - The summary must match `<RegularExpression>`
+* `matchfield transparency <RegularExpression>` - The summary must match `<RegularExpression>`. The API documented values are:
+    * `opaque` - Busy. The API does not seem to return this value; use `"(^$)|opaque"` to match no value or `opaque`.
+    * `transparent` -  Free/Available
+* `matchfield visibility <RegularExpression>` - The summary must match `<RegularExpression>`. The API documented values are:
+    * `default` - The API does not seem to return this value; use `"(^$)|default"` to match no value or `default`.
+    * `public` - The API does not seem to return this value if it is the default; use `"(^$)|public"` to match no value or `public`.
+    * `private` - The API does not seem to return this value if it is the default; use `"(^$)|private"` to match no value or `private`.
+    * `confidential`
+
+## Add and import calendar events
+```
+gam calendar <CalendarEntity> add event [id <String>] <EventAttribute>+ [<EventNotificationAttribute>]
+        [showdayofweek]
+        [csv [todrive <ToDriveAttribute>*] [formatjson [quotechar <Character>]]]
+gam calendar <CalendarEntity> import event icaluid <iCalUID> <EventImportAttribute>+
+        [showdayofweek]
+        [csv [todrive <ToDriveAttribute>*] [formatjson [quotechar <Character>]]]
+```
+By default, when an event is created|imported, GAM outputs the calendar name and event ID.
+* `csv [todrive <ToDriveAttribute>*] [formatjson [quotechar <Character>]]` - Output the event details in CSV format.
+
+You can specify multiple attachments; `<String>` is the title of the attachment and `<URL>` is a sharable link from Google Drive.
+You must specify all attachments in each command, you can not incrementally add attachments.
+
+Importing events is similar to adding events; the principal difference
+is that you must specify an `iCalUID`. All instances of recurring events will have the same
+`iCalUID` but different `EventIDs`. The import command supports two new attributes to set the
+event organizer, but the API doesn't seem to honor the values; the organizer is set to
+the calendar owner.
+
+## Add calendar attendees
+You can specify attendees in the following ways:
+* `attendee <EmailAddress>` - The attendee attendance is required with status `needsaction'
+* `optionalattendee <EmailAddress>` - The attendee attendance is optional with status `needsaction'
+* `attendeestatus [<AttendeeAttendance>] [<AttendeeStatus>] <EmailAddress>` - One attendee
+  * If `<AttendeeAttendance>` is not specified, the attendee is required to attend
+  * If `<AttendeeStatus>` is not specified, `needsaction` is chosen
+* `jsonattendees [charset <Charset>] <String>`
+* `jsonattendees file <FileName> [charset <Charset>]`
+* `selectattendees [<AttendeeAttendance>] [<AttendeeStatus>] <UserTypeEntity>` - Multiple attendees
+  * If `<AttendeeAttendance>` is not specified, all attendees are required to attend
+  * If `<AttendeeStatus>` is not specified, `needsaction` is chosen
+
+To add an attendee to a single recurring calendar event, you need to specify the ID of that specific event. 
+```
+gam calendar <CalendarEntity> update event id xxxxxxx_YYYYMMDDTHHMMSSZ attendee attendee@domain.com
+```
+For `<UserTypeEntity>` See: [Collections of Users](Collections-of-Users)
+
+## Update calendar events
+```
+gam calendar <CalendarEntity> update event [<EventEntity>] <EventUpdateAttribute>+ [<EventNotificationAttribute>]
+        [showdayofweek]
+        [csv [todrive <ToDriveAttribute>*] [formatjson [quotechar <Character>]]]
+```
+If `<EventEntity>` is not specified, all events in `<CalendarEntity>` are selected. This is not typically used
+unless you're trying to change a basic `<EventAttribute>`, e.g., `color`, on all events.
+
+By default, when an event is updated, GAM outputs the calendar name and event ID.
+* `csv [todrive <ToDriveAttribute>*] [formatjson [quotechar <Character>]]` - Output the event details in CSV format.
+
+You can clear/modify existing attributes:
+* `clearattachments` - Delete all attachments
+* `clearhangoutsmeet` - Clear Hangouts/Meet link
+* `clearprivateproperty <PropertyKey>` - Clear private properties
+* `clearsharedproperty <PropertyKey>` - Clear shared properties
+* `replacedescription <RegularExpression> <String>` - Modify the description
+
+## Update calendar attendees
+The default behavior of `gam calendar <CalendarEntity> update events` has been changed regarding attendees.
+In versions of GAM before `5.02.00`, updating attendees in calendar events was complicated because you had to
+supply the complete attendee list even if you just wanted incremental changes.
+The default behavior now is to allow incremental changes to the attendees list;
+the current attendee list is downloaded and the specified changes are applied.
+
+The `replacemode` option invokes the previous behavior from versions before `5.02.00`; the current attendee list is replaced.
+
+You can add attendees in the following ways:
+* `attendee <EmailAddress>` - The attendee attendance is required with status `needsaction'
+* `optionalattendee <EmailAddress>` - The attendee attendance is optional with status `needsaction'
+* `attendeestatus [<AttendeeAttendance>] [<AttendeeStatus>] <EmailAddress>` - One attendee
+  * If `<AttendeeAttendance>` is not specified, the attendee is required to attend
+  * If `<AttendeeStatus>` is not specified, `needsaction` is chosen
+* `jsonattendees [charset <Charset>] <String>`
+* `jsonattendees file <FileName> [charset <Charset>]`
+* `selectattendees [<AttendeeAttendance>] [<AttendeeStatus>] <UserTypeEntity>` - Multiple attendees
+  * If `<AttendeeAttendance>` is not specified, all attendees are required to attend
+  * If `<AttendeeStatus>` is not specified, `needsaction` is chosen
+
+You can remove attendees in the following ways:
+* `clearattendees` - Clear all current attendees from the attendee list
+* `removeattendee <EmailAddress>` - Remove a single attendee from the attendee list
+* `selectremoveattendees <UserTypeEntity>` - Remove a selected collection of attendees from the attendee list
+
+For `<UserTypeEntity>` See: [Collections of Users](Collections-of-Users)
+
+## Specify calendar attendees with JSON data
+You can predefine lists of attendees and use them when creating/updating events. If you set `responseStatus` to `accepted`, no notifications are sent.
+```
+$ more attendees.json
+{"attendees": [{"email": "testuser2@domain.com", "responseStatus": "needsAction", "optional": "True"}, {"email": "testuser3@domain.com", "responseStatus": "accepted"}, {"email": "testuser4@domain.com", "responseStatus": "accepted"}]}
+```
+You can use output the attendee information for an event in a calendar and use that data when defining other events.
+```
+$ gam redirect stdout ./attendees.json calendar testuser1@domain.com info event id 0000h8kk7c9o2tonk73hu2zzzz fields attendees formatjson 
+$ more attendees.json 
+{"calendarId": "testuser1@domain.com", "event": {"attendees": [{"email": "testuser3@domain.com", "responseStatus": "accepted"}, {"email": "testuser4@domain.com", "responseStatus": "accepted"}], "id": "0000h8kk7c9o2tonk73hu2zzzz"}}
+```
+Use `jsonattendees file ./attendees.json` in `create/update event`.
+
+## Delete selected calendar events
+```
+gam calendar <CalendarEntity> delete events [<EventEntity>] [doit] [<EventNotificationAttribute>]
+gam calendar <CalendarEntity> purge events [<EventEntity>] [doit] [<EventNotificationAttribute>]
+```
+If `<EventEntity>` is not specified, all events in `<CalendarEntity>` are selected. This is not typically used.
+
+No events are deleted unless you specify the `doit` option; omit `doit` to verify that you properly selected the events to delete.
+
+When events are deleted from a calendar, they are moved to the calendar's trash and are only permanently deleted (purged) after 30 days.
+Following a suggestion here (https://stackoverflow.com/questions/41043053/how-to-empty-calendar-trash-via-google-services) you can permanently delete
+calendar events with `purge events`. This is achieved by creating a temporary calendar, deleting the events, moving the deleted events to the temporary calendar
+and then deleting the temporary calendar.
+
+## Delete all calendar events
+For a user's primary calendar:
+```
+gam calendar <CalendarEntity> wipe events
+```
+For non-primary calendars:
+```
+gam calendar <CalendarEntity> delete events [doit] [<EventNotificationAttribute>]
+```
+No events are deleted unless you specify the `doit` option; omit `doit` to verify that you properly selected the events to delete.
+
+## Move calendar events to another calendar
+Generally you won't move all events from one calendar to another; typically, you'll move events created by the event creator
+using `matchfield creatoremail <RegularExpression>` in conjunction with other `<EventSelectProperty>` and `<EventMatchProperty>` options.
+```
+gam calendar <CalendarEntity> move event [<EventEntity>] destination|to <CalendarItem> [<EventNotificationAttribute>]
+```
+
+## Empty calendar trash
+A user signed in to Google Calendar can empty the calendar trash but there is no direct API support for this operation.
+To empty the calendar trash a temporary calendar is created, the deleted events are moved to the temporary calendar and then the temporary calendar is deleted.
+```
+gam calendar|calendars <CalendarEntity> empty calendartrash
+```
+
+## Display calendar events
+```
+gam calendar <CalendarEntity> info events [<EventEntity>] [maxinstances <Number>]
+        [fields <EventFieldNameList>] [showdayofweek]
+        [formatjson]
+```
+In `<EventEntity>`, any `<EventSelectProperty>` options must precede all other options.
+
+* `maxinstances -1` - Default, display base event
+* `maxinstances 0` - Display all instances of a recurring event
+* `maxinstances N` - Display first N instances of a recurring event
+
+`showdayofweek` displays `dayOfWeek` when event start and end times are displayed.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam calendar <CalendarEntity> show events [<EventEntity>] <EventDisplayProperty>*
+        [fields <EventFieldNameList>] [showdayofweek]
+        [countsonly] [formatjson]
+```
+In `<EventEntity>`, any `<EventSelectProperty>` options must precede all other options.
+
+By default, only the base event of a recurring event is displayed. Use the `<EventSelectProperty>`
+option `singleevents` to display all instances of a recurring event.
+
+`<EventDisplayProperty> orderby starttime` is only valid with `<EventSelectProperty> singleevents`.
+
+`showdayofweek` displays `dayOfWeek` when event start and end times are displayed.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+By default, Gam displays event details, use `countsonly` to display only the number of events. `formatjson` does not apply in this case.
+
+```
+gam calendar <CalendarEntity> print events [<EventEntity>] <EventDisplayProperty>*
+         [fields <EventFieldNameList>] [showdayofweek]
+         [countsonly] [formatjson [quotechar <Character>]] [todrive <ToDriveAttribute>*]
+```
+In `<EventEntity>`, any `<EventSelectProperty>` options must precede all other options.
+
+By default, only the base event of a recurring event is displayed. Use the `<EventSelectProperty>`
+option `singleevents` to display all instances of a recurring event.
+
+`<EventDisplayProperty> orderby starttime` is only valid with `<EventSelectProperty> singleevents`.
+
+`showdayofweek` displays columns `start.dayOfWeek` and `end.dayOfWeek` when event start and end times are displayed.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, Gam displays event details, use `countsonly` to display only the number of events. `formatjson` does not apply in this case.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+### Old format commands
+These commands are backwards compatible with basic Gam.
+```
+gam calendar <CalendarEntity> addevent <EventAttribute>+ [<EventNotificationAttribute>]
+gam calendar <CalendarEntity> deleteevent (id|eventid <EventID>)+ [doit] [<EventNotificationAttribute>]
+gam calendar <CalendarEntity> moveevent (id|eventid <EventID>)+ destination <CalendarItem> [<EventNotificationAttribute>]
+gam calendar <CalendarEntity> updateevent <EventID> <EventAttribute>+ [<EventNotificationAttribute>]
+gam calendar <CalendarEntity> wipe
+gam calendar <CalendarEntity> printevents <EventSelectProperty>* <EventDisplayProperty>* [fields <EventFieldNameList>]
+         [formatjson [quotechar <Character>]] [todrive <ToDriveAttribute>*]
+```

docs/Calendars.md

@@ -0,0 +1,66 @@
+# Calendars
+- [Notes](#Notes)
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Modify calendar settings](#modify-calendar-settings)
+- [Display calendar settings](#display-calendar-settings)
+
+## Notes
+These commands use Client access for all commands except those that reference user's primary calendars
+where Service Account access is used. When using Client access on user's secondary calendars, some operations are restricted.
+In general, you should use the following commands to manage user's calendars.
+* [Users - Calendars](Users-Calendars)
+
+Client access works when accessing Resource calendars.
+
+## API documentation
+* https://developers.google.com/google-apps/calendar/v3/reference/calendars
+
+## Definitions
+```
+<CalendarItem> ::= <EmailAddress>
+<CalendarList> ::= "<CalendarItem>(,<CalendarItem>)*"
+<CalendarEntity> ::= <CalendarList> | <FileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+
+<TimeZone> ::= <String>
+        See: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
+
+<CalendarSettings> ::=
+        (description <String>)|
+        (location <String>)|
+        (summary <String>)|
+        (timezone <TimeZone>)
+
+<CalendarSettingsField> ::=
+        conferenceproperties|
+        description|
+        id|
+        location|
+        summary|
+        timezone
+<CalendarSettingsFieldList> ::= "<CalendarSettingsField>(,<CalendarSettingsField>)*"
+```
+## Modify calendar settings
+```
+gam calendar <CalendarEntity> modify <CalendarSettings>+
+```
+## Display calendar settings
+```
+gam calendar <CalendarEntity> show settings
+        [fields <CalendarSettingsFieldList>]
+        [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam calendar <CalendarEntity> print settings [todrive <ToDriveAttribute>*]
+        [fields <CalendarSettingsFieldList>]
+        [formatjson [quotechar <Character>]]
+```
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+

docs/Chat-Bot.md

@@ -0,0 +1,251 @@
+# Chat Bot
+
+- [Notes](#notes)
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Set up a Chat Bot](#set-up-a-chat-bot)
+- [Display Rooms and Chats to which your Bot belongs](#display-rooms-and-chats-to-which-your-bot-belongs)
+- [Display Members of a Room or Chat](#display-members-of-a-room-or-chat)
+- [Create a Chat Message](#create-a-chat-message)
+- [Update a Chat Message](#update-a-chat-message)
+- [Delete a Chat Message](#delete-a-chat-message)
+- [Display a Chat Message](#display-a-chat-message)
+
+## Notes
+This Wiki page was built directly from Jay Lee's Wiki page; my sincere thanks for his efforts.
+
+## API documentation
+* https://developers.google.com/chat/concepts
+* https://developers.google.com/chat/reference/rest
+* https://support.google.com/chat/answer/7655820
+
+## Definitions
+* [Drive File Selection](Drive-File-Selection) for symbols not listed here, such as `<DriveFileIDEntity>`
+* [Command data from Google Docs/Sheets/Storage](Command-Data-From-Google-Docs-Sheets-Storage)
+```
+<StorageBucketName> ::= <String>
+<StorageObjectName> ::= <String>
+<StorageBucketObjectName> ::=
+        https://storage.cloud.google.com/<StorageBucketName>/<StorageObjectName>|
+        https://storage.googleapis.com/<StorageBucketName>/<StorageObjectName>|
+        gs://<StorageBucketName>/<StorageObjectName>|
+        <StorageBucketName>/<StorageObjectName>
+
+<UserGoogleDoc> ::=
+        <EmailAddress> <DriveFileIDEntity>|<DriveFileNameEntity>|(<SharedDriveEntity> <SharedDriveFileNameEntity>)
+
+<ChatContent> ::=
+        ((text <String>)|
+         (textfile <FileName> [charset <Charset>])|
+         (gdoc <UserGoogleDoc>)|
+         (gcsdoc <StorageBucketObjectName>))
+
+<ChatMember> ::= spaces/<String>/members/<String>
+<ChatMessage> ::= spaces/<String>/messages/<String>
+<ChatSpace> ::= spaces/<String> | space <String> | space spaces/<String>
+<ChatThread> ::= spaces/<String>/threads/<String>
+<ChatMessageID> ::= client-<String>
+        <String> must contain only lowercase letters, numbers, and hyphens up to 56 characters in length.
+```
+
+## Set up a Chat Bot
+Since GAM 6.04.00, GAM is capable of acting as a Chat Bot and sending messages to Chat Rooms or direct messages to users. You first need to configure your Chat Bot.
+
+* Run the command `gam setup chat`; it will point you to a URL to configure your Chat Bot.
+* Enter an App name and Description of your choosing.
+* For the Avatar URL you can use `https://dummyimage.com/384x256/4d4d4d/0011ff.png&text=+GAM` or a public URL to an image of your own choosing.
+* In Functionality, uncheck both "Receive 1:1 messages" and "Join spaces and group conversations"
+* In Connection settings, choose "Cloud Pub/Sub" and enter "no-topic" for the topic name. GAM doesn't yet listen to pub/sub so this option is not used.
+* In Visibility, uncheck "Make this Chat app available to specific people and groups in  Domain Workspace".
+* Click Save.
+
+----
+
+## Display Rooms and Chats to which your Bot belongs
+Display the spaces to which your Chat Bot can send messages.
+A space can be a direct message to a user, a chat group or a chat room.
+At first you'll have no spaces listed. Try [finding your bot and chatting it](https://support.google.com/chat/answer/7655820) and then your space will be listed.
+
+### Display information about a specific chat space
+```
+gam info chatspace space <ChatSpace>
+        [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+### Display information about all chat spaces
+```
+gam show chatspaces
+        [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+```
+gam print chatspaces [todrive <ToDriveAttribute>*]
+        [formatjson [quotechar <Character>]]
+```
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+----
+
+## Display Members of a Room or Chat
+### Display information about a specific chat member
+```
+gam info chatmember member <ChatMember>
+        [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+### Display information about all chat members in a chat space
+```
+gam show chatmembers space <ChatSpace>
+        [showinvited [<Boolean>]] [showgroups [<Boolean>]] [filter <String>]
+        [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+```
+gam print chatmembers [todrive <ToDriveAttribute>*] space <ChatSpace>
+        [showinvited [<Boolean>]] [showgroups [<Boolean>]] [filter <String>]
+        [formatjson [quotechar <Character>]]
+```
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+By default, only `JOINED` members are displayed; use `showinvited` to also display `INVITED` members.
+
+Use `filter <String>` to filter memberships by a member's role and membertype.
+  * To filter by role, set role to ROLE_MEMBER or ROLE_MANAGER.
+  * To filter by type, set member.type to HUMAN or BOT.
+  * To filter by both role and type, use the AND operator.
+  * To filter by either role or type, use the OR operator.
+
+For example, the following queries are valid:
+```
+role = "ROLE_MANAGER" OR role = "ROLE_MEMBER"
+member.type = "HUMAN" AND role = "ROLE_MANAGER"
+```
+The following queries are invalid:
+```
+member.type = "HUMAN" AND member.type = "BOT"
+role = "ROLE_MANAGER" AND role = "ROLE_MEMBER"
+```
+
+## Create a Chat Message
+Create a chat message in a space. Messages are limited to 4,096 characters and will be trimmed to that length.
+
+Chat supports [simple formatting](https://developers.google.com/chat/reference/message-formats/basic#using_formatted_text_in_messages) allowing you to bold, underline, italics and strikethrough your text.
+```
+gam create chatmessage space <ChatSpace>
+        <ChatContent>
+        [messageId <ChatMessageID>]
+        [(thread <ChatThread>)|(threadkey <String>) [replyoption fail|fallbacktonew]]
+        [returnidonly]
+```
+Specify the text of the message: `<ChatContent>`
+* `text <String>` - The message is `<String>`
+* `textfile <FileName> [charset <Charset>]` - The message is read from a local file
+* `gdoc <UserGoogleDoc>` - The message is read from a Google Doc.
+* `gcsdoc <StorageBucketObjectName>` - The message is read from a Google Cloud Storage file.
+
+By default, a new message thread is created; use `thread <ChatThread>` or `threadkey <String>` to create the message as a reply to an existing thread.
+Use `replyoption` to specify what happens if the specified thread does not exist:
+* `fail` - If the thread soes not exiat, a `Not Found` error is generated
+* `fallbacktonew` - If the thread does not exist, start a new thread
+
+The first time you reply to a thread you must use `thread <ChatThread>`; if you also specify `threadkey <String>`
+then you can use just `threadkey <String>` in subsequent replies.
+
+If you specify `thread` or `threadkey` but not `replyoption`, the default is `fail'.
+
+By default, details about the chat message are displayed.
+* `returnidonly` - Display the chat message name only
+
+### Examples
+This example creates a new chat message in the given room.
+```
+gam create chatmessage space spaces/iEMj8AAAAAE text "Hello Chat"
+```
+This example creates a formatted message and posts it to an existing thread
+```
+gam create chatmessage space spaces/AAAADi-pvqc thread spaces/AAAADi-pvqc/threads/FMNw-iE9jN4 text "*Bold* _Italics_ ~Strikethrough~"
+```
+This example reads the MotD.txt file and posts its contents to Chat.
+```
+gam create chatmessage spaces spaces/AAAADi-pvqc textfile MotD.txt
+```
+This example reads the Google Doc MotD and posts its contents to Chat.
+```
+gam create chatmessage spaces spaces/AAAADi-pvqc gdoc announcements@domain.com name "MotD"
+```
+
+----
+
+## Update a Chat Message
+Updates and rewrites an existing Chat message. Message will show as edited and no notification will be sent to members.
+```
+gam update chatmessage name <ChatMessage>
+        <ChatContent>
+```
+Specify the source of the message:
+* `text <String>` - The message is `<String>`
+* `textfile <FileName> [charset <Charset>]` - The message is read from a local file
+* `gdoc <UserGoogleDoc>` - The message is read from a Google Doc.
+* `gcsdoc <StorageBucketObjectName>` - The message is read from a Google Cloud Storage file.
+
+### Example
+This example updates an existing chat message with new text.
+```
+gam update chatmessage name spaces/AAAADi-pvqc/messages/PKJrx90ooIU.PKJrx90ooIU text "HELLO CHAT?"
+```
+
+----
+
+## Delete a Chat Message
+Deletes the given Chat message. Members will no longer see the message.
+
+```
+gam delete chatmessage name <ChatMessage>
+```
+
+### Example
+```
+gam delete chatmessage name spaces/AAAADi-pvqc/messages/PKJrx90ooIU.PKJrx90ooIU
+```
+
+----
+
+## Display a Chat Message
+Display the given Chat message.
+
+```
+gam info chatmessage name <ChatMessage>
+        [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+### Example
+```
+gam info chatmessage name spaces/AAAADi-pvqc/messages/PKJrx90ooIU.PKJrx90ooIU
+```
+
+----

docs/Chrome-AUE-Counts.md

@@ -0,0 +1,90 @@
+# Chrome Auto Update Expiration Counts
+
+- [Chrome Auto Update Expiration Counts](#chrome-auto-update-expiration-counts)
+  - [API documentation](#api-documentation)
+  - [Definitions](#definitions)
+  - [Quoting rules](#quoting-rules)
+  - [Display Chrome auto update expiration counts](#display-chrome-auto-update-expiration-counts)
+
+## API documentation
+
+* https://developers.google.com/chrome/management/reference/rest/v1/customers.reports/countChromeDevicesReachingAutoExpirationDate
+
+## Notes
+To use these features you must add the `Chrome Management API` to your project and authorize
+the appropriate scope: `Chrome Management API - read only`.
+```
+gam update project
+gam oauth create
+```
+
+## Definitions
+```
+<Date> ::=
+        <Year>-<Month>-<Day> |
+        (+|-)<Number>(d|w|y) |
+        never|
+        today
+<OrgUnitID> ::= id:<String>
+<OrgUnitPath> ::= /|(/<String>)+
+<OrgUnitItem> ::= <OrgUnitID>|<OrgUnitPath>
+<OrgUnitList> ::= "<OrgUnitItem>(,<OrgUnitItem>)*"
+```
+## Quoting rules
+Items in a list can be separated by commas or spaces; if an item itself contains a comma, a space or a single quote, special quoting must be used.
+Typically, you will enclose the entire list in double quotes and quote each item in the list as detailed below.
+
+- Items, separated by commas, without spaces, commas or single quotes in the items themselves
+   * ```"item,item,item"```
+- Items, separated by spaces, without spaces, commas or single quotes in the items themselves
+   * ```"item item item"```
+- Items, separated by commas, with spaces, commas or single quotes in the items themselves
+   * ```"'it em','it,em',\"it'em\""```
+- Items, separated by spaces, with spaces, commas or single quotes in the items themselves
+   * ```"'it em' 'it,em' \"it'em\""```
+
+## Display Chrome auto update expiration counts
+These counts are for provisioned devices.
+```
+gam show chromeaues
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [minauedate <Date>] [maxauedate <Date>]
+        [formatjson]
+```
+Use these options to select Chrome devices; if none are chosen, all Chrome devices in the account are selected.
+
+- `ou <OrgUnitItem>` - Select devices directly in the OU `<OrgUnitItem>`
+- `ou_and_children <OrgUnitItem>` - Select devices in the OU `<OrgUnitItem>` and its sub OUs
+- `ous <OrgUnitList>` - Select devices directly in the OUs `<OrgUnitList>`
+- `ous_and_children <OrgUnitList>` - Select devices in the OUs `<OrgUnitList>` and their sub OUs
+- `minauedate <Date>` - Devices that have already expired and devices with auto expiration date equal to or later than the minimum date
+- `maxauedate <Date>` - Devices that have already expired and devices with auto expiration date equal to or earlier than the maximum date
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+```
+gam print chromeaues [todrive <ToDriveAttribute>*]
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [minauedate <Date>] [maxauedate <Date>]
+        [formatjson [quotechar <Character>]]
+```
+Use these options to select Chrome devices; if none are chosen, all Chrome devices in the account are selected.
+
+- `ou <OrgUnitItem>` - Select devices directly in the OU `<OrgUnitItem>`
+- `ou_and_children <OrgUnitItem>` - Select devices in the OU `<OrgUnitItem>` and its sub OUs
+- `ous <OrgUnitList>` - Select devices directly in the OUs `<OrgUnitList>`
+- `ous_and_children <OrgUnitList>` - Select devices in the OUs `<OrgUnitList>` and their sub OUs
+- `minauedate <Date>` - Devices that have already expired and devices with auto expiration date equal to or later than the minimum date
+- `maxauedate <Date>` - Devices that have already expired and devices with auto expiration date equal to or earlier than the maximum date
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Chrome-Browser-Cloud-Management.md

@@ -0,0 +1,425 @@
+# Chrome Browser Cloud Management
+
+- [Chrome Browser Cloud Management](#chrome-browser-cloud-management)
+  - [API documentation](#api-documentation)
+  - [Query documentation](#query-documentation)
+  - [Definitions](#definitions)
+  - [Manage Chrome browsers](#manage-chrome-browsers)
+    - [Update Chrome browsers](#update-chrome-browsers)
+      - [Example: Add a new note to existing notes](#example-add-a-new-note-to-existing-notes)
+    - [Move Chrome browsers from one OU to another](#move-chrome-browsers-from-one-ou-to-another)
+    - [Delete Chrome browsers](#delete-chrome-browsers)
+  - [Display Chrome browsers](#display-chrome-browsers)
+    - [Examples](#examples)
+  - [Browser Query Searchable Fields](#browser-query-searchable-fields)
+  - [Manage Chrome browser enrollment tokens](#manage-chrome-browser-enrollment-tokens)
+  - [Display Chrome browser enrollment tokens](#display-chrome-browser-enrollment-tokens)
+
+## API documentation
+
+* https://support.google.com/chrome/a/answer/9681204
+* https://support.google.com/chrome/a/answer/9949706
+
+## Query documentation
+* https://support.google.com/chrome/a/answer/9681204#retrieve_all_chrome_devices_for_an_account
+
+## Definitions
+* [`<CrOSTypeEntity>`](Collections-of-ChromeOS-Devices)
+
+```
+<BrowserTokenPermanentID> ::= <String>
+<OrgUnitPath> ::= /|(/<String)+
+<QueryBrowser> ::= <String> See: https://support.google.com/chrome/a/answer/9681204#retrieve_all_chrome_devices_for_an_account
+<QueryBrowserList> ::= "<QueryBrowser>(,<QueryBrowser>)*"
+<QueryBrowserToken> ::= <String> https://support.google.com/chrome/a/answer/9949706, scroll down to Filter Query Language
+<QueryBrowserTokenList> ::= "<QueryBrowserToken>(,<QueryBrowserToken>)*"
+<DeviceID> ::= <String>
+<DeviceIDList> ::= "<DeviceID>(,<DeviceID>)*"
+
+<BrowserEntity> ::=
+        <DeviceIDList> | 
+        (query:<QueryBrowser>)|(query:orgunitpath:<OrgUnitPath>)|(query <QueryBrowser>) |
+        (browserou <OrgUnitItem>) | (browserous <OrgUnitList>) |
+        <FileSelector> | <CSVFileSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+
+<BrowserAttribute> ::=
+        (annotatedassetid|asset|assetid <String>)|
+        (annotatedlocation|location <String>)|
+        (annotatednotes|notes <String>)|(updatenotes <String>)|
+        (annotateduser|user <String>
+
+<BrowserFieldName> ::=
+        annotatedassetid|asset|assetid|
+        annotatedlocation|location|
+        annotatednotes|notes|
+        annotateduser|user|
+        browsers|
+        browserversions|
+        deviceid|
+        deviceidentifiershistory|
+        extensioncount|
+        lastactivitytime|
+        lastdeviceuser|
+        lastdeviceusers|
+        lastpolicyfetchtime|
+        lastregistrationtime|
+        laststatusreporttime|
+        machinename|
+        machinepolicies|
+        orgunitpath|org|orgunit|ou|
+        osarchitecture|
+        osplatform|
+        osplatformversion|
+        osversion|
+        policycount|
+        safebrowsingclickthroughcount|
+        serialnumber|
+        virtualdeviceid
+<BrowserFieldNameList> ::= "<BrowseFieldName>(,<BrowserFieldName>)*"
+
+<BrowserOrderByFieldName> ::=
+        annotatedassetid|assetassetid|
+        annotatedlocation|location|
+        annotatednotes|notes|
+        annotateduser|user|
+        browserversionchannel|
+        browserversionsortable|
+        deviceid|id|
+        enrollmentdate|
+        extensioncount|
+        lastactivity|
+        lastsignedinuser|
+        lastsync|
+        machinename|
+        orgunit|ou|org|
+        osversion|
+        osversionsortable|
+        platformmajorversion|
+        policycount
+```
+```
+<BrowserTokenFieldName> ::=
+        createtime|
+        creatorid|
+        customerid|
+        expiretime|
+        org|
+        orgunit|
+        orgunitpath|
+        revoketime|
+        revokerid|
+        state|
+        token|
+        tokenpermanentid
+<BrowserTokenFieldNameList> ::= "<BrowseTokenFieldName>(,<BrowserTokenFieldName>)*"
+```
+## Manage Chrome browsers
+## Update Chrome browsers
+There are four attributes that can be set for a browser.
+```
+gam update browser <BrowserDeviceEntity> <BrowserAttibute>+
+```
+
+### Example: Add a new note to existing notes
+
+If you specify the `updatenotes <String>` option and it contains the string `#notes#`, the existing notes value will replace `#notes#`.
+This requires an additional API to get the existing value.
+
+If you have a CSV file, UpdateBrowsers.csv with two columns: deviceId,notes
+this command will add a new line of notes to the front of the existing notes:
+
+```
+gam csv UpdateBrowsers.csv gam update browser ~deviceId updatenotes "~~notes~~\n#notes#"
+```
+
+## Move Chrome browsers from one OU to another
+```
+gam move browsers ou|org|orgunit <OrgUnitPath>
+        ((ids <DeviceIDList>) |
+         (queries <QueryBrowserList> [querytime<String> <Time>]) |
+         (browserou <OrgUnitItem>) | (browserous <OrgUnitList>) |
+         <FileSelector> | <CSVFileSelector>)
+        [batchsize <Integer>]
+```
+
+Batches of devices are processed to minimize the number of API calls; `batch_size` controls the number of deviceIds handled in each batch
+`batch_size` defaults to the value from `gam.cfg`, its maximum value is 600.
+
+Google performs  error checking of the browser  deviceIDs, if any deviceID in a batch is invalid, none of the browsers in the batch are moved.
+
+### Example: Move Chrome browsers from one OU to another
+
+```
+gam move browsers ou /Students/2021 browserou /Students/2020
+```
+
+## Delete Chrome browsers
+Deletes a browser; the browser will be removed from Google's admin console and no longer sync policy or reporting. However, existing policies will still be applied until the device registration and dm tokens are removed.
+```
+gam delete browser <BrowserDeviceEntity>
+```
+
+## Display Chrome browsers
+```
+gam info browser <BrowserEntity>
+        [basic|full|annotated] <BrowserFieldName>* [fields <BrowserFieldNameList>]
+        [formatjson]
+```
+Select the fields to be displayed:
+* `annotated` - Display these fields: deviceId,annotatedAssetId,annotatedLocation,annotatedNotes,annotatedUser
+* `basic` - Display all fields except: browsers, lastDeviceUsers, lastStatusReportTime, machinePolicies; this is the default
+* `allfields/full` - Display all fields
+* `<BrowserFieldName>* [fields <BrowserFieldNameList>]` - Display a selected list of fields
+
+By default, Gam displays the information as an indented list of keys and values:
+- `formatjson` - Display the fields in JSON format.
+
+```
+gam show browsers
+        ([ou|org|orgunit|browserou <OrgUnitPath>] [(query <QueryBrowser>)|(queries <QueryBrowserList>))|(select <BrowserEntity>))
+        [querytime<String> <Time>]
+        [orderby <BrowserOrderByFieldName> [ascending|descending]]
+        [basic|full|allfields|annotated] <BrowserFieldName>* [fields <BrowserFieldNameList>]
+        [formatjson]
+```
+
+Use these options to select Chrome browsers; if none are chosen, all Chrome browsers in the account are selected:
+* `ou|org|orgunit|browserou <OrgUnitPath>` - Limit browsers to those in the specified OU; this option can be used in conjunction with query
+* `(query <QueryBrowser>)|(queries <QueryBrowserList>)` - Limit browsers to those that match a query
+* `select <BrowserEntity>` - Select a specific set of browsers to display
+
+Select the fields to be displayed:
+* `annotated` - Display these fields: deviceId,annotatedAssetId,annotatedLocation,annotatedNotes,annotatedUser
+* `basic` - Display all fields except: browsers, lastDeviceUsers, lastStatusReportTime, machinePloicies; this is the default
+* `allfields/full` - Display all fields
+* `<BrowserFieldName>* [fields <BrowserFieldNameList>]` - Displaya selected list of fields
+
+By default, Gam displays the information as an indented list of keys and values:
+- `formatjson` - Display the fields in JSON format.
+
+Use the `querytime<String> <Time>` option to allow times, usually relative, to be substituted into the `query <QueryBrowser>` and `queries <QueryBrowserList>` options.
+The `querytime<String> <Time>` value replaces the string `#querytime<String>#` in any queries.
+The characters following `querytime` can be any combination of lowercase letters and numbers.
+
+```
+gam print browsers [todrive <ToDriveAttribute>*]
+        ([ou|org|orgunit|browserou <OrgUnitPath>] [(query <QueryBrowser>)|(queries <QueryBrowserList>))|(select <BrowserEntity>))
+        [querytime<String> <Time>]
+        [orderby <BrowserOrderByFieldName> [ascending|descending]]
+        [basic|full|allfields|annotated] <BrowserFieldName>* [fields <BrowserFieldNameList>]
+        [sortheaders] [formatjson [quotechar <Character>]]
+```
+
+Use these options to select Chrome browsers; if none are chosen, all Chrome browsers in the account are selected:
+* `ou|org|orgunit|browserou <OrgUnitPath>` - Limit browsers to those in the specified OU; this option can be used in conjunction with query
+* `(query <QueryBrowser>)|(queries <QueryBrowserList>)` - Limit browsers to those that match a query
+* `select <BrowserEntity>` - Select a specific set of browsers to display
+
+Use the `querytime<String> <Time>` option to allow times, usually relative, to be substituted into the `query <QueryBrowser>` and `queries <QueryBrowserList>` options.
+The `querytime<String> <Time>` value replaces the string `#querytime<String>#` in any queries.
+The characters following `querytime` can be any combination of lowercase letters and numbers.
+
+For example, query for Chrome browsers last synced more than a year ago:
+```
+querytime1year -1y query "sync:..#querytime1year#"
+```
+
+The first column will always be deviceId; the remaining field names will be sorted if `allfields`, `basic`, `full` or `sortheders` is specified;
+otherwise, the remaining field names will appear in the order specified.
+
+Select the fields to be displayed:
+* `annotated` - Display these fields: deviceId,annotatedAssetId,annotatedLocation,annotatedNotes,annotatedUser
+* `basic` - Display all fields except: browsers, lastDeviceUsers, lastStatusReportTime, machinePloicies; this is the default
+* `allfields/full` - Display all fields
+* `<BrowserFieldName>* [fields <BrowserFieldNameList>]` - Displaya selected list of fields
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format:
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+### Examples
+
+Print information about Chrome browsers synced more than 30 days ago:
+
+```
+gam print browsers query "sync:..#querytime1#" querytime1 -30d
+```
+
+Print information about Chrome browsers synced in the last 30 days:
+
+```
+gam print browsers query "sync:#querytime1#.." querytime1 -30d
+```
+
+Print information about Chrome browsers synced between 45 days ago and 30 days ago:
+
+```
+gam print browsers query "sync:#querytime1#..#querytime2#" querytime1 -45d querytime2 -30d
+```
+
+## Browser Query Searchable Fields
+
+These are the fields that can be used in a query:
+```
+Field             Description
+arch              The CPU architecture for the Chrome browser device. (e.g. x86_64)
+asset_id          The annotated asset ID for the Chrome browser device.
+browser_version   A reported Chrome browser installed on the Chrome browser device (e.g. 73)
+enrollment_token  The enrollment token used to register the Chrome browser device.
+last_activity     The last time the Chrome browser device has shown activity (policy fetch or reporting).
+location          The annotated location for the Chrome browser device.
+machine_name      The machine name for the Chrome browser device.
+machine_user      The last reported user of the Chrome browser device.
+note              The annotated note for the Chrome browser device.
+num_extensions    The number of extensions reported by the Chrome browser device.
+num_policies      The number of policies reported by the Chrome browser device.
+os                The combine OS platform and major OS version for the Chrome browser device (e.g. "Windows 10")
+os_platform       The OS platform for the Chrome browser device. (e.g. Windows)
+os_version        The OS version for the chrome browser device. (e.g. 10.0.16299.904)
+register          The registration time for the Chrome browser device.
+report            The last report time for the Chrome browser device
+sync              The last policy sync time for the Chrome browser device.
+user              The annotated user for the Chrome browser device.
+```
+
+For fields that accept time (register, report, sync, last_activity) the time format is YYYY-MM-DDThh:mm:ss (e.g. 2020-01-01T12:00:00). You may also specify open or closed ranges for the time:
+```
+datetime           exactly on the given date or time, e.g., 2011-03-23 2011-04-26T14:23:05
+
+datetime..datetime within (inclusive) the given interval of date or time, e.g., 2011-03-23..2011-04-26
+
+datetime..         on or after the given date or time; e.g., 2011-04-26T14:23:05..
+
+..datetime         on or before the given date or time; e.g., ..2011-04-26T14:23:05
+```
+To search within a specific field only (for example, to search for a specific user), you can enter an operator followed by an argument -- for example, `user:jsmith`. You can use single words or quoted lists of words as an argument when running an operator query.
+
+To run an operator query, follow these guidelines for each field:
+
+### User
+Enter user: as the operator. For example, to match the name Joe, but not Joey, enter the following:
+
+`gam print browsers query "user:joe"`
+
+To match the name Tom Sawyer or A. Tom Sawyer, but not Tom A. Sawyer, enter with quotation marks:
+
+`gam print browsers query "user:'tom sawyer'"`
+
+### Location
+Enter location: as the operator. For example, to match Seattle, enter the following:
+
+`gam print browsers query "location:seattle"`
+
+Notes
+Enter note: as the operator. For example, to match loaned from John, enter the following with quotation marks:
+
+`gam print browsers query "note:'loaned from john'"`
+
+### Register
+This field is not displayed on the Chrome OS settings page. However, you can search for devices that were registered on a given date, or within a given time range.
+
+Enter register: as the operator, and enter a date and time (or time range) as the argument. For example, to search for all devices registered on April 15, 2020, enter the following:
+
+`gam print browsers query "register:2020-04-15"`
+
+For additional examples using dates, times, and ranges, see "Format for date searches" below.
+
+### Last Sync
+Enter sync: as the operator and a date or time range as the argument. For example, to search for all devices that were last synced with policy settings on April 15, 2020, enter the following:
+
+`gam print browsers query "sync:2020-04-15"`
+
+For additional examples using dates, times, and ranges, see "Format for date searches" below.
+
+### Format for date searches
+* `YYYY-MM-DD` - A single date
+* `YYYY-MM-DD..YYYY-MM-DD` - A date range
+* `..YYYY-MM-DD` - All dates on or before a date
+* `YYYY-MM-DD..` - All dates on or after a date
+
+### Asset ID
+Enter asset_id: as the operator. For example, to match the partial Asset ID 1234, enter the following:
+
+`gam print browsers query "asset_id:1234"`
+
+## Manage Chrome browser enrollment tokens
+Create a browser enrollment token. The Google API that supports this call always returns an error.
+```
+gam create browsertoken
+        [ou|org|orgunit|browserou <OrgUnitPath>] [expire|expires <Time>]
+        [formatjson]
+```
+By default, the enrollment token is created for the root OU; use `ou|org|orgunit|browserou <OrgUnitPath>`
+to create the token for a specific OU.
+
+By default, Gam displays the created token as an indented list of keys and values:
+- `formatjson` - Display the token in JSON format.
+
+Revoke a browser enrollment token.
+An enrollment token is revoked by referencing its `tokenPermanentId` which can be obtained
+from `gam show|print browsertokens`.
+```
+gam revoke browsertoken <BrowserTokenPermanentID>
+
+```
+## Display Chrome browser enrollment tokens
+```
+gam show browsertokens
+        ([ou|org|orgunit|browserou <OrgUnitPath>] [(query <QueryBrowserToken)|(queries <QueryBrowserTokenList>)))
+        [querytime<String> <Time>]
+        [orderby <BrowserTokenFieldName> [ascending|descending]]
+        [allfields] <BrowserTokenFieldName>* [fields <BrowserTokenFieldNameList>]
+        [formatjson]
+```
+Use these options to select Chrome browsers; if none are chosen, all Chrome browsers in the account are selected:
+* `ou|org|orgunit|browserou <OrgUnitPath>` - Limit browsers to those in the specified OU; this option can be used in conjunction with query
+* `(query <QueryBrowserToken>)|(queries <QueryBrowserTokenList>)` - Limit browsers  to those that match a query
+
+Use the `querytime<String> <Time>` option to allow times, usually relative, to be substituted into the `query <QueryBrowserToken>` and `queries <QueryBrowserTokenList>` options.
+The `querytime<String> <Time>` value replaces the string `#querytime<String>#` in any queries.
+The characters following `querytime` can be any combination of lowercase letters and numbers.
+
+Select the fields to be displayed:
+* `allfields` - Display all fields; this is the default
+* `<BrowserTokenFieldName>* [fields <BrowserTokenFieldNameList>]` - Displaya selected list of fields
+
+By default, Gam displays the information as an indented list of keys and values:
+- `formatjson` - Display the fields in JSON format.
+
+```
+gam print browsertokens [todrive <ToDriveAttribute>*]
+        ([ou|org|orgunit|browserou <OrgUnitPath>] [(query <QueryBrowserToken)|(queries <QueryBrowserTokenList>)))
+        [querytime<String> <Time>]
+        [orderby <BrowserTokenFieldName> [ascending|descending]]
+        [allfields] <BrowserTokenFieldName>* [fields <BrowserTokenFieldNameList>]
+        [sortheaders] [formatjson [quotechar <Character>]]
+```
+Use these options to select Chrome browsers; if none are chosen, all Chrome browsers in the account are selected:
+* `ou|org|orgunit|browserou <OrgUnitPath>` - Limit browsers to those in the specified OU; this option can be used in conjunction with query
+* `(query <QueryBrowserToken>)|(queries <QueryBrowserTokenList>)` - Limit browser s to those that match a query
+
+Use the `querytime<String> <Time>` option to allow times, usually relative, to be substituted into the `query <QueryBrowserToken>` and `queries <QueryBrowserTokenList>` options.
+The `querytime<String> <Time>` value replaces the string `#querytime<String>#` in any queries.
+The characters following `querytime` can be any combination of lowercase letters and numbers.
+
+The first column will always be deviceId; the remaining field names will be sorted if `allfields`, `basic`, `full` or `sortheders` is specified;
+otherwise, the remaining field names will appear in the order specified.
+
+Select the fields to be displayed:
+* `allfields` - Display all fields; this is the default
+* `<BrowserTokenFieldName>* [fields <BrowserTokenFieldNameList>]` - Displaya selected list of fields
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format:
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Chrome-Installed-Apps.md

@@ -0,0 +1,142 @@
+# Chrome Installed Apps Counts
+
+- [Chrome Policies](#chrome-policies)
+  - [API documentation](#api-documentation)
+  - [Definitions](#definitions)
+  - [Quoting rules](#quoting-rules)
+  - [Display Chrome installed app details](#display-chrome-installed-app-details)
+  - [Display Chrome installed apps counts](#display-chrome-installed-apps-counts)
+  - [Display Chrome devices with a specific installed application](#display-chrome-devices-with-a-specific-installed-application)
+
+## API documentation
+
+* https://developers.google.com/chrome/management/reference/rest/v1/customers.reports/countInstalledApps
+* https://developers.google.com/chrome/management/reference/rest/v1/customers.reports/findInstalledAppDevices
+
+## Notes
+To use these features you must add the `Chrome Management API` to your project and authorize
+the appropriate scope: `Chrome Management API - read only`.
+```
+gam update project
+gam oauth create
+```
+To get installed app details you must authorize the scope: `Chrome Management API - AppDetails read only`.
+
+## Definitions
+```
+<AppID> ::= <String>
+<AppType> ::= extension|app|theme|hostedapp|androidapp
+<Date> ::=
+        <Year>-<Month>-<Day> |
+        (+|-)<Number>(d|w|y) |
+        never|
+        today
+<OrgUnitID> ::= id:<String>
+<OrgUnitPath> ::= /|(/<String>)+
+<OrgUnitItem> ::= <OrgUnitID>|<OrgUnitPath>
+```
+
+## Quoting rules
+Items in a list can be separated by commas or spaces; if an item itself contains a comma, a space or a single quote, special quoting must be used.
+Typically, you will enclose the entire list in double quotes and quote each item in the list as detailed below.
+
+- Items, separated by commas, without spaces, commas or single quotes in the items themselves
+   * ```"item,item,item"```
+- Items, separated by spaces, without spaces, commas or single quotes in the items themselves
+   * ```"item item item"```
+- Items, separated by commas, with spaces, commas or single quotes in the items themselves
+   * ```"'it em','it,em',\"it'em\""```
+- Items, separated by spaces, with spaces, commas or single quotes in the items themselves
+   * ```"'it em' 'it,em' \"it'em\""```
+
+## Display Chrome installed app details
+```
+gam info chromeapp android|chrome|web <AppID>
+        [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+## Display Chrome installed apps counts
+```
+gam show chromeapps
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [filter <String>]
+        [orderby appname|apptype|installtype|numberofpermissions|totalinstallcount]
+        [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print chromeapps [todrive <ToDriveAttribute>*]
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [filter <String>]
+        [orderby appname|apptype|installtype|numberofpermissions|totalinstallcount]
+        [formatjson [quotechar <Character>]] [delimiter <Character>]
+```
+Use these options to select Chrome devices; if none are chosen, all Chrome devices in the account are selected.
+
+- `ou <OrgUnitItem>` - Select devices directly in the OU `<OrgUnitItem>`
+- `ou_and_children <OrgUnitItem>` - Select devices in the OU `<OrgUnitItem>` and its sub OUs
+- `ous <OrgUnitList>` - Select devices directly in the OUs `<OrgUnitList>`
+- `ous_and_children <OrgUnitList>` - Select devices in the OUs `<OrgUnitList>` and their sub OUs
+- `filter <String>` - The minimum `last_active_date` for the devices
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display Chrome devices with a specific installed application
+```
+gam show chromeappdevices
+        appid <AppID> apptype <AppType>
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [start <Date>] [end <Date>]
+        [orderby deviceid|machine]
+        [formatjson]
+```
+Use these options to select Chrome devices; if none are chosen, all Chrome devices in the account are selected.
+
+- `ou <OrgUnitItem>` - Select devices directly in the OU `<OrgUnitItem>`
+- `ou_and_children <OrgUnitItem>` - Select devices in the OU `<OrgUnitItem>` and its sub OUs
+- `ous <OrgUnitList>` - Select devices directly in the OUs `<OrgUnitList>`
+- `ous_and_children <OrgUnitList>` - Select devices in the OUs `<OrgUnitList>` and their sub OUs
+- `start <Date>` - The minimum `last_active_date` for the devices
+- `end <Date>` - The maximum `last_active_date` for the devices
+  
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print chromeappdevices [todrive <ToDriveAttribute>*]
+        appid <AppID> apptype <AppType)
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [start <Date>] [end <Date>]
+        [orderby deviceid|machine]
+        [formatjson [quotechar <Character>]]
+```
+Use these options to select Chrome devices; if none are chosen, all Chrome devices in the account are selected.
+
+- `ou <OrgUnitItem>` - Select devices directly in the OU `<OrgUnitItem>`
+- `ou_and_children <OrgUnitItem>` - Select devices in the OU `<OrgUnitItem>` and its sub OUs
+- `ous <OrgUnitList>` - Select devices directly in the OUs `<OrgUnitList>`
+- `ous_and_children <OrgUnitList>` - Select devices in the OUs `<OrgUnitList>` and their sub OUs
+- `start <Date>` - The minimum `last_active_date` for the devices
+- `end <Date>` - The maximum `last_active_date` for the devices
+  
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Chrome-Needs-Attention-Counts.md

@@ -0,0 +1,78 @@
+# Chrome Device Needs Attention Counts
+
+- [Chrome Device Needs Attention Counts](#chrome-device-needs-attention-counts)
+  - [API documentation](#api-documentation)
+  - [Definitions](#definitions)
+  - [Quoting rules](#quoting-rules)
+  - [Display Chrome Device needs attention counts](#display-chrome-device-needs-attention-counts)
+
+## API documentation
+
+* https://developers.google.com/chrome/management/reference/rest/v1/customers.reports/countChromeDevicesThatNeedAttention
+
+## Notes
+To use these features you must add the `Chrome Management API` to your project and authorize
+the appropriate scope: `Chrome Management API - read only`.
+```
+gam update project
+gam oauth create
+```
+
+## Definitions
+```
+<OrgUnitID> ::= id:<String>
+<OrgUnitPath> ::= /|(/<String>)+
+<OrgUnitItem> ::= <OrgUnitID>|<OrgUnitPath>
+<OrgUnitList> ::= "<OrgUnitItem>(,<OrgUnitItem>)*"
+```
+## Quoting rules
+Items in a list can be separated by commas or spaces; if an item itself contains a comma, a space or a single quote, special quoting must be used.
+Typically, you will enclose the entire list in double quotes and quote each item in the list as detailed below.
+
+- Items, separated by commas, without spaces, commas or single quotes in the items themselves
+   * ```"item,item,item"```
+- Items, separated by spaces, without spaces, commas or single quotes in the items themselves
+   * ```"item item item"```
+- Items, separated by commas, with spaces, commas or single quotes in the items themselves
+   * ```"'it em','it,em',\"it'em\""```
+- Items, separated by spaces, with spaces, commas or single quotes in the items themselves
+   * ```"'it em' 'it,em' \"it'em\""```
+
+## Display Chrome device needs attention counts
+```
+gam show chromeneedsattn
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [formatjson]
+```
+Use these options to select Chrome devices; if none are chosen, all Chrome devices in the account are selected.
+
+- `ou <OrgUnitItem>` - Select devices directly in the OU `<OrgUnitItem>`
+- `ou_and_children <OrgUnitItem>` - Select devices in the OU `<OrgUnitItem>` and its sub OUs
+- `ous <OrgUnitList>` - Select devices directly in the OUs `<OrgUnitList>`
+- `ous_and_children <OrgUnitList>` - Select devices in the OUs `<OrgUnitList>` and their sub OUs
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+```
+gam print chromeneedsattn [todrive <ToDriveAttribute>*]
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [formatjson [quotechar <Character>]]
+```
+Use these options to select Chrome devices; if none are chosen, all Chrome devices in the account are selected.
+
+- `ou <OrgUnitItem>` - Select devices directly in the OU `<OrgUnitItem>`
+- `ou_and_children <OrgUnitItem>` - Select devices in the OU `<OrgUnitItem>` and its sub OUs
+- `ous <OrgUnitList>` - Select devices directly in the OUs `<OrgUnitList>`
+- `ous_and_children <OrgUnitList>` - Select devices in the OUs `<OrgUnitList>` and their sub OUs
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Chrome-Printers.md

@@ -0,0 +1,182 @@
+# Chrome Printers
+- [API documentation](#api-documentation)
+- [Notes](#notes)
+- [Definitions](#definitions)
+- [Quoting rules](#quoting-rules)
+- [Manage printers](#manage-printers)
+- [Display printers](#display-printers)
+- [Display printer models](#display-printer-models)
+- [Bulk printer updates](#bulk-printer-updates)
+
+## API documentation
+* https://developers.google.com/admin-sdk/chrome-printer/reference/rest
+
+## Notes
+To use these features you must authorize the appropriate scope: `Directory API - Printers (supports readonly)`.
+
+As of 2021-10-05, `gam update printer` does not work due to some API problem. To update a printer,
+you'll have to delete it and create it.
+
+```
+gam oauth create
+```
+
+## Definitions
+```
+<OrgUnitID> ::= id:<String>
+<OrgUnitPath> ::= /|(/<String)+
+<OrgUnitItem> ::= <OrgUnitID>|<OrgUnitPath>
+<OrgUnitList> ::= "<OrgUnitItem>(,<OrgUnitItem>)*"
+
+<PrinterID> ::= <String>
+<PrinterIDList> ::= "<PrinterID>(,<PrinterID>)*"
+
+<PrinterAttribute> ::=
+        (description <String>)|
+        (displayname <String>)|
+        (json [charset <Charset>] <JSONData>)|(json file <FileName> [charset <Charset>])|
+        (makeandmodel <String>)|
+        (ou|org|orgunit <OrgUnitItem>)|
+        (uri <String>)|
+        (driverless [<Boolean>])
+
+<PrinterFieldName> ::=
+        auxiliarymessages|
+        createtime|
+        description|
+        displayname|
+        id|
+        makeandmodel|
+        name|
+        ou|org|orgunit|orgunitid|
+        uri|
+        usedriverlessconfig|
+<PrinterFieldNameList> ::= "<PrinterFieldName>(,<PrinterFieldName>)*"
+```
+```
+<StorageBucketName> ::= <String>
+<StorageObjectName> ::= <String>
+<StorageBucketObjectName> ::=
+        https://storage.cloud.google.com/<StorageBucketName>/<StorageObjectName>|
+        https://storage.googleapis.com/<StorageBucketName>/<StorageObjectName>|
+        gs://<StorageBucketName>/<StorageObjectName>|
+        <StorageBucketName>/<StorageObjectName>
+
+<UserGoogleDoc> ::=
+        <EmailAddress> <DriveFileIDEntity>|<DriveFileNameEntity>|(<SharedDriveEntity> <SharedDriveFileNameEntity>)
+
+<FileSelector> ::=
+        file ((<FileName> [charset <Charset>])|
+              (gdoc <UserGoogleDoc>)|
+              (gcsdoc <StorageBucketObjectName>))
+             [delimiter <Character>]
+
+<CSVFileSelector> ::=
+        csvfile ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+                 (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+                 (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+                 (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+                 (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+                [warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
+                [endcsv|(fields <FieldNameList>)]
+                (matchfield|skipfield <FieldName> <RegularExpression>)*
+                [delimiter <Character>]
+
+```
+## Quoting rules
+Items in a list can be separated by commas or spaces; if an item itself contains a comma, a space or a single quote, special quoting must be used.
+Typically, you will enclose the entire list in double quotes and quote each item in the list as detailed below.
+
+- Items, separated by commas, without spaces, commas or single quotes in the items themselves
+   * ```"item,item,item"```
+- Items, separated by spaces, without spaces, commas or single quotes in the items themselves
+   * ```"item item item"```
+- Items, separated by commas, with spaces, commas or single quotes in the items themselves
+   * ```"'it em','it,em',\"it'em\""```
+- Items, separated by spaces, with spaces, commas or single quotes in the items themselves
+   * ```"'it em' 'it,em' \"it'em\""```
+
+## Manage printers
+When creating a printer you must specify: `displayname`, `ou`, `uri` and `makeandmodel` or `driverless`.
+```
+gam create printer <PrinterAttribute>+ [nodetails]
+gam update printer <PrinterID> <PrinterAttribute>+ [nodetails]
+gam delete printer
+        <PrinterIDList>|
+        <FileSelector>|
+	<CSVFileSelector>
+```
+By default, when a printer is created/updated, GAM outputs details of the printer; the `nodetails` option suppresses this output.
+
+## Display printers
+Display information about a single printer.
+
+```
+gam info printer <PrinterID>
+        [fields <PrinterFieldNameList>] [formatjson]
+```
+Display information about multiple printers.
+```
+gam show printers
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [filter <String>] [showinherited [<Boolean>]]
+        [fields <PrinterFieldNameList>] [formatjson]
+gam print printers [todrive <ToDriveAttribute>*]
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [filter <String>] [showinherited [<Boolean>]]
+        [fields <PrinterFieldNameList>] [[formatjson [quotechar <Character>]]
+```
+Use these options to select printers; if none are chosen, all printers in the account are selected.
+
+If only `filter <String>` is specified, the query applies to all printers. If one of the `ou` options
+is also specified, the filter applies to printers within the OUs. The `filter <String>` is applied
+to the printer `displayName` and `description` fields.
+
+- `filter <String>` - Filter on printer `description` and `displayName'.
+- `ou <OrgUnitItem>` - Select printers directly in the OU `<OrgUnitItem>`
+- `ou_and_children <OrgUnitItem>` - Select printers in the OU `<OrgUnitItem>` and its sub OUs
+- `ous <OrgUnitList>` - Select printers directly in the OUs `<OrgUnitList>`
+- `ous_and_children <OrgUnitList>` - Select printers in the OUs `<OrgUnitList>` and their sub OUs
+
+By default, only printers defined in the specified OUs are displayed. Use the `showinherited` option
+to display inherited printers in the OUs; three additional fields are displayed.
+- `inherited` - False if the printer is defined in the OU, True if the printer is inherited by the OU
+- `parentOrgUnitId` - Blank if the printer is defined in the OU, the ID of the defining OU if the printer is inherited by the OU
+- `parentOrgUnitPath` - Blank if the printer is defined in the OU, the path of the defining OU if the printer is inherited by the OU
+
+## Display printer models
+```
+gam show printermodels
+        [filter <String>]
+        [formatjson]
+gam print printermodels [todrive <ToDriveAttribute>*]
+        [filter <String>]
+        [[formatjson [quotechar <Character>]]
+```
+If `filter <String>` isn't specified, all printer models are displayed.
+You can filter by manufacturer: `filter "manufacturer:XYX"`
+
+## Bulk printer updates
+Suppose you have replaced one model of printer with another and have to update the make and model.
+
+As of 2021-10-05, you'll have to delete and create the updated printer as `gam update printer` does not work due to some API problem.
+
+Get the list of printers.
+```
+gam redirect csv ./StudentPrinters.csv print printers formatjson quotechar "'" ou /Students
+```
+Edit StudentPrinters.csv and add a new column labelled `action`; it does not matter where you place the column.
+In each row's JSON data there will be an entry like this: `"makeAndModel": "vendor1 xy abcd"`; replace `vendor1 xy abcd`
+with `vendor2 ab wxyz` for the rows of interest and put an `x` in the `action` column.
+
+Delete the marked printers.
+```
+gam config csv_input_row_filter "action:regex:x" redirect stdout ./DeletePrinters.txt multiprocess redirect stderr stdout csv ./StudentPrinters.csv quotechar "'" gam delete printer "~id"
+```
+
+Recreate the marked printers with the updated `makeAndModel`.
+```
+gam config csv_input_row_filter "action:regex:x" redirect stdout ./CreatetePrinters.txt multiprocess redirect stderr stdout csv ./StudentPrinters.csv quotechar "'" gam create printer json "~JSON"
+```

docs/Chrome-Version-Counts.md

@@ -0,0 +1,96 @@
+# Chrome Version Counts
+
+- [Chrome Version Counts](#chrome-version-counts)
+  - [API documentation](#api-documentation)
+  - [Definitions](#definitions)
+  - [Quoting rules](#quoting-rules)
+  - [Display Chrome version counts](#display-chrome-version-counts)
+
+## API documentation
+
+* https://developers.google.com/chrome/management/reference/rest/v1/customers.reports/countChromeVersions
+
+## Notes
+To use these features you must add the `Chrome Management API` to your project and authorize
+the appropriate scope: `Chrome Management API - read only`.
+```
+gam update project
+gam oauth create
+```
+
+## Definitions
+```
+<Date> ::=
+        <Year>-<Month>-<Day> |
+        (+|-)<Number>(d|w|y) |
+        never|
+        today
+<OrgUnitID> ::= id:<String>
+<OrgUnitPath> ::= /|(/<String>)+
+<OrgUnitItem> ::= <OrgUnitID>|<OrgUnitPath>
+<OrgUnitList> ::= "<OrgUnitItem>(,<OrgUnitItem>)*"
+```
+## Quoting rules
+Items in a list can be separated by commas or spaces; if an item itself contains a comma, a space or a single quote, special quoting must be used.
+Typically, you will enclose the entire list in double quotes and quote each item in the list as detailed below.
+
+- Items, separated by commas, without spaces, commas or single quotes in the items themselves
+   * ```"item,item,item"```
+- Items, separated by spaces, without spaces, commas or single quotes in the items themselves
+   * ```"item item item"```
+- Items, separated by commas, with spaces, commas or single quotes in the items themselves
+   * ```"'it em','it,em',\"it'em\""```
+- Items, separated by spaces, with spaces, commas or single quotes in the items themselves
+   * ```"'it em' 'it,em' \"it'em\""```
+
+## Display Chrome version counts
+These counts are for provisioned devices.
+```
+gam show chromeversions
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [start <Date>] [end <Date>]
+        [recentfirst [<Boolean>]]
+        [formatjson]
+```
+Use these options to select Chrome devices; if none are chosen, all Chrome devices in the account are selected.
+
+- `ou <OrgUnitItem>` - Select devices directly in the OU `<OrgUnitItem>`
+- `ou_and_children <OrgUnitItem>` - Select devices in the OU `<OrgUnitItem>` and its sub OUs
+- `ous <OrgUnitList>` - Select devices directly in the OUs `<OrgUnitList>`
+- `ous_and_children <OrgUnitList>` - Select devices in the OUs `<OrgUnitList>` and their sub OUs
+- `start <Date>` - The minimum `last_active_date` for the devices
+- `end <Date>` - The maximum `last_active_date` for the devices
+
+By default, the versions are displayed from oldest to most recent; use the `recentfirst` option to reverse this order.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+```
+gam print chromeversions [todrive <ToDriveAttribute>*]
+        [(ou <OrgUnitItem>)|(ou_and_children <OrgUnitItem>)|
+         (ous <OrgUnitList>)|(ous_and_children <OrgUnitList>)]
+        [start <Date>] [end <Date>]
+        [recentfirst [<Boolean>]]
+        [formatjson [quotechar <Character>]]
+```
+Use these options to select Chrome devices; if none are chosen, all Chrome devices in the account are selected.
+
+- `ou <OrgUnitItem>` - Select devices directly in the OU `<OrgUnitItem>`
+- `ou_and_children <OrgUnitItem>` - Select devices in the OU `<OrgUnitItem>` and its sub OUs
+- `ous <OrgUnitList>` - Select devices directly in the OUs `<OrgUnitList>`
+- `ous_and_children <OrgUnitList>` - Select devices in the OUs `<OrgUnitList>` and their sub OUs
+- `start <Date>` - The minimum `last_active_date` for the devices
+- `end <Date>` - The maximum `last_active_date` for the devices
+
+By default, the versions are displayed from oldest to most recent; use the `recentfirst` option to reverse this order.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Chrome-Version-History.md

@@ -0,0 +1,166 @@
+# Chrome Version History
+
+- [Chrome Version History](#chrome-version-history)
+  - [API documentation](#api-documentation)
+  - [Definitions](#definitions)
+  - [Display Chrome platforms](#display-chrome-platforms)
+  - [Display Chrome channels](#display-chrome-channels)
+  - [Display Chrome versions](#display-chrome-versions)
+  - [Display Chrome releases](#display-chrome-releases)
+
+## API documentation
+
+* https://developer.chrome.com/docs/versionhistory/guide/
+* https://developer.chrome.com/docs/versionhistory/reference/#filter
+* https://developer.chrome.com/docs/versionhistory/reference/#order
+
+## Definitions
+```
+<ChromePlatfornType> ::=
+        all|
+        android|
+        ios|
+        lacros|
+        linux|
+        mac|
+        macarm64|
+        sebview|
+        win|
+        win64
+<ChromeChannelType> ::=
+        beta|
+        canary|
+        canaryasan|
+        dev|
+        stable
+<ChromeVersionsOrderByFieldName> ::=
+        channel|
+        name|
+        platform|
+        version|
+<ChromeReleasesOrderByFieldName> ::= 
+        channel|
+        endtime|
+        fraction|
+        name|
+        platform|
+        starttime|
+        version
+```
+## Display Chrome platforms
+```
+gam show chromehistory platforms
+        [formatjson]
+```
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+```
+gam print chromehistory platforms [todrive <ToDriveAttribute>*]
+        [formatjson [quotechar <Character>]]
+```
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display Chrome channels
+```
+gam show chromehistory channels
+        [platform <ChromePlatformType>]
+        [formatjson]
+```
+
+By default, channels for all platforms are displayed; use `platform <ChromePlatformType>]`
+to select a specific platform.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+```
+gam print chromehistory channels [todrive <ToDriveAttribute>*]
+        [platform <ChromePlatformType>]
+        [formatjson [quotechar <Character>]]
+```
+By default, channels for all platforms are displayed; use `platform <ChromePlatformType>]`
+to select a specific platform.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display Chrome versions
+```
+gam show chromehistory versions
+        [platform <ChromePlatformType>] [channel <ChromeChannelType>]
+        [filter <String>]
+        (orderby <ChromeVersionsOrderByFieldName> [ascending|descending])*
+        [formatjson]
+```
+By default, versions for all platforms and channels are displayed; use `platform <ChromePlatformType>]`
+and/or `channel <ChromeChannelType>` to select a specific platform and/or channel.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+```
+gam print chromehistory versions [todrive <ToDriveAttribute>*]
+        [platform <ChromePlatformType>] [channel <ChromeChannelType>]
+        [filter <String>]
+        (orderby <ChromeVersionsOrderByFieldName> [ascending|descending])*
+        [formatjson [quotechar <Character>]]
+```
+By default, versions for all platforms and channels are displayed; use `platform <ChromePlatformType>]`
+and/or `channel <ChromeChannelType>` to select a specific platform and/or channel.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display Chrome releases
+```
+gam show chromehistory releases
+        [platform <ChromePlatformType>] [channel <ChromeChannelType>] [version <String>]
+        [filter <String>]
+        (orderby <ChromeReleasessOrderByFieldName> [ascending|descending])*
+        [formatjson]
+```
+By default, versions for all platforms, channels and versions are displayed; use `platform <ChromePlatformType>]`
+and/or `channel <ChromeChannelType>` and/or `version <String>` to select a specific platform and/or channel and/or version.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+```
+gam print chromehistory releases [todrive <ToDriveAttribute>*]
+        [platform <ChromePlatformType>] [channel <ChromeChannelType>] [version <String>]
+        [filter <String>]
+        (orderby <ChromeReleasessOrderByFieldName> [ascending|descending])*
+        [formatjson [quotechar <Character>]]
+```
+By default, versions for all platforms, channels and versions are displayed; use `platform <ChromePlatformType>]`
+and/or `channel <ChromeChannelType>` and/or `version <String>` to select a specific platform and/or channel and/or version.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Classroom-Courses.md

@@ -0,0 +1,693 @@
+# Classroom - Courses
+- [API documentation](#api-documentation)
+- [Notes](#notes)
+- [Python Regular Expressions](Python-Regular-Expressions) Match function
+- [Definitions](#definitions)
+- [Special quoting for course aliases and topics](#special-quoting-for-course-aliases-and-topics)
+- [Updating course owner](#updating-course-owner)
+- [Create and update courses](#create-and-update-courses)
+- [Delete courses](#delete-courses)
+- [Manage course aliases](#manage-course-aliases)
+- [Manage course topics](#manage-course-topics)
+- [Display courses](#display-courses)
+- [Display course counts](#display-course-counts)
+- [Display course announcements](#display-course-announcements)
+- [Display course materials](#display-course-materials)
+- [Display course topics](#display-course-topics)
+- [Display course work](#display-course-work)
+- [Display course submissions](#display-course-submissions)
+
+## API documentation
+* https://developers.google.com/classroom/reference/rest/
+* https://developers.google.com/classroom/reference/rest/v1/courses.students
+* https://developers.google.com/classroom/reference/rest/v1/courses.teachers
+* https://developers.google.com/classroom/reference/rest/v1/courses.announcements/list
+* https://developers.google.com/classroom/reference/rest/v1/courses.topics/list
+* https://developers.google.com/classroom/reference/rest/v1/courses.courseWork/list
+* https://developers.google.com/classroom/reference/rest/v1/courses.courseWorkMaterials/list
+* https://developers.google.com/classroom/reference/rest/v1/courses.courseWork.studentSubmissions/list
+
+## Notes
+In this document, `course materials` refers to stand-alone materials, not the materials associated with
+`course announcements` or `course work`. Google added support for stand-alone materials in early 2021.
+
+To use the course materials features you must authorize the appropriate scope: `Classroom API - Course Work/Materials`.
+```
+gam oauth create
+gam user user@domain.com check|update serviceaccount
+```
+
+## Definitions
+```
+<DomainName> ::= <String>(.<String>)+
+<EmailAddress> ::= <String>@<DomainName>
+<UniqueID> ::= id:<String>
+<UserItem> ::= <EmailAddress>|<UniqueID>|<String>
+
+<CourseAlias> ::= <String>
+<CourseAliasList> ::= "<CourseAlias>(,<CourseAlias>)*"
+<CourseAliasEntity> ::=
+        <CourseAliasList>|<FileSelector>|<CSVFileSelector>|<CSVkmdSelector>|<CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<CourseAnnouncementID> ::= <Number>
+<CourseAnnouncementIDList> ::= "<CourseAnnouncementID>(,<CourseAnnouncementID>)*"
+<CourseAnnouncementIDEntity> ::=
+        <CourseAnnouncementIDList>|<FileSelector>|<CSVFileSelector>|<CSVkmdSelector>|<CSVSubkeySelector>|<CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<CourseAnnouncementState> ::= draft|published|deleted
+<CourseAnnouncementStateList> ::= all|"<CourseAnnouncementState>(,<CourseAnnouncementState>)*"
+<CourseID> ::= <Number>|d:<CourseAlias>
+<CourseIDList> ::= "<CourseID>(,<CourseID>)*"
+<CourseEntity> ::=
+        <CourseIDList>|<FileSelector>|<CSVFileSelector>|<CSVkmdSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<CourseMaterialID> ::= <Number>
+<CourseMaterialIDList> ::= "<CourseMaterialID>(,<CourseMaterialID>)*"
+<CourseMaterialState> ::= draft|published|deleted
+<CourseMaterialStateList> ::= all|"<CourseMaterialState>(,<CourseMaterialState>)*"
+<CourseMaterialIDEntity> ::=
+        <CourseMaterialIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVSubkeySelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<CourseState> ::= active|archived|provisioned|declined|suspended
+<CourseStateList> ::= all|"<CourseState>(,<CourseState>)*"
+<CourseSubmissionID> ::= <Number>
+<CourseSubmissionIDList> ::= "<CourseSubmissionID>(,<CourseSubmissionID>)*"
+<CourseSubmissionIDEntity> ::=
+        <CourseSubmissionIDList>|<FileSelector>|<CSVFileSelector>|<CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<CourseSubmissionState> ::= new|created|turned_in|returned|reclaimed_by_student
+<CourseSubmissionStateList> ::= all|"<CourseSubmissionState>(,<CourseSubmissionState>)*"
+<CourseTopic> ::= <String>
+<CourseTopicList> ::= "<CourseTopic>(,<CourseTopic>)*"
+<CourseTopicEntity> ::=
+        <CourseTopicList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<CourseTopicID> ::= <Number>
+<CourseTopicIDList> ::= "<CourseTopicID>(,<CourseTopicID>)*"
+<CourseTopicIDEntity> ::=
+        <CourseTopicIDList>|<FileSelector>|<CSVFileSelector>|<CSVkmdSelector>|<CSVSubkeySelector>|<CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<CourseWorkID> ::= <Number>
+<CourseWorkIDList> ::= "<CourseWorkID>(,<CourseWorkID>)*"
+<CourseWorkIDEntity> ::=
+        <CourseWorkIDList>|<FileSelector>|<CSVFileSelector>|<CSVkmdSelector>|<CSVSubkeySelector>|<CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<CourseWorkState> ::= draft|published|deleted
+<CourseWorkStateList> ::= all|"<CourseWorkState>(,<CourseWorkState>)*"
+
+<CourseAttribute> ::=
+        (description <String>)|
+        (descriptionheading|heading <String>)|
+        (name <String>)|
+        (room <String>)|
+        (section <string>)|
+        (state|status <CourseState>)|
+        (owner|ownerid|teacher <UserItem>)
+
+<CourseFieldName> ::=
+        alternatelink|
+        coursegroupemail|
+        coursematerialsets|
+        coursestate|
+        creationtime|
+        description|
+        descriptionheading|heading|
+        enrollmentcode|
+        gradebooksettings|
+        guardiansenabled|
+        id|
+        name|
+        owneremail|
+        ownerid|
+        room|
+        section|
+        teacherfolder|
+        teachergroupemail|
+        updatetime
+<CourseFieldNameList> ::= '<CourseFieldName>(,<CourseFieldName>)*'
+
+<CourseAnnouncementFieldName> ::=
+        alternatelink|
+        assigneemode|
+        courseid|
+        courseannouncementid|
+        creationtime|
+        creator|creatoruserid|
+        id|
+        materials|
+        scheduledtime|
+        state|
+        text|
+        updatetime
+<CourseAnnouncementFieldNameList> ::= "<CourseAnnouncementFieldName>(,<CourseAnnouncementFieldName>)*"
+
+<CourseAnnouncementOrderByFieldName> ::=
+        updatetime|
+        updatedate
+
+<CourseMaterialFieldName> ::=
+        alternatelink|
+        assigneemode|
+        courseid|
+        courseworkmaterialid|
+        creationtime|
+        creator|creatoruserid|
+        description|
+        id|
+        materials|
+        scheduledtime|
+        state|
+        title|
+        topicid|
+        updatetime|
+        workmaterialid
+<CourseMaterialFieldNameList> ::= "<CourseMaterialFieldName>(,<CourseMaterialFieldName>)*"
+
+<CourseMaterialOrderByFieldName> ::=
+        updatetime|
+        updatedate
+
+<CourseWorkFieldName> ::=
+        alternatelink|
+        assigneemode|
+        courseid|
+        courseworkid|
+        courseworktype|
+        creationtime|
+        creator|creatoruserid|
+        description|
+        duedate|
+        duetime|
+        id|
+        materials|
+        maxpoints|
+        scheduledtime|
+        state|
+        submissionmodificationmode|
+        title|
+        topicid|
+        updatetime|
+        worktype
+<CourseWorkFieldNameList> ::= "<CourseWorkFieldName>(,<CourseWorkFieldName>)*"
+
+<CourseWorkOrderByFieldName> ::=
+        duedate|
+        updatetime|
+        updatedate
+
+<CourseSubmissionFieldName> ::=
+        alternatelink|
+        assignedgrade|
+        courseid|
+        courseworkid|
+        courseworktype|
+        creationtime|
+        draftgrade|
+        id|
+        late|
+        state|
+        submissionhistory|
+        updatetime|
+        userid|
+        worktype
+<CourseSubmissionFieldNameList> ::= "<CourseSubmissionFieldName>(,<CourseSubmissionFieldName>)*"
+```
+## Special quoting for course aliases and topics
+As course aliases and topics can contain spaces, some care must be used when entering `<CourseAliasList>` and `<CourseTopicList>`.
+
+Suppose you have a course with the alias `Math Class`. To get information about it you enter the command: `gam info course "d:Math Class"`
+
+The shell strips the `"` leaving a single argument `d:Math Class`; gam correctly processes the argument as it is expecting a single course.
+
+Suppose you enter the command: `gam info courses "d:Math Class"`
+
+The shell strips the `"` leaving a single argument `d:Math Class`; as gam is expecting a list, it splits the argument on space leaving two items and then tries to process `d:Math` and `Class`, not what you want.
+
+You must enter: `gam info courses "'d:Math Class'"`
+
+The shell strips the `"` leaving a single argument `'d:Math Class'`; as gam is expecting a list, it splits the argument on space while honoring the `'` leaving one item `d:Math Class` and correctly processes the item.
+
+For multiple aliases you must enter: `gam info courses "'d:Math Class','d:Science Class'"`
+
+See: [Lists and Collections](Lists-and-Collections)
+
+## Updating course owner
+When updating a course owner, the Classroom API generates an error if the new owner is not a co-teacher
+or is the current owner.
+
+Prior to version 5.31.08, if `<UserItem>` was not a co-teacher, you got this error:
+```
+$ gam update course 123929046789 teacher newteacher@domain.com
+Course: 123929046789, Update Failed: @IneligibleOwner Only a co-teacher can be invited as owner of the course
+```
+GAM now adds `<UserItem>` as a co-teacher of the course, pauses 10 seconds, and then updates them to be the owner.
+```
+$ gam update course 123929046789 teacher newteacher@domain.com
+Course Name: Test, Course: 123929046789, Updated with new teacher as owner: newteacher@domain.com
+```
+
+Prior to version 5.31.08, if `<UserItem>` is the current owner, you got this error:
+```
+$ gam update course 123929046789 teacher newteacher@domain.com
+Course: 123929046789, Update Failed: @UserAlreadyOwner Cannot transfer course to the user who is already the owner
+
+```
+GAM now reports that the current owner was retained.
+```
+$ gam update course 123929046789 teacher newteacher@domain.com
+Course Name: Test, Course: 123929046789, Updated with current owner: newteacher@domain.com
+```
+
+In the normal case when `<UserItem>` is a co-teacher, GAM now reports the change.
+```
+$ gam update course 123929046789 teacher newteacher@domain.com
+Course Name: Test, Course: 123929046789, Updated with co-teacher as owner: newteacher@domain.com
+```
+
+## Create and update courses
+The options `name <String>` and `teacher <UserItem>` are required when creating a class.
+```
+gam create|add course [id|alias <CourseAlias>] <CourseAttribute>*
+        [copyfrom <CourseID>
+            [announcementstates <CourseAnnouncementStateList>]
+            [materialstates <CourseMaterialStateList>]
+            [workstates <CourseWorkStateList>]
+                [removeduedate [<Boolean>]]
+                [mapsharemodestudentcopy edit|none|view]
+            [copymaterialsfiles [<Boolean>]]
+            [copytopics [<Boolean>]]
+            [markdraftaspublished [<Boolean>]]
+            [markpublishedasdraft [<Boolean>]]
+            [members none|all|students|teachers]]
+            [logdrivefileids [<Boolean>]]
+
+gam update course <CourseID> <CourseAttribute>+
+        [copyfrom <CourseID>
+            [announcementstates <CourseAnnouncementStateList>]
+            [materialstates <CourseMaterialStateList>]
+            [workstates <CourseWorkStateList>]
+                [removeduedate [<Boolean>]]
+                [mapsharemodestudentcopy edit|none|view]
+            [copymaterialsfiles [<Boolean>]]
+            [copytopics [<Boolean>]]
+            [markdraftaspublished [<Boolean>]]
+            [markpublishedasdraft [<Boolean>]]
+            [members none|all|students|teachers]]
+            [logdrivefileids [<Boolean>]]
+gam update courses <CourseEntity> <CourseAttribute>+
+        [copyfrom <CourseID>
+            [announcementstates <CourseAnnouncementStateList>]
+            [materialstates <CourseMaterialStateList>]
+            [workstates <CourseWorkStateList>]
+                [removeduedate [<Boolean>]]
+                [mapsharemodestudentcopy edit|none|view]
+            [copymaterialsfiles [<Boolean>]]
+            [copytopics [<Boolean>]]
+            [markdraftaspublished [<Boolean>]]
+            [markpublishedasdraft [<Boolean>]]
+            [members none|all|students|teachers]]
+            [logdrivefileids [<Boolean>]]
+```
+`copyfrom <CourseID>` allows copying of course announcements, work, topics and members from one course to another.
+* Accouncements - By default, no course announcements are copied
+    * `announcementstates <CourseAnnouncementStateList>` - Copy class announcements with the specified states
+* Materials - By default, no course materials are copied
+    * `materialstates <CourseMaterialsStateList>` - Copy class materials with the specified states
+* Work - By default, no course work is copied
+    * `workstates <CourseWorkStateList>` - Copy class work with the specified states
+        * `removeduedate false` - Remove due dates before the current time; this is the default
+        * `removeduedate|removeduedate true` - Remove all due dates
+* Announcements, Materials and Work Materials files
+    * `copymaterialsfiles false` - Copy links to files referenced by materials in the `copyfrom` course; this is the default
+    * `copymaterialsfiles|copymaterialsfiles true` - Copy files referenced by materials in the `copyfrom` course
+        * You must verify that the teacher of the course being created/updated has access to the files in the `copyfrom` course
+        * Files can only be copied to a course that is ACTIVE; GAM will adjust the course state as necessary
+* Topics - By default, no course topics are copied; if topics are not copied, references to them will be deleted from class work that is copied
+    * `copytopics false` - No course topics are copies
+    * `copytopics|copytopics true` - Copy topics
+* Published Material and Work - By default, published material and work is not relabeled
+    * `markdraftaspublished false` - Do not relabel draft material/work as published; this is the default
+    * `markdraftaspublished|markpublishedasdraft true` - Relabel draft material/work as published
+    * `markpublishedasdraft false` - Do not relabel published material/work as draft; this is the default
+    * `markpublishedasdraft|markpublishedasdraft true` - Relabel published material/work as draft
+* Members - By default, no course members are copied
+    * `members none` - No course members are copied
+    * `members all` - Copy course students and teachers
+    * `members students` - Copy students
+    * `members teachers` - Copy teachers
+
+When true, `logdrivefileids [<Boolean>]` generates a CSV file with headers `courseId,ownerId,fileId' that
+lists all drive files in the course.
+
+The Classroom API does not support course materials of type `form`, they will not be copied.
+
+Drive files with `shareMode` `Each student will get a copy` don't seem to be able to be copied.
+  * `mapsharemodestudentcopy edit` - Map `Each student will get a copy` to `Students can edit file`
+  * `mapsharemodestudentcopy view` - Map `Each student will get a copy` to `Students can view file`
+  * `mapsharemodestudentcopy none` or not specified - No `shareMode` mapping is performed, you may get an error
+
+## Delete courses
+Classes can only be deleted when they are in the ARCHIVED state; to delete a class, you can update its state to ARCHIVED
+and then delete it or you can specify that it be archived as part of the delete command.
+```
+gam delete course <CourseID> [archived]
+gam delete courses <CourseEntity> [archived]
+```
+## Manage course aliases
+These commands can process a single course.
+```
+gam course <CourseID> add alias <CourseAlias>
+gam course <CourseID> delete alias <CourseAlias>
+```
+These commands can process multiple courses.
+```
+gam courses <CourseEntity> add alias <CourseAliasEntity>
+gam courses <CourseEntity> delete alias <CourseAliasEntity>
+```
+## Manage course topics
+These commands can process a single course.
+```
+gam course <CourseID> add topic <CourseTopic>
+gam course <CourseID> delete topic <CourseTopicID>
+```
+These commands can process multiple courses.
+```
+gam courses <CourseEntity> add topic <CourseTopicEntity>
+gam courses <CourseEntity> delete topic <CourseTopicIDEntity>
+```
+## Display courses
+```
+gam info course <CourseID> [owneremail] [alias|aliases] [show all|students|teachers] [countsonly]
+        [fields <CourseFieldNameList>] [skipfields <CourseFieldNameList>] [formatjson]
+gam info courses <CourseEntity> [owneremail] [alias|aliases] [show all|students|teachers] [countsonly]
+        [fields <CourseFieldNameList>] [skipfields <CourseFieldNameList>] [formatjson]
+
+gam print courses [todrive <ToDriveAttribute>*]
+        (course|class <CourseEntity>)*|([teacher <UserItem>] [student <UserItem>] [states <CourseStateList>])
+        [owneremail] [owneremailmatchpattern <RegularExpression>]
+        [alias|aliases|aliasesincolumns [delimiter <Character>]]
+        [show all|students|teachers] [countsonly]
+        [fields <CourseFieldNameList>] [skipfields <CourseFieldNameList>] [formatjson [quotechar <Character>]]
+        [timefilter creationtime|updatetime] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>]
+```
+By default, the `print courses` command displays information about all courses.
+
+To get information about a specific set of courses, use the following option; it can be repeated to select multiple courses.
+* `(course|class <CourseEntity>)*` - Display courses with the IDs specified in `<CourseEntity>`.
+
+To get information about courses based on its owner's emailaddress, use the `owneremailmatchpattern <RegularExpression>` option.
+* `foo@bar.com` - Display courses with a specific owner emailaddress.
+* `.*test.*` - Display courses with an owner emailaddress that matches a pattern.
+* `Unknown user` - Display courses where the owner emailaddress has been deleted.
+
+To get information about courses based on their having a particular participant, use the following options. Both options can be specified.
+* `teacher <UserItem>` - Display courses with the specified teacher.
+* `student <UserItem>` - Display courses with the specified student.
+
+To get information about courses based on their state, use the following option. This option can be combined with the `teacher` and `student` options.
+By default, all course states are selected.
+* `states <CourseStateList>` - Display courses with any of the specified states.
+
+To get information about courses created/updated within a particular time frame, use the following options.
+* `timefilter creationtime|updatetime` - select which event to filter
+* `start|starttime <Date>|<Time>` - specify the start of the time frame; if not specified, the time frame will be open ended at the start
+* `end|endtime <Date>|<Time>` - specify the end of the time frame; if not specified, the time frame will be open ended at the end
+For the filter to apply, `timefilter` and at least one of `start|starttime` and `end|endtime` must be specified.
+
+By default, all basic course fields are displayed; use the following options to modify the output.
+* `owneremail` - Display course owner email; requires an additional API call per course.
+* `alias|aliases` - Display course aliases; all aliases are in the single column `Aliases` separated by a delimiter; requires an additional API call per course.
+    * `delimiter <Character>` - Delimiter between aliases with `print` command.
+* `aliasesincolumn` - Display course aliases; the `Aliases` column contains the number of aliases and `Aliases.0`, `Aliases.1`, ... contain the individual aliases; requires an additional API call per course.
+* `show all|students|teachers` - Show class participants profile information; requires an additional API call per course.
+    * `countsonly` - Eliminates the student/teacher profile information and outputs only the student/teacher counts.
+* `fields <CourseFieldNameList>` - Select specific basic fields to display.
+* `skipfields <CourseFieldNameList>` - Select specific basic fields to eliminate from display; typically used with `coursematerialsets`.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display course counts
+Display the number of courses.
+```
+gam print courses
+        (course|class <CourseEntity>)*|([teacher <UserItem>] [student <UserItem>] [states <CourseStateList>])
+        [owneremailmatchpattern <RegularExpression>]
+        showitemcountonly
+```
+Example
+```
+$ gam print courses states active showitemcountonly
+Getting all Courses that match query (Course State: ACTIVE), may take some time on a large Google Workspace Account...
+Got 268 Courses...
+Got 272 Courses...
+Got 272 Courses...
+272
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print courses states active showitemcountonly)
+Windows PowerShell
+count = & gam print courses states active showitemcountonly
+```
+
+## Display course announcements
+```
+gam print course-announcements [todrive <ToDriveAttribute>*]
+        (course|class <CourseEntity>)*|([teacher <UserItem>] [student <UserItem>] states <CourseStateList>])
+        (courseannouncementids <CourseAnnouncementIDEntity>)|(announcementstates <CourseAnnouncementStateList>)*
+        (orderby <CourseAnnouncementOrderByFieldName> [ascending|descending])*)
+        [creatoremail] [fields <CourseAnnouncementFieldNameList>] [formatjson [quotechar <Character>]]
+        [timefilter creationtime|updatetime|scheduledtime] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>]
+```
+By default, the `print course-announcements` command displays course announcement information for all courses.
+
+To get course announcements for a specific set of courses, use the following option; it can be repeated to select multiple courses.
+* `(course|class <CourseEntity>)*` - Display courses with the IDs specified in `<CourseEntity>`.
+
+To get course announcements for courses based on their having a particular participant, use the following options. Both options can be specified.
+* `teacher <UserItem>` - Display courses with the specified teacher.
+* `student <UserItem>` - Display courses with the specified student.
+
+To get course announcements for courses based on their state, use the following option. This option can be combined with the `teacher` and `student` options.
+By default, all course states are selected.
+* `states <CourseStateList>` - Display courses with any of the specified states.
+
+By default, all published course announcements for a course are displayed; use the following options to select specific course announcements.
+* `courseannouncementids <CourseAnnouncementIDEntity>` - Display course announcements with the IDs specified in `<CourseAnnouncementIDEntity>`.
+* `announcementstates <CourseAnnouncementStateList>` - Display course announcements with any of the specified states.
+
+To get information about course announcements created/updated/scheduled within a particular time frame, use the following options.
+* `timefilter creationtime|updatetime|scheduledtime` - select which event to filter
+* `start|starttime <Date>|<Time>` - specify the start of the time frame; if not specified, the time frame will be open ended at the start
+* `end|endtime <Date>|<Time>` - specify the end of the time frame; if not specified, the time frame will be open ended at the end
+For the filter to apply, `timefilter` and at least one of `start|starttime` and `end|endtime` must be specified.
+
+By default, all course announcement fields are displayed; use the following options to modify the output.
+* `creatoremail` - Display course announcement creator email; requires an additional API call per course announcement.
+* `fields <CourseAnnouncementFieldNameList>` - Select specific fields to display.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display course materials
+```
+gam print course-materials [todrive <ToDriveAttribute>*]
+        (course|class <CourseEntity>)*|([teacher <UserItem>] [student <UserItem>] states <CourseStateList>])
+        (materialids <CourseMaterialIDEntity>)|(materialstates <CourseMaterialStateList>)*
+        (orderby <CourseMaterialOrderByFieldName> [ascending|descending])*)
+        [showcreatoremails|creatoremail] [showtopicnames] [fields <CourseMaterialFieldNameList>] [formatjson [quotechar <Character>]]
+        [timefilter creationtime|updatetime|scheduledtime] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>]
+```
+By default, the `print course-materials` command displays course materials information for all courses.
+
+To get course materials information for a specific set of courses, use the following option; it can be repeated to select multiple courses.
+* `(course|class <CourseEntity>)*` - Display courses with the IDs specified in `<CourseEntity>`.
+
+To get course materials information for courses based on their having a particular participant, use the following options. Both options can be specified.
+* `teacher <UserItem>` - Display courses with the specified teacher.
+* `student <UserItem>` - Display courses with the specified student.
+
+To get course materials information for courses based on their state, use the following option. This option can be combined with the `teacher` and `student` options.
+By default, all course states are selected.
+* `states <CourseStateList>` - Display courses with any of the specified states.
+
+To get information about course materials created/updated/scheduled within a particular time frame, use the following options.
+* `timefilter creationtime|updatetime|scheduledtime` - select which event to filter
+* `start|starttime <Date>|<Time>` - specify the start of the time frame; if not specified, the time frame will be open ended at the start
+* `end|endtime <Date>|<Time>` - specify the end of the time frame; if not specified, the time frame will be open ended at the end
+For the filter to apply, `timefilter` and at least one of `start|starttime` and `end|endtime` must be specified.
+
+By default, all published course materials for a course are displayed; use the following options to select specific course materials.
+* `materialsids <CourseMaterialsIDEntity>` - Display course materials with the IDs specified in `<CourseMaterialsIDEntity>`.
+* `materialsstates <CourseMaterialsStateList>` - Display course materials with any of the specified states.
+
+By default, all course materials fields are displayed; use the following options to modify the output.
+* `showcreatoremails` - Display course materials creator email; requires an additional API call per course materials.
+* `showtopicnames` - Display topic names; requires and additional API call per course.
+* `fields <CourseMaterialsFieldNameList>` - Select specific fields to display.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display course topics
+```
+gam print course-topics [todrive <ToDriveAttribute>*]
+        (course|class <CourseEntity>)*|([teacher <UserItem>] [student <UserItem>] states <CourseStateList>])
+        (coursetopicids <CourseTopicIDEntity>)
+        [formatjson [quotechar <Character>]]
+        [timefilter updatetime] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>]
+```
+By default, the `print course-topics` command displays course topic information for all courses.
+
+To get course topics for a specific set of courses, use the following option; it can be repeated to select multiple courses.
+* `(course|class <CourseEntity>)*` - Display courses with the IDs specified in `<CourseEntity>`.
+
+To get course topics for courses based on their having a particular participant, use the following options. Both options can be specified.
+* `teacher <UserItem>` - Display courses with the specified teacher.
+* `student <UserItem>` - Display courses with the specified student.
+
+To get course topics for courses based on their state, use the following option. This option can be combined with the `teacher` and `student` options.
+By default, all course states are selected.
+* `states <CourseStateList>` - Display courses with any of the specified states.
+
+By default, all published course topics for a course are displayed; use the following options to select specific course topics.
+* `coursetopicids <CourseTopicIDEntity>` - Display course topics with the IDs specified in `<CourseTopicIDEntity>`.
+* `topicstates <CourseTopicStateList>` - Display course topics with any of the specified states.
+
+To get information about course topics updated within a particular time frame, use the following options.
+* `timefilter updatetime` - select which event to filter
+* `start|starttime <Date>|<Time>` - specify the start of the time frame; if not specified, the time frame will be open ended at the start
+* `end|endtime <Date>|<Time>` - specify the end of the time frame; if not specified, the time frame will be open ended at the end
+For the filter to apply, `timefilter` and at least one of `start|starttime` and `end|endtime` must be specified.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display course work
+```
+gam print course-work [todrive <ToDriveAttribute>*]
+        (course|class <CourseEntity>)*|([teacher <UserItem>] [student <UserItem>] states <CourseStateList>])
+        (workids <CourseWorkIDEntity>)|(workstates <CourseWorkStateList>)*
+        (orderby <CourseWorkOrderByFieldName> [ascending|descending])*)
+        [showcreatoremails] [showtopicnames] [fields <CourseWorkFieldNameList>] [formatjson [quotechar <Character>]]
+        [timefilter creationtime|updatetime|scheduledtime] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>]
+```
+By default, the `print course-work` command displays course work information for all courses.
+
+To get course work information for a specific set of courses, use the following option; it can be repeated to select multiple courses.
+* `(course|class <CourseEntity>)*` - Display courses with the IDs specified in `<CourseEntity>`.
+
+To get course work information for courses based on their having a particular participant, use the following options. Both options can be specified.
+* `teacher <UserItem>` - Display courses with the specified teacher.
+* `student <UserItem>` - Display courses with the specified student.
+
+To get course work information for courses based on their state, use the following option. This option can be combined with the `teacher` and `student` options.
+By default, all course states are selected.
+* `states <CourseStateList>` - Display courses with any of the specified states.
+
+To get information about course work created/updated/scheduled within a particular time frame, use the following options.
+* `timefilter creationtime|updatetime|scheduledtime` - select which event to filter
+* `start|starttime <Date>|<Time>` - specify the start of the time frame; if not specified, the time frame will be open ended at the start
+* `end|endtime <Date>|<Time>` - specify the end of the time frame; if not specified, the time frame will be open ended at the end
+For the filter to apply, `timefilter` and at least one of `start|starttime` and `end|endtime` must be specified.
+
+By default, all pub`lished course work for a course is displayed; use the following options to select specific course work.
+* `workids <CourseWorkIDEntity>` - Display course work with the IDs specified in `<CourseWorkIDEntity>`.
+* `workstates <CourseWorkStateList>` - Display course work with any of the specified states.
+
+By default, all course work fields are displayed; use the following options to modify the output.
+* `showcreatoremails` - Display course work creator email; requires an additional API call per course work.
+* `showtopicnames` - Display topic names; requires and additional API call per course.
+* `fields <CourseWorkFieldNameList>` - Select specific fields to display.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display course submissions
+```
+gam print course-submissions [todrive <ToDriveAttribute>*]
+        (course|class <CourseEntity>)*|([teacher <UserItem>] [student <UserItem>] states <CourseStateList>])
+        (workids <CourseWorkIDEntity>)|(workstates <CourseWorkStateList>)*
+        (orderby <CourseWorkOrderByFieldName> [ascending|descending])*)
+        (submissionids <CourseSubmissionIDEntity>)|(submissionstates <CourseSubmissionStateList>)*) [late|notlate]
+        [fields <CourseSubmissionFieldNameList>] [showuserprofile] [formatjson [quotechar <Character>]]
+        [timefilter creationtime|updatetime] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>]
+```
+By default, the `print course-submissions` command displays course submission information for all course work for all courses.
+
+To get course submission information for a specific set of courses, use the following option; it can be repeated to select multiple courses.
+* `(course|class <CourseEntity>)*` - Display courses with the IDs specified in `<CourseEntity>`.
+
+To get course submission information for courses based on their having a particular participant, use the following options. Both options can be specified.
+* `teacher <UserItem>` - Display courses with the specified teacher.
+* `student <UserItem>` - Display courses with the specified student.
+
+To get course submission information for courses based on their state, use the following option. This option can be combined with the `teacher` and `student` options.
+By default, all course states are selected.
+* `states <CourseStateList>` - Display courses with any of the specified states.
+
+By default, all course work for a course is displayed; use the following options to select specific course work.
+* `workids <CourseWorkIDEntity>` - Display course work with the IDs specified in `<CourseWorkIDEntity>`.
+* `workstates <CourseWorkStateList>` - Display course work with any of the specified states.
+
+By default, all course submissions for a course work is displayed; use the following options to select specific course submissions.
+* `submissionids <CourseSubmissionIDEntity>` - Display course submissions with the IDs specified in `<CourseSubmissionIDEntity>`.
+* `submissionstates <CourseSubmissionStateList>` - Display course submissions with any of the specified states.
+* `late` - Display course submissions marked late.
+* `notlate` - Display course submissions not marked late.
+
+To get information about course submissionss created/updated within a particular time frame, use the following options.
+* `timefilter creationtime|updatetime` - select which event to filter
+* `start|starttime <Date>|<Time>` - specify the start of the time frame; if not specified, the time frame will be open ended at the start
+* `end|endtime <Date>|<Time>` - specify the end of the time frame; if not specified, the time frame will be open ended at the end
+For the filter to apply, `timefilter` and at least one of `start|starttime` and `end|endtime` must be specified.
+
+By default, all course submission fields are displayed; use the following options to modify the output.
+* `fields <CourseSubmissionFieldNameList>` - Select specific fields to display.
+
+By default, only the numeric userId is displayed; use the `showuserprofile` option to get the user email address and name.
+You can only get profile information if the scope `https://www.googleapis.com/auth/classroom.profile.emails` is enabled
+for service account access; verify with `gam <UserTypeEntity> update serviceaccount`.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Classroom-Guardians.md

@@ -0,0 +1,189 @@
+# Classroom - Guardians
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Create guardian invitations](#create-guardian-invitations)
+- [Delete guardian invitations](#delete-guardian-invitations)
+- [Display guardian invitations](#display-guardian-invitations)
+- [Delete guardians](#delete-guardians)
+- [Synchronize guardians](#synchronize-guardians)
+- [Display guardians, indented keys and values](#display-guardians-indented-keys-and-values)
+- [Display guardians, CSV format](#display-guardians-csv-format)
+
+## API documentation
+* https://developers.google.com/classroom/reference/rest/v1/userProfiles.guardianInvitations
+* https://developers.google.com/classroom/reference/rest/v1/userProfiles.guardians
+
+## Definitions
+```
+<DomainName> ::= <String>(.<String>)+
+<EmailAddress> ::= <String>@<DomainName>
+<UniqueID> ::= id:<String>
+<GuardianItem> ::= <EmailAddress>|<UniqueID>|<String>
+<GuardianItemList> ::= "<GuardianItem>(,<GuardianItem>)*"
+<GuardianEntity> ::=
+        <GuardianList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<StudentItem> ::= <EmailAddress>|<UniqueID>|<String>
+<GuardianInvitationID> ::= <String>
+<GuardianInvitationIDList> ::= "<GuardianInvitationId>(,<GuardianInvitationID>)*"
+<GuardianInvitationIDEntity> ::=
+        <GuardianInvitationIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<GuardianState> ::= complete|pending
+<GuardianStateList> ::= "<GuardianState>(,<GuardianState>)*"
+```
+## Create guardian invitations
+### Selected students, new style
+```
+gam <UserTypeEntity> create|add guardian|guardianinvite|inviteguardian <GuardianEntity>
+```
+### Selected students, old style
+```
+gam create guardian|guardianinvite|inviteguardian <EmailAddress> <StudentItem>
+```
+## Delete guardian invitations
+### Selected students, new style
+```
+gam <UserTypeEnfity> cancel guardianinvitation|guardianinvitations <GuardianInvitationIDEntity>
+gam <UserTypeEntity> delete guardian|guardians <GuardianEntity> invitations
+gam <UserTypeEntity> clear guardian|guardians invitations
+```
+### Selected students, old style
+```
+gam cancel guardianinvitation|guardianinvitations <GuardianInvitationID> <StudentItem>
+gam delete guardian|guardians <GuardianItem> <StudentItem> invitations
+```
+## Display guardian invitations
+### All students
+```
+gam show guardian|guardians invitations [states <GuardianInvitationStateList>] [invitedguardian <EmailAddress>]
+        [showstudentemails] [formatjson]
+gam print guardian|guardians [todrive <ToDriveAttribute>*] invitations [states <GuardianInvitationStateList>] [invitedguardian <EmailAddress>]
+        [showstudentemails] [formatjson [quotechar <Character>]]
+```
+The Classroom API does not return the student email address, use the `showstudentemails` option to get the student email address. This requires an additional API call per student.
+
+### Selected students, new style
+```
+gam <UserTypeEntity> show guardian|guardians invitations [states <GuardianInvitationStateList>] [invitedguardian <EmailAddress>]
+        [formatjson]
+gam <UserTypeEntity> print guardian|guardians [todrive <ToDriveAttribute>*] invitations [states <GuardianInvitationStateList>] [invitedguardian <EmailAddress>]
+        [formatjson [quotechar <Character>]]
+```
+### Selected students, old style
+```
+gam show guardian|guardians invitations [showstudentemails] [states <GuardianStateList>] [invitedguardian <EmailAddress>]
+        [student <StudentItem>] [<UserTypeEntity>]
+        [formatjson]
+gam print guardian|guardians [todrive <ToDriveAttribute>*] invitations [showstudentemails] [states <GuardianStateList>] [invitedguardian <EmailAddress>]
+        [student <StudentItem>] [<UserTypeEntity>]
+        [formatjson [quotechar <Character>]]
+```
+By default, Gam displays informations for all guardian invitations; you can limit the display with the following options.
+* `states <GuardianStateList>` - Display guardian invitations with the specified state
+* `invitedguardian <EmailAddress>` - Display guardians invitations with `<EmailAddress>`
+
+## Delete guardians
+### Selected students, new style
+```
+gam <UserTypeEntity> delete guardian|guardians <GuardianEntity> [accepted|invitations|all]
+gam <UserTypeEntity> clear guardian|guardians [accepted|invitations|all]
+```
+* `accepted` - Delete accepted invitations
+* `invitations` - Delete pending invitations
+* `all` - Delete accepted and pending invitations
+
+### Selected students, old style
+```
+gam delete guardian|guardians <GuardianItem> <StudentItem>
+```
+
+## Synchronize guardians
+Gam deletes any pending guardian invitations and accepted guardians that are not in `<GuardianEntity>` and sends
+invitations to the members in `<GuardianEntity>` that don't have a pending invitation or have not accepted.
+```
+gam <UserTypeEntity> sync guardian|guardians <GuardianEntity>
+```
+### Example
+Your school SIS produces a CSV file, StudentGuardians.csv, each evening with two columns: Student,Guardian.
+There is no indication as to what changes have been made from the night before. The following command will perform the
+necessary changes.
+```
+gam csvkmd users StudentGuardians.csv keyfield Student datafield Guardian sync guardians csvdata Guardian
+```
+
+## Display guardians, indented keys and values
+### All students
+```
+gam show guardian|guardians [accepted|invitations|all]
+        [states <GuardianInvitationStateList>] [invitedguardian <EmailAddress>]
+        [showstudentemails] [formatjson]
+```
+### Selected students, new style
+```
+gam <UserTypeEntity> show guardian|guardians [accepted|invitations|all]
+        [states <GuardianInvitationStateList>] [invitedguardian <EmailAddress>]
+        [formatjson]
+```
+### Selected students, old style
+```
+gam show guardian|guardians [accepted|invitations|all] [invitedguardian <EmailAddress>]
+        [states <GuardianInvitationStateList>] [invitedguardian <EmailAddress>]
+        [student <StudentItem>] [<UserTypeEntity>]
+        [showstudentemails] [formatjson]
+```
+Use these options to control what information is displayed:
+* `accepted` - Display accepted guardians; this is the default
+* `invitations` - Display invitations
+    * `states <GuardianInvitationStateList>` - Filter the invitations by state
+* `all` - Display accepted guardians and pending invitations
+    * `states <GuardianInvitationStateList>` - Filter the invitations by state
+
+By default, Gam displays informations for all guardians; you can limit the display with the following option:
+* `invitedguardian <EmailAddress>` - Display guardians with `<EmailAddress>`.
+
+The Classroom API does not return the student email address, use the `showstudentemails` option to get the student email address. This requires an additional API call per student.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+## Display guardians, CSV format
+### All students
+```
+gam print guardian|guardians [todrive <ToDriveAttribute>*] [accepted|invitations|all]
+        [states <GuardianInvitationStateList>] [invitedguardian <EmailAddress>]
+        [showstudentemails] [formatjson [quotechar <Character>]]
+```
+### Selected students, new style
+```
+gam <UserTypeEntity> print guardian|guardians [todrive <ToDriveAttribute>*] [accepted|invitations|all]
+        [states <GuardianInvitationStateList>] [invitedguardian <EmailAddress>]
+        [formatjson [quotechar <Character>]]
+```
+### Selected students, old style
+```
+gam print guardian|guardians [todrive <ToDriveAttribute>*] [accepted|invitations|all]
+        [states <GuardianInvitationStateList>] [invitedguardian <EmailAddress>]
+        [student <StudentItem>] [<UserTypeEntity>]
+        [showstudentemails] [formatjson [quotechar <Character>]]
+```
+Use these options to control what information is displayed:
+* `accepted` - Display accepted guardians; this is the default
+* `invitations` - Display invitations
+    * `states <GuardianInvitationStateList>` - Filter the invitations by state
+* `all` - Display accepted guardians and pending invitations
+    * `states <GuardianInvitationStateList>` - Filter the invitations by state
+
+By default, Gam displays informations for all guardians; you can limit the display with the following options.
+* `invitedguardian <EmailAddress>` - Display guardians with `<EmailAddress>`.
+
+The Classroom API does not return the student email address, use the `showstudentemails` option to get the student email address. This requires an additional API call per student.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Classroom-Invitations.md

@@ -0,0 +1,163 @@
+# Classroom - Invitations
+- [API documentation](#api-documentation)
+- [Notes](#notes)
+- [Definitions](#definitions)
+- [Create classroom invitations](#create-classroom-invitations)
+- [Accept classroom invitations by user](#accept-classroom-invitations-by-user)
+- [Delete classroom invitations by user](#delete-classroom-invitations-by-user)
+- [Display classroom invitations by user](#display-classroom-invitations-by-user)
+- [Delete classroom invitations by course](#delete-classroom-invitations-by-course)
+- [Display classroom invitations by course](#display-classroom-invitations-by-course)
+
+## API documentation
+* https://developers.google.com/classroom/reference/rest/v1/invitations
+
+## Notes
+
+You must authorize an additional Service Account scope to use these commands.
+Do this command; sustitute a valid email address for user@domain.com.
+```
+gam user user@domain.com check serviceaccount
+```
+You should see the following scope fail:
+```
+Scope: https://www.googleapis.com/auth/classroom.rosters           , Checked: FAIL (6/15)
+```
+Follow the directions to authorize the Service Account scopes.
+
+## Definitions
+```
+<DomainName> ::= <String>(.<String>)+
+<EmailAddress> ::= <String>@<DomainName>
+<UniqueID> ::= id:<String>
+<ClassroomInvitationID> ::= <String>
+<ClassroomInvitationIDList> ::= "<ClassroomInvitationID>(,<ClassroomInvitationID>)*"
+<ClassroomInvitationIDEntity> ::=
+        <ClassroomInvitationIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<CourseAlias> ::= <String>
+<CourseID> ::= <Number>|d:<CourseAlias>
+<CourseIDList> ::= "<CourseID>(,<CourseID>)*"
+<CourseEntity> ::=
+        <CourseIDList> | <FileSelector> | <CSVFileSelector | <CSVkmdSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<CourseState> ::= active|archived|provisioned|declined|suspended
+<CourseStateList> ::= all|"<CourseState>(,<CourseState>)*"
+```
+## Create classroom invitations
+Invite users to classes.
+```
+gam <UserTypeEntity> create classroominvitation courses <CourseEntity> [role owner|student|teacher]
+        [adminaccess|asadmin]
+        [csv|csvformat] [todrive <ToDriveAttributes>*] [formatjson [quotechar <Character>]]
+```
+If `role` is not specified, `student` will be used.
+
+You can only invite a co-teacher to be an owner of a course.
+
+By default, classroom invitations are issued by the owner of the course, the `adminaccess` option causes the invitations to be issued by the admin named in `oauth2.txt`.
+
+By default, when an invitation is created, GAM outputs details of the invitation as indented keywords and values.
+* `csv|csvformat [todrive <ToDriveAttribute>*] [formatjson [quotechar <Character>]]` - Output the details in CSV format.
+
+### Example
+
+Suppose you have a CSV file CourseStudent.csv  with two columns: Course,Student.
+This command will invite all students to their courses serially by student.
+```
+gam redirect stdout ./Invites.out redirect stderr stdout csvkmd users CourseStudent.csv keyfield Student datafield Course create classroominvitation role student course csvdata Course
+```
+This command will invite all students to their courses in parallel
+```
+gam redirect stdout ./Invites.out multiprocess redirect stderr stdout multiprocess csv CourseStudent.csv gam user ~Student create classroominvitation role student course ~Course
+```
+## Accept classroom invitations by user
+Accept classroom invitations for users.
+```
+gam <UserTypeEntity> accept classroominvitation (ids <ClassroomInvitationIDEntity>)|([courses <CourseEntity>] [role all|owner|student|teacher])
+```
+`<UserTypeEntity>` must specify users in your domain.
+
+By default, all invitations for the specified users will be accepted.
+
+Select specific invitations to accept:
+* `ids <ClassroomInvitationIDEntity>` - Specify invitation IDs
+
+Select courses and accept invitations for those courses.
+* `courses <CourseEntity>` - Specify courses
+
+By default, invitations for all roles will be accepted; you can limit the acceptances to invitations of a specific role.
+
+## Delete classroom invitations by user
+Delete classroom invitations for users.
+```
+gam <UserTypeEntity> delete classroominvitation (ids <ClassroomInvitationIDEntity>)|([courses <CourseEntity>] [role all|owner|student|teacher])
+```
+`<UserTypeEntity>` must specify users in your domain.
+
+By default, all invitations for the specified users will be deleted.
+
+Select specific invitations to delete:
+* `ids <ClassroomInvitationIDEntity>` - Specify invitation IDs
+
+Select courses and delete invitations for those courses.
+* `courses <CourseEntity>` - Specify courses
+
+By default, invitations for all roles will be deleted; you can limit the deletions to invitations of a specific role.
+
+## Display classroom invitations by user
+Display classroom invitations for users.
+```
+gam <UserTypeEntity> show classroominvitations [role all|owner|student|teacher]
+        [formatjson]
+gam <UserTypeEntity> print classroominvitations [todrive <ToDriveAttributes>*] [role all|owner|student|teacher]
+        [formatjson [quotechar <Character>]]
+```
+`<UserTypeEntity>` must specify users in your domain.
+
+By default, invitations for all roles will be displayed; you can limit the display to invitations of a specific role.
+
+## Delete classroom invitations by course
+Delete classroom invitations for courses. This command must be used to delete non-domain member invitations.
+```
+gam delete classroominvitation courses <CourseEntity> (ids <ClassroomInvitationIDEntity>)|(role all|owner|student|teacher)
+```
+Select courses and delete invitations for those courses.
+* `courses <CourseEntity>` - Specify courses
+
+Select specific invitations to delete:
+* `ids <ClassroomInvitationIDEntity>` - Specify invitation IDs
+
+Select invitations to delete by role. By default, invitations for all roles will be deleted; you can limit the deletions to invitations of a specific role.
+
+## Display classroom invitations by course
+```
+gam show classroominvitations (course|class <CourseEntity>)*|([teacher <UserItem>] [student <UserItem>] [states <CourseStateList>])
+        [role all|owner|student|teacher] [formatjson]
+gam print classroominvitations [todrive <ToDriveAttributes>*] (course|class <CourseEntity>)*|([teacher <UserItem>] [student <UserItem>] [states <CourseStateList>])
+        [role all|owner|student|teacher] [formatjson [quotechar <Character>]]
+```
+By default, classroom invitations for all courses are displayed.
+
+To get classroom invitations for a specific set of courses, use the following option; it can be repeated to select multiple courses.
+* `(course|class <CourseEntity>)*` - Display classroom invitations from the courses with the IDs specified in `<CourseEntity>`.
+
+To get classroom invitations for courses based on their having a particular participant, use the following options. Both options can be specified.
+* `teacher <UserItem>` - Display courses with the specified teacher.
+* `student <UserItem>` - Display courses with the specified student.
+
+To get classroom invitations for courses based on their state, use the following option. This option can be combined with the `teacher` and `student` options.
+By default, all course states are selected.
+* `states <CourseStateList>` - Display courses with any of the specified states.
+
+By default, for `show`, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+By default, for `print`, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Classroom-Membership.md

@@ -0,0 +1,166 @@
+# Classroom - Membership
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Special quoting for course aliases](#special-quoting-for-course-aliases)
+- [Manage membership for courses](#manage-membership-for-courses)
+- [Legacy manage membership](#legacy-manage-membership)
+- [Bulk membership changes](#bulk-membership-changes)
+- [Display course membership](#display-course-membership)
+- [Display course membership counts](#display-course-membership-counts)
+
+## API documentation
+* https://developers.google.com/classroom/reference/rest/
+* https://developers.google.com/classroom/reference/rest/v1/courses.students
+* https://developers.google.com/classroom/reference/rest/v1/courses.teachers
+
+## Definitions
+```
+<DomainName> ::= <String>(.<String>)+
+<EmailAddress> ::= <String>@<DomainName>
+<UniqueID> ::= id:<String>
+<UserItem> ::= <EmailAddress>|<UniqueID>|<String>
+
+<CourseAlias> ::= <String>
+<CourseID> ::= <Number>|d:<CourseAlias>
+<CourseIDList> ::= "<CourseID>(,<CourseID>)*"
+<CourseEntity> ::=
+        <CourseIDList> | <FileSelector> | <CSVFileSelector | <CSVkmdSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<CourseState> ::= active|archived|provisioned|declined|suspended
+<CourseStateList> ::= all|"<CourseState>(,<CourseState>)*"
+```
+## Special quoting for course aliases
+As course aliases can contain spaces, some care must be used when entering `<CourseAliasList>`, `<CourseID>`, `<CourseIDList>` and `<CourseEntity>`.
+
+Suppose you have a course with the alias `Math Class`. To get information about it you enter the command: `gam info course "d:Math Class"`
+
+The shell strips the `"` leaving a single argument `d:Math Class`; gam correctly processes the argument as it is expecting a single course.
+
+Suppose you enter the command: `gam info courses "d:Math Class"`
+
+The shell strips the `"` leaving a single argument `d:Math Class`; as gam is expecting a list, it splits the argument on space leaving two items and then tries to process `d:Math` and `Class`, not what you want.
+
+You must enter: `gam info courses "'d:Math Class'"`
+
+The shell strips the `"` leaving a single argument `'d:Math Class'`; as gam is expecting a list, it splits the argument on space while honoring the `'` leaving one item `d:Math Class` and correctly processes the item.
+
+For multiple aliases you must enter: `gam info courses "'d:Math Class','d:Science Class'"`
+
+See: [Lists and Collections](Lists-and-Collections)
+
+## Manage membership for courses
+
+These commands can process multiple courses and `add` and `delete` can process multiple students/teachers.
+```
+gam courses <CourseEntity> add teachers [makefirstteacherowner] <UserTypeEntity>
+gam courses <CourseEntity> add students <UserTypeEntity>
+gam courses <CourseEntity> delete|remove teachers|students <UserTypeEntity>
+gam courses <CourseEntity> clear teachers|students
+gam courses <CourseEntity> sync teachers [addonly|removeonly] [makefirstteacherowner] <UserTypeEntity>
+gam courses <CourseEntity> sync students [addonly|removeonly] <UserTypeEntity>
+```
+When `makefirstteacherowner` is specified, the first/only user in `<UserTypeEntity>` will be updated to be the
+owner of the Course(s).
+
+### Clear
+A `clear` operation deletes all of the members of the specified type. The owner teacher will not deleted.
+
+### Sync
+A `sync` operation gets the current roster for a course and compares it to the proposed roster.
+
+Current/Default:
+* members in the proposed roster that are not in the current roster will be added
+* members in the current roster that are not in the proposed roster will deleted
+
+When the `addonly` option is specified:
+* members in the proposed roster that are not in the current roster will be added
+* members in the current roster that are not in the proposed roster will not be deleted
+
+When the `removeonly` option is specified:
+* members in the proposed roster that are not in the current roster will not be added
+* members in the current roster that are not in the proposed roster will be deleted
+
+## Bulk membership changes
+Suppose you have a CSV file (CourseStudents.csv) with headers: courseId,email
+
+Each row contains a course ID and a student email address.
+
+The following command will synchronize the membership for all courses.
+```
+gam redirect stdout ./CourseUpdates.txt redirect stderr stdout courses csvkmd CourseStudents.csv keyfield courseId datafield email sync students csvdata email
+```
+You can also do `add` and `delete` in this manner.
+
+## Legacy manage membership
+
+These commands are for backward compatibility; only one course can be processed and `add` and `delete` can only process a single student/teacher.
+```
+gam course <CourseID> add [makefirstteacherowner] teachers <UserItem>
+gam course <CourseID> add students <UserItem>
+gam course <CourseID> delete|remove teachers|students <UserItem>
+gam course <CourseID> clear teachers|students
+gam course <CourseID> sync teachers [addonly|removeonly] [makefirstteacherowner] <UserTypeEntity>
+gam course <CourseID> sync students [addonly|removeonly] <UserTypeEntity>
+```
+When `makefirstteacherowner` is specified, the only/first user in `<UserItem>` or `<UserTypeEntity>` will be updated to be the
+owner of the Course.
+
+## Display course membership
+```
+gam print course-participants [todrive <ToDriveAttribute>*]
+        (course|class <CourseID>)*|([teacher <UserItem>] [student <UserItem>]) [states <CourseStateList>]
+        [show all|students|teachers] [formatjson [quotechar <Character>]]
+```
+By default, the `print course-participants` command displays participant information about all courses.
+
+To get participant information for a specific set of courses, use the following option; it can be repeated to select multiple courses.
+* `(course|class <CourseID>)*` - Display courses with the specified `<CourseID>`.
+
+To get participant information for courses based on their having a particular participant, use the following options. Both options can be specified.
+* `teacher <UserItem>` - Display courses with the specified teacher.
+* `student <UserItem>` - Display courses with the specified student.
+
+To get participant information for courses based on their state, use the following option. This option can be combined with the `teacher` and `student` options.
+By default, all course states are selected.
+* `states <CourseStateList>` - Display courses with any of the specified states.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display course membership counts
+Display the number of course participants.
+```
+gam print course-participants
+        (course|class <CourseID>)*|([teacher <UserItem>] [student <UserItem>]) [states <CourseStateList>]
+        [show all|students|teachers]
+        showitemcountonly
+```
+Example
+```
+$ gam print course-participants teacher asmith states active show students showitemcountonly
+Getting all Courses that match query (Teacher: asmith@domain.com, Course State: ACTIVE), may take some time on a large Google Workspace Account...
+Got 3 Courses...
+Getting Students for Course: 636981507234 (1/3)
+Got 30 Students...
+Got 43 Students...
+Getting Students for Course: 589346784341 (2/3)
+Got 22 Students...
+Getting Students for Course: 589345535881 (3/3)
+Got 23 Students...
+88
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print course-participants teacher asmith states active show students showitemcountonly)
+Windows PowerShell
+count = & gam print course-participants teacher asmith states active show students showitemcountonly
+```

docs/Cloud-Channel.md

@@ -0,0 +1,318 @@
+# Cloud Channel
+- [API documentation](#api-documentation)
+- [Notes](#notes)
+- [Definitions](#definitions)
+- [Display Channel Customers](#display-channel-customers)
+- [Display Channel Customer Entitlements](#display-channel-customer-entitlements)
+- [Display Channel Offers](#display-channel-offers)
+- [Display Channel Products](#display-channel-products)
+- [Display Channel SKUs](#display-channel-skus)
+
+## API documentation
+* https://cloud.google.com/channel/docs/reference/rest
+* https://cloud.google.com/channel/docs/concepts/google-cloud/filter-customers
+
+## Notes
+To use these commands you must add the 'Cloud Channel API' to your project and update your client authorization.
+```
+gam update project
+gam oauth create
+```
+
+The Customer ID value that the Cloud Channel API describes is not the Google Workspace Customer ID value; it is unique to the Cloud Channel API.
+
+## Definitions
+```
+<ChannelCustomerID> ::= <String>
+<ProductID> ::= <String>
+<ResellerID> ::= <String>
+
+<LanguageCode> ::=
+        ach|af|ag|ak|am|ar|az|be|bem|bg|bn|br|bs|ca|chr|ckb|co|crs|cs|cy|da|de|
+        ee|el|en|en-gb|en-us|eo|es|es-419|et|eu|fa|fi|fil|fo|fr|fr-ca|fy|
+        ga|gaa|gd|gl|gn|gu|ha|haw|he|hi|hr|ht|hu|hy|ia|id|ig|in|is|it|iw|ja|jw|
+        ka|kg|kk|km|kn|ko|kri|ku|ky|la|lg|ln|lo|loz|lt|lua|lv|
+        mfe|mg|mi|mk|ml|mn|mo|mr|ms|mt|my|ne|nl|nn|no|nso|ny|nyn|oc|om|or|
+        pa|pcm|pl|ps|pt-br|pt-pt|qu|rm|rn|ro|ru|rw|
+        sd|sh|si|sk|sl|sn|so|sq|sr|sr-me|st|su|sv|sw|
+        ta|te|tg|th|ti|tk|tl|tn|to|tr|tt|tum|tw|
+        ug|uk|ur|uz|vi|wo|xh|yi|yo|zh-cn|zh-hk|zh-tw|
+
+<ChannelCustomerField> ::=
+        alternateemail |
+        channelpartnerid |
+        cloudidentityid |
+        cloudidentityinfo |
+        createtime |
+        domain |
+        languagecode |
+        name |
+        orgdisplayname |
+        orgpostaladdress |
+        primarycontactinfo |
+        updatetime
+<ChannelCustomerFieldList> ::= "<ChannelCustomerField>(,<ChannelCustomerField>)*"
+
+<ChannelCustomerEntitlementField> ::=
+        associationinfo |
+        commitmentsettings |
+        createtime |
+        name |
+        offer |
+        parameters |
+        provisionedservice |
+        provisioningstate |
+        purchaseorderid |
+        suspensionreasons |
+        trialsettings |
+        updatetime
+<ChannelCustomerEntitlementFieldList> ::= "<ChannelCustomerEntitlementField>(,<ChannelCustomerEntitlementField>)*"
+```
+```
+<ChannelCustomerOfferField> ::=
+        constraints |
+        endtime |
+        marketinginfo |
+        name |
+        parameterdefinitions |
+        plan |
+        pricebyresources |
+        sku |
+        starttime
+<ChannelOfferFieldList> ::= "<ChannelOfferField>(,<ChannelOfferField>)*"
+
+<ChannelProductField> ::=
+        marketinginfo |
+        name
+<ChannelProductFieldList> ::= "<ChannelProductField>(,<ChannelProductField>)*"
+
+<ChannelSKUField> ::=
+        marketinginfo |
+        name |
+        product
+<ChannelSKUFieldList> ::= "<ChannelSKUField>(,<ChannelSKUField>)*"
+```
+## Display Channel Customers
+```
+gam show channelcustomers
+        [resellerid <ResellerID>] [filter <String>]
+        [fields <ChannelCustomerFieldList>]
+        [maxresults <Number>]
+        [formatjson]
+```
+If `resellerId <ResellerID>` is omitted, the `reseller_id` value from `gam.cfg` is used.
+
+Cloud Channel API documentation for `filter <String>`:
+* https://cloud.google.com/channel/docs/concepts/google-cloud/filter-customers
+
+The filters will contain `"`, you must quote `<String>` as follows:
+* Linux and MacOS
+  * Surround `<String>` with single quotes `'`
+  * Embedded `"` in `<String>` are entered as is
+  * Example: `gam show channelcustomers filter 'cloud_identity_id="someid"'`
+* Windows Command Prompt
+  * Surround `<String>` with double quotes `"`
+  * Embedded `"` in `<String>` are entered as `\"`
+  * Example: `gam show channelcustomers filter "cloud_identity_id=\"someid\""`
+* Windows PowerShell
+  * Surround `<String>` with single quotes `'`
+  * Embedded `"` in `<String>` are entered as `\"`
+  * Example: `gam show channelcustomers filter "cloud_identity_id=\"someid\""`
+
+When retrieving lists of customers from Cloud Channel API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many customers to retrieve in each API call; default is 50, the maximum.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print channelcustomers [todrive <ToDriveAttribute>*]
+        [resellerid <ResellerID>] [filter <String>]
+        [fields <ChannelCustomerFieldList>]
+        [maxresults <Number>]
+        [formatjson [quotechar <Character>]]
+```
+If `resellerId <ResellerID>` is omitted, the `reseller_id` value from `gam.cfg` is used.
+
+Cloud Channel API documentation for `filter <String>`:
+* https://cloud.google.com/channel/docs/concepts/google-cloud/filter-customers
+
+The filters will contain `"`, you must quote `<String>` as follows:
+* Linux and MacOS
+  * Surround `<String>` with single quotes `'`
+  * Embedded `"` in `<String>` are entered as is
+* Windows Command Prompt
+  * Surround `<String>` with double quotes `"`
+  * Embedded `"` in `<String>` are entered as `\"`
+* Windows PowerShell
+  * Surround `<String>` with single quotes `'`
+  * Embedded `"` in `<String>` are entered as `\"`
+
+When retrieving lists of customers from Cloud Channel API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many customers to retrieve in each API call; default is 50, the maximum.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display Channel Customer Entitlements
+```
+gam show channelcustomerentitlements
+        ([resellerid <ResellerID>] [customerid <ChannelCustomerID>])|
+        (name accounts/<ResellerID>/customers/<ChannelCustomerID>)
+        [fields <ChannelCustomerEntitlementsFieldList>]
+        [maxresults <Number>]
+        [formatjson]
+```
+If `name accounts/<ResellerID>/customers/<ChannelCustomerID>` is specified, `resellerId <ResellerID>` and `customerid <ChannelCustomerID>`
+are ignored.
+
+If `resellerId <ResellerID>` is omitted, the `reseller_id` value from `gam.cfg` is used.
+
+If `customerid <ChannelCustomerID>` is omitted, the `channel_customer_id` value from `gam.cfg` is used.
+
+When retrieving lists of customer entitlements from Cloud Channel API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many customer entitlements to retrieve in each API call; default is 100, the maximum.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print channelcustomerentitlements [todrive <ToDriveAttribute>*]
+        ([resellerid <ResellerID>] [customerid <ChannelCustomerID>])|
+        (name accounts/<ResellerID>/customers/<ChannelCustomerID>)
+        [fields <ChannelCustomerEntitlementsFieldList>]
+        [maxresults <Number>]
+        [formatjson [quotechar <Character>]]
+```
+If `name accounts/<ResellerID>/customers/<ChannelCustomerID>` is specified, `resellerId <ResellerID>` and `customerid <ChannelCustomerID>`
+are ignored.
+
+If `resellerId <ResellerID>` is omitted, the `reseller_id` value from `gam.cfg` is used.
+
+If `customerid <ChannelCustomerID>` is omitted, the `channel_customer_id` value from `gam.cfg` is used.
+
+When retrieving lists of customer entitlements from Cloud Channel API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many customer entitlements to retrieve in each API call; default is 100, the maximum.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display Channel Offers
+```
+gam show channeloffers
+        [resellerid <ResellerID>] [filter <String>] [language <LanguageCode>]
+        [fields <ChannelOfferFieldList>]
+        [maxresults <Number>]
+        [formatjson]
+```
+If `resellerId <ResellerID>` is omitted, the `reseller_id` value from `gam.cfg` is used.
+
+Cloud Channel API documentation for `filter <String>`:
+```
+The expression to filter results by name (name of the Offer), sku.name (name of the SKU), or sku.product.name (name of the Product).
+* Example 1: sku.product.name=products/p1 AND sku.name!=products/p1/skus/s1
+* Example 2: name=accounts/a1/offers/o1
+```
+
+When retrieving lists of offers from Cloud Channel API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many offers to retrieve in each API call; default is 1000, the maximum.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print channeloffers [todrive <ToDriveAttribute>*]
+        [resellerid <ResellerID>] [filter <String>] [language <LanguageCode>]
+        [fields <ChannelOfferFieldList>]
+        [maxresults <Number>]
+        [formatjson [quotechar <Character>]]
+```
+If `resellerId <ResellerID>` is omitted, the `reseller_id` value from `gam.cfg` is used.
+
+Cloud Channel API documentation for `filter <String>`:
+```
+The expression to filter results by name (name of the Offer), sku.name (name of the SKU), or sku.product.name (name of the Product).
+* Example 1: sku.product.name=products/p1 AND sku.name!=products/p1/skus/s1
+* Example 2: name=accounts/a1/offers/o1
+```
+When retrieving lists of offers from Cloud Channel API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many offers to retrieve in each API call; default is 1000, the maximum.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display Channel Products
+```
+gam show channelproducts
+        [resellerid <ResellerID>] [language <LanguageCode>]
+        [fields <ChannelProductFieldList>]
+        [maxresults <Number>]
+        [formatjson]
+```
+If `resellerId <ResellerID>` is omitted, the `reseller_id` value from `gam.cfg` is used.
+
+When retrieving lists of products from Cloud Channel API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many products to retrieve in each API call; default is 1000, the maximum.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print channelproducts [todrive <ToDriveAttribute>*]
+        [resellerid <ResellerID>] [language <LanguageCode>]
+        [fields <ChannelProductFieldList>]
+        [maxresults <Number>]
+        [formatjson [quotechar <Character>]]
+```
+If `resellerId <ResellerID>` is omitted, the `reseller_id` value from `gam.cfg` is used.
+
+When retrieving lists of products from Cloud Channel API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many products to retrieve in each API call; default is 1000, the maximum.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display Channel SKUs
+```
+gam show channelskus
+        [resellerid <ResellerID>] [language <LanguageCode>] [productid <ProductID>]
+        [fields <ChannelSKUFieldList>]
+        [maxresults <Number>]
+        [formatjson]
+```
+If `resellerId <ResellerID>` is omitted, the `reseller_id` value from `gam.cfg` is used.
+
+If `productid <ProductID>` is omitted, SKUs for all products are displayed.
+
+When retrieving lists of SKUs from Cloud Channel API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many SKUs to retrieve in each API call; default is 1000, the maximum.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print channelskus [todrive <ToDriveAttribute>*]
+        [resellerid <ResellerID>] [language <LanguageCode>] [productid <ProductID>]
+        [fields <ChannelSKUFieldList>]
+        [maxresults <Number>]
+        [formatjson [quotechar <Character>]]
+```
+If `resellerId <ResellerID>` is omitted, the `reseller_id` value from `gam.cfg` is used.
+
+If `productid <ProductID>` is omitted, SKUs for all products are displayed.
+
+When retrieving lists of SKUs from Cloud Channel API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many SKUs to retrieve in each API call; default is 1000, the maximum.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Cloud-Identity-Devices.md

@@ -0,0 +1,368 @@
+# Cloud Identity Devices
+- [API documentation](#api-documentation)
+- [Query documentation](#query-documentation)
+- [Definitions](#definitions)
+- [Create a company device](#create-a-company-device)
+- [Delete devices](#delete-devices)
+- [Wipe devices](#wipe-devices)
+- [Perform device actions](#perform-device-actions)
+- [Synchronize devices](#synchronize-devices)
+- [Display devices](#display-devices)
+- [Print devices](#print-devices)
+- [Display device counts](#display-device-counts)
+- [Approve or block device users](#approve-or-block-device-users)
+- [Delete device users](#delete-device-users)
+- [Wipe device users](#wipe-device-users)
+- [Perform device user actions](#perform-device-user-actions)
+- [Display device users](#display-device-users)
+- [Display device user counts](#display-device-user-counts)
+- [Print device users](#print-device-users)
+- [Display device user client state](#display-device-user-client-state)
+- [Update device user client state](#update-device-user-client-state)
+
+## API documentation
+* https://cloud.google.com/identity/docs/reference/rest/v1/devices
+* https://cloud.google.com/identity/docs/reference/rest/v1/devices.deviceUsers
+* https://cloud.google.com/identity/docs/reference/rest/v1/devices.deviceUsers.clientStates
+* https://cloud.google.com/endpoint-verification/docs/overview
+
+## Query documentation
+* https://developers.google.com/admin-sdk/directory/v1/search-operators
+* https://support.google.com/a/answer/7549103
+
+## Definitions
+```
+<AssetTag> ::= <String>
+<AssetTagList> ::= "<AssetTag>(,<AssetTag>)*"
+<QueryDevice> ::= <String>
+        See: https://support.google.com/a/answer/7549103
+<QueryDeviceList> ::= "<QueryDevice>(,<QueryDevice>)*"
+<DeviceID> ::= devices/<String>
+<DeviceIDList> ::= "<DeviceID>(,<DeviceID>)*"
+<DeviceEntity> ::=
+        <DeviceIDList> | devicesn <String> |
+        (query:<QueryDevice>)|(query <QueryDevice>)
+<DeviceType> ::= android|chrome_os|google_sync|linux|mac_os|windows
+<DeviceUserID> ::= devices/<String>/deviceUsers/<String>
+<DeviceUserEntity> ::=
+        <DeviceUserIDList> |
+        (query:<QueryDevice>)|(query <QueryDevice>)
+
+<DeviceFieldName> ::=
+        androidspecificattributes|
+        assettag|
+        basebandversion|
+        bootloaderversion|
+        brand|
+        buildnumber|
+        compromisedstate|
+        createtime|
+        devicetype|
+        enableddeveloperoptions|
+        enabledusbdebugging|
+        endpointverificationspecificattributes|
+        encryptionstate|
+        imei|
+        kernelversion|
+        lastsynctime|
+        managementstate|
+        manufacturer|
+        meid|
+        model|
+        name|
+        networkoperator|
+        osversion|
+        otheraccounts|
+        ownertype|
+        releaseversion|
+        securitypatchtime|
+        serialnumber|
+        wifimacaddresses
+<DeviceFieldNameList> ::= "<DeviceFieldName>(,<DeviceFieldName>)*"
+
+<DeviceAction> ::=
+        cancelwipe|
+        wipe
+
+<DeviceUserFieldName> ::=
+        compromisedstate|
+        createtime|
+        firstsynctime|
+        languagecode|
+        lastsynctime|
+        managementstate|
+        name|
+        passwordstate|
+        useragent|
+        useremail
+<DeviceUserFieldNameList> ::= "<DeviceUserFieldName>(,<DeviceUserFieldName>)*"
+
+<DeviceOrderbyFieldName> ::= 
+        createtime|devicetype|lastsynctime|model|osversion|serialnumber
+
+<DeviceUserAction> ::=
+        approve|
+        block|
+        cancelwipe|
+        wipe
+
+```
+## Create a company device
+Adds a new device to the Google company-owned inventory. Once a user is assigned and enrolled on the device the device will be considered company-owned for management purposes.
+The device will also register as company-owned with Google services like [Context-Aware Access (CAA)](https://support.google.com/a/answer/9275380).
+```
+gam create device serialnumber <String> devicetype <DeviceType> [assettag <String>]
+```
+Arguments `serialnumber <String>` and `devicetype <DeviceType>` are required; you can optionally specify `assettag <String>`.
+
+## Delete devices
+Delete a device from appearing in the Admin console, stop syncing for the device user.
+No user data should be removed.
+```
+gam delete device <DeviceEntity> [doit]
+```
+If `<DeviceEntity>` uses a query, the `doit` option must be used to enable execution.
+
+## Wipe devices
+Wiping a device performs a factory reset, all device data is removed.
+```
+gam cancelwipe device <DeviceEntity> [doit]
+gam wipe device <DeviceEntity> [removeresetlock] [doit]
+```
+If `<DeviceEntity>` uses a query, the `doit` option must be used to enable execution.
+
+Specifying `removeresetlock` will remove the account lock on the Android or iOS device.
+This lock is enabled by default and requires the existing device user to log in after the wipe in order to unlock the device.
+* See: https://support.google.com/android/answer/9459346
+
+## Perform device actions
+This is an alternative form of the above commands
+```
+gam update device <DeviceEntity> action <DeviceAction> [removeresetlock] [doit]
+```
+If `<DeviceEntity>` uses a query, the `doit` option must be used to enable execution.
+
+Specifying `removeresetlock` when `<DeviceAction>` is `wipe` will remove the account lock on the Android or iOS device.
+This lock is enabled by default and requires the existing device user to log in after the wipe in order to unlock the device.
+* See: https://support.google.com/android/answer/9459346
+
+## Synchronize devices
+This command generates a list of your current company devices, either a complete list
+or a subset based on a query. A CSV file is read to generate another list of devices.
+
+At a minimum, two values are required for devices in the CSV file list; a device type and a serial number.
+For the device type, you can either specify a static device type or specify the column in the CSV file that contains a device type.
+* `static_devicetype <DeviceType>` - A fixed device type
+* `devicetype_column <String>` - The name of the column containing device types; if not specified, `deviceType` is used
+
+For the serial number, you must specify the column in the CSV file that contains a serial number.
+* `serialnumber_column <String>` - The name of the column containing serial numbers; if not specified, `serialNumber` is used
+
+You can optionally specify the column in the CSV file that contains an asset tag.
+* `assettag_column <String>` - The name of the column containing asset tags; the typical value is `assetTag`
+
+These two/three columns are used to match current company devices against the CSV file devices.
+* Devices in the CSV device list will be created if they are not the the current company device list.
+* Devices in the current company device list that are not in the CSV device list will have an optional operation performed on them.
+  * `unassigned_missing_action delete|wipe|none` - Perform this operation if the company device has never been assigned; default action is `delete`
+  * `assigned_missing_action delete|wipe|none` - Perform this operation if the company device has been assigned; default action is `none`
+
+If `preview` is specified, the operations that would be performed are previewed but are not performed; use this to test.
+```
+gam sync devices
+        [(query <QueryDevice>)|(queries <QueryDeviceList>) (querytime<String> <Time>)*]
+        csvfile <FileName>
+        (devicetype_column <String>)|(static_devicetype <DeviceType>)
+        (serialnumber_column <String>)
+        [assettag_column <String>]
+        [unassigned_missing_action delete|wipe|none]
+        [assigned_missing_action delete|wipe|none]
+        [preview]
+```
+
+## Display devices
+```
+gam info device <DeviceEntity>
+        <DeviceFieldName>* [fields <DeviceFieldNameList>] [userfields <DeviceUserFieldNameList>]
+        [nodeviceusers]
+        [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+## Print devices
+```
+gam print devices [todrive <ToDriveAttribute>*]
+        [(query <QueryDevice>)|(queries <QueryDeviceList>) (querytime<String> <Time>)*]
+        <DeviceFieldName>* [fields <DeviceFieldNameList>] [userfields <DeviceUserFieldNameList>]
+        [orderby <DeviceOrderByFieldName> [ascending|descending]]
+        [all|company|personal|nocompanydevices|nopersonaldevices]
+        [nodeviceusers]
+        [formatjson [quotechar <Character>]]
+```
+By default, all devices are displayed; use the query options to limit the display.
+
+To AND query terms, put all of your terms in one query:
+```
+gam print devices query "manufacturer:Meizu os:Android 7.0.0"
+```
+To OR query terms, put the terms im multiple queries:
+```
+gam print devices queries "'model:iPhone 6','model:samsung'"
+```
+Select the view of devices to display:
+* `all` - Company and personal devices; this is the default
+* `company|nopersonaldevices` - Company devices
+* `personal|nocompanydevices` - Personal devices
+
+By default, Gam makes additional API calls to display the device users for the devices;
+use `nodeviceuser` to suppress making the additional calls.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display device counts
+Display the number of devices.
+```
+gam print devices
+        [(query <QueryDevice>)|(queries <QueryDeviceList>) (querytime<String> <Time>)*]
+        [all|company|personal|nocompanydevices|nopersonaldevices]
+        showitemcountonly
+```
+Example
+```
+$ gam print devices queries "'model:Mac'" showitemcountonly
+Getting all Devices that match query (model:Mac), may take some time on a large Google Workspace Account...
+Got 100 Devices...
+Got 200 Devices...
+Got 300 Devices...
+...
+Got 900 Devices...
+Got 995 Devices...
+Got 995 Devices...
+995
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print devices queries "'model:Mac'" showitemcountonly)
+Windows PowerShell
+count = & gam print devices queries "'model:Mac'" showitemcountonly
+```
+
+## Approve or block device users
+Approve or block user profiles on a device.
+```
+gam approve deviceuser <DeviceUserEntity> [doit]
+gam block deviceuser <DeviceUserEntity> [doit]
+```
+If `<DeviceUserEntity>` uses a query, the `doit` option must be used to enable execution.
+
+## Delete device users
+Delete a device user from appearing in the Admin console, stop syncing for the device user.
+No user data should be removed.
+```
+gam delete deviceuser <DeviceUserEntity> [doit]
+```
+If `<DeviceUserEntity>` uses a query, the `doit` option must be used to enable execution.
+
+## Wipe device users
+Wipe a device user profile from a device.
+In the case of Android for Work, the work profile will be removed but the personal profile left alone.
+```
+gam wipe deviceuser <DeviceUserEntity> [doit]
+gam cancelwipe deviceuser <DeviceUserEntity> [doit]
+```
+If `<DeviceUserEntity>` uses a query, the `doit` option must be used to enable execution.
+
+## Perform device user actions
+This is an alternative form of the above commands.
+```
+gam update deviceuser <DeviceUserEntity> action <DeviceUserAction> [doit]
+```
+If `<DeviceUserEntity>` uses a query, the `doit` option must be used to enable execution.
+
+## Display device users
+```
+gam info deviceuser <DeviceUserEntity>
+        <DeviceUserFieldName>* [fields <DeviceUserFieldNameList>]
+        [formatjson]
+```
+## Print device users
+```
+gam print deviceusers [todrive <ToDriveAttribute>*]
+        [select <DeviceID>]
+        [(query <QueryDevice>)|(queries <QueryDeviceList>) (querytime<String> <Time>)*]
+        <DeviceUserFieldName>* [fields <DeviceUserFieldNameList>]
+        [orderby <DeviceOrderByFieldName> [ascending|descending]]
+        [formatjson [quotechar <Character>]]
+```
+By default, Gam displays device users for all devices;
+* `select <DeviceID>` - Display users for a specific device
+* `(query <QueryDevice>)|(queries <QueryDeviceList>)` - Display users that match queries.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display device user counts
+Display the number of device users.
+```
+gam print deviceusers [todrive <ToDriveAttribute>*]
+        [select <DeviceID>]
+        [(query <QueryDevice>)|(queries <QueryDeviceList>) (querytime<String> <Time>)*]
+        showitemcountonly
+```
+Example
+```
+$ gam print deviceusers queries "'model:Mac'" showitemcountonly
+Getting all Device Users that match query (model:Mac), may take some time on a large Google Workspace Account...
+Got 20 Device Users...
+Got 40 Device Users...
+Got 60 Device Users...
+...
+Got 980 Device Users...
+Got 995 Device Users...
+Got 995 Device Users...
+995
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print deviceusers queries "'model:Mac'" showitemcountonly)
+Windows PowerShell
+count = & gam print deviceusers queries "'model:Mac'" showitemcountonly
+```
+
+
+## Display device user client state
+```
+gam info deviceuserstate <DeviceUserEntity> [clientid <String>] 
+```
+
+## Update device user client state
+The API that supports this command is in beta mode. In particular, setting `assettags` and `customvalues`
+works if you set the values once; each additional time you set values they are added to the existing values
+and they is no way at the moment to clear values.
+```
+gam update deviceuserstate <DeviceUserEntity> [clientid <String>] 
+        [customid <String>] [assettags clear|<AssetTagList>]
+        [compliantstate|compliancestate compliant|noncompliant] [managedstate clear|managed|unmanaged]
+        [healthscore very_poor|poor|neutral|good|very_good] [scorereason clear|<String>]
+        (customvalue (bool|boolean <Boolean>)|(number <Integer>)|(string <String>))*
+```

docs/Cloud-Identity-Groups-Membership.md

@@ -0,0 +1,486 @@
+# Cloud Identity Groups - Membership
+- [API documentation](#api-documentation)
+- [Query documentation](#query-documentation)
+- [Cloud Identity Group Documentation](#cloud-identity-group-documentation)
+- [Security Group Documentation](#security-group-documentation)
+- [Python Regular Expressions](Python-Regular-Expressions) Match function
+- [Definitions](#definitions)
+- [Notes](#Notes)
+- [Collections of Users](#collections-of-users)
+- [Add members to a group](#add-members-to-a-group)
+- [Delete members from a group](#delete-members-from-a-group)
+- [Synchronize members in a group](#synchronize-members-in-a-group)
+- [Delete members from a group by role](#delete-members-from-a-group-by-role)
+- [Update member roles and expiration time](#update-member-roles-and-expiration-time)
+- [Bulk membership changes](#bulk-membership-changes)
+- [Display user group member options](#display-user-group-member-options)
+- [Display group membership in CSV format](#display-group-membership-in-csv-format)
+- [Display group membership in hierarchical format](#display-group-membership-in-hierarchical-format)
+
+## API documentation
+* https://cloud.google.com/identity/docs/groups
+* https://cloud.google.com/identity/docs/reference/rest/v1/groups
+* https://cloud.google.com/identity/docs/reference/rest/v1/groups.memberships
+
+## Query documentation
+* https://cloud.google.com/identity/docs/reference/rest/v1/groups#dynamicgroupquery
+
+## Cloud Identity Group Documentation
+* https://gsuiteupdates.googleblog.com/2020/08/new-api-cloud-identity-groups-google.html
+
+## Security Group Documentation
+* https://gsuiteupdates.googleblog.com/2020/09/security-groups-beta.html
+
+## Notes
+
+In the Admin Directory API a group has the following characteristics:
+* `id` - The unique ID of a group
+* `email` - The group's email address
+* `name` - The group's display name
+
+In the Cloud Indentity Groups API a group has the following characteristics:
+* `name` - The unique ID of a group
+* `groupKey.id` - The group's email address
+* `displayName` - The group's display name
+
+The Admin Directory API group characteristic names will be used.
+
+Dynamic Groups require Cloud Identity Premium accounts.
+
+* https://cloud.google.com/identity/docs/how-to/create-dynamic-groups
+
+The `cimember <UserItem>` option of `gam print|show cigroup-members` requires a Google Workspace Enterprise Standard, Enterprise Plus, and Enterprise for Education;
+and Cloud Identity Premium accounts. Unfortunately, even if you have the required account, the API call that supports the query doesn't work.
+
+* https://cloud.google.com/identity/docs/reference/rest/v1/groups.memberships/searchTransitiveGroups
+
+## Definitions
+```
+<DomainName> ::= <String>(.<String>)+
+<EmailAddress> ::= <String>@<DomainName>
+<UniqueID> ::= id:<String>
+<GroupItem> ::= <EmailAddress>|<UniqueID>|groups/<String>
+<GroupList> ::= "<GroupItem>(,<GroupItem>)*"
+<GroupEntity> ::=
+        <GroupList> | <FileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<GroupRole> ::= owner|manager|member
+<GroupRoleList> ::= "<GroupRole>(,<GroupRole>)*"
+<CIGroupType> ::= customer|group|other|serviceaccount|user
+<CIGroupTypeList> ::= "<CIGroupType>(,<CIGroupType>)*"
+
+<CIGroupMembersFieldName> ::=
+        createtime
+        expiretime|
+        memberkey|
+        name|
+        preferredmemberkey|
+        role|
+        type|
+        updatetime|
+        useremail
+<CIGroupMembersFieldNameList> ::= "<CIGroupMembersFieldName>(,<CIGroupMembersFieldName>)*"
+```
+
+## Collections of Users
+Group membership commands involve specifying collections of users;
+for `<UserTypeEntity>`, see: [Collections of Users](Collections-of-Users)
+
+## Add members to a group
+```
+gam update cigroups <GroupEntity> create|add [<GroupRole>]
+        [usersonly|groupsonly]
+        [notsuspended|suspended] [notarchived|archived]
+        [expire|expires <Time>] [preview] [actioncsv]
+        <UserTypeEntity>
+```
+When `<UserTypeEntity>` specifies a group or groups:
+* `usersonly` - Only the user members from the specified groups are added
+* `groupsonly` - Only the group members from the specified groups are added
+
+By default, when adding members from organization units, all users, whether suspended or not, are included.
+* `notsuspended` - Do not include suspended users, this is common
+* `suspended` - Only include suspended users, this is not common but allows creating groups that allow easy identification of suspended users
+
+By default, when adding members from groups, all users, whether suspended/archived or not, are included.
+* `notsuspended` - Do not include suspended users, this is common
+* `suspended` - Only include suspended users, this is not common but allows creating groups that allow easy identification of suspended users
+* `notarchived` - Do not include archived users
+* `archived` - Only include archived users, this is not common but allows creating groups that allow easy identification of archived users
+* `notsuspended notarchived` - Do not include suspended and archived users
+* `suspended archived` - Include only suspended or archived users
+* `notsuspended archived` - Only include archived users, this is not common but allows creating groups that allow easy identification of archived users
+* `suspended notarchived` - Only include suspended users, this is not common but allows creating groups that allow easy identification of suspended users
+
+If `preview` is specified, the changes will be previewed but not executed.
+
+If `actioncsv` is specified, a CSV file with columns `group,email,role,action,message` is generated
+that shows the actions performed when updating the group.
+
+### `actioncsv` Example
+Using `actioncsv` produces a CSV file showing the actions taken.
+```
+$ gam redirect csv AddUpdates.csv update cigroup testgroup add members actioncsv users testuser2,testuser3
+Group: testgroup@domain.com, Add 2 Members
+  Group: testgroup@domain.com, Member: testuser2@domain.com, Added: Role: MEMBER (1/2)
+  Group: testgroup@domain.com, Member: testuser3@domain.com, Add Failed: Member already exists. (2/2)
+$ more AddUpdates.csv 
+group,email,role,action,message
+testgroup@domain.com,testuser2@domain.com,MEMBER,Added,Success
+testgroup@domain.com,testuser3@domain.com,MEMBER,Add Failed,Member already exists.
+```
+
+## Delete members from a group
+```
+gam update cigroups <GroupEntity> delete|remove [<GroupRole>]
+        [usersonly|groupsonly]
+        [notsuspended|suspended] [notarchived|archived]
+        [preview] [actioncsv]
+        <UserTypeEntity>
+```
+`<GroupRole>` is ignored, deletions take place regardless of role.
+
+When `<UserTypeEntity>` specifies a group or groups:
+* `usersonly` - Only the user members from the specified groups are deleted
+* `groupsonly` - Only the group members from the specified groups are deleted
+
+By default, when deleting members from organization units, all users, whether suspended or not, are included.
+* `notsuspended` - Do not include suspended users, this is common
+* `suspended` - Only include suspended users, this is not common but allows creating groups that allow easy identification of suspended users
+
+By default, when deleting members from groups, all users, whether suspended/archived or not, are included.
+* `notsuspended` - Do not include suspended users, this is common
+* `suspended` - Only include suspended users, this is not common but allows creating groups that allow easy identification of suspended users
+* `notarchived` - Do not include archived users
+* `archived` - Only include archived users, this is not common but allows creating groups that allow easy identification of archived users
+* `notsuspended notarchived` - Do not include suspended and archived users
+* `suspended archived` - Include only suspended or archived users
+* `notsuspended archived` - Only include archived users, this is not common but allows creating groups that allow easy identification of archived users
+* `suspended notarchived` - Only include suspended users, this is not common but allows creating groups that allow easy identification of suspended users
+
+If `preview` is specified, the changes will be previewed but not executed.
+
+If `actioncsv` is specified, a CSV file with columns `group,email,role,action,message` is generated
+that shows the actions performed when updating the group.
+
+### `actioncsv` Example
+Using `actioncsv` produces a CSV file showing the actions taken.
+```
+$ gam redirect csv DeleteUpdates.csv update cigroup testgroup delete members actioncsv users testuser2,testuser4
+Group: testgroup@domain.com, Remove 2 Members
+  Group: testgroup@domain.com, Member: testuser2@domain.com, Removed: Role: MEMBER (1/2)
+  Group: testgroup@domain.com, Member: testuser4@domain.com, Remove Failed: Does not exist (2/2)
+$ more DeleteUpdates.csv 
+group,email,role,action,message
+testgroup@domain.com,testuser2@domain.com,MEMBER,Removed,Success
+testgroup@domain.com,testuser4@domain.com,MEMBER,Remove Failed,Does not exist
+```
+
+## Synchronize members in a group
+A synchronize operation gets the current membership for a group and does adds and deletes as necessary to make it match `<UserTypeEntity>`.
+This is done by specific role except for a special case where role is ignored.
+```
+gam update cigroups <GroupEntity> sync [<GroupRole>|ignorerole]
+        [usersonly|groupsonly] [addonly|removeonly]
+        [notsuspended|suspended] [notarchived|archived]
+        [expire|expires <Time>] [preview] [actioncsv]
+        <UserTypeEntity>
+```
+If `ignorerole` is specified, GAM removes members regardless of role and adds new members with role MEMBER.
+This is a special purpose option, use with caution and ensure that `<UserTypeEntity>` specifies the full desired membership list of all roles.
+
+If neither `<GroupRole>` nor `ignorerole` is specified, `member` is assumed.
+
+When `<UserTypeEntity>` specifies a group or groups:
+* `usersonly` - Only the user members from the specified groups are added/deleted
+* `groupsonly` - Only the group members from the specified groups are added/deleted
+
+By default, when synchronizing members from organization units, all users, whether suspended or not, are included.
+* `notsuspended` - Do not include suspended users, this is common
+* `suspended` - Only include suspended users, this is not common but allows creating groups that allow easy identification of suspended users
+
+By default, when synchronizing members from groups, all users, whether suspended/archived or not, are included.
+* `notsuspended` - Do not include suspended users, this is common
+* `suspended` - Only include suspended users, this is not common but allows creating groups that allow easy identification of suspended users
+* `notarchived` - Do not include archived users
+* `archived` - Only include archived users, this is not common but allows creating groups that allow easy identification of archived users
+* `notsuspended notarchived` - Do not include suspended and archived users
+* `suspended archived` - Include only suspended or archived users
+* `notsuspended archived` - Only include archived users, this is not common but allows creating groups that allow easy identification of archived users
+* `suspended notarchived` - Only include suspended users, this is not common but allows creating groups that allow easy identification of suspended users
+
+Default:
+* members in `<UserTypeEntity>` that are not in the current membership will be added
+* members in the current membership that are not in `<UserTypeEntity>` will deleted
+
+When the `addonly` option is specified:
+* members in `<UserTypeEntity>` that are not in the current membership will be added
+* members in the current membership that are not in `<UserTypeEntity>` will not be deleted
+
+When the `removeonly` option is specified:
+* members in `<UserTypeEntity>` that are not in the current membership will not be added
+* members in the current membership that are not in `<UserTypeEntity>` will be deleted
+
+If `preview` is specified, the changes will be previewed but not executed.
+
+If `actioncsv` is specified, a CSV file with columns `group,email,role,action,message` is generated
+that shows the actions performed when updating the group.
+
+### Examples using CSV file and Google sheets:
+* https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Users#examples-using-csv-files-and-google-sheets-to-update-the-membership-of-a-group
+
+### Example
+Assume that at your school there is a group for each grade level and the members come from an OU; here is a sample CSV file GradeOU.csv
+```
+Grade,OU
+seniors@domain.org,/Students/ClassOf2018
+juniors@domain.org,/Students/ClassOf2019
+...
+```
+This allows you to do: `gam csv GradeOU.csv gam update cigroup ~Grade sync members ou ~OU`
+But suppose that at each grade level there are additional group members that are groups of faculty/staff; e.g., senioradvisors@domain.org.
+In this scenario, you can't do the `update cigroup sync` command as the members that are groups will be deleted; the `usersonly` option allows
+the `update cigroup sync` command to work: `gam csv GradeOU.csv gam update cigroup ~Grade sync members usersonly ou ~OU`
+The users from the OU are matched against the user members of the group and adds/deletes are done as necessary to synchronize them;
+the group members of the group are unaffected.
+
+### `actioncsv` Example
+Using `actioncsv` produces a CSV file showing the actions taken.
+```
+$ gam redirect csv SyncUpdates.csv update cigroup testgroup sync members actioncsv users testuser1,testuser3,testuser4
+Getting all Members for testgroup@domain.com, may take some time on a large Group...
+Got 3 Members for testgroup@domain.com...
+Group: testgroup@domain.com, Remove 1 Member
+  Group: testgroup@domain.com, Member: testuser2@domain.com, Removed: Role: MEMBER
+Group: testgroup@domain.com, Add 1 Member
+  Group: testgroup@domain.com, Member: testuser4@domain.com, Added: Role: MEMBER
+$ more SyncUpdates.csv 
+group,email,role,action,message
+testgroup@domain.com,testuser2@domain.com,MEMBER,Removed,Success
+testgroup@domain.com,testuser4@domain.com,MEMBER,Added,Success
+```
+## Delete members from a group by role
+```
+gam update cigroups <GroupEntity> clear [member] [manager] [owner]
+        [usersonly|groupsonly]
+        [emailclearpattern|emailretainpattern <RegularExpression>]
+        [preview] [actioncsv]
+```
+If none of `member`, `manager`, or `owner` are specified, `member` is assumed.
+
+By default, when clearing members from a group, all members, whether users or groups, are included.
+* `usersonly` - Clear only the user members
+* `groupsonly` - Clear only the group members
+
+Members that have met the above qualifications to be cleared can be further qualifed by their email address.
+* `emailclearpattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be cleared; others will be retained
+* `emailretainpattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be retained; others will be cleared
+
+If `preview` is specified, the deletes will be previewed but not executed.
+
+If `actioncsv` is specified, a CSV file with columns `group,email,role,action,message` is generated
+that shows the actions performed when updating the group.
+
+## Update member roles and expiration time
+```
+gam update cigroups <GroupEntity> update [<GroupRole>]
+        [usersonly|groupsonly]
+        [notsuspended|suspended] [notarchived|archived]
+        [expire|expires <Time>] [preview] [actioncsv]
+        <UserTypeEntity>
+```
+There are two items that can be updated: role and expiration time. If neither option is specified,
+the users are updated to members; this is the behavior from previous versions. Otherwise,
+only the specified items are updated.
+
+When `<UserTypeEntity>` specifies a group or groups:
+* `usersonly` - Only the user members from the specified groups are added
+* `groupsonly` - Only the group members from the specified groups are added
+
+By default, when updating members from organization units, all users, whether suspended or not, are included.
+* `notsuspended` - Do not include suspended users
+* `suspended` - Only include suspended users
+
+By default, when updating members from groups, all users, whether suspended/archived or not, are included.
+* `notsuspended` - Do not include suspended users
+* `suspended` - Only include suspended users
+* `notarchived` - Do not include archived users
+* `archived` - Only include archived users
+* `notsuspended notarchived` - Do not include suspended and archived users
+* `suspended archived` - Include only suspended or archived users
+* `notsuspended archived` - Only include archived users
+* `suspended notarchived` - Only include suspended users
+
+If `preview` is specified, the changes will be previewed but not executed.
+
+If `actioncsv` is specified, a CSV file with columns `group,email,role,action,message` is generated
+that shows the actions performed when updating the group.
+
+## Bulk membership changes
+Suppose you have a CSV file (GroupMembers.csv) with headers: group,role,email
+
+Each row contains a group email address, member role (OWNER, MEMBER, MANAGER) and a member email address.
+
+The following command will synchronize the membership for all groups and roles.
+```
+gam redirect stdout ./MemberUpdates.txt redirect stderr stdout update cigroup csvkmd GroupMembers.csv keyfield group subkeyfield role datafield email sync csvdata email
+```
+You can also do `create|add`, `delete` and `update` in this manner.
+
+If you want to update a specific role, you can do one of the following.
+```
+gam redirect stdout ./MemberUpdates.txt redirect stderr stdout update cigroup csvkmd ./GroupMembers.csv keyfield group matchfield role MEMBER datafield email sync member csvdata email
+gam redirect stdout ./ManagerUpdates.txt redirect stderr stdout update cigroup csvkmd ./GroupMembers.csv keyfield group matchfield role MANAGER datafield email sync manager csvdata email
+gam redirect stdout ./OwnerUpdates.txt redirect stderr stdout update cigroup csvkmd ./GroupMembers.csv keyfield group matchfield role OWNER datafield email sync owner csvdata email
+```
+
+## Display user group member options
+
+Display user's group membership information.
+```
+gam <UserTypeEntity> info cimember <GroupEntity>
+gam info cimember <UserTypeEntity> <GroupEntity>
+```
+
+## Display group membership in CSV format
+```
+gam print cigroup-members [todrive <ToDriveAttribute>*]
+        [(cimember|showownedby <UserItem>)|(cigroup <GroupItem>)|(select <GroupEntity>)]
+        [emailmatchpattern [not] <RegularExpression>] [namematchpattern [not] <RegularExpression>]
+        [descriptionmatchpattern [not] <RegularExpression>]
+        [roles <GroupRoleList>] [members] [managers] [owners]
+        [types <CIGroupTypeList>]
+        <CIGroupMembersFieldName>* [fields <CIGroupMembersFieldNameList>]
+        [(recursive [noduplicates])||includederivedmembership] [nogroupeemail]
+        [memberemaildisplaypattern|memberemailskippattern <RegularExpression>]
+```
+By default, the group membership of all groups in the account are displayed, these options allow selection of subsets of groups:
+* `cimember <UserItem>` - Limit display to groups that contain `<UserItem>` as a member
+* `showownedby <UserItem>` - Limit display to groups owned by `<UserItem>`
+* `cigroup <GroupItem>` - Limit display to the single group `<GroupItem>`
+* `select <GroupEntity>` - Limit display to the groups specified in `<GroupEntity>`
+
+These options further limit the list of groups selected above:
+* `emailmatchpattern <RegularExpression>` - Limit display to groups whose email address matches `<RegularExpression>`
+* `emailmatchpattern not <RegularExpression>` - Limit display to groups whose email address does not match `<RegularExpression>`
+* `namematchpattern <RegularExpression>` - Limit display to groups whose name matches `<RegularExpression>`
+* `namematchpattern not <RegularExpression>` - Limit display to groups whose name does not match `<RegularExpression>`
+* `descriptionmatchpattern <RegularExpression>` - Limit display to groups whose description matches `<RegularExpression>`
+* `descriptionmatchpattern not <RegularExpression>` - Limit display to groups whose description does not match `<RegularExpression>`
+
+By default, all members, managers and owners in the group are displayed; these options modify that behavior:
+* `roles <GroupRoleList>` - Display specified roles
+* `members` - Display members
+* `managers` - Display managers
+* `owners` - Display owners
+
+By default, all types of members (customer, group, serviceaccoun, user) in the group are displayed; when `recursive` is specified,
+the default is to only display type user members. This option modifies those behaviors:
+* `types <CIGroupTypeList>` - Display specified types
+
+Members that have met the above qualifications to be displayed can be further qualifed by their email address.
+* `memberemaildisplaypattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be displayed; others will not be displayed
+* `memberemailskippattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will not be displayed; others will be displayed
+
+By default, the ID, role, email address, type, createTime, updateTime and expireTime of each member is displayed along with the group email address;
+these options specify which fields to display:
+* `<CIGroupMembersFieldName>*` - Individual field names
+* `fields <CIGroupMembersFieldNameList>` - A comma separated list of field names
+
+By default, the group email address is always shown, you can suppress it with the `nogroupemail` option.
+
+By default, members that are groups are displayed as a single entry of type GROUP; this option recursively expands group members to display their user members.
+* `recursive` - Recursively expand group members
+
+The `recursive` option does not expand or display members of type CUSTOMER.
+
+The `recursive` option adds two columns, level and subgroup, to the output:
+* `level` - At what level of the expansion does the user appear; level 0 is the top level
+* `subgroup` - The group that contained the user
+
+Displaying membership of multiple groups or recursive expansion may result in multiple instances of the same user being displayed; these multiple instances can be reduced to one entry.
+* `noduplicates` - Reduce multiple instances of the same user to the first instance
+
+The `includederivedmembership` option is an alternative to `recursive`; it causes the API to expand type GROUP
+members to display their constituent members. The role displayed for a user is the highest role it
+has in any constituent group, it is not necessarily its role in the top group.
+
+The options `recursive noduplicates` and `includederivedmembership types user` return the same list of users.
+The `includederivedmembership` option makes less API calls but doesn't show level and subgroup information.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display group membership in hierarchical format
+```
+gam show cigroup-members
+        [(cimember|showownedby <UserItem>)|(cigroup <GroupItem>)|(select <GroupEntity>)]
+        [emailmatchpattern [not] <RegularExpression>] [namematchpattern [not] <RegularExpression>]
+        [descriptionmatchpattern [not] <RegularExpression>]
+        [roles <GroupRoleList>] [members] [managers] [owners] [depth <Number>]
+        [types <CIGroupTypeList>]
+        [memberemaildisplaypattern|memberemailskippattern <RegularExpression>]
+        [includederivedmembership]
+        [formatjson [quotechar <Character>]]
+```
+By default, the group membership of all groups in the account are displayed, these options allow selection of subsets of groups:
+* `cimember <UserItem>` - Limit display to groups that contain `<UserItem>` as a member
+* `showownedby <UserItem>` - Limit display to groups owned by `<UserItem>`
+* `cigroup <GroupItem>` - Limit display to the single group `<GroupItem>`
+* `select <GroupEntity>` - Limit display to the groups specified in `<GroupEntity>`
+
+These options further limit the list of groups selected above:
+* `emailmatchpattern <RegularExpression>` - Limit display to groups whose email address matches `<RegularExpression>`
+* `emailmatchpattern not <RegularExpression>` - Limit display to groups whose email address does not match `<RegularExpression>`
+* `namematchpattern <RegularExpression>` - Limit display to groups whose name matches `<RegularExpression>`
+* `namematchpattern not <RegularExpression>` - Limit display to groups whose name does not match `<RegularExpression>`
+* `descriptionmatchpattern <RegularExpression>` - Limit display to groups whose description matches `<RegularExpression>`
+* `descriptionmatchpattern not <RegularExpression>` - Limit display to groups whose description does not match `<RegularExpression>`
+
+By default, all members, managers and owners in the group are displayed; these options modify that behavior:
+* `roles <GroupRoleList>` - Display specified roles
+* `members` - Display members
+* `managers` - Display managers
+* `owners` - Display owners
+
+By default, all types of members (customer, group, serviceaccount, user) in the group are displayed; this option modifies that behavior:
+* `types <CIGroupTypeList>` - Display specified types
+
+Members that have met the above qualifications to be displayed can be further qualifed by their email address.
+* `memberemaildisplaypattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be displayed; others will not be displayed
+* `memberemailskippattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will not be displayed; others will be displayed
+
+By default, members of type GROUP are recursively expanded to show their constituent members. (Members of
+type CUSTOMER are not expanded.) The `depth <Number>` argument controls the depth to which nested groups are displayed.
+* `depth -1` - all groups in the selected group and below are displayed; this is the default.
+* `depth 0` - the groups within a selected group are displayed, no descendants are displayed.
+* `depth N` - the groups within the selected group and those groups N levels below the selected group are displayed.
+
+The `includederivedmembership` option causes the API to expand type GROUP
+members to display their constituent members. The role displayed for a user is the highest role it
+has in any constituent group, it is not necessarily its role in the top group.
+
+The options `types user` and `includederivedmembership types user` return the same list of users.
+The `includederivedmembership` option makes less API calls but doesn't show hierarchy.
+
+### Display group structure
+To see a group's structure of nested groups use the `type group` option.
+```
+$ gam show cigroup-members group testgroup5 types group
+Group: testgroup5@domain.com
+  MEMBER, GROUP, testgroup1@domain.com, ACTIVE
+    MEMBER, GROUP, testgroup2@domain.com, ACTIVE
+  MEMBER, GROUP, testgroup3@domain.com, ACTIVE
+    MEMBER, GROUP, testgroup2@domain.com, ACTIVE
+    MEMBER, GROUP, testgroup4@domain.com, ACTIVE
+```
+To show the structure of all groups you can do the following; it will be time consuming for a large number of groups.
+```
+gam redirect stdout ./groups.txt show group-members types group
+```

docs/Cloud-Identity-Groups.md

@@ -0,0 +1,405 @@
+# Cloud Identity Groups
+- [API documentation](#api-documentation)
+- [Python Regular Expressions](Python-Regular-Expressions) Match function
+- [Query documentation](#query-documentation)
+- [Cloud Identity Group Documentation](#cloud-identity-group-documentation)
+- [Security Group Documentation](#security-group-documentation)
+- [Notes](#Notes)
+- [Definitions](#definitions)
+- [Manage groups](#manage-groups)
+- [Display information about individual groups](#display-information-about-individual-groups)
+- [Display information about multiple groups](#display-information-about-multiple-groups)
+- [Display group counts](#display-group-counts)
+
+## API documentation
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/groups
+* https://developers.google.com/admin-sdk/groups-settings/v1/reference/groups
+* https://cloud.google.com/identity/docs/groups
+* https://cloud.google.com/identity/docs/reference/rest/v1/groups
+* https://support.google.com/a/answer/11192679
+
+## Query documentation
+* https://cloud.google.com/identity/docs/reference/rest/v1/groups#dynamicgroupquery
+* https://cloud.google.com/identity/docs/reference/rest/v1/SecuritySettings#MemberRestriction
+
+## Cloud Identity Group Documentation
+* https://gsuiteupdates.googleblog.com/2020/08/new-api-cloud-identity-groups-google.html
+
+## Security Group Documentation
+* https://gsuiteupdates.googleblog.com/2020/09/security-groups-beta.html
+
+## Notes
+
+In the Admin Directory API a group has the following characteristics:
+* `id` - The unique ID of a group
+* `email` - The group's email address
+* `name` - The group's display name
+
+In the Cloud Indentity Groups API a group has the following characteristics:
+* `name` - The unique ID of a group
+* `groupKey.id` - The group's email address
+* `displayName` - The group's display name
+
+The Admin Directory API group characteristic names will be used.
+
+Dynamic Groups require Cloud Identity Premium accounts.
+
+* https://cloud.google.com/identity/docs/how-to/create-dynamic-groups
+
+The `cimember <UserItem>` option of `gam print cigroups` requires a Google Workspace Enterprise Standard, Enterprise Plus, and Enterprise for Education;
+and Cloud Identity Premium accounts. Unfortunately, even if you have the required account, the API call that supports the query doesn't work.
+
+* https://cloud.google.com/identity/docs/reference/rest/v1/groups.memberships/searchTransitiveGroups
+
+## Definitions
+```
+<DomainName> ::= <String>(.<String>)+
+<EmailAddress> ::= <String>@<DomainName>
+<UniqueID> ::= id:<String>
+<GroupItem> ::= <EmailAddress>|<UniqueID>|<String>
+<GroupList> ::= "<GroupItem>(,<GroupItem>)*"
+<GroupEntity> ::=
+        <GroupList> | <FileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<GroupRole> ::= owner|manager|member
+<GroupRoleList> ::= "<GroupRole>(,<GroupRole>)*"
+<CIGroupType> ::= customer|group|other|serviceaccount|user
+<CIGroupTypeList> ::= "<CIGroupType>(,<CIGroupType>)*"
+<QueryDynamicGroup> ::= <String>
+        See: https://cloud.google.com/identity/docs/reference/rest/v1/groups#dynamicgroupquery
+<QueryMemberRestrictions> ::= <String>
+        See: https://cloud.google.com/identity/docs/reference/rest/v1/SecuritySettings#MemberRestriction
+
+<JSONData> ::= (json [charset <Charset>] <String>) | (json file <FileName> [charset <Charset>]) |
+
+<GroupSettingsAttribute> ::=
+        (allowexternalmembers <Boolean>)|
+        (allowwebposting <Boolean>)|
+        (archiveonly <Boolean>)|
+        (customfootertext <String>)|
+        (customreplyto <EmailAddress>)|
+        (defaultmessagedenynotificationtext <String>)|
+        (description <String>)|
+        (enablecollaborativeinbox|collaborative <Boolean>)|
+        (includeinglobaladdresslist|gal <Boolean>)|
+        (includecustomfooter <Boolean>)|
+        (isarchived <Boolean>)|
+        (memberscanpostasthegroup <Boolean>)|
+        (messagemoderationlevel moderate_all_messages|moderate_non_members|moderate_new_members|moderate_none)|
+        (name|displayname <String>)|
+        (primarylanguage <Language>)|
+        (replyto reply_to_custom|reply_to_sender|reply_to_list|reply_to_owner|reply_to_ignore|reply_to_managers)|
+        (sendmessagedenynotification <Boolean>)|
+        (spammoderationlevel allow|moderate|silently_moderate|reject)|
+        (whocanadd all_members_can_add|all_managers_can_add|all_owners_can_add|none_can_add)|
+        (whocancontactowner anyone_can_contact|all_in_domain_can_contact|all_members_can_contact|all_managers_can_contact)|
+        (whocanjoin anyone_can_join|all_in_domain_can_join|invited_can_join|can_request_to_join)|
+        (whocanleavegroup all_members_can_leave|all_managers_can_leave|all_owners_can_leave|none_can_leave)|
+        (whocanpostmessage none_can_post|all_managers_can_post|all_members_can_post|all_owners_can_post|all_in_domain_can_post|anyone_can_post)|
+        (whocanviewgroup anyone_can_view|all_in_domain_can_view|all_members_can_view|all_managers_can_view|all_owners_can_view)|
+        (whocanviewmembership all_in_domain_can_view|all_members_can_view|all_managers_can_view|all_owners_can_view)
+<GroupWhoCanDiscoverGroupDeprecatedAttribute> ::=
+        (showingroupdirectory <Boolean>)
+<GroupWhoCanAssistContentDeprecatedAttribute> ::=
+        (whocanassigntopics all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanenterfreeformtags all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanhideabuse all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmaketopicssticky all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmarkduplicate all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmarkfavoritereplyonanytopic all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmarknoresponseneeded all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmodifytagsandcategories all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocantaketopics all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanunassigntopic all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanunmarkfavoritereplyonanytopic all_members|owners_and_managers|managers_only|owners_only|none)
+<GroupWhoCanModerateContentDeprecatedAttribute> ::=
+        (whocanapprovemessages all_members|owners_and_managers|owners_only|none)|
+        (whocandeleteanypost all_members|owners_and_managers|owners_only|none)|
+        (whocandeletetopics all_members|owners_and_managers|owners_only|none)|
+        (whocanlocktopics all_members|owners_and_managers|owners_only|none)|
+        (whocanmovetopicsin all_members|owners_and_managers|owners_only|none)|
+        (whocanmovetopicsout all_members|owners_and_managers|owners_only|none)|
+        (whocanpostannouncements all_members|owners_and_managers|owners_only|none)
+<GroupWhoCanModerateMembersDeprecatedAttribute> ::=
+        (whocanadd all_members_can_add|all_managers_can_add|none_can_add)|
+        (whocanapprovemembers all_members_can_approve|all_managers_can_approve|all_owners_can_approve|none_can_approve)|
+        (whocanbanusers all_members|owners_and_managers|owners_only|none)|
+        (whocaninvite all_members_can_invite|all_managers_can_invite|all_owners_can_invite|none_can_invite)|
+        (whocanmodifymembers all_members|owners_and_managers|owners_only|none)
+<GroupDeprecatedAttribute> ::=
+        (allowgooglecommunication <Boolean>)|
+        (favoriterepliesontop <Boolean>)|
+        (maxmessagebytes <ByteCount>)|
+        (messagedisplayfont default_font|fixed_width_font)|
+        (whocanaddreferences all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmarkfavoritereplyonowntopic all_members|owners_and_managers|managers_only|owners_only|none)
+<GroupAttribute> ::=
+        <JSONData>|
+        <GroupSettingsAttribute>|
+        (whocandiscovergroup allmemberscandiscover|allindomaincandiscover|anyonecandiscover)|
+        (whocanassistcontent all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmoderatecontent all_members|owners_and_managers|owners_only|none)|
+        (whocanmoderatemembers all_members|owners_and_managers|owners_only|none)|
+        <GroupWhoCanDiscoverGroupDeprecatedAttribute>|
+        <GroupWhoCanAssistContentDeprecatedAttribute>|
+        <GroupWhoCanModerateContentDeprecatedAttribute>|
+        <GroupWhoCanModerateMembersDeprecatedAttribute>|
+        <GroupDeprecatedAttribute>
+```
+```
+<GroupFieldName> ::=
+        admincreated|
+        aliases|
+        allowexternalmembers|
+        allowgooglecommunication|
+        allowwebposting|
+        archiveonly|
+        customfootertext|
+        customreplyto|
+        customrolesenabledforsettingstobemerged|
+        defaultmessagedenynotificationtext|
+        description|
+        directmemberscount|
+        email|
+        enablecollaborativeinbox|collaborative|
+        favoriterepliesontop|
+        id|
+        includecustomfooter|
+        includeinglobaladdresslist|gal|
+        isarchived|
+        maxmessagebytes|
+        memberscanpostasthegroup|
+        messagedisplayfont|
+        messagemoderationlevel|
+        name|
+        primarylanguage|
+        replyto|
+        sendmessagedenynotification|
+        showingroupdirectory|
+        spammoderationlevel|
+        whocanaddreferences|
+        whocanadd|
+        whocanapprovemessages|
+        whocanassigntopics|
+        whocanassistcontent|
+        whocancontactowner|
+        whocandeleteanypost|
+        whocandeletetopics|
+        whocandiscovergroup|
+        whocanenterfreeformtags|
+        whocanhideabuse|
+        whocaninvite|
+        whocanjoin|
+        whocanleavegroup|
+        whocanlocktopics|
+        whocanmaketopicssticky|
+        whocanmarkduplicate|
+        whocanmarkfavoritereplyonanytopic|
+        whocanmarkfavoritereplyonowntopic|
+        whocanmarknoresponseneeded|
+        whocanmoderatecontent|
+        whocanmodifytagsandcategories|
+        whocanmovetopicsin|
+        whocanmovetopicsout|
+        whocanpostannouncements|
+        whocanpostmessage|
+        whocantaketopics|
+        whocanunassigntopic|
+        whocanunmarkfavoritereplyonanytopic|
+        whocanviewgroup|
+        whocanviewmembership
+<GroupFieldNameList> ::= "<GroupFieldName>(,<GroupFieldName>)*"
+```
+```
+<CIGroupFieldName> ::=
+        additionalgroupkeys|
+        createtime|
+        description|
+        displayname|
+        dynamicgroupmetadata|
+        email|
+        groupkey|
+        id|
+        labels|
+        name|
+        parent|
+        updatetime
+<CIGroupFieldNameList> ::= "<CIGroupFieldName>(,<CIGroupFieldName>)*"
+```
+## Manage groups
+
+These commands allow you to create, update and delete groups. They use the Admin SDK Groups Settings API
+to set `<GroupAttribute>`.
+```
+gam create cigroup <EmailAddress> [copyfrom <GroupItem>] <GroupAttribute>
+        [makeowner]
+        [alias|aliases <EmailAddressList>] [dynamic <QueryDynamicGroup>]
+gam update cigroup <GroupEntity> [copyfrom <GroupItem>] <GroupAttribute>
+        [makesecuritygroup|security] [makedynamicsecuritygroup|dynamicsecurity] [dynamic <QueryDynamicGroup>]
+        [memberrestrictions <QueryMemberRestrictions>]
+gam delete cigroups <GroupEntity>
+```
+The `copyfrom <GroupItem>` allows copying of group attributes from one group to another.
+The following attributes are not copied: name, description, email, admincreated, aliases, noneditablealiases.
+Any `<GroupAttribute>` specified will override the copied attributes.
+
+You can update a non-dynamic group to a non-dynamic security group with the `makesecuritygroup` option. To update a dynamic group to a security group, use the `makedynamicsecuritygroup` option instead.
+* Warning: A Security Group cannot be changed back to a Google Group.
+
+You can update a group to restrict its membership with the `memberrestrictions <QueryMemberRestrictions>`option.
+* https://cloud.google.com/identity/docs/reference/rest/v1/SecuritySettings#MemberRestriction
+
+The `makeowner` option makes the administrator in `oauth2.txt` the initial owner of the group.
+
+## Display information about individual groups
+This command displays information as an indented list of keys and values.
+```
+gam info cigroups <GroupEntity>
+        [nousers|membertree] [quick] [noaliases]
+        [nosecurity|nosecuritysettings]
+        [allfields|<CIGroupFieldName>*|(fields <CIGroupFieldNameList>)]
+        [roles <GroupRoleList>] [members] [managers] [owners]
+        [types <CIGroupTypeList>]
+        [memberemaildisplaypattern|memberemailskippattern <RegularExpression>]
+        [formatjson]
+```
+
+By default, all direct members, managers and owners in the group are displayed; these options modify that behavior:
+* `members` - Display members
+* `managers` - Display managers
+* `owners` - Display owners
+* `nousers` or `quick` - Do not display any members, managers or owners
+* `membertree` - Display all roles; expand all groups
+
+By default, when displaying members from a group, all types of members (customer, group, serviceaccount, user) in the group are displayed; this option modifies that behavior:
+* `types <CIGroupTypeList>` - Display specified types
+
+Members that have met the above qualifications to be displayed can be further qualifed by their email address.
+* `memberemaildisplaypattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be displayed; others will not be displayed
+* `memberemailskippattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will not be displayed; others will be displayed
+
+By default, all group aliases are displayed, these options modify that behavior:
+* `noaliases` or `quick` - Do not display group aliases
+
+By default, GAM makes an additional API call to get the `SecuritySettings` for the group.
+* `nosecuritysettings` - Do not make API and display `SecuritySettings`
+
+* `allfields` - All Cloud Identity Group fields
+* `<CIGroupFieldName>*` - Individual fields to display
+* `fields <CIGroupFieldNameList>` - A comma separated list of fields to display
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the output in JSON notation
+
+## Display information about multiple groups
+This command displays information in CSV format.
+```
+gam print cigroups [todrive <ToDriveAttribute>*]
+        [(cimember|showownedby <UserItem>)|(select <GroupEntity>)|(query <String>)]
+        [emailmatchpattern [not] <RegularExpression>] [namematchpattern [not] <RegularExpression>]
+        [descriptionmatchpattern [not] <RegularExpression>]
+        [basic|allfields|(<CIGroupFieldName>* [fields <CIGroupFieldNameList>])]
+        [roles <GroupRoleList>] [memberrestrictions]
+        [members|memberscount] [managers|managerscount] [owners|ownerscount] [totalcount] [countsonly]
+        [types <CIGroupTypeList>]
+        [memberemaildisplaypattern|memberemailskippattern <RegularExpression>]
+        [convertcrnl] [delimiter <Character>]
+        [formatjson [quotechar <Character>]]
+```
+By default, all groups in the account are displayed, these options allow selection of subsets of groups:
+* `cimember <UserItem>` - Limit display to groups that contain `<UserItem>` as a member
+* `showownedby <UserItem>` - Limit display to groups owned by `<UserItem>`
+* `select <GroupEntity>` - Limit display to the groups specified in `<GroupEntity>`
+* `query <String>` - Limit display to the groups that match the query
+
+These options further limit the list of groups selected above:
+* `emailmatchpattern <RegularExpression>` - Limit display to groups whose email address matches `<RegularExpression>`
+* `emailmatchpattern not <RegularExpression>` - Limit display to groups whose email address does not match `<RegularExpression>`
+* `namematchpattern <RegularExpression>` - Limit display to groups whose name matches `<RegularExpression>`
+* `namematchpattern not <RegularExpression>` - Limit display to groups whose name does not match `<RegularExpression>`
+* `descriptionmatchpattern <RegularExpression>` - Limit display to groups whose description matches `<RegularExpression>`
+* `descriptionmatchpattern not <RegularExpression>` - Limit display to groups whose description does not match `<RegularExpression>`
+
+By default, GAM does not make an additional API call todisplay the member restrictions from `SecuritySettings`.
+* `memberrestrictions` - Make an additional API call and display the member restrictions from `SecuritySettings`
+
+When retrieving lists of Google Groups from API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many groups to retrieve in each API call; default is 500.
+
+By default, only the group email address is displayed, these options specify what group fields to display:
+* `basic` - Only Cloud Identity Group basic fields are displayed; no additional API calls are required
+* `allfields|ciallfields` - All Cloud Identity Group fields are displayed; an additional API call per group is required
+* `<GroupFieldName>*` - Individual fields to display
+* `fields|cifields <CIGroupFieldNameList>` - A comma separated list of fields to display
+
+As of 2020-12-24, a separate API call is required for each group to get the following fields:
+`additionalgroupkeys,createtime,dynamicgroupmetadata,parent,updatetime`
+
+Some text fields may contain carriage returns or line feeds, displaying fields containing these characters will make processing the CSV file with a script hard; this option converts those characters to a text form.
+The default value is `csv_output_convert_cr_nl` from `gam.cfg`
+* `convertcrnl` - Convert carriage return to \r and line feed to \n
+
+When lists of items are displayed, the delimiter between items defaults to the `csv_output_column_delimiter` value in gam.cfg; you can specify a different delimiter:
+* `delimiter <Character>` - Use `<Character>` as the list item delimiter, `<Character>` must be a single character after processing any escape character
+
+By default, no members, managers or owners in the group are displayed; these options modify that behavior:
+* `members` - Display list of members
+* `memberscount` - Display count of members but not individual members
+* `managers` - Display list of managers
+* `managerscount` - Display count of managers but not individual managers
+* `owners` - Display list of owners
+* `ownerscount` - Display count of owners but not individual owners
+* `countsonly` - Change any `members`, `managers` or `owners` options to `memberscount`, `managerscount` or `ownerscount`
+* `totalcount` - Display sum of counts of members, managers, owners.
+
+By default, when displaying members from a group, all types of members (customer, group, serviceaccount, user) in the group are displayed; this option modifies that behavior:
+* `types <CIGroupTypeList>` - Display specified types
+
+Members that have met the above qualifications to be displayed can be further qualifed by their email address.
+* `memberemaildisplaypattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be displayed; others will not be displayed
+* `memberemailskippattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will not be displayed; others will be displayed
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+### Display dynamic groups
+```
+gam print cigroups query "'cloudidentity.googleapis.com/groups.dynamic' in labels"
+```
+
+### Display security groups
+```
+gam print cigroups query "'cloudidentity.googleapis.com/groups.security' in labels"
+```
+
+## Display group counts
+Display the number of groups.
+```
+gam print cigroups
+        [(cimember|showownedby <UserItem>)|(select <GroupEntity>)|(query <String>)]
+        [emailmatchpattern [not] <RegularExpression>] [namematchpattern [not] <RegularExpression>]
+        [descriptionmatchpattern [not] <RegularExpression>]
+        showitemcountonly
+```
+Example
+```
+$ gam print cigroups showitemcountonly
+Getting all Cloud Identity Groups, may take some time on a large Google Workspace Account...
+Got 242 Cloud Identity Groups: td.current@domain.com - postmaster@domain.com
+242
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print cigroups showitemcountonly)
+Windows PowerShell
+count = & gam print cidgroups showitemcountonly
+```

docs/Cloud-Storage.md

@@ -0,0 +1,57 @@
+# Cloud Storage
+- [API documentation](#api-documentation)
+- [Notes](#notes)
+- [Definitions](#definitions)
+- [Download a Cloud Storage Bucket Object](#download-a-cloud-storage-bucket-object)
+
+## API documentation
+* https://cloud.google.com/storage/docs/json_api/v1/objects
+
+## Notes
+To use these commands you must add the 'Cloud Storage API' to your project and update your client access authorization.
+Enable `Cloud Storage API (Read, Vault/Takeout Download)`.
+```
+gam update project
+gam oauth create
+```
+
+## Definitions
+```
+<StorageBucketName> ::= <String>
+<StorageObjectName> ::= <String>
+<StorageBucketObjectName> ::=
+        https://storage.cloud.google.com/<StorageBucketName>/<StorageObjectName>|
+        https://storage.googleapis.com/<StorageBucketName>/<StorageObjectName>|
+        gs://<StorageBucketName>/<StorageObjectName>|
+        <StorageBucketName>/<StorageObjectName>
+```
+## Download a Cloud Storage Bucket Object
+```
+gam download storagefile <StorageBucketObjectName>
+        [targetfolder <FilePath>] [overwrite [<Boolean>]] [nogcspath [<Boolean>]]
+```
+By default, the takeout files will be downloaded to the directory specified by `drive_dir` in gam.cfg.
+* `targetfolder <FilePath>` - The takeout files will be downloaded to `<FilePath>`
+
+By default, when getting a document, an existing local file will not be overwritten; a numeric prefix is added to the filename.
+* `overwrite false` - Do not overwite an existing file; add a numeric prefix and create a new file
+* `overwrite | overwrite true` - Overwite an existing file
+
+By default, when getting a document, its Google Cloud Storage path is preserved.
+* `nogcspath false` - Preserve the Google Cloud Storage path
+* `nogcspath | nogcspath true` - Do not preserve the Google Cloud Storage path
+
+### Example
+This example downloads a Google Cloud Storage file preserving its path
+```
+$ gam download storagefile gs://gam-bucket/SubFolder/SimpleText.txt
+Getting File SubFolder/SimpleText.txt
+Cloud Storage File: SubFolder/SimpleText.txt, Downloaded to: /Users/admin/Documents/GamWork/SubFolder/SimpleText.txt
+```
+This example downloads a Google Cloud Storage file removing its path
+```
+$ gam download storagefile gs://gam-bucket/SubFolder/SimpleText.txt nogcspath
+Getting File SubFolder/SimpleText.txt
+Cloud Storage File: SubFolder/SimpleText.txt, Downloaded to: /Users/admin/Documents/GamWork/SimpleText.txt
+
+```

docs/Collections-of-ChromeOS-Devices.md

@@ -0,0 +1,464 @@
+# Collections of ChromeOS Devices
+- [Python Regular Expressions](Python-Regular-Expressions) Match function
+- [Definitions](#definitions)
+- [Organization Unit Quoting](#organization-unit-quoting)
+- [Query Quoting](#query-quoting)
+- [Query Notes](#query-notes)
+- [CrOS Type Entity](#cros-type-entity)
+  - [All ChromeOS devices](#all-chromeos-devices)
+  - [A list of ChromeOS deviceIds](#a-list-of-chromeos-deviceids)
+  - [A list of ChromeOS device serial numbers](#a-list-of-chromeos-device-serial-numbers)
+  - [ChromeOS devices directly in the Organization Unit `<OrgUnitItem>`](#chromeos-devices-directly-in-the-organization-unit-orgunititem)
+  - [ChromeOS devices in the Organization Unit `<OrgUnitItem>` and all of its sub Organization Units](#chromeos-devices-in-the-organization-unit-orgunititem-and-all-of-its-sub-organization-units)
+  - [ChromeOS devices directly in the Organization Units `<OrgUnitList>`](#chromeos-devices-directly-in-the-organization-units-orgunitlist)
+  - [ChromeOS devices in the Organization Units `<OrgUnitList>` and all of their sub Organization Units](#chromeos-devices-in-the-organization-units-orgunitlist-and-all-of-their-sub-organization-units)
+  - [ChromeOS devices directly in the Organization Unit `<OrgUnitItem>` that also match a query](#chromeos-devices-directly-in-the-organization-unit-orgunititem-that-also-match-a-query)
+  - [ChromeOS devices in the Organization Unit `<OrgUnitItem>` and all of its sub Organization Units that also match a query](#chromeos-devices-in-the-organization-unit-orgunititem-and-all-of-its-sub-organization-units-that-also-match-a-query)
+  - [ChromeOS devices directly in the Organization Units `<OrgUnitList>` that also match a query](#chromeos-devices-directly-in-the-organization-units-orgunitlist-that-also-match-a-query)
+  - [ChromeOS devices in the Organization Units `<OrgUnitList>` and all of their sub Organization Units that also match a query](#chromeos-devices-in-the-organization-units-orgunitlist-and-all-of-their-sub-organization-units-that-also-match-a-query)
+  - [ChromeOS devices directly in the Organization Unit `<OrgUnitItem>` that also match any query in a list of queries](#chromeos-devices-directly-in-the-organization-unit-orgunititem-that-also-match-any-query-in-a-list-of-queries)
+  - [ChromeOS devices in the Organization Unit `<OrgUnitItem>` and all of its sub Organization Units that also match any query in a list of queries](#chromeos-devices-in-the-organization-unit-orgunititem-and-all-of-its-sub-organization-units-that-also-match-any-query-in-a-list-of-queries)
+  - [ChromeOS devices directly in the Organization Units `<OrgUnitList>` that also match any query in a list of queries](#chromeos-devices-directly-in-the-organization-units-orgunitlist-that-also-match-any-query-in-a-list-of-queries)
+  - [ChromeOS devices in the Organization Units `<OrgUnitList>` and all of their sub Organization Units that also match any query in a list of queries](#chromeos-devices-in-the-organization-units-orgunitlist-and-all-of-their-sub-organization-units-that-also-match-any-query-in-a-list-of-queries)
+  - [ChromeOS devices that match a query](#chromeos-devices-that-match-a-query)
+  - [ChromeOS devices that match any query in a list of queries](#chromeos-devices-that-match-any-query-in-a-list-of-queries)
+  - [ChromeOS deviceIds in a flat file/Google Doc/Google Cloud Storage Object](#chromeos-deviceids-in-a-flat-filegoogle-docgoogle-cloud-storage-object)
+  - [ChromeOS serial numbers in a flat file/Google Doc/Google Cloud Storage Object](#chromeos-serial-numbers-in-a-flat-filegoogle-docgoogle-cloud-storage-object)
+  - [Selected ChromeOS deviceIds in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object](#selected-chromeos-deviceids-in-a-csv-filegoogle-sheetgoogle-docgoogle-cloud-storage-object)
+  - [Selected ChromeOS serial numbers in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object](#selected-chromeos-serial-numbers-in-a-csv-filegoogle-sheetgoogle-docgoogle-cloud-storage-object)
+  - [ChromeOS devices from OUs in a flat file/Google Doc/Google Cloud Storage Object](#chromeos-devices-from-ous-in-a-flat-filegoogle-docgoogle-cloud-storage-object)
+  - [ChromeOS deviceIds from OUs in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object](#chromeos-deviceids-from-ous-in-a-csv-filegoogle-sheetgoogle-docgoogle-cloud-storage-object)
+  - [ChromeOS devices directly in or from OUs in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object](#chromeos-devices-directly-in-or-from-ous-in-a-csv-filegoogle-sheetgoogle-docgoogle-cloud-storage-object)
+  - [ChromeOS deviceIds from data fields identified in a `csvkmd` argument](#chromeos-deviceids-from-data-fields-identified-in-a-csvkmd-argument)
+- [Examples using CSV files](#examples-using-csv-files)
+- [Examples using multiple queries or Org Units](#examples-using-multiple-queries-or-org-units)
+
+## Definitions
+* [Basic Items](Basic-Items)
+
+* [List Items](List-Items)
+
+* [Command data from Google Docs/Sheets/Storage](Command-Data-From-Google-Docs-Sheets-Storage)
+```
+<StorageBucketName> ::= <String>
+<StorageObjectName> ::= <String>
+<StorageBucketObjectName> ::=
+        https://storage.cloud.google.com/<StorageBucketName>/<StorageObjectName>|
+        https://storage.googleapis.com/<StorageBucketName>/<StorageObjectName>|
+        gs://<StorageBucketName>/<StorageObjectName>|
+        <StorageBucketName>/<StorageObjectName>
+
+<UserGoogleDoc> ::=
+        <EmailAddress> <DriveFileIDEntity>|<DriveFileNameEntity>|(<SharedDriveEntity> <SharedDriveFileNameEntity>)
+<UserGoogleSheet> ::=
+        <EmailAddress> <DriveFileIDEntity>|<DriveFileNameEntity>|(<SharedDriveEntity> <SharedDriveFileNameEntity>) <SheetEntity>
+
+<JSONData> ::= (json [charset <Charset>] <String>) | (json file <FileName> [charset <Charset>]) |
+```
+```
+<CrOSTypeEntity> ::=
+        (all cros)|
+        (cros <CrOSIDList>)|
+        (cros_sn <SerialNumberList>)|
+        (cros_ou <OrgUnitItem>)|
+        (cros_ou_and_children <OrgUnitItem>)|
+        (cros_ous <OrgUnitList>)|
+        (cros_ous_and_children <OrgUnitList>)|
+        (cros_ou_query <OrgUnitItem> <QueryCrOS>)|
+        (cros_ou_and_children_query <OrgUnitItem> <QueryCrOS>)|
+        (cros_ous_query <OrgUnitList> <QueryCrOS>)|
+        (cros_ous_and_children_query <OrgUnitList> <QueryCrOS>)|
+        (cros_ou_queries <OrgUnitItem> <QueryCrOSList>)|
+        (cros_ou_and_children_queries <OrgUnitItem> <QueryCrOSList>)|
+        (cros_ous_queries <OrgUnitList> <QueryCrOSList>)|
+        (cros_ous_and_children_queries <OrgUnitList> <QueryCrOSList>)|
+        (crosquery <QueryCrOS>)|
+        (crosqueries <QueryCrOSList>)|
+        (crosfile
+            ((<FileName> [charset <Charset>])|
+             (gdoc <UserGoogleDoc>)|
+             (gcsdoc <StorageBucketObjectName>))
+            [delimiter <Character>])|
+        (crosfile_sn
+            ((<FileName> [charset <Charset>])|
+             (gdoc <UserGoogleDoc>)|
+             (gcsdoc <StorageBucketObjectName>))
+            [delimiter <Character>])|
+        (croscsvfile
+            ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+             (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+             (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+             (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+             (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+            [warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
+            [endcsv|(fields <FieldNameList>)]
+            (matchfield|skipfield <FieldName> <RegularExpression>)*
+            [delimiter <Character>])|
+        (croscsvfile_sn
+            ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+             (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+             (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+             (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+             (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+            [warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
+            [endcsv|(fields <FieldNameList>)]
+            (matchfield|skipfield <FieldName> <RegularExpression>)*
+            [delimiter <Character>])|
+        (datafile
+            cros|cros_sn|cros_ous|cros_ous_and_children
+            ((<FileName> [charset <Charset>])|
+              (gdoc <UserGoogleDoc>)|
+              (gcsdoc <StorageBucketObjectName>))
+             [delimiter <Character>])|
+        (csvdatafile
+            cros|cros_sn|cros_ous|cros_ous_and_children
+            ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+              (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+              (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+              (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+              (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+            [warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
+            [endcsv|(fields <FieldNameList>)]
+            (matchfield|skipfield <FieldName> <RegularExpression>)*
+            [delimiter <Character>])|
+        (csvkmd
+            cros|cros_sn|cros_ous|cros_ous_and_children
+            ((<FileName>|
+              (gsheet <UserGoogleSheet>)|
+              (gdoc <UserGoogleDoc>)|
+              (gcscsv <StorageBucketObjectName>)|
+              (gcsdoc <StorageBucketObjectName>))
+             [charset <Charset>] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>] [fields <FieldNameList>])
+            keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
+            subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
+            (matchfield|skipfield <FieldName> <RegularExpression>)*
+            [datafield <FieldName>(:<FieldName>)* [delimiter <Character>]])
+        (croscsvdata <FieldName>(:<FieldName>*))
+```
+## Organization Unit Quoting
+* `<OrgUnitItem>` should be enclosed in `"` if it contains a space, comma or single quote.
+* `<OrgUnitList>` may require special quoting based on whether the OUs contain spaces, commas or single quotes.
+
+For quoting rules, see: [List Items](List-Items)
+
+## Query Quoting
+`<QueryCrOSList>` may require special quoting based on whether the queries contain spaces, commas or single quotes.
+
+* Surround `<QueryCrOSList>` with `" "`
+* Surround each query with `\" \"`, separate the queries with commas.
+
+```
+queries "\"orgUnitPath='/Path/To/OU 1'\",\"orgUnitPath='/Path/To/OU 2'\",\"orgUnitPath='/Path/To/OU 3'\""
+```
+
+## Query Notes
+
+See https://support.google.com/chrome/a/answer/1698333
+
+Undocumented API query terms.
+```
+<QueryDate> ::=
+        YYYY-MM-DD    # Specific date
+        ..YYYY-MM-DD  # Before a date
+        YYYY-MM-DD..  # After a date
+        YYYY-MM-DD..YYYY-MM-DD  # Range of dates
+        
+aue:<QueryDate>
+compliance:compliant|pending_update|not_compliant
+last_user_activity:<QueryDate>
+policy_status:true|false
+public_model_name:<String>
+update_status:default_os_up_to_date|pending_update|os_image_download_not_started|os_image_download_in_progress|os_update_need_reboot
+```
+
+## CrOS Type Entity
+
+Use these options to select Chrome OS devices for GAM commands.
+
+## All ChromeOS devices
+* `all cros`
+
+## A list of ChromeOS deviceIds
+* `cros <CrOSList>`
+
+## A list of ChromeOS device serial numbers
+* `cros_sn <SerialNumberList>`
+
+## ChromeOS devices directly in the Organization Unit `<OrgUnitItem>`
+* `cros_ou <OrgUnitItem>`
+
+## ChromeOS devices in the Organization Unit `<OrgUnitItem>` and all of its sub Organization Units
+* `cros_ou_and_children <OrgUnitItem>`
+
+## ChromeOS devices directly in the Organization Units `<OrgUnitList>`
+* `cros_ous <OrgUnitList>`
+
+## ChromeOS devices in the Organization Units `<OrgUnitList>` and all of their sub Organization Units
+* `cros_ous_and_children <OrgUnitList>`
+
+## ChromeOS devices directly in the Organization Unit `<OrgUnitItem>` that also match a query
+* `cros_ou_query <OrgUnitItem> <QueryCrOS>`
+
+## ChromeOS devices in the Organization Unit `<OrgUnitItem>` and all of its sub Organization Units that also match a query
+* `cros_ou_and_children_query <OrgUnitItem> <QueryCrOS>`
+
+## ChromeOS devices directly in the Organization Units `<OrgUnitList>` that also match a query
+* `cros_ous_query <OrgUnitList> <QueryCrOS>`
+
+## ChromeOS devices in the Organization Units `<OrgUnitList>` and all of their sub Organization Units that also match a query
+* `cros_ous_and_children_query <OrgUnitList> <QueryCrOS>`
+
+## ChromeOS devices directly in the Organization Unit `<OrgUnitItem>` that also match any query in a list of queries
+* `cros_ou_queries <OrgUnitItem> <QueryCrOSList>`
+
+## ChromeOS devices in the Organization Unit `<OrgUnitItem>` and all of its sub Organization Units that also match any query in a list of queries
+* `cros_ou_and_children_queries <OrgUnitItem> <QueryCrOSList>`
+
+## ChromeOS devices directly in the Organization Units `<OrgUnitList>` that also match any query in a list of queries
+* `cros_ous_queries <OrgUnitList> <QueryCrOSList>`
+
+
+## ChromeOS devices in the Organization Units `<OrgUnitList>` and all of their sub Organization Units that also match any query in a list of queries
+* `cros_ous_and_children_queries <OrgUnitList> <QueryCrOSList>`
+
+## ChromeOS devices that match a query
+* `crosquery <QueryCrOS>`
+
+## ChromeOS devices that match any query in a list of queries
+* `crosqueries <QueryCrOSList>`
+
+## ChromeOS deviceIds in a flat file/Google Doc/Google Cloud Storage Object
+```
+crosfile
+   ((<FileName> [charset <Charset>])|
+    (gdoc <UserGoogleDoc>)|
+    (gcsdoc <StorageBucketObjectName>))
+   [delimiter <Character>]
+```
+* `<FileName>` - A flat file containing a single ChromeOS deviceId per row
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gdoc <UserGoogleDoc>` - A Google Doc containing a single ChromeOS deviceId per row
+* `gcsdoc <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object containing a single ChromeOS deviceId per row
+* `delimiter <Character>` - There are multiple deviceIds per row separated by `<Character>`; if not specified, there is single deviceId per row
+
+## ChromeOS serial numbers in a flat file/Google Doc/Google Cloud Storage Object
+```
+crosfile_sn
+   ((<FileName> [charset <Charset>])|
+    (gdoc <UserGoogleDoc>)|
+    (gcsdoc <StorageBucketObjectName>))
+   [delimiter <Character>]
+```
+* `<FileName>` - A flat file containing a single ChromeOS serial number per row
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gdoc <UserGoogleDoc>` - A Google Doc containing a single ChromeOS serial number per row
+* `gcsdoc <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object containing a single ChromeOS serial number per row
+* `delimiter <Character>` - There are multiple serial numbers per row separated by `<Character>`; if not specified, there is single serial number per row
+
+## Selected ChromeOS deviceIds in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object
+```
+croscsvfile
+   ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+    (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+    (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+    (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+    (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+   [warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
+   [endcsv|(fields <FieldNameList>)]
+   (matchfield|skipfield <FieldName> <RegularExpression>)*
+   [delimiter <Character>]
+```
+* `<FileName>(:<FieldName>)+` - A CSV file and the one or more columns that contain ChromeOS deviceIds
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gsheet(:<FieldName>)+ <UserGoogleSheet>` - A Google Sheet and the one or more columns that contain ChromeOS deviceIds
+* `gdoc(:<FieldName>)+ <UserGoogleDoc>` - A Google Doc and the one or more columns that contain ChromeOS deviceIds
+* `gcscsv(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain ChromeOS deviceIds
+* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain ChromeOS deviceIds
+* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
+* `columndelimiter <Character>` - Columns are  separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
+* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
+* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
+* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
+* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
+* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
+* `delimiter <Character>` - There are multiple deviceIds per column separated by `<Character>`; if not specified, there is single deviceId per column
+
+## Selected ChromeOS serial numbers in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object
+```
+croscsvfile_sn
+   ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+    (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+    (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+    (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+    (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+   [warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
+   [endcsv|(fields <FieldNameList>)]
+   (matchfield|skipfield <FieldName> <RegularExpression>)*
+   [delimiter <Character>]
+```
+* `<FileName>(:<FieldName>)+` - A CSV file and the one or more columns that contain ChromeOS serial numbers
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gsheet(:<FieldName>)+ <UserGoogleSheet>` - A Google Sheet and the one or more columns that contain ChromeOS serial numbers
+* `gdoc(:<FieldName>)+ <UserGoogleDoc>` - A Google Doc and the one or more columns that contain ChromeOS serial numbers
+* `gcscsv(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain ChromeOS serial numbers
+* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain ChromeOS serial numbers
+* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
+* `columndelimiter <Character>` - Columns are  separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
+* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
+* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
+* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
+* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
+* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
+* `delimiter <Character>` - There are multiple serial numbers per column separated by `<Character>`; if not specified, there is single deviceId per column
+
+## ChromeOS devices from OUs in a flat file/Google Doc/Google Cloud Storage Object
+```
+datafile
+   cros|cros_sn|cros_ous|cros_ous_and_children
+   ((<FileName> [charset <Charset>])|
+     (gdoc <UserGoogleDoc>)|
+     (gcsdoc <StorageBucketObjectName>))
+    [delimiter <Character>]
+```
+* `cros|cros_sn|cros_ous|cros_ous_and_children` - The type of item in the file
+* `<FileName>` - A flat file containing a single item per row
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gdoc <UserGoogleDoc>` - A Google Doc containing a single item per row
+* `gcsdoc <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object containing a item per row
+* `delimiter <Character>` - There are multiple items per row separated by `<Character>`; if not specified, there is single item per row
+
+## ChromeOS deviceIds from OUs in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object
+```
+csvdatafile
+   cros|cros_sn|cros_sn|cros_ous|cros_ous_and_children
+   ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+     (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+     (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+     (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+     (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+   [warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
+   [endcsv|(fields <FieldNameList>)]
+   (matchfield|skipfield <FieldName> <RegularExpression>)*
+   [delimiter <Character>]
+```
+* `cros|cros_ous|cros_ous_and_children` - The type of item in the file
+* `<FileName>(:<FieldName>)+` - A CSV file and the one or more columns that contain ChromeOS deviceIds
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gsheet(:<FieldName>)+ <UserGoogleSheet>` - A Google Sheet and the one or more columns that contain ChromeOS deviceIds
+* `gdoc(:<FieldName>)+ <UserGoogleDoc>` - A Google Doc and the one or more columns that contain ChromeOS deviceIds
+* `gcscsv(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain ChromeOS deviceIds
+* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain ChromeOS deviceIds
+* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
+* `columndelimiter <Character>` - Columns are  separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
+* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
+* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
+* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
+* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
+* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
+* `delimiter <Character>` - There are multiple deviceIds per column separated by `<Character>`; if not specified, there is single deviceId per column
+
+## ChromeOS devices directly in or from OUs in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object
+```
+csvkmd
+   cros|cros_sn|cros_ous|cros_ous_and_children
+   ((<FileName>|
+     (gsheet <UserGoogleSheet>)|
+     (gdoc <UserGoogleDoc>)|
+     (gcscsv <StorageBucketObjectName>)|
+     (gcsdoc <StorageBucketObjectName>))
+    [charset <Charset>] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>] [fields <FieldNameList>])
+   keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
+   subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
+   (matchfield|skipfield <FieldName> <RegularExpression>)*
+   [datafield <FieldName>(:<FieldName>)* [delimiter <Character>]]
+```
+* `cros|cros_sn|cros_ous|cros_ous_and_children` - The type of item in the file
+* `<FileName>` - A CSV file containing rows with columns of the type of item specified
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gsheet <UserGoogleSheet>` - A Google Sheet containing rows with columns of the type of item specified
+* `gdoc <UserGoogleDoc>` - A Google Doc containing rows with columns of the type of item specified
+* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
+* `columndelimiter <Character>` - Columns are  separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
+* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
+* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
+* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
+* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
+* `(keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>])+`
+    * `keyfield <FieldName>` - The column containing key values
+    * `[keypattern <RegularExpression>] [keyvalue <String>]` - Allows transforming the value(s) in the `keyfield` column. If only `keyvalue <String>` is specified, all instances of `<FieldName>` in `keyvalue <String>` will be replaced by the item value. If `keypattern <RegularExpression>` is specified, the item value is matched against `<RegularExpression>` and the matched segments are substituted into `keyvalue <String>`
+    * `delimiter <Character>` - There are multiple values per keyfield column separated by `<Character>`; if not specified, there is single value per keyfield column
+* `(subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>])*`
+    * `subkeyfield <FieldName>` - The column containing subkey values
+    * `[keypattern <RegularExpression>] [keyvalue <String>]` - Allows transforming the value(s) in the `subkeyfield` column. If only `keyvalue <String>` is specified, all instances of `<FieldName>` in `keyvalue <String>` will be replaced by the item value. If `keypattern <RegularExpression>` is specified, the item value is matched against `<RegularExpression>` and the matched segments are substituted into `keyvalue <String>`
+    * `delimiter <Character>` - There are multiple values per subkeyfield column separated by `<Character>`; if not specified, there is single value per subkeyfield column
+* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
+* `(datafield <FieldName>(:<FieldName)* [delimiter <Character>])*`
+    * `datafield <FieldName>(:<FieldName)*` - The column(s) containing data values
+    * `delimiter <Character>` - There are multiple values per datafield column separated by `<Character>`; if not specified, there is single value per datafield column
+
+## ChromeOS deviceIds from data fields identified in a `csvkmd` argument
+* `croscsvdata <FieldName>(:<FieldName>*)` - Data fields identified in a `csvkmd` argument
+
+## Examples using CSV files
+
+You want to print information about ChromeOS devices at your school from Org Units based on graduation year.
+
+Example 1
+CSV File OrgUnit.csv, exactly the data you want, `keypattern` and `keyvalue` are not required.
+```
+OrgUnit
+/Students/2020
+/Students/2021
+...
+```
+For each row, the value from the OrgUnit column is used as the Org Unit  name.
+```
+gam csvkmd cros_ous OrgUnit.csv keyfield OrgUnit print cros
+```
+
+Example 2
+CSV File GradYear.csv, you have to convert GradYear to Org Unit name `/Students/GradYear`, `keyvalue` is required.
+```
+GradYear
+2020
+2021
+...
+```
+For each row, the value from the GradYear column replaces the keyField name in the `keyvalue` argument and that value is used as the Org Unit name.
+```
+gam csvkmd cros_ous GradYear.csv keyfield GradYear keyvalue "/Students/GradYear" print cros
+```
+
+Example 3
+CSV File GradYear.csv, you have to convert GradYear to Org Unit name `/Students/LastTwoDigitsOfGradYear`, `keypattern` and `keyvalue` are required.
+```
+GradYear
+2020
+2021
+...
+```
+For each row, the value from the GradYear column is matched against the `keypattern` and the matched segments are substituted into the `keyvalue` argument and that value is used as the Org Unit name.
+```
+gam csvkmd cros_ous GradYear.csv keyfield GradYear keypattern '20(..)' keyvalue '/Students/\1' print cros
+```
+
+## Examples using multiple queries or Org Units
+
+Example 1
+Print information about all ChromeOS devices with a serial number that starts with HY3 or 5CD.
+```
+gam crosqueries "id:HY3,id:5CD" print cros allfields nolists
+```
+
+Example 2
+Print information about all ChromeOS devices in two Org Units that contain spaces in their names.
+```
+gam crosqueries "\"orgUnitPath='/Students/Middle School/2021'\",\"orgUnitPath='/Students/Middle School/2020'\"" print cros allfields nolists
+```
+
+This is equivaluent to:
+```
+gam cros_ous "'/Students/Middle School/2021','/Students/Middle School/2020'" print cros allfields nolists
+```
+
+

docs/Collections-of-Items.md

@@ -0,0 +1,387 @@
+# Collections of Items
+- [Python Regular Expressions](Python-Regular-Expressions) Match function
+- [Definitions](#definitions)
+- [ListSelector](#listselector)
+- [FileSelector](#fileselector)
+- [CSVFileSelector](#csvfileselector)
+- [CSVkmdSelector](#csvkmdselector)
+- [CSVSubkeySelector](#csvsubkeyselector)
+- [CSVDataSelector](#csvdataselector)
+- [Named Collections](#named-collections)
+- [Examples](#examples)
+
+## Definitions
+* [Basic Items](Basic-Items)
+
+* [List Items](List-Items)
+
+* [Command data from Google Docs/Sheets/Storage](Command-Data-From-Google-Docs-Sheets-Storage)
+```
+<StorageBucketName> ::= <String>
+<StorageObjectName> ::= <String>
+<StorageBucketObjectName> ::=
+        https://storage.cloud.google.com/<StorageBucketName>/<StorageObjectName>|
+        https://storage.googleapis.com/<StorageBucketName>/<StorageObjectName>|
+        gs://<StorageBucketName>/<StorageObjectName>|
+        <StorageBucketName>/<StorageObjectName>
+
+<UserGoogleDoc> ::=
+        <EmailAddress> <DriveFileIDEntity>|<DriveFileNameEntity>|(<SharedDriveEntity> <SharedDriveFileNameEntity>)
+<UserGoogleSheet> ::=
+        <EmailAddress> <DriveFileIDEntity>|<DriveFileNameEntity>|(<SharedDriveEntity> <SharedDriveFileNameEntity>) <SheetEntity>
+
+<JSONData> ::= (json [charset <Charset>] <String>) | (json file <FileName> [charset <Charset>]) |
+```
+## ListSelector
+A list of items
+```
+<Item> ::= <String>
+<ItemList> ::= "<Item>(,<Item>)*"
+<ListSelector> ::= list <ItemList>
+```
+
+## FileSelector
+A flat file containing a single Item per row.
+```
+<FileSelector> ::=
+        file ((<FileName> [charset <Charset>])|
+              (gdoc <UserGoogleDoc>)|
+              (gcsdoc <StorageBucketObjectName>))
+             [delimiter <Character>]
+```
+* `<FileName>` - A flat file containing Items
+* `gdoc <UserGoogleDoc>` - A Google Doc containing Items
+* `gcsdoc <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object containing Items
+* `delimiter <Character>` - There are multiple Items per row separated by `<Character>`; if not specified, there is single item per row
+
+## CSVFileSelector
+A CSV file with one or more columns per row that contain Items.
+```
+<CSVFileSelector> ::=
+        csvfile ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+                 (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+                 (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+                 (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+                 (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+                [warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
+                [endcsv|(fields <FieldNameList>)]
+                (matchfield|skipfield <FieldName> <RegularExpression>)*
+                [delimiter <Character>]
+```
+* `<FileName>(:<FieldName>)+` - A CSV file and the one or more columns that contain Items
+* `gsheet(:<FieldName>)+ <UserGoogleSheet>` - A Google Sheet and the one or more columns that contain Items
+* `gdoc(:<FieldName>)+ <UserGoogleDoc>` - A Google Doc and the one or more columns that contain Items
+* `gcscsv(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain Items
+* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain Items
+* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
+* `columndelimiter <Character>` - Columns are  separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
+* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
+* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
+* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
+* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
+* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
+* `delimiter <Character>` - There are multiple Items per column separated by `<Character>`; if not specified, there is single item per column
+
+## CSVkmdSelector
+A CSV file with a key column that contains an Item and optional subkey and data columns that contain data related to the key Item.
+```
+<CSVkmdSelector> ::=
+        csvkmd ((<FileName>|
+                 (gsheet <UserGoogleSheet>)|
+                 (gdoc <UserGoogleDoc>)|
+                 (gcscsv <StorageBucketObjectName>)|
+                 (gcsdoc <StorageBucketObjectName>))
+                 [charset <Charset>] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>] [fields <FieldNameList>])
+                keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
+                subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
+                (matchfield|skipfield <FieldName> <RegularExpression>)*
+                [datafield <FieldName>(:<FieldName>)* [delimiter <Character>]]
+```
+* `<FileName>` - A CSV file containing rows with columns of items
+* `gsheet <UserGoogleSheet>` - A Google Sheet containing rows with columns of items
+* `gdoc <UserGoogleDoc>` - A Google Doc containing rows with columns of items
+* `gcscsv <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object containing rows with columns of items
+* `gcsdoc <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object containing rows with columns of items
+* `columndelimiter <Character>` - Columns are  separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
+* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
+* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
+* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
+* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
+* `(keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>])+`
+    * `keyfield <FieldName>` - The column containing key values
+    * `[keypattern <RegularExpression>] [keyvalue <String>]` - Allows transforming the value(s) in the `keyfield` column. If only `keyvalue <String>` is specified, all instances of `<FieldName>` in `keyvalue <String>` will be replaced by the item value. If `keypattern <RegularExpression>` is specified, the item value is matched against `<RegularExpression>` and the matched segments are substituted into `keyvalue <String>`
+    * `delimiter <Character>` - There are multiple values per keyfield column separated by `<Character>`; if not specified, there is single value per keyfield column
+* `(subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>])*`
+    * `subkeyfield <FieldName>` - The column containing subkey values
+    * `[keypattern <RegularExpression>] [keyvalue <String>]` - Allows transforming the value(s) in the `subkeyfield` column. If only `keyvalue <String>` is specified, all instances of `<FieldName>` in `keyvalue <String>` will be replaced by the item value. If `keypattern <RegularExpression>` is specified, the item value is matched against `<RegularExpression>` and the matched segments are substituted into `keyvalue <String>`
+    * `delimiter <Character>` - There are multiple values per subkeyfield column separated by `<Character>`; if not specified, there is single value per subkeyfield column
+* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
+* `(datafield <FieldName>(:<FieldName)* [delimiter <Character>])*`
+    * `datafield <FieldName>(:<FieldName)*` - The column(s) containing data values
+    * `delimiter <Character>` - There are multiple values per datafield column separated by `<Character>`; if not specified, there is single value per datafield column
+
+## CSVSubkeySelector
+A subkey field identified in a `csvkmd` argument.
+```
+<CSVSubkeySelector> ::=
+        csvsubkey <FieldName>
+```
+
+## CSVDataSelector
+Data fields identified in a `csvkmd` argument.
+```
+<CSVDataSelector> ::=
+        csvdata <FieldName>(:<FieldName)*
+```
+## Named Collections
+```
+<BrowserEntity> ::=
+        <DeviceIDList> |
+        (query:<QueryBrowser>)|(query:orgunitpath:<OrgUnitPath>)|(query <QueryBrowser>) |
+        (browserou <OrgUnitItem>) | (browserous <OrgUnitList>) |
+        <FileSelector> | <CSVFileSelector>
+<CalendarACLScopeEntity> ::=
+        <CalendarACLScopeList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<CalendarEntity> ::=
+        <CalendarList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<ClassroomInvitationIDEntity> ::=
+        <ClassroomInvitationIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<ContactEntity> ::=
+        <ContactIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<ContactGroupEntity> ::=
+        <ContactGroupList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<CourseAliasEntity> ::=
+        <CourseAliasList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<CourseEntity> ::=
+        <CourseIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector>
+<CourseAnnouncementIDEntity> ::=
+        <CourseAnnouncementIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVSubkeySelector> | <CSVDataSelector>
+<CourseSubmissionIDEntity> ::=
+        <CourseSubmissionIDList> | <FileSelector> | <CSVFileSelector> | <CSVDataSelector>
+<CourseTopicIDEntity> ::=
+        <CourseTopicIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVSubkeySelector> | <CSVDataSelector>
+<CourseWorkIDEntity> ::=
+        <CourseWorkIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVSubkeySelector> | <CSVDataSelector>
+<CourseMaterialIDEntity> ::=
+        <CourseMaterialIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVSubkeySelector> | <CSVDataSelector>
+<CrOSEntity> ::=
+        <CrOSIDList> | (cros_sn <SerialNumberList>) |
+        (query:<QueryCrOS>) | (query:orgunitpath:<OrgUnitPath>) | (query <QueryCrOS>)
+<DeviceIDEntity> ::=
+        <DeviceIDList> | (device_sn <SerialNumber>)
+        (query:<QueryDevice>) | (query <QueryDevice>)
+<DeviceFileEntity> ::=
+        <TimeList> |
+        (first|last|allexceptfirst|allexceptlast <Number>) |
+        (before|after <Time>) | (range <Time> <Time>) |
+        <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<DomainNameEntity> ::=
+        <DomainNameList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<DriveFileIDEntity> ::=
+        <DriveFileItem> |
+        (id <DriveFileItem>) | (id:<DriveFileItem>) |
+        (ids <DriveFileList>) | (ids:<DriveFileList>)
+<DriveFileNameEntity> ::=
+        (anyname <DriveFileName>) | (anyname:<DriveFileName>) |
+        (anydrivefilename <DriveFileName>) | (anydrivefilename:<DriveFileName>) |
+        (name <DriveFileName>) | (name:<DriveFileName>) |
+        (drivefilename <DriveFileName>) | (drivefilename:<DriveFileName>) |
+        (othername <DriveFileName>) | (othername:<DriveFileName>) |
+        (otherdrivefilename <DriveFileName>) | (otherdrivefilename:<DriveFileName>)
+<DriveFileQueryEntity> ::=
+        (query <QueryDriveFile>) | (query:<QueryDriveFile>) |
+        (fullquery <QueryDriveFile>)
+<DriveFileQueryShortcut> ::=
+        all_files |
+        all_folders |
+        all_forms |
+        all_google_files |
+        all_non_google_files |
+        all_shortcuts |
+        all_3p_shortcuts |
+        all_items |
+        my_files |
+        my_folders |
+        my_forms |
+        my_google_files |
+        my_non_google_files |
+        my_shortcuts |
+        my_3p_shortcuts |
+        my_items |
+        my_top_files |
+        my_top_folders |
+        my_top_items |
+        others_files |
+        others_folders |
+        others_forms |
+        others_google_files |
+        others_non_google_files |
+        others_shortcuts |
+        others_3p_shortcuts |
+        others_items |
+        writable_files
+<DriveFileEntityShortcut> ::=
+        alldrives |
+        mydrive_any |
+        mydrive_me |
+        mydrive_others |
+        onlyteamdrives|onlyshareddrives |
+        orphans |
+        ownedby_any |
+        ownedby_me |
+        ownedby_others |
+        root | mydrive |
+        rootwithorphans|mydrivewithorphans |
+        sharedwithme_all |
+        sharedwithme_mydrive |
+        sharedwithme_notmydrive
+<DriveFileEntity> ::=
+        <DriveFileIDEntity> |
+        <DriveFileNameEntity> |
+        <DriveFileQueryEntity> |
+        <DriveFileQueryShortcut> |
+        mydrive | mydriveid |
+        root | rootid |
+        <SharedDriveIDEntity> [<SharedDriveFileQueryShortcut>] |
+        <SharedDriveNameEntity> [<SharedDriveFileQueryShortcut>] |
+        <SharedDriveFileNameEntity> |
+        <SharedDriveFileQueryEntity> |
+        <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVSubkeySelector> | <CSVDataSelector>)
+<DriveFilePermissionEntity> ::=
+        <DriveFilePermissionList> |
+        <JSONData> |
+        <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<DriveFilePermissionIDEntity> ::=
+        <DriveFilePermissionIDList> |
+        <JSONData> |
+        <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<DriveFileRevisionIDEntity> ::=
+        (<DriveFileRevisionID>) | (id[ |:]<DriveFileRevisionID>) (ids[ |:]<DriveFileRevisionIDList>)
+        (first|last|allexceptfirst|allexceptlast <Number>)|
+        (before|after <Time>) | (range <Time> <Time>)|
+        <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<DriveLabelNameEntity> ::=
+        <DriveLabelNameList> | <FileSelector> | <CSVFileSelector> | <CSVDataSelector>
+<EmailAddressEntity> ::=
+        <EmailAddressList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<FilterIDEntity> ::=
+        <FilterIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<EventIDEntity> ::=
+        (id|eventid <EventId>) |
+        (event|events <EventIdList> |
+        <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVSubkeySelector> | <CSVDataSelector>)
+<GroupEntity> ::=
+        <GroupList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<GuardianEntity> ::=
+        <GuardianList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<GuardianInvitationIDEntity> ::=
+        <GuardianInvitationIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<LabelIDEntity> ::=
+        <LabelIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<LabelNameEntity> ::=
+        <LabelNameList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<LookerStudioAssetIDEntity> ::=
+        <LookerStudioAssetIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<LookerStudioPermissionEntity> ::=
+        <LookerStudioPermissionList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<MessageIDEntity> ::=
+        <MessageIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<MobileEntity> ::=
+        <ResourceIDList> |
+        (query:<QueryMobile>) | (query <QueryMobile>)
+<NotesNameEntity> ::=
+        <NotesNameList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<OrgUnitEntity> ::=
+        <OrgUnitList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector>
+<OtherContactsResourceNameEntity> ::=
+        <OtherContactsResourceNameNameList> | <FileSelector> | <CSVFileSelector> | <CSVDataSelector>
+<PeopleResourceNameEntity> ::=
+        <PeopleResourceNameList> | <FileSelector> | <CSVFileSelector> | <CSVDataSelector>
+<ProjectIDEntity> ::=
+        current | gam | <ProjectID> | (filter <String>) |
+        (select <ProjectIDList> | <FileSelector> | <CSVFileSelector>)
+<PrinterIDEntity> ::=
+        <PrinterIDList> | <FileSelector> | <CSVFileSelector>
+<RecipientEntity> ::=
+        <EmailAddressEntity> | (select <UserTypeEntity>)
+<ResourceEntity> ::=
+        <ResourceIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector>
+<SchemaEntity> ::=
+        <SchemaNameList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector>
+<SerialNumberEntity> ::=
+        <SerialNumberList> | <FileSelector> | <CSVFileSelector>
+<SharedDriveIDEntity> ::=
+        <DriveFileItem> |
+        (teamdriveid <DriveFileItem>) | (teamdriveid:<DriveFileItem>)
+<SharedDriveNameEntity> ::=
+        (teamdrive <SharedDriveName>) | (teamdrive:<SharedDriveName>)
+<SharedDriveEntity> ::=
+        <SharedDriveIDEntity> |
+        <SharedDriveNameEntity>
+<SharedDriveAdminQueryEntity> ::=
+        (teamdriveadminquery <QueryTeamDrive>) | (teamdriveadminquery:<QueryTeamDrive>)
+<SharedDriveEntityAdmin> ::=
+        <SharedDriveIDEntity> |
+        <SharedDriveNameEntity>|
+        <SharedDriveAdminQueryEntity>
+<SharedDriveFileNameEntity> ::=
+        (teamdrivefilename <DriveFileName>) | (teamdrivefilename:<DriveFileName>)
+<SharedDriveFileQueryEntity> ::=
+        (teamdrivequery <QueryDriveFile>) | (teamdrivequery:<QueryDriveFile>)
+<SharedDriveFileQueryShortcut> ::=
+        all_files | all_folders | all_google_files | all_non_google_files | all_items
+<SiteACLScopeEntity> ::=
+        <SiteACLScopeList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<SiteEntity> ::=
+        <SiteList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<TasklistEntity> ::=
+        <TasklistIDList> | <TaskListTitleList> | <FileSelector> | <CSVFileSelector>
+<TasklistIDTaskIDEntity> ::=
+        <TasklistIDTaskIDList> | <FileSelector> | <CSVFileSelector>
+<ThreadIDEntity> ::=
+        <ThreadIDList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+<UserEntity> ::=
+        <UserList> | <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+```
+## Examples
+
+You want to update the membership of a collection of parent groups at your school, the data is coming from a database in a fixed format.
+
+Example 1, CSV File GroupP1P2.csv, exactly the data you want, `keypattern` and `keyvalue` are not required.
+```
+Group,P1Email,P2Email
+2017-parents@domain.com,g1member11@domain.com,g1member12@domain.com
+2017-parents@domain.com,g1member21@domain.com,g1member22@domain.com
+2018-parents@domain.com,g2member11@domain.com,g2member11@domain.com
+2018-parents@domain.com,g2member21@domain.com,g2member22@domain.com
+...
+```
+For each row, the value from the Group column is used as the group name.
+
+`gam update groups csvkmd GroupP1P2.csv keyfield Group datafield P1Email:P2Email sync member csvdata P1Email:P2Email`
+
+Example 2, CSV File GradYearP1P2.csv, you have to convert GradYear to group name `GradYear-parents@domain.com`, `keyvalue` is required.
+```
+GradYear,P1Email,P2Email
+2017,g1member11@domain.com,g1member12@domain.com
+2017,g1member21@domain.com,g1member22@domain.com
+2018,g2member11@domain.com,g2member11@domain.com
+2018,g2member21@domain.com,g2member22@domain.com
+...
+```
+For each row, the value from the GradYear column replaces the keyField name in the `keyvalue` argument and that value is used as the group name.
+
+`gam update groups csvkmd GradYearP1P2.csv keyfield GradYear keyvalue GradYear-parents@domain.com datafield P1Email:P2Email sync member csvdata P1Email:P2Email`
+
+Example 3, CSV File GradYearP1P2.csv, you have to convert GradYear to group name `LastTwoDigitsOfGradYear-parents@domain.com`, `keypattern` and `keyvalue` are required.
+```
+GradYear,P1Email,P2Email
+2017,g1member11@domain.com,g1member12@domain.com
+2017,g1member21@domain.com,g1member22@domain.com
+2018,g2member11@domain.com,g2member11@domain.com
+2018,g2member21@domain.com,g2member22@domain.com
+...
+```
+For each row, the value from the GradYear column is matched against the `keypattern`, the matched segments are substituted into the `keyvalue` argument and that value is used as the group name.
+
+`gam update groups csvkmd GradYearP1P2.csv keyfield GradYear keypattern '20(..)' keyvalue '\1-parents@domain.com' datafield P1Email:P2Email sync member csvdata P1Email:P2Email`

docs/Collections-of-Users.md

@@ -0,0 +1,681 @@
+# Collections of Users
+- [Python Regular Expressions](Python-Regular-Expressions) Match function
+- [Definitions](#definitions)
+- [List quoting rules](#list-quoting-rules)
+- [User Type Entity](#user-type-entity)
+  - [All non-suspended Users](#all-non-suspended-users)
+  - [All suspended Users](#all-suspended-Users)
+  - [All non-suspended and suspended Users](#all-non-suspended-and-suspended-users)
+  - [A single User](#a-single-user)
+  - [A list of Users](#a-list-of-users)
+  - [The admin user referenced in oauth2.txt](#the-admin-user-referenced-in-oauth2txt)
+  - [Users in the domains `<DomainNameList>`](#users-in-the-domains-domainnamelist)
+  - [Users directly in the group `<GroupItem>`](#users-directly-in-the-group-groupitem)
+  - [Users directly in the groups `<GroupList>`](#users-directly-in-the-groups-grouplist)
+  - [Users directly and indirectly in the group `<GroupItem>`](#users-directly-and-indirectly-in-the-group-groupitem)
+  - [Users directly and indirectly in the groups `<GroupList>`](#users-directly-and-indirectly-in-the-groups-grouplist)
+  - [Selected Users from groups](#selected-users-from-groups)
+  - [Users directly in the Cloud Identity group `<GroupItem>`](#users-directly-in-the-cloud-identity-group-groupitem)
+  - [Users directly in the Cloud Identity groups `<GroupList>`](#users-directly-in-the-cloud-identity-groups-grouplist)
+  - [Selected Users from Cloud Identity groups](#selected-users-from-cloud-identity-groups)
+  - [Users directly in the Organization Unit `<OrgUnitItem>`](#users-directly-in-the-organization-unit-orgunititem)
+  - [Users in the Organization Unit `<OrgUnitItem>` and all of its sub Organization Units](#users-in-the-organization-unit-orgunititem-and-all-of-its-sub-organization-units)
+  - [Users directly in the Organization Units `<OrgUnitList>`](#users-directly-in-the-organization-units-orgunitlist)
+  - [Users in the Organization Units `<OrgUnitList>` and all of their sub Organization Units](#users-in-the-organization-units-orgunitlist-and-all-of-their-sub-organization-units)
+  - [All of the students and teachers in the courses specified in `<CourseIDList>`](#all-of-the-students-and-teachers-in-the-courses-specified-in-courseidlist)
+  - [All of the students in the courses specified in `<CourseIDList>`](#all-of-the-students-in-the-courses-specified-in-courseidlist)
+  - [All of the teachers in the courses specified in `<CourseIDList>`](#all-of-the-teachers-in-the-courses-specified-in-courseidlist)
+  - [All Users with any of the licenses specified in `<SKUIDList>`](#all-users-with-any-of-the-licenses-specified-in-skuidlist)
+  - [Users that match a query](#users-that-match-a-query)
+  - [Users that match any query in a list of queries](#users-that-match-any-query-in-a-list-of-queries)
+  - [Users in a flat file/Google Doc/Google Cloud Storage Object](#users-in-a-flat-filegoogle-docgoogle-cloud-storage-object)
+  - [Selected users in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object](#selected-users-in-a-csv-filegoogle-sheetgoogle-docgoogle-cloud-storage-object)
+  - [Users from groups/OUs/courses in a flat file/Google Doc/Google Cloud Storage Object](#users-from-groupsouscourses-in-a-flat-filegoogle-docgoogle-cloud-storage-object)
+  - [Users from groups/OUs/courses in a CSV file/Google Sheet/Google Doc](#users-from-groupsouscourses-in-a-csv-filegoogle-sheetgoogle-docgoogle-cloud-storage-object)
+  - [Users directly in or from groups/OUs/courses in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object](#users-directly-in-or-from-groupsouscourses-in-a-csv-filegoogle-sheetgoogle-docgoogle-cloud-storage-object)
+  - [Users from data fields identified in a `csvkmd` argument](#users-from-data-fields-identified-in-a-csvkmd-argument)
+- [Examples using CSV files and Google Sheets to update the membership of a group](#examples-using-csv-files-and-google-sheets-to-update-the-membership-of-a-group)
+- [Examples using CSV files to print users from groups](#examples-using-CSV-files-to-print-users-from-groups)
+- [Examples using multiple queries](#examples-using-multiple-queries)
+
+## Definitions
+* [Basic Items](Basic-Items)
+
+* [List Items](List-Items)
+
+* [Command data from Google Docs/Sheets/Storage](Command-Data-From-Google-Docs-Sheets-Storage)
+```
+<StorageBucketName> ::= <String>
+<StorageObjectName> ::= <String>
+<StorageBucketObjectName> ::=
+        https://storage.cloud.google.com/<StorageBucketName>/<StorageObjectName>|
+        https://storage.googleapis.com/<StorageBucketName>/<StorageObjectName>|
+        gs://<StorageBucketName>/<StorageObjectName>|
+        <StorageBucketName>/<StorageObjectName>
+
+<UserGoogleDoc> ::=
+        <EmailAddress> <DriveFileIDEntity>|<DriveFileNameEntity>|(<SharedDriveEntity> <SharedDriveFileNameEntity>)
+<UserGoogleSheet> ::=
+        <EmailAddress> <DriveFileIDEntity>|<DriveFileNameEntity>|(<SharedDriveEntity> <SharedDriveFileNameEntity>) <SheetEntity>
+```
+```
+<DriveFileID> ::= <String>
+<DriveFileURL> ::=
+        https://drive.google.com/open?id=<DriveFileID>
+        https://drive.google.com/drive/files/<DriveFileID>
+        https://drive.google.com/drive/folders/<DriveFileID>
+        https://drive.google.com/drive/folders/<DriveFileID>?resourcekey=<String>
+        https://drive.google.com/file/d/<DriveFileID>/<String>
+        https://docs.google.com>/document/d/<DriveFileID>/<String>
+        https://docs.google.com>/drawings/d/<DriveFileID>/<String>
+        https://docs.google.com>/forms/d/<DriveFileID>/<String>
+        https://docs.google.com>/presentation/d/<DriveFileID>/<String>
+        https://docs.google.com>/spreadsheets/d/<DriveFileID>/<String>
+<DriveFileItem> ::= <DriveFileID>|<DriveFileURL>
+<DriveFileList> ::= "<DriveFileItem>(,<DriveFileItem>)*"
+<DriveFileName> ::= <String>
+<DriveFileIDEntity> ::=
+        (<DriveFileItem>)|(id( |:)<DriveFileItem>)|(ids( |:)<DriveFileList>)
+<DriveFileNameEntity> ::=
+        (drivefilename <DriveFileName>)|(drivefilename:<DriveFileName>)|
+        (anydrivefilename <DriveFileName>)|(anydrivefilename:<DriveFileName>)
+<SharedDriveID> ::= <String>
+<SharedDriveName> ::= <String>
+<SharedDriveIDEntity> ::= (teamdriveid <DriveFileItem>) | (teamdriveid:<DriveFileItem>)
+<SharedDriveNameEntity> ::= (teamdrive <SharedDriveName>) | (teamdrive:<SharedDriveName>)
+<SharedDriveFileNameEntity> ::= (teamdrivefilename <DriveFileName>) | (teamdrivefilename:<DriveFileName>)
+<SharedDriveEntity> ::=
+        <SharedDriveIDEntity> |
+        <SharedDriveNameEntity>
+
+<SheetEntity> ::= <String>|id:<Number>
+
+<UserTypeEntity> ::=
+        (all users|users_ns|users_susp|users_ns_susp)|
+        (user <UserItem>)|
+        (users <UserList>)|
+        (oauthuser)
+        (domains|domains_ns|domains_susp <DomainNameList>)|
+        (group|group_ns|group_susp|group_inde <GroupItem>)|
+        (groups|groups_ns|groups_susp|groups_inde <GroupList>)|
+        (group_inde <GroupItem>)|(groups_inde <GroupList>)|
+        (group_users|group_users_ns|group_users_susp <GroupList>
+                [members] [managers] [owners]
+                [primarydomain] [domains <DomainNameList>] [recursive|includederivedmembership] end)|
+        (group_users_select <GroupList>
+                [members] [managers] [owners]
+                [notsuspended|suspended] [notarchived|archived]
+                [primarydomain] [domains <DomainNameList>] [recursive|includederivedmembership] end)|
+        (ou|ou_ns|ou_susp <OrgUnitItem>)|
+        (ou_and_children|ou_and_children_ns|ou_and_children_susp <OrgUnitItem>)|
+        (ous|ous_ns|ous_susp <OrgUnitList>)|
+        (ous_and_children|ous_and_children_ns|ous_and_children_susp <OrgUnitList>)|
+        (courseparticipants <CourseIDList>)|
+        (students <CourseIDList>)|
+        (teachers <CourseIDList>)|
+        (license|licenses|licence|licences <SKUIDList>)|
+        (query <QueryUser>)|
+        (queries <QueryUserList>)|
+        (file
+            ((<FileName> [charset <Charset>])|
+             (gdoc <UserGoogleDoc>)|
+             (gcsdoc <StorageBucketObjectName>))
+            [delimiter <Character>])|
+        (csvfile
+            ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+             (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+             (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+             (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+             (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+            [warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>]
+            [endcsv|(fields <FieldNameList>)]
+            (matchfield|skipfield <FieldName> <RegularExpression>)*
+            [delimiter <Character>])|
+        (datafile
+            users|groups|groups_ns|groups_susp|groups_inde|ous|ous_ns|ous_susp|
+            ous_and_children|ous_and_children_ns|ous_and_children_susp|
+            courseparticipants|students|teachers
+            ((<FileName> [charset <Charset>])|
+              (gdoc <UserGoogleDoc>)|
+              (gcsdoc <StorageBucketObjectName>))
+             [delimiter <Character>])|
+        (csvdatafile
+            users|groups|groups_ns|groups_susp|groups_inde|ous|ous_ns|ous_susp|
+            ous_and_children|ous_and_children_ns|ous_and_children_susp|
+            courseparticipants|students|teachers
+            ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+              (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+              (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+              (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+              (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+            [warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>]
+            [endcsv|(fields <FieldNameList>)]
+            (matchfield|skipfield <FieldName> <RegularExpression>)*
+            [delimiter <Character>])|
+        (csvkmd
+            users|groups|groups_ns|groups_susp|groups_inde|ous|ous_ns|ous_susp|
+            ous_and_children|ous_and_children_ns|ous_and_children_susp|
+            courseparticipants|students|teachers
+            ((<FileName>|
+              (gsheet <UserGoogleSheet>)|
+              (gdoc <UserGoogleDoc>)|
+              (gcscsv <StorageBucketObjectName>)|
+              (gcsdoc <StorageBucketObjectName>))
+             [charset <Charset>] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>] [fields <FieldNameList>])
+            keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
+            subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
+            (matchfield|skipfield <FieldName> <RegularExpression>)*
+            [datafield <FieldName>(:<FieldName>)* [delimiter <Character>]])
+        (csvdata <FieldName>(:<FieldName>*))
+```
+## List quoting rules
+Items in a list can be separated by commas or spaces; if an item itself contains a comma, a space or a single quote, special quoting must be used.
+Typically, you will enclose the entire list in double quotes and quote each item in the list as detailed below.
+
+- Items, separated by commas, without spaces, commas or single quotes in the items themselves
+   * ```"item,item,item"```
+- Items, separated by spaces, without spaces, commas or single quotes in the items themselves
+   * ```"item item item"```
+- Items, separated by commas, with spaces, commas or single quotes in the items themselves
+   * ```"'it em','it,em',\"it'em\""```
+- Items, separated by spaces, with spaces, commas or single quotes in the items themselves
+   * ```"'it em' 'it,em' \"it'em\""```
+
+Typical places where these rules apply are lists of OUs and Contact Groups.
+## User Type Entity
+
+Use these options to select users for GAM commands.
+
+## All non-suspended Users
+* `all users`
+* `all users_ns`
+
+## All suspended Users
+* `all users_susp`
+
+## All non-suspended and suspended Users
+* `all users_ns_susp`
+
+## A single User
+* `user <UserItem>`
+
+## A list of Users
+* `users <UserList>`
+
+## The admin user referenced in oauth2.txt
+* `oauthuser`
+
+## Users in the domains `<DomainNameList>`
+* `domains|domains_ns|domains_susp <DomainNameList>`
+    * `domains` - All users
+    * `domains_ns` - Non-suspended users
+    * `domains_susp` - Suspended users
+
+## Users directly in the group `<GroupItem>`
+* `group|group_ns|group_susp <GroupItem>`
+    * `group` - All user members
+    * `group_ns` - Non-suspended user members
+    * `group_susp` - Suspended user members
+
+## Users directly in the groups `<GroupList>`
+* `groups|groups_ns|groups_susp <GroupList>`
+    * `groups` - All user members
+    * `groups_ns` - Non-suspended user members
+    * `groups_susp` - Suspended user members
+
+## Users directly and indirectly in the group `<GroupItem>`
+    * `group_inde` - All user members including those from all subgroups
+
+## Users directly and indirectly in the groups `<GroupList>`
+    * `groups_inde` - All user members including those from all subgroups
+
+## Selected Users from groups
+* `group_users|group_users_ns|group_users_susp <GroupList> [members] [managers] [owners] [primarydomain] [domains <DomainNameList>] [recursive|includederivedmembership] end`
+    * `group_users` - All user members
+    * `group_users_ns` - Non-suspended user members
+    * `group_users_susp` - Suspended user members
+    * `[members] [managers] [owners]` - The desired roles; if roles are not specified, all roles are included
+    * `primarydomain` - Select Users from the primary domain
+    * `domains <DomainNameList>` - Select Users from the list of domains
+    * `recursive` - Select Users from all subgroups; do not select Users from a member of type CUSTOMER (all users in a domain); GAM performs the recursion
+    * `includederivedmembership` - Select Users from all subgroups; do select Users from a member of type CUSTOMER (all users in a domain); the API performs the recursion but produces inconsistent results, use with caution
+    * `end` - Terminate the selection
+* `group_users_select <GroupList> [members] [managers] [owners] [notsuspended|suspended] [notarchived|archived] [primarydomain] [domains <DomainNameList>] [recursive|includederivedmembership] end`
+    * `[members] [managers] [owners]` - The desired roles; if roles are not specified, all roles are included
+    * By default, memebers of all statuses are included
+      * `notsuspended` - Do not include suspended users, this is common
+      * `suspended` - Only include suspended users, this is not common but allows creating groups that allow easy identification of suspended users
+      * `notarchived` - Do not include archived members
+      * `archived` - Only include archived members, this is not common but allows creating groups that allow easy identification of archived users
+      * `notsuspended notarchived` - Do not include suspended and archived members
+      * `suspended archived` - Include only suspended or archived members
+      * `notsuspended archived` - Only include archived members, this is not common but allows creating groups that allow easy identification of archived users
+      * `suspended notarchived` - Only include suspended users, this is not common but allows creating groups that allow easy identification of suspended users
+    * `primarydomain` - Select Users from the primary domain
+    * `domains <DomainNameList>` - Select Users from the list of domains
+    * `recursive` - Select Users from all subgroups; do not select Users from a member of type CUSTOMER (all users in a domain); GAM performs the recursion
+    * `includederivedmembership` - Select Users from all subgroups; do select Users from a member of type CUSTOMER (all users in a domain); the API performs the recursion but produces inconsistent results, use with caution
+    * `end` - Terminate the selection
+
+## Users directly in the Cloud Identity group `<GroupItem>`
+* `cigroup <GroupItem>`
+    * `cigroup` - All user members
+
+## Users directly in the Cloud Identity groups `<GroupList>`
+* `cigroups <GroupList>`
+    * `cigroups` - All user members
+
+## Selected Users from Cloud Identity groups
+* `cigroup_users <GroupList> [members] [managers] [owners>] [recursive] end`
+    * `cigroup_users` - All user members
+    * `[members] [managers] [owners]` - The desired roles; if roles are not specified, all roles are included
+    * `recursive` - Select Users from all subgroups; do not select Users from a member of type CUSTOMER (all users in a domain); GAM performs the recursion
+    * `end` - Terminate the selection
+
+## Users directly in the Organization Unit `<OrgUnitItem>`
+* `ou|ou_ns|ou_susp <OrgUnitItem>`
+    * `ou` - All users
+    * `ou_ns` - Non-Suspended users
+    * `ou_susp` - Suspended users
+
+## Users in the Organization Unit `<OrgUnitItem>` and all of its sub Organization Units
+* `ou_and_children|ou_and_children_ns|ou_and_children_susp <OrgUnitItem>`
+    * `ou_and_children` - All  users
+    * `ou_and_children_ns` - Non-suspended users
+    * `ou_and_children_susp` - Suspended users
+
+## Users directly in the Organization Units `<OrgUnitList>`
+* `ous|ous_ns|ous_susp <OrgUnitList>` - Users directly in the Organization Units `<OrgUnitList>`
+    * `ous` - All users
+    * `ous_ns` - Non-suspended users
+    * `ous_susp` - Suspended users
+
+`<OrgUnitList>` may require special quoting based on whether the OUs contain spaces, commas or single quotes.
+
+For quoting rules, see: [List Items](List-Items)
+
+## Users in the Organization Units `<OrgUnitList>` and all of their sub Organization Units
+* `ous_and_children|ous_and_children_ns|ous_and_children_susp <OrgUnitList>` - Users in the Organization Units `<OrgUnitList>` and all of their sub Organization Units
+    * `ous_and_children` - All users
+    * `ous_and_children_ns` - Non-suspended users
+    * `ous_and_children_susp` - Suspended users
+
+`<OrgUnitList>` may require special quoting based on whether the OUs contain spaces, commas or single quotes.
+
+For quoting rules, see: [List Items](List-Items)
+
+## All of the students and teachers in the courses specified in `<CourseIDList>`
+* `courseparticipants <CourseIDList>`
+
+## All of the students in the courses specified in `<CourseIDList>`
+* `students <CourseIDList>`
+
+## All of the teachers in the courses specified in `<CourseIDList>`
+* `teachers <CourseIDList>`
+
+## All Users with any of the licenses specified in `<SKUIDList>`
+* `license|licenses|licence|licences <SKUIDList>`
+
+## Users that match a query
+* `query <QueryUser>`
+
+See https://developers.google.com/admin-sdk/directory/v1/guides/search-users
+
+## Users that match any query in a list of queries
+* `queries <QueryUserList>`
+
+See https://developers.google.com/admin-sdk/directory/v1/guides/search-users
+
+`<QueryUserList>` may require special quoting based on whether the queries contain spaces, commas or single quotes.
+
+* Surround `<QueryCrOSList>` with `" "`
+* Surround each query with `\" \"`, separate the queries with commas.
+
+```
+queries "\"orgUnitPath='/Path/To/OU 1' isSuspended=False\",\"orgUnitPath='/Path/To/OU 2' isSuspended=False\",\"orgUnitPath='/Path/To/OU 3' isSuspended=False\""
+```
+Note that the results are all users who match one or more of the queries. In other words this is "OR" logic, and you get the union of all matching results.
+
+For quoting rules, see: [List Items](List-Items)
+
+## Users in a flat file/Google Doc/Google Cloud Storage Object
+```
+file
+   ((<FileName> [charset <Charset>])|
+    (gdoc <UserGoogleDoc>)|
+    (gcsdoc <StorageBucketObjectName>))
+   [delimiter <Character>]
+```
+* `<FileName>` - A flat file containing a single User per row
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gdoc <UserGoogleDoc>` - A Google Doc containing a single User per row
+* `gcsdoc <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object containing a single User per row
+* `delimiter <Character>` - There are multiple Users per row separated by `<Character>`; if not specified, there is single user per row
+
+## Selected users in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object
+```
+csvfile
+   ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+    (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+    (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+    (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+    (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+   [warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>]
+   [endcsv|(fields <FieldNameList>)]
+   (matchfield|skipfield <FieldName> <RegularExpression>)*
+   [delimiter <Character>]
+```
+* `<FileName>(:<FieldName>)+` - A CSV file and the one or more columns that contain Users
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gsheet(:<FieldName>)+ <UserGoogleSheet>` - A Google Sheet and the one or more columns that contain Users
+* `gdoc(:<FieldName>)+ <UserGoogleDoc>` - A Google Doc and the one or more columns that contain Users
+* `gcscsv(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain Users
+* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain Users
+* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
+* `columndelimiter <Character>` - Columns are  separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
+* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
+* `quotechar <Character>` - The column quote character is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
+* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
+* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
+* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
+* `delimiter <Character>` - There are multiple Users per column separated by `<Character>`; if not specified, there is single user per column
+
+## Users from groups/OUs/courses in a flat file/Google Doc/Google Cloud Storage Object
+```
+datafile
+   users|groups|groups_ns|groups_susp|groups_inde|ous|ous_ns|ous_susp|
+   ous_and_children|ous_and_children_ns|ous_and_children_susp|
+   courseparticipants|students|teachers
+   ((<FileName> [charset <Charset>])|
+     (gdoc <UserGoogleDoc>)|
+     (gcsdoc <StorageBucketObjectName>))
+    [delimiter <Character>]
+```
+* `users|groups|groups_ns|groups_susp|groups_inde|ous|ous_ns|ous_susp|ous_and_children|ous_and_children_ns|ous_and_children_susp|courseparticipants|students|teachers` - The type of item in the file
+* `<FileName>` - A flat file containing rows of the type of item specified
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gdoc <UserGoogleDoc>` - A Google Doc containing rows of the type of item specified
+* `gcsdoc <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object containing rows of the type of item specified
+* `delimiter <Character>` - There are multiple items per row separated by `<Character>`; if not specified, there is single item per row
+
+## Users from groups/OUs/courses in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object
+```
+csvdatafile
+   users|groups|groups_ns|groups_susp|groups_inde|ous|ous_ns|ous_susp|
+   ous_and_children|ous_and_children_ns|ous_and_children_susp|
+   courseparticipants|students|teachers
+   ((<FileName>(:<FieldName>)+ [charset <Charset>] )|
+     (gsheet(:<FieldName>)+ <UserGoogleSheet>)|
+     (gdoc(:<FieldName>)+ <UserGoogleDoc>)|
+     (gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
+     (gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
+   [warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>]
+   [endcsv|(fields <FieldNameList>)]
+   (matchfield|skipfield <FieldName> <RegularExpression>)*
+   [delimiter <Character>]
+```
+* `users|groups|groups_ns|groups_susp|groups_inde|ous|ous_ns|ous_susp|ous_and_children|ous_and_children_ns|ous_and_children_susp|courseparticipants|students|teachers` - The type of item in the file
+* `<FileName>(:<FieldName>)+` - A CSV file and the one or more columns contain the type of item specified
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gsheet(:<FieldName>)+ <UserGoogleSheet>` - A Google Sheet and the one or more columns contain the type of item specified
+* `gdoc(:<FieldName>)+ <UserGoogleDoc>` - A Google Doc and the one or more columns contain the type of item specified
+* `gcscsv(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns contain the type of item specified
+* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns contain the type of item specified
+* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
+* `columndelimiter <Character>` - Columns are  separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
+* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
+* `quotechar <Character>` - The column quote character is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
+* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
+* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
+* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
+* `delimiter <Character>` - There are multiple Users per column separated by `<Character>`; if not specified, there is single user per column
+
+## Users directly in or from groups/OUs/courses in a CSV file/Google Sheet/Google Doc/Google Cloud Storage Object
+```
+csvkmd
+   users|groups|groups_ns|groups_susp|groups_inde|ous|ous_ns|ous_susp|
+   ous_and_children|ous_and_children_ns|ous_and_children_susp|
+   courseparticipants|students|teachers
+   ((<FileName>|
+     (gsheet <UserGoogleSheet>)|
+     (gdoc <UserGoogleDoc>)|
+     (gcscsv <StorageBucketObjectName>)|
+     (gcsdoc <StorageBucketObjectName>))
+    [charset <Charset>] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>] [fields <FieldNameList>])
+   keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
+   subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
+   (matchfield|skipfield <FieldName> <RegularExpression>)*
+            [datafield <FieldName>(:<FieldName>)* [delimiter <Character>]]
+```
+* `users|groups|groups_ns_|groups_susp|groups_inde|ous|ous_ns|ous_susp|ous_and_children|ous_and_children_ns|ous_and_children_susp|courseparticipants|students|teachers` - The type of item in the file
+* `<FileName>` - A CSV file containing rows with columns of the type of item specified
+  * `charset <Charset>` - The character aset of the file if it isn't UTF-8
+* `gsheet <UserGoogleSheet>` - A Google Sheet containing rows with columns of the type of item specified
+* `gdoc <UserGoogleDoc>` - A Google Doc containing rows with columns of the type of item specified
+* `gcscsv <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object with columns of the type of item specified
+* `gcsdoc <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object with columns of the type of item specified
+* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
+* `columndelimiter <Character>` - Columns are  separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
+* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
+* `quotechar <Character>` - The column quote character is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
+* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
+* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
+* `(keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>])+`
+    * `keyfield <FieldName>` - The column containing key values
+    * `[keypattern <RegularExpression>] [keyvalue <String>]` - Allows transforming the value(s) in the `keyfield` column. If only `keyvalue <String>` is specified, all instances of `<FieldName>` in `keyvalue <String>` will be replaced by the item value. If `keypattern <RegularExpression>` is specified, the item value is matched against `<RegularExpression>` and the matched segments are substituted into `keyvalue <String>`
+    * `delimiter <Character>` - There are multiple values per keyfield column separated by `<Character>`; if not specified, there is single value per keyfield column
+* `(subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>])*`
+    * `subkeyfield <FieldName>` - The column containing subkey values
+    * `[keypattern <RegularExpression>] [keyvalue <String>]` - Allows transforming the value(s) in the `subkeyfield` column. If only `keyvalue <String>` is specified, all instances of `<FieldName>` in `keyvalue <String>` will be replaced by the item value. If `keypattern <RegularExpression>` is specified, the item value is matched against `<RegularExpression>` and the matched segments are substituted into `keyvalue <String>`
+    * `delimiter <Character>` - There are multiple values per subkeyfield column separated by `<Character>`; if not specified, there is single value per subkeyfield column
+* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
+* `(datafield <FieldName>(:<FieldName)* [delimiter <Character>])*`
+    * `datafield <FieldName>(:<FieldName)*` - The column(s) containing data values
+    * `delimiter <Character>` - There are multiple values per datafield column separated by `<Character>`; if not specified, there is single value per datafield column
+
+## Users from data fields identified in a `csvkmd` argument
+* `csvdata <FieldName>(:<FieldName>*)`
+
+## Examples using CSV files and Google Sheets to update the membership of a group
+
+### Example 1
+The file Users.csv has a single column of email addresses, there is no header row.
+```
+user1@domain.com
+user2@domain.com
+...
+
+gam update group group@domain.com sync members file Users.csv
+```
+
+The Google Sheet `user@domain.com <DriveFileID> <SheetEntity>` has a single column of email addresses, there is no header row.
+Define an implicit header with the `fields Email` option.
+```
+user1@domain.com
+user2@domain.com
+...
+
+gam update group group@domain.com sync members csvfile gsheet:Email user@domain.com <DriveFileID> <SheetEntity> fields Email
+```
+
+The Google Doc `user@domain.com <DriveFileID>` has a single column of email addresses, there is no header row.
+```
+user1@domain.com
+user2@domain.com
+...
+
+gam update group group@domain.com sync members file gdoc user@domain.com <DriveFileID>
+```
+
+### Example 2
+The CSV file Users.csv has one column of email addresses labelled Email.
+```
+Email
+user1@domain.com
+user2@domain.com
+...
+
+gam update group group@domain.com sync members csvfile Users.csv:Email
+```
+
+The Google Sheet `user@domain.com <DriveFileID> <SheetEntity>` has one column of email addresses labelled Email.
+```
+Email
+user1@domain.com
+user2@domain.com
+...
+
+gam update group group@domain.com sync members csvfile gsheet:Email user@domain.com <DriveFileID> <SheetEntity>
+```
+
+### Example 3
+The CSV file Users.csv has two columns of email addresses labelled Email1 and Email2.
+```
+Email1,Email2
+user1@domain.com,user2@domain.com
+user3@domain.com,user4@domain.com
+...
+
+gam update group group@domain.com sync members csvfile Users.csv:Email1:Email2
+```
+
+The Google Sheet `user@domain.com <DriveFileID> <SheetEntity>` has two columns of email addresses labelled Email1 and Email2.
+```
+Email1,Email2
+user1@domain.com,user2@domain.com
+user3@domain.com,user4@domain.com
+...
+
+gam update group group@domain.com sync members csvfile gsheet:Email1:Email2 user@domain.com <DriveFileID> <SheetEntity>
+```
+
+### Example 4
+The file Groups.txt has a single column of group email addresses, there is no header row.
+You want to sync with the members of those groups.
+```
+group1@domain.com
+group2@domain.com
+...
+
+gam update group group@domain.com sync members datafile groups Groups.txt
+```
+
+The Google Doc `user@domain.com <DriveFileID>` has a single column of group email addresses, there is no header row.
+You want to sync with the members of those groups.
+```
+group1@domain.com
+group2@domain.com
+...
+
+gam update group group@domain.com sync members datafile groups gdoc user@domain.com <DriveFileID>
+```
+
+### Example 5
+The CSV file Groups.csv has a single column of group email addresses labelled Group.
+You want to sync with the members of those groups.
+```
+Group
+group1@domain.com
+group2@domain.com
+...
+
+gam update group group@domain.com sync members csvdatafile groups Groups.csv:Group
+```
+
+The Google Sheet `user@domain.com <DriveFileID> <SheetEntity>` has a single column of group email addresses labelled Group.
+You want to sync with the members of those groups.
+```
+Group
+group1@domain.com
+group2@domain.com
+...
+
+gam update group group@domain.com sync members csvdatafile groups gsheet:Group user@domain.com <DriveFileID> <SheetEntity>
+```
+
+### Example 6
+The CSV file GroupMembers.csv has headers: group,role,email
+
+Each row contains a group email address, member role (OWNER, MEMBER, MANAGER) and a member email address.
+
+The following command will synchronize the membership for all groups and roles.
+```
+gam redirect stdout ./MemberUpdates.txt redirect stderr stdout update group csvkmd GroupMembers.csv keyfield group subkeyfield role datafield email sync csvdata email
+```
+
+The Google Sheet `user@domain.com <DriveFileID> <SheetEntity>` has headers: group,role,email
+
+Each row contains a group email address, member role (OWNER, MEMBER, MANAGER) and a member email address.
+
+The following command will synchronize the membership for all groups and roles.
+```
+gam redirect stdout ./MemberUpdates.txt redirect stderr stdout update group csvkmd gsheet user@domain.com <DriveFileID> <SheetEntity> keyfield group subkeyfield role datafield email sync csvdata email
+```
+
+## Examples using CSV files to print users from groups
+
+You want to print the membership of a collection of parent groups at your school based on graduation year.
+
+### Example 1
+The CSV File Group.csv has exactly the data you want, `keypattern` and `keyvalue` are not required.
+```
+Group
+2020-parents@domain.com
+2021-parents@domain.com
+...
+```
+For each row, the value from the Group column is used as the group name.
+```
+gam csvkmd groups Group.csv keyfield Group print users
+```
+
+### Example 2
+The CSV File GradYear.csv has graduation years; you have to convert GradYear to group name `GradYear-parents@domain.com`, `keyvalue` is required.
+```
+GradYear
+2020
+2021
+...
+```
+For each row, the value from the GradYear column replaces the keyField name in the `keyvalue` argument and that value is used as the group name.
+```
+gam csvkmd group GradYear.csv keyfield GradYear keyvalue GradYear-parents@domain.com print users
+```
+
+### Example 3
+The CSV File GradYear.csv has graduation years; you have to convert GradYear to group name `LastTwoDigitsOfGradYear-parents@domain.com`, `keypattern` and `keyvalue` are required.
+```
+GradYear
+2020
+2021
+...
+```
+For each row, the value from the GradYear column is matched against the `keypattern` and the matched segments are substituted into the `keyvalue` argument and that value is used as the group name.
+```
+gam csvkmd group GradYear.csv keyfield GradYear keypattern '20(..)' keyvalue '\1-parents@domain.com' print users
+```
+
+## Examples using multiple queries
+
+### Example 1
+Print users who are specialists or technicians:
+```
+gam queries "orgTitle=Specialist,orgTitle=Technician" print users allfields
+```
+
+### Example 2
+Print users who are have the title Manager in the sales org or anyone in the marketing org:
+```
+gam queries "\"orgName='Sales Org' orgTitle=Manager\",\"orgName='Marketing Org'\"" print users allfields
+````
+
+### Example 3
+Print users in either of two Org Units that contain spaces in their names.
+```
+gam queries "\"orgUnitPath='/Students/Middle School/2021'\",\"orgUnitPath='/Students/Middle School/2020'\"" print users allfields
+```
+
+This is equivaluent to:
+```
+gam ous "'/Students/Middle School/2021','/Students/Middle School/2020'" print users allfields
+```

docs/Command-Data-From-Google-Docs-Sheets-Storage.md

@@ -0,0 +1,109 @@
+# Command data from Google Docs, Sheets and Cloud Storage
+- [Introduction](#introduction)
+- [Definitions](#definitions)
+- [Read data from a Google Doc or Drive File](#read-data-from-a-google-doc-or-drive-file)
+  - [Plain Text](#plain-text)
+  - [HTML](#html)
+- [Read data from a Google Sheet](#read-data-from-a-google-sheet)
+- [Read data from a Google Cloud Storage File](#read-data-from-a-google-cloud-storage-file)
+  - [Plain Text](#plain-text)
+  - [CSV](#csv)
+  - [HTML](#html)
+
+## Introduction
+Google Sheets can be used in `gam csv ...` commands.
+* [Bulk Processing](Bulk-Processing)
+
+Google Docs and Sheets can be used to specify collections of data.
+* [Collections of ChromeOS Devices](Collections-of-ChromeOS-Devices)
+* [Collections of Items](Collections-of-Items)
+* [Collections of Users](Collections-of-Users)
+
+Google Docs and Drive Files can be used to specify notes, messages and signatures.
+* [Domain Shared Contacts - Global Address List](Contacts-GAL)
+* [Send Email](Send-Email)
+* [Users](Users)
+* [Users - Contacts](Users-Contacts)
+* [Users - Gmail - Messages/Threads](Users-Gmail-Messages-Threads)
+* [Users - Gmail - SendAs/Signature/Vacation](Users-Gmail-Send-As-Signature-Vacation)
+
+## Definitions
+* [Drive Items](Drive-Items)
+
+## Read data from a Google Doc or Drive File
+```
+<UserGoogleDoc> ::=
+        <EmailAddress> <DriveFileIDEntity>|<DriveFileNameEntity>|(<SharedDriveEntity> <SharedDriveFileNameEntity>)
+```
+* `<EmailAddress>` - The email address of a user with at least read access to the document
+
+Use one of the following to specify the file:
+* `<DriveFileIDEntity>` - The ID of the file on a Drive or Shared Drive
+* `<DriveFileNameEntity>` - The name of the file
+* `<SharedDriveEntity> <SharedDriveFileNameEntity>` - A Shared Drive and the name of the file on that drive
+
+## Plain Text
+Interpret a Google Doc as plain text or read a Drive file with MIME type text/plain.
+```
+gdoc <UserGoogleDoc>
+```
+
+## HTML
+Read a Drive file with MIME type text/html.
+```
+ghtml <UserGoogleDoc>
+```
+
+## Read data from a Google Sheet
+```
+<SheetEntity> ::= <String>|id:<Number>
+
+<UserGoogleSheet> ::=
+        <EmailAddress> <DriveFileIDEntity>|<DriveFileNameEntity>|(<SharedDriveEntity> <SharedDriveFileNameEntity>) <SheetEntity>
+```
+* `<EmailAddress>` - The email address of a user with at least read access to the document
+
+Use one of the following to specify the file:
+* `<DriveFileIDEntity>` - The ID of the file on a Drive or Shared Drive
+* `<DriveFileNameEntity>` - The name of the file
+* `<SharedDriveEntity> <SharedDriveFileNameEntity>` - A Shared Drive and the name of the file on that drive
+
+If a file name is specified, it must resolve to a single file ID; otherwise an error is generated.
+
+If a Shared Drive name is specified, it must resolve to a single Shared Drive ID; otherwise an error is generated.
+
+Select a sheet/tab from the Google Sheet with its ID or name; it is verified to exist within the Google Sheet.
+
+Example:
+
+```
+gam csv gsheet you@exmaple.com <DriveFileIDEntity> "Sheet 1" gam create user firstname "~FirstName" lastname "~lastName" email "~email"
+```
+## Read data from a Google Cloud Storage File
+```
+<StorageBucketName> ::= <String>
+<StorageObjectName> ::= <String>
+<StorageBucketObjectName> ::=
+        https://storage.cloud.google.com/<StorageBucketName>/<StorageObjectName>|
+        https://storage.googleapis.com/<StorageBucketName>/<StorageObjectName>|
+        gs://<StorageBucketName>/<StorageObjectName>|
+        <StorageBucketName>/<StorageObjectName>
+```
+
+## CSV
+Read a Google Cloud Storage file with contentType text/csv.
+```
+gcscsv <StorageBucketObjectName>
+```
+
+## Plain Text
+Read a Google Cloud Storage file with contentType text/plain.
+```
+gcsdoc <StorageBucketObjectName>
+```
+
+## HTML
+Read a Google Cloud Storage file with contentType text/html.
+```
+gcshtml <StorageBucketObjectName>
+```

docs/Command-Line-Parsing.md

@@ -0,0 +1,79 @@
+# Command Line Parsing
+- [Linux and MacOS](#linux-and-macos)
+- [Windows Command Prompt](#windows-command-prompt)
+- [Windows PowerShell](#windows-powershell)
+- [List quoting rules](#list-quoting-rules)
+- [Queries example](#queries-example)
+
+## Linux and MacOS
+
+When entering `gam csv` commands, you should enclose references to CSV file headers in `"`; e.g., `name "~name"`.
+
+In bash, if an argument contains a `~`, `|`, `>`, or `<`, you must enclose the argument in `"`; e.g., `name "Test|Group"`.
+
+In zsh, if an argument contains a `~`, `|`, `!`, `>`, or `<`, you must enclose the argument in `'`; e.g., `name 'Test|Group'`.
+
+To embed a `'` in a string enclosed in `"`, enter `'`; `name "Test'Group"`.
+
+To embed a `"` in a string enclosed in `'`, enter `"`; `name 'Test"Group'`.
+
+To embed a `'` in a string enclosed in `'`, enter `'\''`; `name 'Test'\''Group'`.
+
+To embed a `"` in a string enclosed in `"`, enter `\"`; `name "Test\"Group"`.
+
+Linux and MacOS do not recognize smart or curly quotes, `“` and `”`, they can not be used to enclose arguments.
+
+## Windows Command Prompt
+
+Command Prompt does not recognize smart or curly quotes, `“` and `”`, they can not be used to enclose arguments.
+
+Command Prompt does not recognize single quotes, `'`, they can not be used to enclose arguments.
+
+To embed a `'` in a string enclosed in `"`, enter `'`; `name "Test'Group"`.
+
+To embed a `"` in a string enclosed in `"`, enter `\"`; `name "Test\"Group"`.
+
+## Windows PowerShell
+
+In PowerShell, if you want an empty string argument, you must enter: ``` `"`" ```
+
+PowerShell does not recognize smart or curly quotes, `“` and `”`, they can not be used to enclose arguments.
+
+To embed a `'` in a string enclosed in `"`, enter `'`; `name "Test'Group"`.
+
+To embed a `"` in a string enclosed in `"`, enter ``` `" ```; ```name "Test`"Group"```.
+
+To embed a `'` in a string enclosed in `'`, enter `''`; `name 'Test''Group'`.
+
+To embed a `"` in a string enclosed in `'`, enter `\"`; `name 'Test\"Group'`.
+
+## List quoting rules
+Items in a list can be separated by commas or spaces; if an item itself contains a comma, a space or a single quote, special quoting must be used.
+Typically, you will enclose the entire list in double quotes and quote each item in the list as detailed below.
+
+- Items, separated by commas, without spaces, commas or single quotes in the items themselves
+   * ```"item,item,item"```
+- Items, separated by spaces, without spaces, commas or single quotes in the items themselves
+   * ```"item item item"```
+- Items, separated by commas, with spaces, commas or single quotes in the items themselves
+   * ```"'it em','it,em',\"it'em\""```
+- Items, separated by spaces, with spaces, commas or single quotes in the items themselves
+   * ```"'it em' 'it,em' \"it'em\""```
+
+Typical places where these rules apply are lists of OUs and Contact Groups.
+
+## Queries example
+### Linux and MacOS
+```
+gam print users queries "\"orgUnitPath='/Students/Lower School/2027'\",\"orgUnitPath='/Students/Lower School/2028'\""
+```
+
+### Windows Command Prompt
+```
+gam print users queries "\"orgUnitPath='/Students/Lower School/2027'\",\"orgUnitPath='/Students/Lower School/2028'\""
+```
+
+### Windows Power Shell
+```
+gam print users queries "`"orgUnitPath=\'/Students/Lower\ School/2027\'`",`"orgUnitPath=\'/Students/Lower\ School/2028\'`""
+```

docs/Command-Logging-Progress.md

@@ -0,0 +1,88 @@
+# Command Logging and Progress
+- [Introduction](#introduction)
+- [GAM Configuration](gam.cfg)
+- [Command Logging](#command-logging)
+- [Command Progress](#command-progress)
+
+## Introduction
+Starting with version 6.07.00, GAM can log its commands to a file.
+
+Display of `gam batch|tbatch|csv|loop` progress messages has been improved.
+
+## Command Logging
+The following keywords in `gam.cfg` control logging of GAM commands.
+```
+cmdlog
+        Path to GAM Log file; there is no logging if cmdlog is empty
+        Default: ''
+cmdlog_max_backups
+        Maximum number of backup log files
+        Default: 5
+        Range: 1 - 10
+cmdlog_max_kilo_bytes
+        Maximum kilobytes per log file
+        Default: 1000
+        Range: 100 - 10000
+```
+
+If `cmdlog` specifies a relative file path, it is appended to `config_dir` in the current section if defined or `config_dir` in `[DEFAULT]`.
+This makes it easy to have distinct log files when you have multiple clients/tenants defined in `gam.cfg`
+
+You use the `cmdlog_max_kilo_bytes` and `cmdlog_max_backups` values to cause the log file to rollover at a predetermined size.
+When the log file is nearly `cmdlog_max_kilo_bytes` in length, it is closed and a new log file is silently opened for output.
+The system will save old log files by appending `.N`, to the filename. For example, with a `cmdlog_max_backups` of 5 and a base log file name of `gam.log`, you would get `gam.log`, `gam.log.1`, `gam.log.2`, up to `gam.log.5`.
+The log file being written to is always `gam.log`. When this log file is filled, it is closed and renamed to `gam.log.1`, and if files `gam.log.1`, `gam.log.2`, etc. exist, then they are renamed to `gam.log.2`, `gam.log.3` etc. respectively.
+
+Commands are logged at completion with a timestamp, return code and the command line
+```
+2021-08-01T19:350:30.777-07:00,0,/Users/admin/bin/gamadv-xtd3/gam info domain
+```
+
+Commands that generate sub-commands, `gam batch|tbatch|csv|loop`, log the initial command with a return code of `*`,
+the sub-command lines and the initial command with a numeric return code.
+```
+$ gam redirect stdout usernames.csv multiprocess redirect stderr stdout csv users.csv gam info user "~primaryEmail" quick name
+2021-08-01T19:50:38.151-07:00,0/6,Using 6 processes...
+$ more ~/.gam/gam.log
+2021-08-01T19:50:38.120-07:00,*,/Users/admin/bin/gamadv-xtd3/gam redirect stdout usernames.csv multiprocess redirect stderr stdout csv users.csv showcmds false gam info user ~primaryEmail quick name
+2021-08-01T19:50:39.144-07:00,0,gam info user testuser2 quick name
+2021-08-01T19:50:39.358-07:00,0,gam info user testuser3 quick name
+2021-08-01T19:50:39.358-07:00,0,gam info user testuser1 quick name
+2021-08-01T19:50:39.401-07:00,0,gam info user testuser5 quick name
+2021-08-01T19:50:39.459-07:00,56,gam info user testuserx quick name
+2021-08-01T19:50:39.470-07:00,0,gam info user testuser4 quick name
+2021-08-01T19:50:39.483-07:00,0,/Users/admin/bin/gamadv-xtd3/gam redirect stdout usernames.csv multiprocess redirect stderr stdout csv users.csv showcmds false gam info user ~primaryEmail quick name
+```
+
+## Command Progress
+Added the following keyword to `gam.cfg` to display sub-commands to stderr when executing `gam batch|tbatch|csv|loop`.
+The commands are displayed when initiated/completed so you can monitor GAM's progress.
+```
+show_commands
+        Display commands to stderr when executing `gam batch|tbatch|csv|loop`.
+        Default: False
+```
+This value will be used when not overridden by the `showcmds [<Boolean>]` command line option; see [Bulk Processing](Bulk-Processing).
+
+Sub-commands are displayed at initiation with a timestamp, index/total, Start, 0 and the sub-command line.
+
+Sub-commands are displayed at completion with a timestamp, index/total, End, return code and the sub-command line.
+
+```
+$ gam redirect stdout usernames.csv multiprocess redirect stderr stdout csv users.csv showcmds true gam info user "~primaryEmail" quick name
+2021-08-01T19:46:07.845-07:00,0/6,Using 6 processes...
+2021-08-01T19:46:07.846-07:00,1/6,Start,0,gam info user testuser1 quick name
+2021-08-01T19:46:07.846-07:00,2/6,Start,0,gam info user testuser2 quick name
+2021-08-01T19:46:07.846-07:00,3/6,Start,0,gam info user testuser3 quick name
+2021-08-01T19:46:07.846-07:00,4/6,Start,0,gam info user testuser4 quick name
+2021-08-01T19:46:07.846-07:00,5/6,Start,0,gam info user testuser5 quick name
+2021-08-01T19:46:07.846-07:00,6/6,Start,0,gam info user testuserx quick name
+2021-08-01T19:46:08.827-07:00,3/6,End,0,gam info user testuser3 quick name
+2021-08-01T19:46:08.983-07:00,2/6,End,0,gam info user testuser2 quick name
+2021-08-01T19:46:08.983-07:00,1/6,End,0,gam info user testuser1 quick name
+2021-08-01T19:46:09.049-07:00,6/6,End,56,gam info user testuserx quick name
+2021-08-01T19:46:09.059-07:00,5/6,End,0,gam info user testuser5 quick name
+2021-08-01T19:46:09.079-07:00,4/6,End,0,gam info user testuser4 quick name
+2021-08-01T19:46:09.083-07:00,0/6,Complete
+```
+

docs/Context-Aware-Access-Levels.md

@@ -0,0 +1,454 @@
+# Context-Aware Access Levels
+
+- [Notes](#Notes)
+- [Context-Aware Access documentation](https://support.google.com/a/answer/9275380)
+- [API documentation](#api-documentation)
+- [Grant Service Account Rights to Manage CAA](#grant-service-account-rights-to-manage-caa)
+- [Definitions](#definitions)
+- [Parameters for Basic Levels](#parameters-for-basic-levels)
+- [Create an Access Level](#create-an-access-level)
+- [Update an Access Level](#update-an-access-level)
+- [Delete an Access Level](#delete-an-access-level)
+- [Display all Access Levels](#display-all-access-levels)
+- [CAA Region Codes](#caa-region-codes)
+
+## Notes
+This Wiki page was built directly from Jay Lee's Wiki page; my sincere thanks for his efforts.
+
+GAM 6.20.00 and newer can create and manage access levels which can be assigned to Workspace services for your users.
+
+To use these features you must update your project.
+```
+gam update project
+```
+
+## API documentation
+* https://cloud.google.com/access-context-manager/docs/reference/rest/v1/accessPolicies/list
+
+## Grant Service Account Rights to Manage CAA
+In order for GAM to manage CAA access levels, you need to grant your service account a special role for your GCP organization.
+1. Run a GAM command like `gam print caalevels`. This will show you the service account email and role you need to grant it. Copy the service account email.
+2. You can also get the value from oauth2service.json: `"client_email": "gam-project-abc-123-xyz@gam-project-abc-123-xyz.iam.gserviceaccount.com"`
+3. As an organization admin (Workspace Super Admin should work) go to [https://console.cloud.google.com/iam-admin/iam](https://console.cloud.google.com/iam-admin/iam).
+4. In the top blue bar, to the right of `Google Cloud Platform` click the desired `<Project Name>`.
+5. If the page shows `Permissions for organization <Primary Domain>`", skip the next step.
+6. If the page shows `Permissions for project <Project Name>`", click the building icon immediately to the left of your `<Primary Domain>` in the Inheritance column.
+7. Near the top click `Add`.
+8. Enter the service account email address you recorded earlier into the `New principals*` box.
+9. In the `Select a role*` box, select Access Context Manager > Access Context Manager Editor.
+10. Click `Save`. It may take 15 minutes or more for the role permissions to propagate.
+11. Confirm the role is in place by re-running `gam print caalevels`
+
+## Definitions
+```
+<QueryCEL> ::= <String>
+        See: https://cloud.google.com/access-context-manager/docs/custom-access-level-spec
+
+<CAALevelName> ::= <String>
+
+<CAAAllowedEncryptionStatus> ::=
+        encryption_unsupported |
+        encrypted |
+        unencrypted
+<CAAAllowedEncryptionStatusList> ::= "<CAAAllowedEncryptionStatus>(,<CAAAllowedEncryptionStatus>)"
+
+<CAAAllowedDeviceManagementLevel> ::=
+        basic |
+        advanced|complete |
+        none
+<CAAAllowedDeviceManagementLevelList> ::= "<CAAAllowedDeviceManagementLevel>(,<CAAAllowedDeviceManagementLevel>)"
+
+<CAACombiningFunction> ::=
+        and |
+        or
+
+<CAAIPSubNetwork> ::=
+        <CIDRnetmask>
+<CAAIPSubNetworkList> ::= "<CAAIPSubNetwork>(,<CAAIPSubNetwork>)"
+
+<CAAMember> ::=
+        user:<EmailAddress> |
+        serviceAccount:<EmailAddress>
+<CAAMemberList> ::= "<CAAMember>(,<CAAMember>)"
+
+<CAAOsType> ::=
+        DESKTOP_MAC |
+        DESKTOP_WINDOWS |
+        DESKTOP_LINUX |
+        DESKTOP_CHROME_OS |
+        VERIFIED_DESKTOP_CHROME_OS |
+        ANDROID |
+        IOS
+
+<CAAOsConstraint> ::=
+        <CAAOsType> |
+        <CAAOsType>:<String>.<String>.<String>
+<CAAOsConstraintList> ::= "<CAAOsConstraint>(,<CAAOsConstraint>)"
+
+<CAARegion> ::=
+        <Character><Character>
+<CAARegionList> ::= "<CAARegion>(,<CAARegion>)"
+
+<CAADevicePolicyAttribute> ::=
+        (requirescreenlock <Boolean>) |
+        (allowedencryptionstatuses <CAAAllowedEncryptionStatusList>) |
+        (osconstraints <CAAOsConstraintList>) |
+        (alloweddevicemanagementlevels <CAAAllowedDeviceManagementLevelList>) |
+        (requireadminapproval <Boolean>) |
+        (requirecorpowned <Boolean>)    # See: https://www.iso.org/obp/ui/#search
+
+<CAAConditionAttribute> ::=
+        (ipsubnetworks <CAAIPSubNetworkList>) |
+        (devicepolicy <CAADevicePolicyAttribute> enddevicepolicy) |
+        (requiredaccesslevels <StringList>) |
+        (negate <Boolean>) |
+        (members <CAAMemberList>) |
+        (regions <CAARegionList>)
+        
+<CAABasicAttribute> ::+
+        (combiningfunction <CAACombiningFunction>) |
+        (condition <CAAConditionAttribute>+ endcondition)
+```
+
+# Parameters for Basic Levels
+
+```
+basic
+  combiningfunction and|or
+  condition
+    negate true|false
+    ipsubnetworks ip4range,ip6range,...
+    regions <country code>,country code>,...
+    devicepolicy
+      requirescreenlock true|false
+      allowedencryptionstatuses ENCRYPTION_UNSUPPORTED,ENCRYPTED,UNENCRYPTED
+      alloweddevicemanagementlevels NONE,BASIC,COMPLETE
+      requireadminapproval true|false
+      requirecorpowned true|false
+      osconstraints DESKTOP_MAC:version,DESKTOP_WINDOWS:version,DESKTOP_LINUX:version,
+                    DESKTOP_CHROME_OS:version,VERIFIED_DESKTOP_CHROME_OS:version,
+                    ANDROID:version,IOS:version
+    enddevicepolicy
+  endcondition
+  condition
+    ...
+  endcondition
+```
+* The combiningfunction argument specifies if a user must pass all 2+ conditions (AND) or only one (OR).
+* The negate argument specifies whether a user that matches the condition passes it or fails.
+* The ipsubnetworks argument specifies a comma-separated list of IPv4 or IPv6 networks the user must be coming from to match.
+* The regions argument specifies a comma-separated list of country/regions the user must be coming from to match.
+* The device policy argument specifies characteristics of the user's device that must be present to match.
+
+
+## Create an Access Level
+Create a new access level. CAA supports basic and custom conditions.
+```
+gam create caalevel <String> [description <String>] (basic <CAABasicAttribute>+)|(custom <QueryCEL>)
+```
+
+## Example
+This example defines a custom access level that requires the user to use a Cloud-managed Chrome browser (CBCM) or be logged into a Cloud-managed Chrome profile.
+```
+gam create caalevel custom "device.chrome.management_state == ChromeManagementState.CHROME_MANAGEMENT_STATE_BROWSER_MANAGED | ChromeManagementState.CHROME_MANAGEMENT_STATE_PROFILE_MANAGED"
+```
+
+This example creates a basic access level that requires the user to come from the US or Canada regions
+```
+gam create caalevel CORP_COUNTRIES basic condition regions US,CA endcondition
+```
+
+This example creates a basic access level that requires the user come from one of the given IP ranges
+```
+gam create caalevel CORP_IPS basic condition ipsubnetworks 1.2.3.0/24,4.5.6.0/24 endcondition
+```
+----
+## Update an Access Level
+Updates an existing access level. CAA supports basic and custom conditions.
+```
+gam update caalevel <CAALevelName> [description <String>] (basic <CAABasicAttribute>+)|(custom <QueryCEL>)
+```
+
+## Examples
+This example adds UK to the allowed regions for CORP_COUNTRIES
+```
+gam update caalevel CORP_COUNTRIES basic condition regions US,CA,UK endcondition
+```
+
+## Delete an Access Level
+Deletes the specified access level.
+```
+gam delete caalevel <CAALevelName>
+```
+# Display all access levels
+```
+gam show caalevels
+        [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values:
+* `formatjson` - Display the fields in JSON format.
+```
+gam print caalevels [todrive <ToDriveAttribute>*]
+        [formatjson [quotechar <Character>]]
+```
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format:
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## CAA Region Codes
+```
+AD: Andorra
+AE: United Arab Emirates
+AF: Afghanistan
+AG: Antigua and Barbuda
+AI: Anguilla
+AL: Albania
+AM: Armenia
+AO: Angola
+AQ: Antarctica
+AR: Argentina
+AS: American Samoa
+AT: Austria
+AU: Australia
+AW: Aruba
+AX: Åland Islands
+AZ: Azerbaijan
+BA: Bosnia and Herzegovina
+BB: Barbados
+BD: Bangladesh
+BE: Belgium
+BF: Burkina Faso
+BG: Bulgaria
+BH: Bahrain
+BI: Burundi
+BJ: Benin
+BL: Saint Barthélemy
+BM: Bermuda
+BN: Brunei Darussalam
+BO: Bolivia Plurinational State of
+BQ: Bonaire Sint Eustatius and Saba
+BR: Brazil
+BS: Bahamas
+BT: Bhutan
+BV: Bouvet Island
+BW: Botswana
+BY: Belarus
+BZ: Belize
+CA: Canada
+CC: Cocos (Keeling) Islands
+CD: Congo The Democratic Republic of the
+CF: Central African Republic
+CG: Congo
+CH: Switzerland
+CI: Côte d'Ivoire
+CK: Cook Islands
+CL: Chile
+CM: Cameroon
+CN: China
+CO: Colombia
+CR: Costa Rica
+CU: Cuba
+CV: Cabo Verde
+CW: Curaçao
+CX: Christmas Island
+CY: Cyprus
+CZ: Czechia
+DE: Germany
+DJ: Djibouti
+DK: Denmark
+DM: Dominica
+DO: Dominican Republic
+DZ: Algeria
+EC: Ecuador
+EE: Estonia
+EG: Egypt
+EH: Western Sahara
+ER: Eritrea
+ES: Spain
+ET: Ethiopia
+FI: Finland
+FJ: Fiji
+FK: Falkland Islands (Malvinas)
+FM: Micronesia Federated States of
+FO: Faroe Islands
+FR: France
+GA: Gabon
+GB: United Kingdom
+GD: Grenada
+GE: Georgia
+GF: French Guiana
+GG: Guernsey
+GH: Ghana
+GI: Gibraltar
+GL: Greenland
+GM: Gambia
+GN: Guinea
+GP: Guadeloupe
+GQ: Equatorial Guinea
+GR: Greece
+GS: South Georgia and the South Sandwich Islands
+GT: Guatemala
+GU: Guam
+GW: Guinea-Bissau
+GY: Guyana
+HK: Hong Kong
+HM: Heard Island and McDonald Islands
+HN: Honduras
+HR: Croatia
+HT: Haiti
+HU: Hungary
+ID: Indonesia
+IE: Ireland
+IL: Israel
+IM: Isle of Man
+IN: India
+IO: British Indian Ocean Territory
+IQ: Iraq
+IR: Iran Islamic Republic of
+IS: Iceland
+IT: Italy
+JE: Jersey
+JM: Jamaica
+JO: Jordan
+JP: Japan
+KE: Kenya
+KG: Kyrgyzstan
+KH: Cambodia
+KI: Kiribati
+KM: Comoros
+KN: Saint Kitts and Nevis
+KP: Korea Democratic People's Republic of
+KR: Korea Republic of
+KW: Kuwait
+KY: Cayman Islands
+KZ: Kazakhstan
+LA: Lao People's Democratic Republic
+LB: Lebanon
+LC: Saint Lucia
+LI: Liechtenstein
+LK: Sri Lanka
+LR: Liberia
+LS: Lesotho
+LT: Lithuania
+LU: Luxembourg
+LV: Latvia
+LY: Libya
+MA: Morocco
+MC: Monaco
+MD: Moldova Republic of
+ME: Montenegro
+MF: Saint Martin (French part)
+MG: Madagascar
+MH: Marshall Islands
+MK: North Macedonia
+ML: Mali
+MM: Myanmar
+MN: Mongolia
+MO: Macao
+MP: Northern Mariana Islands
+MQ: Martinique
+MR: Mauritania
+MS: Montserrat
+MT: Malta
+MU: Mauritius
+MV: Maldives
+MW: Malawi
+MX: Mexico
+MY: Malaysia
+MZ: Mozambique
+NA: Namibia
+NC: New Caledonia
+NE: Niger
+NF: Norfolk Island
+NG: Nigeria
+NI: Nicaragua
+NL: Netherlands
+NO: Norway
+NP: Nepal
+NR: Nauru
+NU: Niue
+NZ: New Zealand
+OM: Oman
+PA: Panama
+PE: Peru
+PF: French Polynesia
+PG: Papua New Guinea
+PH: Philippines
+PK: Pakistan
+PL: Poland
+PM: Saint Pierre and Miquelon
+PN: Pitcairn
+PR: Puerto Rico
+PS: Palestine State of
+PT: Portugal
+PW: Palau
+PY: Paraguay
+QA: Qatar
+RE: Réunion
+RO: Romania
+RS: Serbia
+RU: Russian Federation
+RW: Rwanda
+SA: Saudi Arabia
+SB: Solomon Islands
+SC: Seychelles
+SD: Sudan
+SE: Sweden
+SG: Singapore
+SH: Saint Helena Ascension and Tristan da Cunha
+SI: Slovenia
+SJ: Svalbard and Jan Mayen
+SK: Slovakia
+SL: Sierra Leone
+SM: San Marino
+SN: Senegal
+SO: Somalia
+SR: Suriname
+SS: South Sudan
+ST: Sao Tome and Principe
+SV: El Salvador
+SX: Sint Maarten (Dutch part)
+SY: Syrian Arab Republic
+SZ: Eswatini
+TC: Turks and Caicos Islands
+TD: Chad
+TF: French Southern Territories
+TG: Togo
+TH: Thailand
+TJ: Tajikistan
+TK: Tokelau
+TL: Timor-Leste
+TM: Turkmenistan
+TN: Tunisia
+TO: Tonga
+TR: Turkey
+TT: Trinidad and Tobago
+TV: Tuvalu
+TW: Taiwan Province of China
+TZ: Tanzania United Republic of
+UA: Ukraine
+UG: Uganda
+UM: United States Minor Outlying Islands
+US: United States
+UY: Uruguay
+UZ: Uzbekistan
+VA: Holy See (Vatican City State)
+VC: Saint Vincent and the Grenadines
+VE: Venezuela Bolivarian Republic of
+VG: Virgin Islands British
+VI: Virgin Islands U.S.
+VN: Viet Nam
+VU: Vanuatu
+WF: Wallis and Futuna
+WS: Samoa
+YE: Yemen
+YT: Mayotte
+ZA: South Africa
+ZM: Zambia
+ZW: Zimbabwe
+```

docs/Customer.md

@@ -0,0 +1,47 @@
+# Customer
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Update customer](#update-customer)
+- [Display customer](#display-customer)
+- [Display instance](#display-instance)
+
+## API documentation
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/customers
+
+## Definitions
+```
+<DomainName> ::= <String>(.<String>)+
+<EmailAddress> ::= <String>@<DomainName>
+
+<CustomerAttribute> ::=
+        (primary <DomainName>)|
+        (adminsecondaryemail|alternateemail <EmailAddress>)|
+        (contact|contactname <String>)|
+        (language <LanguageCode>)|
+        (phone|phonenumber <String>)|
+        (name|organizationname <String>)|
+        (address|address1|addressline1 <String>)|
+        (address2|addressline2 <String>)|
+        (address3|addressline3 <String>)|
+        (city|locality <String>)|
+        (state|region <String>)|
+        (zipcode|postal|postalcode <String>)|
+        (country|countrycode <String>)
+```
+## Update customer
+```
+gam update customer <CustomerAttribute>*
+```
+## Display customer
+```
+gam info customer [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+## Display instance
+```
+gam info instance [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.

docs/Domain-People-Contacts-Profiles.md

@@ -0,0 +1,188 @@
+# Domain People - Contacts & Profiles
+- [API documentation](#api-documentation)
+- [Collections of Users](Collections-of-Users)
+- [Notes](#notes)
+- [Definitions](#definitions)
+- [Display Domain Contacts](#display-domain-contacts)
+- [Display Domain Profiles](#display-domain-profiles)
+
+## API documentation
+* https://developers.google.com/contacts/v3/announcement
+* https://developers.google.com/people/contacts-api-migration
+* https://developers.google.com/people
+* https://developers.google.com/people/api/rest/v1/people/listDirectoryPeople
+* https://developers.google.com/people/api/rest/v1/people/searchDirectoryPeople
+
+## Notes
+To use these features you must add the `People API` to your project and authorize the appropriate scopes:
+* `Client Access` - `People Directory API - read only`
+* `Service Account Access`
+  * `People Directory API - read only`: https://www.googleapis.com/auth/directory.readonly
+  * `OAuth2 API`: https://www.googleapis.com/auth/userinfo.profile
+```
+gam update project
+gam oauth create
+gam user user@domain.com check serviceaccount
+```
+
+## Definitions
+```
+<PeopleResourceName> ::= people/<String>
+<PeopleResourceNameList> ::= "<PeopleResourceName>(,<PeopleResourceName>)*"
+<PeopleResourceNameEntity> ::=
+        <PeopleResourceNameNameList> | <FileSelector> | <CSVFileSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+
+<PeopleSourceName> ::=
+        contact|contacts|
+        profile|profiles
+
+<PeopleMergeSourceName> ::=
+        contact|contacts
+
+<PeopleFieldName> ::=
+        addresses|
+        ageranges|
+        biographies|
+        birthdays|
+        calendarurls|
+        clientdata|
+        coverphotos|
+        emailaddresses|
+        events|
+        externalids|
+        genders|
+        imclients|
+        interests|
+        locales|
+        locations|
+        memberships|
+        metadata|
+        misckeywords|
+        names|
+        nicknames|
+        occupations|
+        organizations|
+        phonenumbers|
+        photos|
+        relations|
+        sipaddresses|
+        skills|
+        urls|
+        userdefined
+<PeopleFieldNameList> ::= "<PeopleFieldName>(,<PeopleFieldName>)*"
+```
+
+## Display Domain Contacts
+### Display as an indented list of keys and values.
+```
+gam info domaincontacts <PeopleResourceNameEntity>
+        [allfields|(fields <PeopleFieldNameList>)]
+        [formatjson]
+```
+By default, Gam displays the fields `names,emailaddresses`.
+* `allfields|(fields <PeopleFieldNameList>)` - Select fields to display
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam show domaincontacts
+        [query <String>]
+        [mergesources <PeopleMergeSourceName>]
+        [allfields|(fields <PeopleFieldNameList>)]
+        [formatjson]
+```
+By default, Gam displays all domain contacts.
+* `query <String>` - Display contacts based on the data in their fields.
+
+Google's explanation of `mergesources`: Additional data to merge into the directory sources
+if they are connected through verified join keys such as email addresses or phone numbers.
+
+By default, Gam displays the fields `names,emailaddresses`.
+* `allfields|(fields <PeopleFieldNameList>)` - Select fields to display
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+### Display as a CSV file.
+```
+gam print domaincontacts [todrive <ToDriveAttribute>*]
+        [query <String>]
+        [mergesources <PeopleMergeSourceName>]
+        [allfields|(fields <PeopleFieldNameList>)]
+        [formatjson [quotechar <Character>]]
+```
+By default, Gam displays all domain contacts.
+* `query <String>` - Display contacts based on the data in their fields.
+
+Google's explanation of `mergesources`: Additional data to merge into the directory sources
+if they are connected through verified join keys such as email addresses or phone numbers.
+
+By default, Gam displays the fields `names,emailaddresses`.
+* `allfields|(fields <PeopleFieldNameList>)` - Select fields to display
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display Domain Profiles
+### Display as an indented list of keys and values.
+```
+gam info domainprofiles|people|peopleprofiles <PeopleResourceNameEntity>
+        [allfields|(fields <PeopleFieldNameList>)]
+        [formatjson]
+```
+By default, Gam displays the fields `names,emailaddresses`.
+* `allfields|(fields <PeopleFieldNameList>)` - Select fields to display
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam show domainprofiles|people|peopleprofiles
+        [query <String>]
+        [mergesources <PeopleMergeSourceName>]
+        [allfields|(fields <PeopleFieldNameList>)]
+        [formatjson]
+```
+By default, Gam displays all domain profiles.
+* `query <String>` - Display profiles based on the data in their fields.
+
+Google's explanation of `mergesources`: Additional data to merge into the directory sources
+if they are connected through verified join keys such as email addresses or phone numbers.
+
+By default, Gam displays the fields `names,emailaddresses`.
+* `allfields|(fields <PeopleFieldNameList>)` - Select fields to display
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+
+### Display as a CSV file.
+```
+gam print domainprofiles|people|peopleprofiles [todrive <ToDriveAttribute>*]
+        [query <String>]
+        [mergesources <PeopleMergeSourceName>]
+        [allfields|(fields <PeopleFieldNameList>)]
+        [formatjson [quotechar <Character>]]
+```
+By default, Gam displays all domain profiles.
+* `query <String>` - Display profiles based on the data in their fields.
+
+Google's explanation of `mergesources`: Additional data to merge into the directory sources
+if they are connected through verified join keys such as email addresses or phone numbers.
+
+By default, Gam displays the fields `names,emailaddresses`.
+* `allfields|(fields <PeopleFieldNameList>)` - Select fields to display
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Domain-SharedContacts-GAL.md

@@ -0,0 +1,331 @@
+# Domain Shared Contacts - Global Address List
+- [API documentation](#api-documentation)
+- [Query documentation](#query-documentation)
+- [Python Regular Expressions](Python-Regular-Expressions) Match function
+- [Definitions](#definitions)
+- [Create domain shared contacts](#create-domain-shared-contacts)
+- [Select domain shared contacts](#select-domain-shared-contacts)
+- [Update domain shared contacts](#update-domain-shared-contacts)
+- [Delete domain shared contacts](#delete-domain-shared-contacts)
+- [Clear old email addresses from contacts](#clear-old-email-addresses-from-contacts)
+- [Delete duplicate email addresses from contacts](#delete-duplicate-email-addresses-from-contacts)
+- [Manage domain contact photos](#manage-domain-contact-photos)
+- [Display domain shared contacts](#display-domain-shared-contacts)
+- [Display global address list](#display-global-address-list)
+
+## API documentation
+* https://developers.google.com/admin-sdk/domain-shared-contacts/
+
+## Query documentation
+* https://developers.google.com/google-apps/contacts/v3/reference#contacts-query-parameters-reference
+
+## Definitions
+* [Command data from Google Docs/Sheets/Storage](Command-Data-From-Google-Docs-Sheets-Storage)
+```
+<StorageBucketName> ::= <String>
+<StorageObjectName> ::= <String>
+<StorageBucketObjectName> ::=
+        https://storage.cloud.google.com/<StorageBucketName>/<StorageObjectName>|
+        https://storage.googleapis.com/<StorageBucketName>/<StorageObjectName>|
+        gs://<StorageBucketName>/<StorageObjectName>|
+        <StorageBucketName>/<StorageObjectName>
+
+<UserGoogleDoc> ::=
+        <EmailAddress> <DriveFileIDEntity>|<DriveFileNameEntity>|(<SharedDriveEntity> <SharedDriveFileNameEntity>)
+
+<NoteContent> ::=
+        ((<String>)|
+         (file <FileName> [charset <Charset>])|
+         (gdoc <UserGoogleDoc>)|
+         (gcsdoc <StorageBucketObjectName>))
+
+<Date> ::=
+        <Year>-<Month>-<Day> |
+        (+|-)<Number>(d|w|y) |
+        never|
+        today
+<DomainName> ::= <String>(.<String>)+
+<EmailAddress> ::= <String>@<DomainName>
+<QueryContact> ::= <String>
+        https://developers.google.com/google-apps/contacts/v3/reference#contacts-query-parameters-reference
+
+<JSONData> ::= (json [charset <Charset>] <String>) | (json file <FileName> [charset <Charset>]) |
+
+<ContactID> ::= <String>
+<ContactIDList> ::= "<ContactID>(,<ContactID>)*"
+<ContactEntity> ::=
+        <ContactIDList> | <FileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<ContactSelection> ::=
+        [query <QueryContact>]
+        [emailmatchpattern <RegularExpression> [emailmatchtype work|home|other|<String>]]
+        [updated_min <Date>]
+```
+```
+<ContactBasicAttribute> ::=
+        (additionalname|middlename <String>)|
+        (billinginfo <String>)|
+        (birthday <Date>)|
+        (directoryserver <String>)|
+        (familyname|lastname <String>)|
+        (gender female|male)|
+        (givenname|firstname <String>)|
+        (initials <String>)|
+        (language <Language>)|
+        (location <String>)|
+        (maidenname <String>)|
+        (mileage <String>)|
+        (name <String>)|
+        (nickname <String>)|
+        (note <NoteContent>)|
+        (occupation <String>)|
+        (prefix <String>)|
+        (priority low|normal|high)
+        (sensitivity confidential|normal|personal|private)
+        (shortname <String>)|
+        (subject <String>)|
+        (suffix <String>)
+```
+```
+<ContactMultiAttribute> ::=
+        (address work|home|other|<String>
+                (formatted|unstructured <String>)|(streetaddress <String>)|
+                (pobox <String>)|(neighborhood <String>)|(locality <String>)|
+                (region <String>)|(postalcode <String>)|(country <String>)*
+                notprimary|primary)|
+        (calendar work|home|free-busy|<String> <URL>
+                notprimary|primary)|
+        (email work|home|other|<String> <EmailAddress>
+                notprimary|primary)|
+        (event anniversary|other|<String> <Date>)|
+        (externalid account|customer|network|organization|<String> <String>)|
+        (hobby <String>)|
+        (im work|home|other|<String>
+                aim|gtalk|icq|jabber|msn|net_meeting|qq|skype|yahoo <String>
+                notprimary|primary)|
+        (jot work|home|other|keywords|user> <String>)|
+        (organization work|other|<String> <String>
+                (location <String>)|(department <String>)|(title <String>)|
+                (jobdescription <String>)|(symbol <String>)*
+                notprimary|primary)|
+        (phone work|home|other|fax|work_fax|home_fax|other_fax|main|company_main|
+                assistant|mobile|work_mobile|pager|work_pager|car|radio|callback|
+                isdn|telex|tty_tdd|<String> <String>
+                notprimary|primary)|
+        (relation spouse|child|mother|father|parent|brother|sister|friend|relative|
+                domestic_partner|manager|assistant|referred_by|partner|<String> <String>)|
+        (userdefinedfield <String> <String>)|
+        (website home_page|blog|profile|work|home|other|ftp|reservations|
+                app_install_page|<String> <URL> notprimary|primary)
+
+<ContactClearAttribute> ::=
+        (address clear)|
+        (calendar clear)|
+        (email clear)|
+        (event clear)|
+        (externalid clear)|
+        (hobby clear)|
+        (im clear)|
+        (jot clear)|
+        (organization clear)|
+        (phone clear)|
+        (relation clear)|
+        (userdefinedfield clear)|
+        (website clear)
+```
+```
+<ContactAttribute> ::=
+        <JSONData>|
+        <ContactBasicAttribute>|
+        <ContactMultiAttribute>|
+        <ContactClearAttribute>
+```
+```
+<ContactFieldName> ::=
+        additionalname|middlename|
+        address|
+        billinginfo|
+        birthday|
+        calendar|
+        directoryserver|
+        email|
+        event|
+        externalid|
+        familyname|lastname|
+        gender|
+        givenname|firstname|
+        hobby|
+        im|
+        initials|
+        jot|
+        language|
+        location|
+        maidenname|
+        mileage|
+        name|
+        nickname|
+        note|
+        occupation|
+        organization|
+        phone|
+        prefix|
+        priority|
+        relation|
+        sensitivity|
+        shortname|
+        subject|
+        suffix|
+        updated|
+        userdefinedfield|
+        website
+<ContactFieldNameList> ::= "<ContactFieldName>(,<ContactFieldName>)*"
+
+<ContactOrderByFieldName> ::=
+        lastmodified
+```
+## Create domain shared contacts
+```
+gam create contact <ContactAttribute>+
+        [(csv [todrive <ToDriveAttribute>*] (addcsvdata <FieldName> <String>)*))| returnidonly]
+```
+By default, the domain name and contact ID are displayed on stdout.
+* `csv [todrive <ToDriveAttribute>*]` - Write domain name and contact ID values to a CSV file.
+  * `addcsvdata <FieldName> <String>` - Add additional columns of data from the command line to the output
+* `returnidonly` - Display just the contact ID on stdout
+
+To retrieve the contact ID with `returnidonly`:
+```
+Linux/MacOS
+contactId=$(gam create contact ... returnidonly)
+Windows PowerShell
+$contactId = & gam create contact ... returnidonly
+```
+
+## Select domain shared contacts
+You specify contacts by ID or by selection qualifiers.
+```
+<ContactID> ::= <String>
+<ContactIDList> ::= "<ContactID>(,<ContactID>)*"
+<ContactEntity> ::=
+        <ContactIDList> | <FileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<ContactSelection> ::=
+        [query <QueryContact>]
+        [emailmatchpattern <RegularExpression> [emailmatchtype work|home|other|<String>]]
+        [updated_min <Date>]
+```
+Selection qualifiers may be combined.
+* `query <QueryContact>` - Fulltext query on contacts data fields. See: https://developers.google.com/contacts/v3/reference#contacts-query-parameters-reference
+* `emailmatchpattern <RegularExpression>` - Select contacts that have an email address matching `<RegularExpression>`
+* `emailmatchpattern <RegularExpression> emailmatchtype work|home|other|<String>` - Select contacts that have an email address matching `<RegularExpression>` and a specific type
+* `emailmatchpattern ".*" emailmatchtype work|home|other|<String>` - Select contacts that have any email address with a specific type
+* `updated_min <Date>` - Select contacts updated since `<Date>`
+
+## Update domain shared contacts
+```
+gam update contacts <ContactEntity>|<ContactSelection> <ContactAttribute>+
+```
+## Delete domain shared contacts
+```
+gam delete contacts <ContactEntity>|<ContactSelection>
+```
+## Clear old email addresses from contacts
+```
+gam clear contacts <ContactEntity>|<ContactSelection>
+        [emailclearpattern <RegularExpression> [emailcleartype work|home|other|<String>]]
+        [delete_cleared_contacts_with_no_emails]
+```
+Typically, you would select contacts by `emailmatchpattern <RegularExpression>` (and optionally `emailmatchtype work|home|other|<String>`),
+then the matching email addresses will be cleared from the domiain contact's email list. The contact itself is updated, not deleted.
+Email addresses that don't match will be unaffected. If you want to clear all email addresses of a particular type,
+use `emailmatchpattern ".*" emailmatchtype work|home|other|<String>`.
+
+You can specify `emailclearpattern <RegularExpression>` (and optionally `emailcleartype work|home|other|<String>`) if you want to
+clear email addresses other than the ones used to match the contacts or if you specify `<ContactEntity>`.
+
+A contact may contain no email addresses after matching email addresses are cleared. If you do not want to keep contacts with no
+email addresses after clearing, use the `delete_cleared_contacts_with_no_emails` option and they will be deleted.
+Contacts with no email addresses before clearing will not be affected.
+
+## Delete duplicate email addresses from contacts
+If the same email address appears multiple times within a contact, all but the first will be deleted.
+```
+gam dedup contacts [<ContactEntity>|<ContactSelection>] [matchType [<Boolean>]]
+```
+If neither `<ContactEntity>` or `<ContactSelection>` is specified, all contacts are checked for duplicates.
+
+By default, the email type `work|home|other|<String>` is ignored, all duplicates, regardless of type,
+will be deleted. If `matchtype` is true, only duplicate email addresses with the same type will be deleted.
+
+## Manage domain contact photos
+```
+gam update contactphotos <ContactEntity>|<ContactSelection>
+        [drivedir|(sourcefolder <FilePath>)] [filename <FileNamePattern>]
+gam get contactphotos <ContactEntity>|<ContactSelection>
+        [drivedir|(targetfolder <FilePath>)] [filename <FileNamePattern>]
+gam delete contactphotos <ContactEntity>|<ContactSelection>
+```
+The default directory is the current working directory, `drivedir` specifies the value of drive_dir from gam.cfg and
+`sourcefolder/targetfolder <FilePath>` specifies a user-chosen path.
+`<FileNamePattern>` can contain the strings `#email#` and `#contactid#` which will be replaced by the the contact's primary emailaddress or the contact ID.
+If not specified, `<FileNamePattern>` defaults to `#contactid#.jpg`.
+
+## Display domain shared contacts
+```
+gam info contacts <ContactEntity>
+        [basic|full]
+        [fields <ContactFieldNameList>] [formatjson]
+gam show contacts [<ContactSelection>]
+        [basic|full|countsonly] [showdeleted]
+        [orderby <ContactOrderByFieldName> [ascending|descending]]
+        [fields <ContactFieldNameList>] [formatjson]
+```
+If `<ContactSelection>` is not specified, all contacts are displayed.
+
+If `countsonly` is specified, no contact fields are displayed, just the number of contacts.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print contacts [todrive <ToDriveAttribute>*] [<ContactSelection>]
+        [basic|full|countsonly] [showdeleted]
+        [orderby <ContactOrderByFieldName> [ascending|descending]]
+        [fields <ContactFieldNameList>] [formatjson [quotechar <Character>]]
+```
+If `<ContactSelection>` is not specified, all contacts are displayed.
+
+If `countsonly` is specified, no contact fields are displayed, just the number of contacts.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display global address list
+```
+gam info gal <ContactEntity>
+        [basic|full]
+        [fields <ContactFieldNameList>] [formatjson]
+gam show gal <ContactSelection>
+        [basic|full] [orderby <ContactOrderByFieldName> [ascending|descending]]
+        [fields <ContactFieldNameList>] [formatjson]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print gal [todrive <ToDriveAttribute>*] <ContactSelection>
+        [basic|full] [orderby <ContactOrderByFieldName> [ascending|descending]]
+        [fields <ContactFieldNameList>] [formatjson [quotechar <Character>]]
+```
+By default, Gam displays the information as columns of fields.
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.

docs/Domains-Verification.md

@@ -0,0 +1,31 @@
+# Domains - Verification
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Introduction](#introduction)
+- [Create site verification tokens](#create-site-verification-tokens)
+- [Test site verification token](#test-site-verification-token)
+- [Display site verification information](#display-site-verification-information)
+
+## API documentation
+* https://developers.google.com/site-verification/v1/getting_started
+* https://developers.google.com/site-verification/v1/
+
+## Definitions
+```
+<DomainName> ::= <String>(.<String>)+
+```
+## Introduction
+To use Google Apps Gmail and other Web services, your account's site ownership must be verified.
+
+## Create site verification tokens
+```
+gam create verify|verification <DomainName>
+```
+## Test site verification token
+```
+gam update verify|verification <DomainName> cname|txt|text|site|file
+```
+## Display site verification information
+```
+gam info verify|verification
+```

docs/Domains.md

@@ -0,0 +1,75 @@
+# Domains
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Create a domain](#create-a-domain)
+- [Promote a domain to be primary](#promote-a-domain-to-be-primary)
+- [Delete a domain](#delete-a-domain)
+- [Display domains](#display-domains)
+- [Create and delete domain aliases](#create-and-delete-domain-aliases)
+- [Display domain aliases](#display-domain-aliases)
+
+## API documentation
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/domains
+
+## Definitions
+```
+<DomainAlias> ::= <String>
+<DomainName> ::= <String>(.<String>)+
+```
+## Create a domain
+```
+gam create domain <DomainName>
+```
+## Promote a domain to be primary
+```
+gam update domain <DomainName> primary
+```
+## Delete a domain
+```
+gam delete domain <DomainName>
+```
+## Display domains
+```
+gam info domain [<DomainName>] [formatjson]
+gam show domains [formatjson]
+```
+For `info`, if `<DomainName>` is omitted, information about the primary domain will be displayed.
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print domains [todrive <ToDriveAttribute>*] [formatjson [quotechar <Character>]]
+```
+By default, Gam displays the information as columns of fields.
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Create and delete domain aliases
+```
+gam create domainalias|aliasdomain <DomainAlias> <DomainName>
+gam delete domainalias|aliasdomain <DomainAlias>
+```
+## Display domain aliases
+```
+gam info domainalias|aliasdomain <DomainAlias> [formatjson]
+gam show domainaliases|aliasdomains [formatjson] [formatjson [quotechar <Character>]]
+```
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the fields in JSON format.
+```
+gam print domainaliases|aliasdomains [todrive <ToDriveAttribute>*] [formatjson [quotechar <Character>]]
+```
+By default, Gam displays the information as columns of fields.
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+

docs/Downloads.md

@@ -0,0 +1,74 @@
+# Downloads
+You can download the current GAMADV-XTD3 release from the [GitHub Releases](https://github.com/taers232c/GAMADV-XTD3/releases) page. Choose one of the following:
+
+* Executable Archive, Automatic, Linux/Mac OS/Google Cloud Shell/Raspberry Pi/ChromeOS
+  - Start a terminal session and execute one of the following commands:
+  - New install, default path `$HOME/bin`
+    - `bash <(curl -s -S -L https://raw.githubusercontent.com/taers232c/GAMADV-XTD3/master/src/gam-install.sh)`
+  - New install, specify a path
+    - `bash <(curl -s -S -L https://raw.githubusercontent.com/taers232c/GAMADV-XTD3/master/src/gam-install.sh) -d <Path>`
+  - Update to latest version, do not create project or authorizations, default path `$HOME/bin`
+    - `bash <(curl -s -S -L https://raw.githubusercontent.com/taers232c/GAMADV-XTD3/master/src/gam-install.sh) -l`
+  - Update to latest version, do not create project or authorizations, specify a path
+    - `bash <(curl -s -S -L https://raw.githubusercontent.com/taers232c/GAMADV-XTD3/master/src/gam-install.sh) -l -d <Path>`
+
+By default, a folder, `gamadv-xtd3`, is created in the default or specified path and the files are downloaded into that folder.
+Add the `-s` option to the end of the above commands to suppress creating the `gamadv-xtd3` folder; the files are downloaded directly into the default or specified path.
+
+* Executable Archive, Manual, Linux/Google Cloud Shell
+  - `gamadv-xtd3-6.wx.yz-linux-x86_64-glibc2.35.tar.xz`
+  - `gamadv-xtd3-6.wx.yz-linux-x86_64-glibc2.31.tar.xz`
+  - `gamadv-xtd3-6.wx.yz-linux-x86_64-glibc2.27.tar.xz`
+  - `gamadv-xtd3-6.wx.yz-linux-x86_64-glibc2.23.tar.xz`
+  - `gamadv-xtd3-6.wx.yz-linux-x86_64-glibc2.19.tar.xz`
+  - `gamadv-xtd3-6.wx.yz-linux-x86_64-legacy.tar.xz`
+  - Download the archive, extract the contents into some directory.
+  - Start a terminal session and cd to the install directory.
+
+* Executable Archive, Manual, Raspberry Pi/ChromeOS ARM devices
+  - `gamadv-xtd3-6.wx.yz-linux-arm64-glibc2.31.tar.xz`
+  - `gamadv-xtd3-6.wx.yz-linux-arm64-glibc2.27.tar.xz`
+  - `gamadv-xtd3-6.wx.yz-linux-arm64-glibc2.23.tar.xz`
+  - Download the archive, extract the contents into some directory.
+  - Start a terminal session and cd to the install directory.
+
+* Executable Archive, Manual, Mac OS versions Big Sur, Monterey, Ventura - M1/M2
+  - `gamadv-xtd3-6.wx.yz-macos-arm64.tar.xz`
+  - Download the archive, extract the contents into some directory.
+  - Start a terminal session and cd to the install directory.
+
+* Executable Archive, Manual, Mac OS, versions Big Sur, Monterey, Ventura - Intel
+  - `gamadv-xtd3-6.wx.yz-macos-x86_64.tar.xz`
+  - Download the archive, extract the contents into some directory.
+  - Start a terminal session and cd to the install directory.
+
+* Executable Archive, Manual, Mac OS, versions prior to Big Sur
+  - `gamadv-xtd3-6.wx.yz-macos-x86_64-legacy.tar`
+  - Download the archive, extract the contents into some directory.
+  - Start a terminal session and cd to the install directory.
+
+* Executable Archive, Manual, Windows 64 bit
+  - `gamadv-xtd3-6.wx.yz-windows-x86_64.zip`
+  - Download the archive, extract the contents into some directory.
+  - Start a terminal session and cd to the install directory.
+
+* Executable Installer, Manual, Windows 64 bit
+  - `gamadv-xtd3-6.wx.yz-windows-x86_64.msi`
+  - Download the installer and run it.
+  - Start a Command Prompt/PowerShell session and cd to the install directory.
+
+* Executable Archive, Manual, Windows 32 bit
+  - `gamadv-xtd3-6.wx.yz-windows-x86.zip`
+  - Download the archive, extract the contents into some directory.
+  - Start a terminal session and cd to the install directory.
+
+* Executable Installer, Manual, Windows 32 bit
+  - `gamadv-xtd3-6.wx.yz-windows-x86.msi`
+  - Download the installer and run it.
+  - Start a Command Prompt/PowerShell session and cd to the install directory.
+
+* Source, all platforms
+  - `Source code(zip)`
+  - `Source code(tar.gz)`
+  - Download the archive, extract the contents into some directory.
+  - Start a terminal/Command Prompt/PowerShell session and cd to the install directory.

docs/Drive-File-Selection.md

@@ -0,0 +1,392 @@
+# Drive File Selection
+- [Definitions](#definitions)
+- [Introduction](#introduction)
+- [Select file by ID](#select-file-by-id)
+- [Select files by their characteristics](#select-files-by-their-characteristics)
+  - [Select with Drive File API query](#select-with-drive-file-api-query)
+  - [Select file by name](#select-file-by-name)
+  - [Select file ownership](#select-file-ownership)
+  - [Select MIME type](#select-MIME-type)
+  - [Select file ownership and MIME type](#select-file-ownership-and-mime-type)
+  - [Select based on file size](#select-based-on-file-size)
+  - [Select based on file name](#select-based-on-file-name)
+  - [Select based on permission matching](#select-based-on-permission-matching)
+- [Select root folder](#select-root-folder)
+- [Select a list of file IDs](#select-a-list-of-file-ids)
+- [Select Shared Drive file by ID](#select-shared-drive-file-by-id)
+- [Select Shared Drive file by name](#select-shared-drive-file-by-name)
+- [Select Shared Drive file by query](#select-shared-drive-file-by-query)
+- [Select root folder of a Shared Drive by ID](#select-root-folder-of-a-shared-drive-by-id)
+- [Select root folder of a Shared Drive by name](#select-root-folder-of-a-shared-drive-by-name)
+
+## Definitions
+```
+<DriveFileID> ::= <String>
+        https://drive.google.com/open?id=<DriveFileID>
+        https://drive.google.com/drive/files/<DriveFileID>
+        https://drive.google.com/drive/folders/<DriveFileID>
+        https://drive.google.com/drive/folders/<DriveFileID>?resourcekey=<String>
+        https://drive.google.com/file/d/<DriveFileID>/<String>
+        https://docs.google.com>/document/d/<DriveFileID>/<String>
+        https://docs.google.com>/drawings/d/<DriveFileID>/<String>
+        https://docs.google.com>/forms/d/<DriveFileID>/<String>
+        https://docs.google.com>/presentation/d/<DriveFileID>/<String>
+        https://docs.google.com>/spreadsheets/d/<DriveFileID>/<String>
+<DriveFileItem> ::= <DriveFileID>|<DriveFileURL>
+<DriveFileList> ::= "<DriveFileItem>(,<DriveFileItem>)*"
+<DriveFileIDEntity> ::=
+        (<DriveFileItem>)|(id( |:)<DriveFileItem>)|(ids( |:)<DriveFileList>)
+<DriveFileName> ::= <String>
+<DriveFileNameEntity> ::=
+        (drivefilename <DriveFileName>)|(drivefilename:<DriveFileName>)|
+        (anydrivefilename <DriveFileName>)|(anydrivefilename:<DriveFileName>)
+<DriveFolderID> ::= <String>
+<DriveFolderIDList> ::= "<DriveFolderID>(,<DriveFolderID>)*"
+<DriveFolderName> ::= <String>
+<QueryDriveFile> :: = <String> See: https://developers.google.com/drive/api/v3/search-files
+<DriveFileQueryEntity> ::= 
+        (query <QueryDriveFile>) | (query:<QueryDriveFile>)
+<DriveFileQueryShortcut> ::=
+        all_files |
+        all_folders |
+        all_forms |
+        all_google_files |
+        all_non_google_files |
+        all_shortcuts |
+        all_3p_shortcuts |
+        all_items |
+        my_files |
+        my_folders |
+        my_forms |
+        my_google_files |
+        my_non_google_files |
+        my_shortcuts |
+        my_3p_shortcuts |
+        my_items |
+        my_top_files |
+        my_top_folders |
+        my_top_items |
+        others_files |
+        others_folders |
+        others_forms |
+        others_google_files |
+        others_non_google_files |
+        others_shortcuts |
+        others_3p_shortcuts |
+        others_items |
+        writable_files
+
+<SharedDriveID> ::= <String>
+<SharedDriveName> ::= <String>
+<SharedDriveIDEntity> ::= (teamdriveid <DriveFileItem>) | (teamdriveid:<DriveFileItem>)
+<SharedDriveNameEntity> ::= (teamdrive <SharedDriveName>) | (teamdrive:<SharedDriveName>)
+<SharedDriveFileNameEntity> ::= (teamdrivefilename <DriveFileName>) | (teamdrivefilename:<DriveFileName>)
+
+<SharedDriveEntity> ::=
+        <SharedDriveIDEntity> |
+        <SharedDriveNameEntity>
+<SharedDriveAdminQueryEntity> ::=
+        (teamdriveadminquery <QueryTeamDrive>) | (teamdriveadminquery:<QueryTeamDrive>)
+<SharedDriveFileQueryEntity> ::= 
+        (query <QueryDriveFile>) | (query:<QueryDriveFile>)
+<SharedDriveFileQueryShortcut> ::=
+        all_files | all_folders | all_google_files | all_non_google_files | all_items
+<SharedDriveEntityAdmin> ::=
+        <SharedDriveIDEntity> |
+        <SharedDriveNameEntity>|
+        <SharedDriveAdminQueryEntity>
+<DriveFileEntity> ::=
+        <DriveFileIDEntity> |
+        <DriveFileNameEntity> |
+        <DriveFileQueryEntity> |
+        <DriveFileQueryShortcut> |
+        mydrive | mydriveid |
+        root | rootid |
+        <SharedDriveIDEntity> [<SharedDriveFileQueryShortcut>] |
+        <SharedDriveNameEntity> [<SharedDriveFileQueryShortcut>] |
+        <SharedDriveFileNameEntity> |
+        <SharedDriveFileQueryEntity> |
+        <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVSubkeySelector>) | <CSVDataSelector>)
+```
+## Introduction
+Many Gam commands operate on Google Drive files, there are multiple ways to specify the file on which to operate.
+The Google Drive REST API can only manipulate files by ID; you either specify an ID or an option that will produce an ID.
+
+## Select file by ID
+Select a file by giving its unique ID.
+
+There are multiple formats for backwards compatibility with old Gam commands that used different formats to specify the same data.
+```
+<DriveFileIDEntity> ::=
+        <DriveFileItem> |
+        (id <DriveFileItem>) | (id:<DriveFileItem>) |
+        (ids <DriveFileList>) | (ids:<DriveFileList>)
+```
+### Examples
+```
+gam user testuser show fileinfo 1234ABCD
+gam user testuser show fileinfo id 1234ABCD
+gam user testuser show fileinfo id:1234ABCD
+gam user testuser show fileinfo https://drive.google.com/a/domain.com/file/d/1234ABCD
+gam user testuser show fileinfo ids "1234ABCD,5678EFGH"
+gam user testuser show fileinfo ids:"1234ABCD,5678EFGH"
+```
+## Select files by their characteristics
+The `print|show filetree|filelist` have variety of options for choosing the files to display.
+
+## Select with Drive File API query
+The Google Drive API has a query option that you can use to select files.
+* https://developers.google.com/drive/api/v3/search-files
+* https://developers.google.com/drive/api/v3/ref-search-terms
+
+```
+<DriveFileQueryEntity> ::= 
+        (query <QueryDriveFile>) | (query:<QueryDriveFile>)
+```
+The default query for selecting files is `'me' in owners`; all files and folders in `My Drive` that the user owns.
+You can specify multiple `query <QueryDriveFile>` and `query:<QueryDriveFile>` options.
+Each one is appended to the default/existing query with `and (<QueryDriveFile>)`.
+
+The are several options manipulate the query.
+
+## Select file by name
+If you have a file name, a search must be performed to find the ID that matches the name.
+Remember, searching for a file by name may return several file IDs if you have multiple files with the same name.
+
+There are multiple formats for backwards compatibility with old Gam commands that used different formats to specify the same data.
+If a drive file name contains spaces or commas, it must be enclosed in quotes.
+```
+<DriveFileNameEntity> ::=
+        (anyname <DriveFileName>) | (anyname:<DriveFileName>) | (anydrivefilename <DriveFileName>) | (anydrivefilename:<DriveFileName>) |
+        (name <DriveFileName>) | (name:<DriveFileName>) | (drivefilename <DriveFileName>) | (drivefilename:<DriveFileName>) |
+        (othername <DriveFileName>) | (othername:<DriveFileName>) | (otherdrivefilename <DriveFileName>) | (otherdrivefilename:<DriveFileName>)
+```
+* `anyname <DriveFileName>` - `(name = '<DriveFileName>')`
+* `anyname:<DriveFileName>` - `(name = '<DriveFileName>')`
+* `anydrivefilename <DriveFileName>` - `(name = '<DriveFileName>')`
+* `anydrivefilename:<DriveFileName>` - `(name = '<DriveFileName>')`
+* `name <DriveFileName>` - `('me' in owners and name = '<DriveFileName>')`
+* `name:<DriveFileName>` - `('me' in owners and name = '<DriveFileName>')`
+* `drivefilename <DriveFileName>` - `('me' in owners and name = '<DriveFileName>')`
+* `drivefilename:<DriveFileName>` - `('me' in owners and name = '<DriveFileName>')`
+* `othername <DriveFileName>` - `(not 'me' in owners and name = '<DriveFileName>')`
+* `othername:<DriveFileName>` - `(not 'me' in owners and name = '<DriveFileName>')`
+* `otherdrivefilename <DriveFileName>` - `(not 'me' in owners and name = '<DriveFileName>')`
+* `otherdrivefilename:<DriveFileName>` - `(not 'me' in owners and name = '<DriveFileName>')`
+
+### Examples
+```
+gam user testuser show fileinfo drivefilename "Test File"
+gam user testuser show fileinfo drivefilename:"Test File"
+gam user testuser show fileinfo anydrivefilename "Test File"
+gam user testuser show fileinfo anydrivefilename:"Test File"
+```
+## Select file ownership
+By default, files the user owns are sisplayed; you can select the ownership characteristic.
+```
+anyowner|(showownedby any|me|others)
+```
+* `showownedby any` or `anyowner` - Removes `'me' in owners` and `not 'me' in owners` from the query
+* `showownedby me` - Adds `'me' in owners` to the query
+* `showownedby others` - Adds `not 'me' in owners` to the query
+
+## Select MIME type
+
+By default, all types of files and folders are displayed; you can specify a list of MIME types to display or a list of MIME types to suppress.
+```
+<MimeTypeShortcut> ::=
+        gdoc|gdocument|
+        gdrawing|
+        gfile|
+        gfolder|gdirectory|
+        gform|
+        gfusion|
+        gjam|
+        gmap|
+        gpresentation|
+        gscript|
+        gshortcut|
+        g3pshortcut|
+        gsheet|gspreadsheet|
+        gsite
+<MimeTypeName> ::= application|audio|font|image|message|model|multipart|text|video
+<MimeType> ::= <MimeTypeShortcut>|(<MimeTypeName>/<String>)
+<MimeTypeList> ::= "<MimeType>(,<MimeType>)*"
+```
+This is the mapping from `<MimeTypeShortcut>` to MIME type.
+* `gdoc|gdocument` - 'application/vnd.google-apps.document
+* `gdrawing` - application/vnd.google-apps.drawing
+* `gfile` - application/vnd.google-apps.file
+* `gfolder|gdirectory` - application/vnd.google-apps.folder
+* `gform` - application/vnd.google-apps.form
+* `gfusion|gfusiontable` - application/vnd.google-apps.fusiontable
+* `gjam` - application/vnd.google-apps.jam
+* `gmap` - application/vnd.google-apps.map
+* `gpresentation` - application/vnd.google-apps.presentation
+* `gscript` - application/vnd.google-apps.script
+* `gshortcut` - application/vnd.google-apps.shortcut
+* `g3pshortcut` - application/vnd.google-apps.drive-sdk
+* `gsite` - application/vnd.google-apps.site
+* `gsheet|gspreadsheet` - application/vnd.google-apps.spreadsheet
+
+Display files and folders with specified MIME types
+```
+showmimetype <MimeTypeList>
+```
+Adds `(mimeType = '<MimeType>' or mimeType = '<MimeType>' ...)` to the query,
+
+Display files and folders with MIME types other than those specified
+```
+showmimetype not <MimeTypeList>
+```
+Adds `(mimeType != '<MimeType>' and mimeType != '<MimeType>' ...)` to the query.
+
+## Select file ownership and MIME type
+The options combine ownership and broad MIME type selections.
+```
+<DriveFileQueryShortcut> ::=
+        all_files | all_folders | all_google_files | all_non_google_files | all_items |
+        my_files | my_folders | my_google_files | my_non_google_files | my_items |
+        my_top_files | my_top_folders | my_top_items |
+        others_files | others_folders | others_google_files | others_non_google_files | others_items |
+        writable_files
+```
+* all_files - "mimeType != application/vnd.google-apps.folder"
+* all_folders - "mimeType = application/vnd.google-apps.folder"
+* all_google_files - "mimeType != application/vnd.google-apps.folder and mimeType contains 'vnd.google'"
+* all_non_google_files - "not mimeType contains 'vnd.google'"
+* all_items - "" (An empty query specifies all files and folders)
+* my_files - "'me' in owners and mimeType != application/vnd.google-apps.folder"
+* my_folders - "'me' in owners and mimeType = application/vnd.google-apps.folder"
+* my_google_files - "'me' in owners and mimeType != application/vnd.google-apps.folder and mimeType contains 'vnd.google'"
+* my_non_google_files - "'me' in owners and not mimeType contains 'vnd.google'"
+* my_items - "'me' in owners"
+* my_top_files - "'me' in owners and mimeType != application/vnd.google-apps.folder and 'root' in parents"
+* my_top_folders - "'me' in owners and mimeType = application/vnd.google-apps.folder and 'root' in parents"
+* my_top_items - "'me' in owners and 'root' in parents"
+* others_files - "not 'me' in owners and mimeType != application/vnd.google-apps.folder"
+* others_folders - "not 'me' in owners and mimeType = application/vnd.google-apps.folder"
+* others_google_files - "not 'me' in owners and mimeType != application/vnd.google-apps.folder and mimeType contains 'vnd.google'"
+* others_non_google_files - "not 'me' in owners and not mimeType contains 'vnd.google'"
+* others_items - "not 'me' in owners"
+* writable_files - "'me' in writers and mimeType != application/vnd.google-apps.folder"
+
+## Select based on file size
+For these filters, GAM processes then after the list of files is downloaded. You can combine these
+options `query <QueryDriveFile>` to minimize the number of files downloaded but they also work with other
+file selection options.
+
+Limit the display to files with binary content of size greater than or equal to a number of bytes.
+```
+minimumfilesize <Integer>`
+```
+## Select based on file name
+The Google Drive API has limited name matching in the query; Limit the display to files whose name matches `<RegularExpression>`.
+```
+filenamematchpattern <RegularExpression>`
+```
+## Select based on permission matching
+Use [Permission matches](#permission-matches) to limit the display to files with matching permissions.
+
+### Examples
+```
+gam user testuser show fileinfo query "name='Test File'"
+gam user testuser show fileinfo query:"name='Test Folder' and mimeType=application/vnd.google-apps.folder"
+gam user testuser print filelist my_non_google_files
+```
+## Select root folder
+```
+root|mydrive
+```
+Examples
+```
+gam user testuser show fileinfo root
+```
+## Select a list of file IDs
+You can select a list of file IDs by referencing files that contain file IDs.
+```
+<DriveFileEntity> ::=
+        <FileSelector> | <CSVFileSelector> | <CSVkmdSelector> | <CSVSubkeySelector>) | <CSVDataSelector>)
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+```
+* [Collections of Items](Collections-of-Items)
+
+## Select Shared Drive file by ID
+Select a Shared Drive file by giving its unique ID.
+```
+<SharedDriveIDEntity> ::=
+        <DriveFileItem> |
+        (teamdriveid <DriveFileItem>) | (teamdriveid:<DriveFileItem>)
+```
+### Examples
+```
+gam user testuser show fileinfo 1234ABCD
+gam user testuser show fileinfo id 1234ABCD
+gam user testuser show fileinfo teamdriveid 1234ABCD
+```
+## Select Shared Drive file by name
+If you have the name, a search must be performed to find the ID that matches the name.
+You must specify the Shared Drive, either by ID or name, and the name of the file.
+
+Remember, searching for a file by name may return several file IDs if you have multiple files with the same name.
+```
+<SharedDriveIDEntity> ::=
+        (teamdriveid <DriveFileItem>) | (teamdriveid:<DriveFileItem>)
+<SharedDriveNameEntity> ::=
+        (teamdrive <SharedDriveName>) | (teamdrive:<SharedDriveName>)
+<SharedDriveFileNameEntity> ::=
+        (teamdrivefilename <DriveFileName>) | (teamdrivefilename:<DriveFileName>)
+```
+### Examples
+```
+gam user testuser show fileinfo teamdriveid 1234ABCD teamdrivefilename  "Test File"
+gam user testuser show fileinfo teamdrive "Shared Drive 1"  teamdrivefilename "Test File"
+```
+## Select Shared Drive file by query
+You can use a query to find a file ID. You perform the query on all Shared Drives or a specific Shared Drive.
+
+See: [Drive Query](https://developers.google.com/drive/api/v3/search-files)
+```
+<SharedDriveFileQueryEntity> ::=
+        (teamdrivequery <QueryDriveFile>) | (teamdrivequery:<QueryDriveFile>)
+<SharedDriveFileQueryShortcut> ::=
+        all_files | all_folders | all_google_files | all_non_google_files | all_items
+```
+Keyword to query mappings for `<DriveFileQueryShortcut>`:
+* all_files - "mimeType != application/vnd.google-apps.folder"
+* all_folders - "mimeType = application/vnd.google-apps.folder"
+* all_google_files - "mimeType != application/vnd.google-apps.folder and mimeType contains 'vnd.google'"
+* all_non_google_files - "not mimeType contains 'vnd.google'"
+* all_items - "" (An empty query specifies all files and folders)
+
+### Examples
+```
+gam user testuser show fileinfo teamdrivequery "name='Test File'"
+gam user testuser show fileinfo teamdriveid 1234ABCD teamdrivequery "name='Test File'"
+gam user testuser show fileinfo teamdrive teamdrive "Shared Drive 1" teamdrivequery "name='Test File'"
+gam user testuser show fileinfo teamdriveid 1234ABCD all_non_google_files
+```
+## Select root folder of a Shared Drive by ID
+The root folder of a Shared Drive is a folder, you select it by giving its unique ID.
+```
+<SharedDriveIDEntity> ::=
+        <DriveFileItem> |
+        (teamdriveid <DriveFileItem>) | (teamdriveid:<DriveFileItem>)
+```
+### Examples
+```
+gam user testuser show fileinfo 1234ABCD
+gam user testuser show fileinfo teamdriveid 1234ABCD
+
+```
+## Select root folder of a Shared Drive by name
+If you have a Shared Drive name, a search must be performed to find the ID that matches the name.
+```
+<SharedDriveNameEntity> ::=
+        (teamdrive <SharedDriveName>) | (teamdrive:<SharedDriveName>)
+```
+### Examples
+```
+gam user testuser show fileinfo teamdrive "Shared Drive 1"
+
+```

docs/Drive-Items.md

@@ -0,0 +1,39 @@
+# Drive Items
+- [Basic Items](Basic-Items)
+- [List Items](List-Items)
+```
+<DriveFileID> ::= <String>
+<DriveFileURL> ::=
+        https://drive.google.com/open?id=<DriveFileID>
+        https://drive.google.com/drive/files/<DriveFileID>
+        https://drive.google.com/drive/folders/<DriveFileID>
+        https://drive.google.com/drive/folders/<DriveFileID>?resourcekey=<String>
+        https://drive.google.com/file/d/<DriveFileID>/<String>
+        https://docs.google.com>/document/d/<DriveFileID>/<String>
+        https://docs.google.com>/drawings/d/<DriveFileID>/<String>
+        https://docs.google.com>/forms/d/<DriveFileID>/<String>
+        https://docs.google.com>/presentation/d/<DriveFileID>/<String>
+        https://docs.google.com>/spreadsheets/d/<DriveFileID>/<String>
+<DriveFileItem> ::= <DriveFileID>|<DriveFileURL>
+<DriveFileName> ::= <String>
+<DriveFileIDEntity> ::=
+        <DriveFileItem> |
+        (id <DriveFileItem>) | (id:<DriveFileItem>) |
+        (ids <DriveFileList>) | (ids:<DriveFileList>)
+<DriveFileNameEntity> ::=
+        (name <DriveFileName>) | (name:<DriveFileName>) |
+        (drivefilename <DriveFileName>) | (drivefilename:<DriveFileName>) |
+        (anyname <DriveFileName>) | (anyname:<DriveFileName>) |
+        (anydrivefilename <DriveFileName>) | (anydrivefilename:<DriveFileName>)
+<SharedDriveIDEntity> ::=
+        <DriveFileItem> |
+        (teamdriveid <DriveFileItem>) | (teamdriveid:<DriveFileItem>)
+<SharedDriveName> ::= <String>
+<SharedDriveNameEntity> ::=
+        (teamdrive <SharedDriveName>) | (teamdrive:<SharedDriveName>)
+<SharedDriveEntity> ::=
+        <SharedDriveIDEntity> |
+        <SharedDriveNameEntity>
+<SharedDriveFileNameEntity> ::=
+        (teamdrivefilename <DriveFileName>) | (teamdrivefilename:<DriveFileName>)
+```

docs/Drive-REST-API-v3.md

@@ -0,0 +1,93 @@
+All Google Drive API calls have been converted from v2 to v3, see: https://developers.google.com/drive/v3/web/migration
+Many of the changes are internal to Gam and have no visible effect. Google has modified/renamed many field names and these will affect scripts that parse the output from `gam print/show drivesettings/drivefileacls/fileinfo/filelist/filerevisions`. Additionally, Google has dropped some fields and their values are no longer available. On input, Gam accepts both the old and new field names.
+
+A variable, `drive_v3_native_names` (default value is True), has been added to `gam.cfg` to control the field names on output: when True, the v3 native field names are used; when False, the v3 native field names are mapped to the v2 field names.
+
+If you have scripts that process the output from these print commands, you may have to make modifications to your scripts.
+Run your print/show commands with a version of Standard Gam and save the output.
+With drive_v3_native_names = False, run your print/show commands with this version of Gam and compare the output to that saved in the previous run;
+modify your scripts that process the output as appropriate.
+
+There is a cost to mapping the v3 field names back to the v2 field names; you can avoid this cost by setting drive_v3_native_names = True,
+running your print/show commands, comparing the output and making the appropriate script modifications.
+```
+print/show drivesettings
+Dropped fields:
+        DRIVE
+        GMAIL
+        PHOTOS
+        domainSharingPolicy
+        lauguageCode
+Renamed fields (Old->New):
+        name->displayName,
+        quotaBytesTotal->limit
+        quotaBytesUsed->usageInDrive
+        quotaBytesUsedAggregate->usage
+        quotaBytesUsedInTrash->usageInDriveTrash
+
+print/show drivefileacls
+Dropped fields:
+        authKey
+Renamed fields (Old->New):
+        name->displayName
+        withLink->allowFileDiscovery
+
+print/show fileinfo/filelist
+Dropped fields:
+        defaultOpenWithLink
+        embedLink
+        exportLinks
+        labels(hidden)
+        markedViewedByMeDate
+        openWithLinks
+        selfLink
+        parents(isRoot)
+        parents(parentLink)
+        parents(selfLink)
+        permissions(selfLink)
+        selfLink
+        userPermission(selfLink)
+Renamed fields (Old->New):
+        alternateLink->webViewLink
+        capabilities(canChangeRestrictedDownload)->capabilities(canChangeViewersCanCopyContent)
+        createdDate->createdTime
+        expirationDate->expirationTime
+        fileSize->size
+        lastViewedByMeDate->viewedByMeTime
+        modified->modifiedByMe
+        modifiedByMeDate->modifiedByMeTime
+        modifiedDate->modifiedTime
+        restricted->viewersCanCopyContent
+        sharedWithMeDate->sharedWithMeTime
+        title->name
+        trashedDate->trashedTime
+        viewed->viewedByMe
+        withLink->allowFileDiscovery
+
+print/show filerevisions
+Dropped fields:
+        exportLinks
+        publishedLink
+        selfLink
+Renamed fields (Old->New):
+        fileSize->size
+        isAuthenticatedUser->me
+        modifiedDate->modifiedTie
+        picture.url->photoLink
+        pinned->keepForever
+```
+The parents field of a file has undergone the most change. In Drive v2 it was a list of compound items with three sub-fields per item: id, isRoot, parentLink.
+In Drive v3 the parents field is a list of simple items, the parent ids. The following examples show how the parents field is output in a CSV file for a file with two parents.
+```
+Previous versions of Gam:
+Owner,title,parents,parents.0.isRoot,parents.0.id,parents.0.parentLink,parents.1.isRoot,parents.1.id,parents.1.parentLink
+testuser@domain.com,TestFile,2,True,PPPP1111,https://www.googleapis.com/drive/v2/files/PPPP1111,False,PPPP2222,https://www.googleapis.com/drive/v2/files/PPPP2222
+
+Current version of Gam with drive_v3_name_names = false
+Owner,title,parents,parents.0.id,parents.1.id
+testuser@domain.com,TestFile,2,PPPP1111,PPPP2222
+
+Current version of Gam with drive_v3_name_names = true
+Owner,name,parents
+testuser@domain.com,TestFile,PPPP1111 PPPP2222
+```

docs/Email-Audit-Monitor.md

@@ -0,0 +1,42 @@
+# Email Audit Monitor
+- [API documentation](#api-documentation)
+- [Notes](#notes)
+- [Definitions](#definitions)
+- [Create Email Audit Monitor](#create-email-audit-monitor)
+- [Delete Email Audit Monitor](#delete-email-audit-monitor)
+- [Display Email Audit Monitors](#display-email-audit-monitors)
+
+## API documentation
+* https://developers.google.com/admin-sdk/email-audit
+
+## Notes
+To use these features you must add the `Email Audit API` to your project and authorize the appropriate scopes:
+* `Client Access` - `Email Audit API`
+```
+gam update project
+gam oauth create
+```
+
+## Definitions
+```
+<DateTime> ::=
+        <Year>-<Month>-<Day>(<Space>|T)<Hour>:<Minute> |
+        (+|-)<Number>(m|h|d|w|y) |
+        never|
+        now|today
+<DomainName> ::= <String>(.<String>)+
+<EmailAddress> ::= <String>@<DomainName>
+```
+## Create Email Audit Monitor
+```
+gam audit monitor create <EmailAddress> <DestEmailAddress> [begin <DateTime>] [end <DateTime>]
+        [incoming_headers] [outgoing_headers] [nochats] [nodrafts] [chat_headers] [draft_headers]
+```
+## Delete Email Audit Monitor
+```
+gam audit monitor delete <EmailAddress> <DestEmailAddress>
+```
+## Display Email Audit Monitors
+```
+gam audit monitor list <EmailAddress>
+```

docs/Find-File-Owner.md

@@ -0,0 +1,54 @@
+# Find File Owner
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Display File Ownership](#display-file-ownership)
+- [Display File Ownership for Old files](#display-file-ownership-for-old-files)
+
+## API documentation
+* https://developers.google.com/admin-sdk/reports/v1/reference/activities
+
+## Definitions
+```
+<DriveFileID> ::= <String>
+<DriveFileName> ::= <String>
+```
+
+## Display File Ownership
+These commands use the Reports API audit activity and may not find the owner if the file has not been accessed in 180 days.
+If you specify a `<DriveFileID>`, there will be at most one line of output. If you specify a `<DriveFileName>`, there will be
+one line of output for each distinct file with that name.
+
+The Reports API calls are:
+* `ownership <DriveFileID>` - `gam report drive filter "doc_id==<DriveFileID>"`
+* `ownership drivefilename <DriveFileName>` - `gam report drive filter "doc_title==<DriveFileName>"`
+
+```
+gam show ownership <DriveFileID>|(drivefilename <DriveFileName>)
+        [formatjson]
+```
+By default, Gam displays the information as a list of keys and values.
+* `formatjson` - Display the output in JSON notation
+
+```
+gam print ownership <DriveFileID>|(drivefilename <DriveFileName>) [todrive <ToDriveAttribute>*]
+        (addcsvdata <FieldName> <String>)*
+        [formatjson [quotechar <Character>]]
+```
+* `addcsvdata <FieldName> <String>` - Add additional columns of data from the command line to the output
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format.
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display File Ownership for Old files
+If the above commands fail, you can try to loop through all accounts, however this might take a long time if you are on a large Google Workspace Account.
+
+```
+gam config auto_batch_min 1 multiprocessexit rc=0 redirect csv - multiprocess redirect stderr null multiprocess all users print filelist select id <DriveFileID> fields id,name,owners.emailaddress norecursion showownedby any
+gam config auto_batch_min 1 multiprocessexit rc=0 redirect csv - multiprocess redirect stderr null multiprocess all users print filelist select name <DriveFileName> fields id,name,owners.emailaddress norecursion showownedby any
+```

docs/GAM-Return-Codes.md

@@ -0,0 +1,64 @@
+# GAM Return Codes
+
+These are the return codes used by GAMADV-XTD3.
+
+```
+SUCCESS_RC = 0
+UNKNOWN_ERROR_RC = 1
+USAGE_ERROR_RC = 2
+SOCKET_ERROR_RC = 3
+GOOGLE_API_ERROR_RC = 4
+NETWORK_ERROR_RC = 5
+FILE_ERROR_RC = 6
+MEMORY_ERROR_RC = 7
+KEYBOARD_INTERRUPT_RC = 8
+HTTP_ERROR_RC = 9
+SCOPES_NOT_AUTHORIZED_RC = 10
+DATA_ERROR_RC = 11
+API_ACCESS_DENIED_RC = 12
+CONFIG_ERROR_RC = 13
+SYSTEM_ERROR_RC = 14
+NO_SCOPES_FOR_API_RC = 15
+CLIENT_SECRETS_JSON_REQUIRED_RC = 16
+OAUTH2SERVICE_JSON_REQUIRED_RC = 16
+OAUTH2_TXT_REQUIRED_RC = 16
+INVALID_JSON_RC = 17
+JSON_ALREADY_EXISTS_RC = 17
+AUTHENTICATION_TOKEN_REFRESH_ERROR_RC = 18
+HARD_ERROR_RC = 19
+# Information
+ENTITY_IS_A_USER_RC = 20
+ENTITY_IS_A_USER_ALIAS_RC = 21
+ENTITY_IS_A_GROUP_RC = 22
+ENTITY_IS_A_GROUP_ALIAS_RC = 23
+ENTITY_IS_AN_UNMANAGED_ACCOUNT_RC = 24
+CHECK_USER_GROUPS_ERROR_RC = 29
+ORPHANS_COLLECTED_RC = 30
+# Warnings/Errors
+ACTION_FAILED_RC = 50
+ACTION_NOT_PERFORMED_RC = 51
+INVALID_ENTITY_RC = 52
+BAD_REQUEST_RC = 53
+ENTITY_IS_NOT_UNIQUE_RC = 54
+DATA_NOT_AVALIABLE_RC = 55
+ENTITY_DOES_NOT_EXIST_RC = 56
+ENTITY_DUPLICATE_RC = 57
+ENTITY_IS_NOT_AN_ALIAS_RC = 58
+ENTITY_IS_UKNOWN_RC = 59
+NO_ENTITIES_FOUND_RC = 60
+INVALID_DOMAIN_RC = 61
+INVALID_DOMAIN_VALUE_RC = 62
+INVALID_TOKEN_RC = 63
+JSON_LOADS_ERROR_RC = 64
+MULTIPLE_DELETED_USERS_FOUND_RC = 65
+MULTIPLE_PROJECT_FOLDERS_FOUND_RC = 65
+STDOUT_STDERR_ERROR_RC = 66
+INSUFFICIENT_PERMISSIONS_RC = 67
+REQUEST_COMPLETED_NO_RESULTS_RC = 71
+REQUEST_NOT_COMPLETED_RC = 72
+SERVICE_NOT_APPLICABLE_RC = 73
+TARGET_DRIVE_SPACE_ERROR_RC = 74
+USER_REQUIRED_TO_CHANGE_PASSWORD_ERROR_RC = 75
+USER_SUSPENDED_ERROR_RC = 76
+NO_CSV_DATA_TO_UPLOAD_RC = 77
+```

docs/GAM-with-minimal-GCP-rights.md

@@ -0,0 +1,41 @@
+# GAM setup with minimal GCP permissions.
+
+- GCP Admin can create a project for the Workspace / GAM admin.
+
+- GAM admin needs following permissions on the created project resource:
+
+```
+clientauthconfig.brands.create
+clientauthconfig.brands.update
+clientauthconfig.clients.create
+clientauthconfig.clients.createSecret
+clientauthconfig.clients.delete
+clientauthconfig.clients.get
+clientauthconfig.clients.getWithSecret
+clientauthconfig.clients.list
+clientauthconfig.clients.listWithSecrets
+clientauthconfig.clients.update
+iam.serviceAccountKeys.create
+iam.serviceAccounts.create
+iam.serviceAccounts.list
+iam.serviceAccounts.setIamPolicy
+oauthconfig.testusers.get
+oauthconfig.verification.get
+resourcemanager.projects.get
+serviceusage.services.enable
+serviceusage.services.get
+serviceusage.services.list
+```
+Reasons for permission by service:
+| Service(s) | Reason |
+|---------|--------|
+| clientauthconfig and oauthconfig | Manage the [OAuth Consent Page](https://developers.google.com/workspace/guides/configure-oauth-consent) |
+| iam | Manage service accounts and their keys |
+| serviceusage | Enable Google API services |
+| resourcemanager | Read basic project info |
+
+- Once GAM admin has rights to the new project they can complete setup with:
+```
+gam use project
+```
+admin will be prompted for the project ID.

docs/GAMADV-XTD3-on-Android-Devices.md

@@ -0,0 +1,16 @@
+# GAMADV-XTD3 on Android Devices
+GAMADV-XTD3 now runs on 64-bit Android devices such as Google's Pixel phones. The installation requires an app that adds the Linux environment to Android such as [UserLAnd](https://play.google.com/store/apps/details?id=tech.ula&hl=en_US).
+
+_Note: Chromebooks / Chrome OS devices should install GAMADV-XTD3 using [these instructions](GAMADV-XTD3-on-Chrome-OS-Devices)._
+
+1. Install the [UserLAnd](https://play.google.com/store/apps/details?id=tech.ula&hl=en_US) app.
+2. Click Debian to install a Debian environment.
+3. Set a username and password.
+4. Choose SSH for connection type.
+5. Once setup, login with the password to get to a Linux shell.
+6. Run the following commands to install prerequisites:
+```
+    sudo apt update
+    sudo apt install curl python3
+```
+7. [How to Install Advanced GAM](How-to-Install-Advanced-GAM)

docs/GAMADV-XTD3-on-Chrome-OS-Devices.md

@@ -0,0 +1,14 @@
+# GAMADV-XTD3 on Chrome OS Devices
+Chrome OS devices that [support Linux apps](https://support.google.com/chromebook/answer/9145439?hl=en) can run GAMADV-XTD3. This includes Intel/AMD x86_64 Chromebooks as well as ARM-based Chromebooks with Mediatek or Rockchip 64-bit CPUs.
+
+1. [Set up Linux on your Chromebook](https://support.google.com/chromebook/answer/9145439?hl=en).
+1. From the Terminal app, run the following commands:
+```
+    sudo apt update
+    sudo apt install xz-utils
+```
+3. [How to Install Advanced GAM](How-to-Install-Advanced-GAM)
+
+# Google cloud shell
+
+Note that from a Chrome OS device, it might be just as easy to use [Google Cloud Shell](https://cloud.google.com/shell). Especially if you are concerned about network connectivity and/or bandwidth, using a shell instance within Google's server infrastructure is always going to be less resource intensive than sending data back and forth between a Google API and your local machine on your local network.

docs/Google-Data-Transfers.md

@@ -0,0 +1,82 @@
+# Google Data Transfers
+- [API documentation](#api-documentation)
+- [Definitions](#definitions)
+- [Display transfer apps](#display-transfer-apps)
+- [Create transfers](#create-transfers)
+- [Display transfers](#display-transfers)
+
+## API documentation
+* https://developers.google.com/admin-sdk/data-transfer/v1/reference/transfers
+* https://support.google.com/a/answer/1247799
+    * Explains how background drive transfers work, including orphaned files, trashed file behaviour (not transfered), etc.
+
+## Definitions
+```
+<DataTransferService> ::=
+        calendar|
+        currents|
+        datastudio|lookerstudio|"google data studio"|
+        drive|gdrive|googledrive|"drive and docs"
+<DataTransferServiceList> ::= "<DataTransferService>(,<DataTransferService>)*"
+
+<UniqueID> ::= id:<String>
+<UserItem> ::= <EmailAddress>|<UniqueID>|<String>
+<OldOwnerID> ::= <UserItem>
+<NewOwnerID> ::= <UserItem>
+<TransferID> ::= <String>
+```
+
+## Display transfer apps
+```
+gam print|show transferapps
+```
+
+## Create transfers
+```
+gam create|add datatransfer|transfer <OldOwnerID> <DataTransferServiceList> <NewOwnerID>
+        [private|shared|all] [privacy_level private|shared|private,shared]
+        [releaseresources [<Boolean>]]
+        (<ParameterKey> <ParameterValue>)*
+        [wait <Integer> <Integer>]
+```
+For`datastudio` and `drive`, there are options to control the privacy level of the files to be transferred.
+* `private` or `privacy_level private` - Transfer files that are not shared with anyone
+* `shared` or `privacy_level shared` - Transfer files shared with at least one other user; this is the **default**
+* `all` or `privacy_level private,shared` - Transfer all files
+
+For calendars, there is an option to indicate whether to release resources for future events.
+* `releaseresources false` - Do not release resources for future events; this is the default.
+* `releaseresources` or `releaseresources true` - Release resources for future events
+
+A `<TransferID>` is returned which can be used to monitor the progress of the transfer.
+
+NOTE: For calendars, the behaviour is not sufficiently defined in the API documentation.
+As of 2020-06-10, background transfers only transfer future non-private events with at least one guest/resource.
+
+The option `<ParameterKey> <ParameterValue>` is for future expansion.
+
+By default, GAM does not wait for the transfer to complete. The option `wait <Integer> <Integer>` causes GAM to wait
+for the transfer to complete. The first `<Integer>` must be in the range 5-60 and is the number
+of seconds between checks to see if the transfer has completed. The second `<Integer>` is the maximum number of checks to perform.
+
+## Display transfers
+```
+gam info datatransfer|transfer <TransferID>
+gam show datatransfers|transfers
+        [olduser|oldowner <UserItem>] [newuser|newowner <UserItem>]
+        [status completed|failed|inprogress|<String>] [delimiter <Character>]
+gam print datatransfers|transfers [todrive <ToDriveAttribute>*]
+        [olduser|oldowner <UserItem>] [newuser|newowner <UserItem>]
+        [status completed|failed|inprogress|<String>] [delimiter <Character>]
+        (addcsvdata <FieldName> <String>)*
+```
+By default, all data transfer operations are printed, use these options to select specific transfers.
+* `olduser|oldowner <UserItem>`
+* `newuser|newowner <UserItem>`
+* `status completed|failed|inprogress`
+
+By default, the entries in lists of items are separated by the `csv_output_field_delimiter` from `gam.cfg`.
+* `delimiter <Character>` - Separate list items with `<Character>`
+
+Add additional columns of data from the command line to the output
+* `addcsvdata <FieldName> <String>`

docs/Google-Network-Addresses.md

@@ -0,0 +1,58 @@
+# Google Network Addresses
+
+All GAM calls are made on port 443 (HTTPS) to the following addresses:
+```
+https://dns.google
+https://accounts.google.com
+
+https://accesscontextmanager.googleapis.com
+https://admin.googleapis.com
+https://alertcenter.googleapis.com
+https://audit.googleapis.com
+https://calendar.googleapis.com
+https://chat.googleapis.com
+https://chromemanagement.googleapis.com
+https://chromepolicy.googleapis.com
+https://classroom.googleapis.com
+https://cloudidentity.googleapis.com
+https://cloudresourcemanager.googleapis.com
+https://contacts.googleapis.com
+https://datastudio.googleapis.com
+https://docs.googleapis.com
+https://drive.googleapis.com
+https://driveactivity.googleapis.com
+https://drivelabels.googleapis.com
+https://forms.googleapis.com
+https://gmail.googleapis.com
+https://groupsmigration.googleapis.com
+https://groupssettings.googleapis.com
+https://keep.googleapis.com
+https://iam.googleapis.com
+https://iap.googleapis.com
+https://licensing.googleapis.com
+https://oauth2.googleapis.com
+https://people.googleapis.com
+https://pubsub.googleapis.com
+https://reseller.googleapis.com
+https://sheets.googleapis.com
+https://siteverification.googleapis.com
+https://storage.googleapis.com
+https://tasks.googleapis.com
+https://vault.googleapis.com
+https://versionhistory.googleapis.com
+https://www.googleapis.com
+```
+Other addresses used to support GAM but not directly accessed by GAM.
+```
+https://admin.google.com
+https://console.cloud.google.com
+https://www.google.com
+
+https://api.github.com
+https://raw.githubusercontent.com
+```
+
+The following command introduced in 6.25.15 can be used to verify the Google connections.
+```
+gam checkconnection
+```

docs/Groups-Membership.md

@@ -0,0 +1,803 @@
+ Groups - Membership
+- [API documentation](#api-documentation)
+- [Python Regular Expressions](Python-Regular-Expressions) Match function
+- [Definitions](#definitions)
+- [Collections of Users](#collections-of-users)
+  - [Select users based on suspension state](#select-users-based-on-suspension-state)
+  - [Select users based on archived state](#select-users-based-on-archived-state)
+- [Add members to a group](#add-members-to-a-group)
+- [Delete members from a group](#delete-members-from-a-group)
+- [Synchronize members in a group](#synchronize-members-in-a-group)
+- [Delete members from a group by role or status](#delete-members-from-a-group-by-role-or-status)
+- [Update member roles and delivery options](#update-member-roles-and-delivery-options)
+- [Bulk membership changes](#bulk-membership-changes)
+- [Display user group member options](#display-user-group-member-options)
+- [Display group membership in CSV format](#display-group-membership-in-csv-format)
+- [Display group membership in hierarchical format](#display-group-membership-in-hierarchical-format)
+
+## API documentation
+* https://developers.google.com/admin-sdk/directory/v1/reference/members
+
+## Definitions
+See [Collections of Items](Collections-of-Items)
+```
+<DeliverySetting> ::=
+        allmail|
+        abridged|daily|
+        digest|
+        disabled|
+        none|nomail
+<DomainName> ::= <String>(.<String>)+
+<DomainNameList> ::= "<DomainName>(,<DomainName>)*"
+<DomainNameEntity> ::=
+        <DomainNameList> | <FileSelector> | <CSVFileSelector>
+<EmailAddress> ::= <String>@<DomainName>
+<EmailItem> ::= <EmailAddress>|<UniqueID>|<String>
+<UniqueID> ::= id:<String>
+<GroupItem> ::= <EmailAddress>|<UniqueID>|<String>
+<GroupList> ::= "<GroupItem>(,<GroupItem>)*"
+<GroupEntity> ::=
+        <GroupList> | <FileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<GroupRole> ::= owner|manager|member
+<GroupRoleList> ::= "<GroupRole>(,<GroupRole>)*"
+<GroupType> ::= customer|group|user
+<GroupTypeList> ::= "<GroupType>(,<GroupType>)*"
+<QueryGroup> ::= <String>
+        See: https://developers.google.com/admin-sdk/directory/v1/guides/search-groups
+<QueryGroupList> ::= "<QueryGroup>(,<QueryGroup>)*"
+
+<MembersFieldName> ::=
+        delivery|deliverysettings|
+        email|useremail|
+        group|groupemail|
+        id|
+        name|
+        role|
+        status|
+        type
+<MembersFieldNameList> ::= "<MembersFieldName>(,<MembersFieldName>)*"
+```
+
+## Collections of Users
+Group membership commands involve specifying collections of users;
+for `<UserTypeEntity>`, see: [Collections of Users](Collections-of-Users)
+
+### Select users based on suspension state
+When adding, deleting or synchronizing group members, to select only suspended or non-suspended users, use the following`<UserTypeEntity>`:
+```
+        (all users_ns|users_susp)|
+        (domains_ns|domains_susp <DomainNameList>)|
+        (group_ns|group_susp <GroupItem>)|
+        (groups_ns|groups_susp <GroupList>)|
+        (group_users_ns|group_users_susp <GroupList>
+                [members] [managers] [owners]
+                [primarydomain] [domains <DomainNameList>] [recursive|includederivedmembership] end)|
+        (ou_ns|ou_susp <OrgUnitItem>)|
+        (ou_and_children_ns|ou_and_children_susp <OrgUnitItem>)|
+        (ous_ns|ous_susp <OrgUnitList>)|
+        (ous_and_children_ns|ous_and_children_susp <OrgUnitList>)
+```
+
+When adding, deleting or synchronizing group members, the `notsuspended|suspended` option can be used to select
+users in a particular suspension state. This option can be used with the following `<UserTypeEntity>`:
+```
+        (all users)|
+        (domains <DomainNameList>)|
+        (group <GroupItem>)|
+        (groups <GroupList>)|
+        (group_users <GroupList>
+                [members] [managers] [owners]
+                [primarydomain] [domains <DomainNameList>] [recursive|includederivedmembership] end)|
+        (ou <OrgUnitItem>)|
+        (ou_and_children <OrgUnitItem>)|
+        (ous <OrgUnitList>)|
+        (ous_and_children <OrgUnitList>)
+```
+
+### Select users based on archived state
+When adding, deleting or synchronizing group members, the `notarchived|archived` option can be used to select
+users in a particular archived state. This option can be used with the following `<UserTypeEntity>`:
+```
+        (all users|users_ns|users_susp|users_ns_susp)|
+        (domains|domains_ns|domains_susp <DomainNameList>)|
+        (group|group_ns|group_susp <GroupItem>)|
+        (groups|groups_ns|groups_susp <GroupList>)|
+        (group_users|group_users_ns|group_users_susp <GroupList>
+                [members] [managers] [owners]
+                [primarydomain] [domains <DomainNameList>] [recursive|includederivedmembership] end)|
+        (ou|ou_ns|ou_susp <OrgUnitItem>)|
+        (ou_and_children|ou_and_children_ns|ou_and_children_susp <OrgUnitItem>)|
+        (ous|ous_ns|ous_susp <OrgUnitList>)|
+        (ous_and_children|ous_and_children_ns|ous_and_children_susp <OrgUnitList>)|
+        (query <QueryUser>)|
+        (queries <QueryUserList>)
+```
+Prior to version `6.20.05`, the `notarchived|archived` option could only be used with the following `<UserTypeEntity>`:
+```
+        (group|group_ns|group_susp <GroupItem>)|
+        (groups|groups_ns|groups_susp <GroupList>)|
+        (group_users|group_users_ns|group_users_susp <GroupList>
+                [members] [managers] [owners]
+                [primarydomain] [domains <DomainNameList>] [recursive|includederivedmembership] end)
+```
+
+## Add members to a group
+```
+gam update group|groups <GroupEntity> create|add [<GroupRole>]
+        [usersonly|groupsonly]
+        [notsuspended|suspended] [notarchived|archived]
+        [[delivery] <DeliverySetting>]
+        [preview] [actioncsv]
+        <UserItem>|<UserTypeEntity>
+```
+When `<UserTypeEntity>` specifies a group or groups:
+* `usersonly` - Only the user members from the specified groups are added
+* `groupsonly` - Only the group members from the specified groups are added
+
+For `notsuspended|suspended`, see: [Select users based on suspension state](#select-users-based-on-suspension-state)
+
+For `notarchived|archived`, see: [Select users based on archived state](#select-users-based-on-archived-state)
+
+You can set the `delivery` option for the new members:
+* `allmail` - All messages, delivered as soon as they arrive
+* `abridged|daily` - No more than one message a day
+* `digest` - Up to 25 messages bundled into a single message
+* `none|nomail` - No messages
+* `disabled` - Remove subscription; this is what the documentation says, it's not clear what it means
+
+If `preview` is specified, the changes will be previewed but not executed.
+
+If `actioncsv` is specified, a CSV file with columns `group,email,role,action,message` is generated
+that shows the actions performed when updating the group.
+
+Gam adds the members in batches and pauses between batches in order to avoid exceeding Google's quota limits. The size of the batch
+is set in `gam.cfg/batch_size` and the pause in `gam.cfg/inter_watch_wait`. For add, values of 20 and 1 seem to give reasonable results.
+For each batch, if the quota rate limit is exceeded, Gam increases inter_batch_wait by .25 seconds.
+
+For example,
+```
+gam config batch_size 20 inter_batch_wait 1 update group testgroup@domain.com add members file users.lst
+```
+### `actioncsv` Example
+Using `actioncsv` produces a CSV file showing the actions taken.
+```
+$ gam redirect csv AddUpdates.csv update group testgroup add members actioncsv users testuser2,testuser3
+Group: testgroup@domain.com, Add 2 Members
+  Group: testgroup@domain.com, Member: testuser2@domain.com, Added: Role: MEMBER (1/2)
+  Group: testgroup@domain.com, Member: testuser3@domain.com, Add Failed: Member already exists. (2/2)
+$ more AddUpdates.csv 
+group,email,role,action,message
+testgroup@domain.com,testuser2@domain.com,MEMBER,Added,Success
+testgroup@domain.com,testuser3@domain.com,MEMBER,Add Failed,Member already exists.
+```
+
+## Delete members from a group
+```
+gam update group|groups <GroupEntity> delete|remove [<GroupRole>]
+        [usersonly|groupsonly]
+        [notsuspended|suspended] [notarchived|archived]
+        [preview] [actioncsv]
+        <UserItem>|<UserTypeEntity>
+```
+`<GroupRole>` is ignored, deletions take place regardless of role.
+
+When `<UserTypeEntity>` specifies a group or groups:
+* `usersonly` - Only the user members from the specified groups are deleted
+* `groupsonly` - Only the group members from the specified groups are deleted
+
+For `notsuspended|suspended`, see: [Select users based on suspension state](#select-users-based-on-suspension-state)
+
+For `notarchived|archived`, see: [Select users based on archived state](#select-users-based-on-archived-state)
+
+If `preview` is specified, the changes will be previewed but not executed.
+
+If `actioncsv` is specified, a CSV file with columns `group,email,role,action,message` is generated
+that shows the actions performed when updating the group.
+
+Gam deletes the members in batches and pauses between batches in order to avoid exceeding Google's quota limits. The size of the batch
+is set in `gam.cfg/batch_size` and the pause in `gam.cfg/inter_watch_wait`. For delete, values of 20 and 2 seem to give reasonable results.
+For each batch, if the quota rate limit is exceeded, Gam increases inter_batch_wait by .25 seconds.
+
+For example,
+```
+gam config batch_size 20 inter_batch_wait 2 update group testgroup@domain.com delete members file users.lst
+```
+### `actioncsv` Example
+Using `actioncsv` produces a CSV file showing the actions taken.
+```
+$ gam redirect csv DeleteUpdates.csv update group testgroup delete members actioncsv users testuser2,testuser4
+Group: testgroup@domain.com, Remove 2 Members
+  Group: testgroup@domain.com, Member: testuser2@domain.com, Removed: Role: MEMBER (1/2)
+  Group: testgroup@domain.com, Member: testuser4@domain.com, Remove Failed: Does not exist (2/2)
+$ more DeleteUpdates.csv 
+group,email,role,action,message
+testgroup@domain.com,testuser2@domain.com,MEMBER,Removed,Success
+testgroup@domain.com,testuser4@domain.com,MEMBER,Remove Failed,Does not exist
+```
+
+## Synchronize members in a group
+A synchronize operation gets the current membership for a group and does adds and deletes as necessary to make it match `<UserTypeEntity>`.
+This is done by specific role except for a special case where role is ignored.
+```
+gam update group|groups <GroupEntity> sync [<GroupRole>|ignorerole]
+        [usersonly|groupsonly] [addonly|removeonly]
+        [notsuspended|suspended] [notarchived|archived]
+        [remove_domain_nostatus_members]
+        [[delivery] <DeliverySetting>]
+        [preview] [actioncsv]
+        (additionalmembers [<GroupRole>] <EmailAddressEntity>)*
+        <UserItem>|<UserTypeEntity>
+```
+If `ignorerole` is specified, GAM removes members regardless of role and adds new members with role MEMBER.
+This is a special purpose option, use with caution and ensure that `<UserTypeEntity>` specifies the full desired membership list of all roles.
+
+If neither `<GroupRole>` nor `ignorerole` is specified, `member` is assumed.
+
+When `<UserTypeEntity>` specifies a group or groups:
+* `usersonly` - Only the user members from the specified groups are added/deleted
+* `groupsonly` - Only the group members from the specified groups are added/deleted
+
+For `notsuspended|suspended`, see: [Select users based on suspension state](#select-users-based-on-suspension-state)
+
+For `notarchived|archived`, see: [Select users based on archived state](#select-users-based-on-archived-state)
+
+The `notsuspended|suspended` and `notarchived|archived` not only control what users are selected from `<UserTypeEntity>`
+but they also control what users are selected from `<GroupEntity>`.
+
+The `remove_domain_nostatus_members` option is used to remove members from the group that are in your domain but have no status.
+These members were added to the group before the user or group that they represent was created.
+Your domain is defined as the value from `domain` in `gam.cfg` if it is defined, or, if not, the domain of your Google Workspace Admin in oauth2.txt.
+
+You can set the `delivery` option for the new members:
+* `allmail` - All messages, delivered as soon as they arrive
+* `abridged|daily` - No more than one message a day
+* `digest` - Up to 25 messages bundled into a single message
+* `none|nomail` - No messages
+* `disabled` - Remove subscription; this is what the documentation says, it's not clear what it means
+
+Default:
+* members in `<UserTypeEntity>` that are not in the current membership will be added
+* members in the current membership that are not in `<UserTypeEntity>` will deleted
+
+When the `addonly` option is specified:
+* members in `<UserTypeEntity>` that are not in the current membership will be added
+* members in the current membership that are not in `<UserTypeEntity>` will not be deleted
+
+When the `removeonly` option is specified:
+* members in `<UserTypeEntity>` that are not in the current membership will not be added
+* members in the current membership that are not in `<UserTypeEntity>` will be deleted
+
+If `preview` is specified, the changes will be previewed but not executed.
+
+If `actioncsv` is specified, a CSV file with columns `group,email,role,action,message` is generated
+that shows the actions performed when updating the group.
+
+The option `additionalmembers [<GroupRole>] <EmailAddressEntity>` can be used to specify members in addition to those specified with `<UserTypeEntity>`.
+
+For example,
+```
+gam update group teachers@domain.com sync member additionalmembers counselor@domain.com ou /Teachers
+```
+
+Gam adds/deletes the members in batches and pauses between batches in order to avoid exceeding Google's quota limits. The size of the batch
+is set in `gam.cfg/batch_size` and the pause in `gam.cfg/inter_watch_wait`. For sync, values of 20 and 1 seem to give reasonable results for
+the adds but the inter_batch_wait is probably too low for the deletes; for each batch, if the quota rate limit is exceeded, Gam increases inter_batch_wait by .25 seconds.
+
+For example,
+```
+gam config batch_size 20 inter_batch_wait 1 update group testgroup@domain.com sync members file users.lst
+```
+### Examples using CSV file and Google sheets:
+* https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Users#examples-using-csv-files-and-google-sheets-to-update-the-membership-of-a-group
+
+### Example
+Assume that at your school there is a group for each grade level and the members come from an OU; here is a sample CSV file GradeOU.csv
+```
+Grade,OU
+seniors@domain.org,/Students/ClassOf2023
+juniors@domain.org,/Students/ClassOf2024
+...
+```
+This allows you to do: `gam csv GradeOU.csv gam update group ~Grade sync members ou ~OU`
+But suppose that at each grade level there are additional group members that are groups of faculty/staff; e.g., senioradvisors@domain.org.
+In this scenario, you can't do the `update group sync` command as the members that are groups will be deleted; the `usersonly` option allows
+the `update group sync` command to work: `gam csv GradeOU.csv gam update group ~Grade sync members usersonly ou ~OU`
+The users from the OU are matched against the user members of the group and adds/deletes are done as necessary to synchronize them;
+the group members of the group are unaffected.
+
+### `actioncsv` Example
+Using `actioncsv` produces a CSV file showing the actions taken.
+```
+$ gam redirect csv SyncUpdates.csv update group testgroup sync members actioncsv users testuser1,testuser3,testuser4
+Getting all Members for testgroup@domain.com, may take some time on a large Group...
+Got 3 Members for testgroup@domain.com...
+Group: testgroup@domain.com, Remove 1 Member
+  Group: testgroup@domain.com, Member: testuser2@domain.com, Removed: Role: MEMBER
+Group: testgroup@domain.com, Add 1 Member
+  Group: testgroup@domain.com, Member: testuser4@domain.com, Added: Role: MEMBER
+$ more SyncUpdates.csv 
+group,email,role,action,message
+testgroup@domain.com,testuser2@domain.com,MEMBER,Removed,Success
+testgroup@domain.com,testuser4@domain.com,MEMBER,Added,Success
+```
+## Delete members from a group by role or status
+```
+gam update group|groups <GroupEntity> clear [member] [manager] [owner]
+        [usersonly|groupsonly]
+        [notsuspended|suspended] [notarchived|archived]
+        [emailclearpattern|emailretainpattern <RegularExpression>]
+        [remove_domain_nostatus_members]
+        [preview] [actioncsv]
+```
+If none of `member`, `manager`, or `owner` are specified, `member` is assumed.
+
+By default, when clearing members from a group, all members, whether users or groups, are included.
+* `usersonly` - Clear only the user members
+* `groupsonly` - Clear only the group members
+
+By default, when clearing members from a group, all members, whether suspended/archived or not, are included.
+* `notsuspended` - Clear only non-suspended members
+* `suspended` - Clear only suspended members
+* `notarchived` - Clear only non-archived members
+* `archived` - Clear only archived users
+* `notsuspended notarchived` - Do not clear suspended and archived members
+* `suspended archived` - Clear suspended and archived members
+* `notsuspended archived` - Do not clear archived members
+* `suspended notarchived` - Do not clear suspended members
+
+Members that have met the above qualifications to be cleared can be further qualifed by their email address.
+* `emailclearpattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be cleared; others will be retained
+* `emailretainpattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be retained; others will be cleared
+
+The `remove_domain_nostatus_members` option is used to clear members from the group that are in your domain but have no status.
+These members were added to the group before the user or group that they represent was created.
+Your domain is defined as the value from `domain` in `gam.cfg` if it is defined, or, if not, the domain of your Google Workspace Admin in oauth2.txt.
+
+If `preview` is specified, the deletes will be previewed but not executed.
+
+If `actioncsv` is specified, a CSV file with columns `group,email,role,action,message` is generated
+that shows the actions performed when updating the group.
+
+Gam deletes the members in batches and pauses between batches in order to avoid exceeding Google's quota limits. The size of the batch
+is set in `gam.cfg/batch_size` and the pause in `gam.cfg/inter_watch_wait`. For delete, values of 20 and 2 seem to give reasonable results.
+For each batch, if the quota rate limit is exceeded, Gam increases inter_batch_wait by .25 seconds.
+
+For example,
+```
+gam config batch_size 20 inter_batch_wait 2 update group testgroup@domain.com clear members
+```
+## Update member roles and delivery options
+```
+gam update group|groups <GroupEntity> update [<GroupRole>]
+        [usersonly|groupsonly]
+        [notsuspended|suspended] [notarchived|archived]
+        [[delivery] <DeliverySetting>]
+        [createifnotfound]
+        [preview] [actioncsv]
+        <UserItem>|<UserTypeEntity>
+```
+There are two items that can be updated: role and delivery. If neither option is specified,
+the users are updated to members; this is the behavior from previous versions. Otherwise,
+only the specified items are updated.
+
+When `<UserTypeEntity>` specifies a group or groups:
+* `usersonly` - Only the user members from the specified groups are added
+* `groupsonly` - Only the group members from the specified groups are added
+
+By default, when updating members from organization units, all users, whether suspended or not, are included.
+* `notsuspended` - Do not include suspended users
+* `suspended` - Only include suspended users
+
+By default, when updating members from groups, all users, whether suspended/archived or not, are included.
+* `notsuspended` - Do not include suspended users
+* `suspended` - Only include suspended users
+* `notarchived` - Do not include archived users
+* `archived` - Only include archived users
+* `notsuspended notarchived` - Do not include suspended and archived users
+* `suspended archived` - Include only suspended or archived users
+* `notsuspended archived` - Only include archived users
+* `suspended notarchived` - Only include suspended users
+
+You can set the `delivery` option for the updated members:
+* `allmail` - All messages, delivered as soon as they arrive
+* `abridged|daily` - No more than one message a day
+* `digest` - Up to 25 messages bundled into a single message
+* `none|nomail` - No messages
+* `disabled` - Remove subscription; this is what the documentation says, it's not clear what it means
+
+If, when attempting to update the role of a group member, the group member is not found, the `createifnotfound` option causes Gam to add the member with the specified role.
+
+If `preview` is specified, the changes will be previewed but not executed.
+
+If `actioncsv` is specified, a CSV file with columns `group,email,role,action,message` is generated
+that shows the actions performed when updating the group.
+
+## Bulk membership changes
+### Example 1
+The file Users.csv has a single column of email addresses, there is no header row.
+```
+user1@domain.com
+user2@domain.com
+...
+
+gam update group group@domain.com sync members file Users.csv
+```
+
+The Google Sheet `user@domain.com <DriveFileID> <SheetEntity>` has a single column of email addresses, there is no header row.
+Define an implicit header with the `fields Email` option.
+```
+user1@domain.com
+user2@domain.com
+...
+
+gam update group group@domain.com sync members csvfile gsheet:Email user@domain.com <DriveFileID> <SheetEntity> fields Email
+```
+
+The Google Doc `user@domain.com <DriveFileID>` has a single column of email addresses, there is no header row.
+```
+user1@domain.com
+user2@domain.com
+...
+
+gam update group group@domain.com sync members file gdoc user@domain.com <DriveFileID>
+```
+
+### Example 2
+The CSV file Users.csv has one column of email addresses labelled Email.
+```
+Email
+user1@domain.com
+user2@domain.com
+...
+
+gam update group group@domain.com sync members csvfile Users.csv:Email
+```
+
+The Google Sheet `user@domain.com <DriveFileID> <SheetEntity>` has one column of email addresses labelled Email.
+```
+Email
+user1@domain.com
+user2@domain.com
+...
+
+gam update group group@domain.com sync members csvfile gsheet:Email user@domain.com <DriveFileID> <SheetEntity>
+```
+
+### Example 3
+The CSV file Users.csv has two columns of email addresses labelled Email1 and Email2.
+```
+Email1,Email2
+user1@domain.com,user2@domain.com
+user3@domain.com,user4@domain.com
+...
+
+gam update group group@domain.com sync members csvfile Users.csv:Email1:Email2
+```
+
+The Google Sheet `user@domain.com <DriveFileID> <SheetEntity>` has two columns of email addresses labelled Email1 and Email2.
+```
+Email1,Email2
+user1@domain.com,user2@domain.com
+user3@domain.com,user4@domain.com
+...
+
+gam update group group@domain.com sync members csvfile gsheet:Email1:Email2 user@domain.com <DriveFileID> <SheetEntity>
+```
+
+### Example 4
+The file Groups.txt has a single column of group email addresses, there is no header row.
+You want to sync with the members of those groups.
+```
+group1@domain.com
+group2@domain.com
+...
+
+gam update group group@domain.com sync members datafile groups Groups.txt
+```
+
+The Google Doc `user@domain.com <DriveFileID>` has a single column of group email addresses, there is no header row.
+You want to sync with the members of those groups.
+```
+group1@domain.com
+group2@domain.com
+...
+
+gam update group group@domain.com sync members datafile groups gdoc user@domain.com <DriveFileID>
+```
+
+### Example 5
+The CSV file Groups.csv has a single column of group email addresses labelled Group.
+You want to sync with the members of those groups.
+```
+Group
+group1@domain.com
+group2@domain.com
+...
+
+gam update group group@domain.com sync members csvdatafile groups Groups.csv:Group
+```
+
+The Google Sheet `user@domain.com <DriveFileID> <SheetEntity>` has a single column of group email addresses labelled Group.
+You want to sync with the members of those groups.
+```
+Group
+group1@domain.com
+group2@domain.com
+...
+
+gam update group group@domain.com sync members csvdatafile groups gsheet:Group user@domain.com <DriveFileID> <SheetEntity>
+```
+
+### Example 6
+The CSV file GroupMembers.csv has headers: group,role,email
+
+Each row contains a group email address, member role (OWNER, MEMBER, MANAGER) and a member email address.
+
+The following command will synchronize the membership for all groups and roles.
+```
+gam redirect stdout ./MemberUpdates.txt redirect stderr stdout update group csvkmd GroupMembers.csv keyfield group subkeyfield role datafield email sync csvdata email
+```
+
+The Google Sheet `user@domain.com <DriveFileID> <SheetEntity>` has headers: group,role,email
+
+Each row contains a group email address, member role (OWNER, MEMBER, MANAGER) and a member email address.
+
+The following command will synchronize the membership for all groups and roles.
+```
+gam redirect stdout ./MemberUpdates.txt redirect stderr stdout update group csvkmd gsheet user@domain.com <DriveFileID> <SheetEntity> keyfield group subkeyfield role datafield email sync csvdata email
+```
+You can also do `create|add`, `delete` and `update` in this manner.
+
+If you want to update a specific role, you can do one of the following.
+```
+gam redirect stdout ./MemberUpdates.txt redirect stderr stdout update group csvkmd ./GroupMembers.csv keyfield group matchfield role MEMBER datafield email sync member csvdata email
+gam redirect stdout ./ManagerUpdates.txt redirect stderr stdout update group csvkmd ./GroupMembers.csv keyfield group matchfield role MANAGER datafield email sync manager csvdata email
+gam redirect stdout ./OwnerUpdates.txt redirect stderr stdout update group csvkmd ./GroupMembers.csv keyfield group matchfield role OWNER datafield email sync owner csvdata email
+```
+
+## Display user group member options
+
+Display user's group membership information. Delivery information is displayed; an additional API call per user is required.
+```
+gam <UserTypeEntity> info member|group-members <GroupEntity>
+gam info member|group-members <UserItem>|<UserTypeEntity> <GroupEntity>
+```
+
+## Display group membership in CSV format
+By default, delivery information is not displayed.
+```
+gam print group-members [todrive <ToDriveAttribute>*]
+        [([domain|domains <DomainNameEntity>] ([member|showownedby <EmailItem>]|[(query <QueryGroup>)|(queries <QueryGroupList>)]))|
+         (group|group_ns|group_susp <GroupItem>)|
+         (select <GroupEntity>)]
+        [emailmatchpattern [not] <RegularExpression>] [namematchpattern [not] <RegularExpression>]
+        [descriptionmatchpattern [not] <RegularExpression>]
+        [admincreatedmatch <Boolean>]
+        [roles <GroupRoleList>] [members] [managers] [owners]
+        [membernames] [showdeliverysettings]
+        <MembersFieldName>* [fields <MembersFieldNameList>]
+        [notsuspended|suspended] [notarchived|archived]
+        [types <GroupTypeList>]
+        [memberemaildisplaypattern|memberemailskippattern <RegularExpression>]
+        [userfields <UserFieldNameList>]
+        [(recursive [noduplicates])|includederivedmembership] [nogroupemail]
+        [peoplelookup|(peoplelookupuser <EmailAddress>)]
+        [unknownname <String>] [cachememberinfo [Boolean]]
+        [formatjson [quotechar <Character>]]
+```
+By default, the group membership of all groups in the account are displayed, these options allow selection of subsets of groups:
+* `domain|domains <DomainNameEntity>` - Limit display to groups in the domains specified by `<DomainNameEntity>`
+  * You can predefine this list with the `print_agu_domains` variable in `gam.cfg`.
+* `member <EmailItem>` - Limit display to groups that contain `<EmailItem>` as a member; mutually exclusive with `query <QueryGroup>`
+* `showownedby <EmailItem>` - Limit display to groups that contain `<EmailItem>` as an owner; mutually exclusive with `query <QueryGroup>`
+* `(query <QueryGroup>)|(queries <QueryGroupList>)` - Limit groups to those that match a query; each query is run against each domain
+* `group <GroupItem>` - Limit display to the single group `<GroupItem>`
+* `group_ns <GroupItem>` - Limit display to the single group `<GroupItem>`, display non-suspended members
+* `group_susp <GroupItem>` - Limit display to the single group `<GroupItem>`, display suspended members
+* `select <GroupEntity>` - Limit display to the groups specified in `<GroupEntity>`
+* `showownedby <UserItem>` - Limit display to groups owned by `<UserItem>`
+
+When using `query <QueryGroup>` with the `name:{PREFIX}*` query, `PREFIX` must contain at least three characters.
+
+You can identify groups with the `All users in the organization` member with:
+* `query "memberKey=<CustomerID>"` - All versions
+* `member id:<CustomerID>` - Version 6.10.06 or later
+
+These options further limit the list of groups selected above:
+* `emailmatchpattern <RegularExpression>` - Limit display to groups whose email address matches `<RegularExpression>`
+* `emailmatchpattern not <RegularExpression>` - Limit display to groups whose email address does not match `<RegularExpression>`
+* `namematchpattern <RegularExpression>` - Limit display to groups whose name matches `<RegularExpression>`
+* `namematchpattern not <RegularExpression>` - Limit display to groups whose name does not match `<RegularExpression>`
+* `descriptionmatchpattern <RegularExpression>` - Limit display to groups whose description matches `<RegularExpression>`
+* `descriptionmatchpattern not <RegularExpression>` - Limit display to groups whose description does not match `<RegularExpression>`
+* `admincreatedmatch True` - Limit display to groups created by administrators
+* `admincreatedmatch False` - Limit display to groups created by users
+
+By default, all members, managers and owners in the group are displayed; these options modify that behavior:
+* `roles <GroupRoleList>` - Display specified roles
+* `members` - Display members
+* `managers` - Display managers
+* `owners` - Display owners
+
+By default, all types of members (customer, group, user) in the group are displayed; when `recursive` is specified,
+the default is to only display type user members. This option modifies those behaviors:
+* `types <GroupTypeList>` - Display specified types
+
+By default, when displaying members from a group, all members, whether suspended/archived or not, are included.
+* `notsuspended` - Display only non-suspended members
+* `suspended` - Display only suspended members
+* `notarchived` - Do not include archived members
+* `archived` - Only include archived members, this is not common but allows creating groups that allow easy identification of archived users
+* `notsuspended notarchived` - Do not include suspended and archived members
+* `suspended archived` - Include only suspended or archived members
+* `notsuspended archived` - Only include archived members, this is not common but allows creating groups that allow easy identification of archived users
+* `suspended notarchived` - Only include suspended members, this is not common but allows creating groups that allow easy identification of suspended users
+
+Members that have met the above qualifications to be displayed can be further qualifed by their email address.
+* `memberemaildisplaypattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be displayed; others will not be displayed
+* `memberemailskippattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will not be displayed; others will be displayed
+
+By default, the ID, role, email address, type and status of each member are displayed along with the group email address;
+these options specify which fields to display:
+* `membernames` - Display members full name; an additional API call per member is required
+* `showdeliverysettings` - Display delivery settings; an additional API call per member is required
+* `<MembersFieldName>*` - Individual field names
+* `fields <MembersFieldNameList>` - A comma separated list of field names
+    * `delivery|deliverysettings` - Specify this field to get delivery information; an additional API call per member is required
+* `userfields <UserFieldNameList>` - For members that are users, display these user fields; an additional API call per member is required
+
+The additional API calls can be reduced with the `cachememberinfo` option; a single API call is made for each user/group
+and the data is cached to eliminate to need to repeat the API call; this consumes more memory but dramatically reduces the number of API calls.
+
+If member names are requested, names are not available for users not in the domain; you can request that GAM use the People API to retrieve
+names for these users. Names are not retrieved in all cases and success is dependent on what user is used to perform the retrievals.
+* `peoplelookup` - Use the administrator named in oauth2.txt to perform the retrievals
+* `peoplelookupuser <EmailAddress>` - Use `<EmailAddress>` to perform the retrievals
+
+By default, when `membernames` is specified, GAM displays `Unknown` for members whose names can not be determined.
+Use `unknownname <String>` to specify an alternative value.
+
+By default, the group email address is always shown, you can suppress it with the `nogroupemail` option.
+
+By default, members that are groups are displayed as a single entry of type GROUP; this option recursively expands group members to display their user members.
+* `recursive` - Recursively expand group members
+
+The `recursive` option does not expand or display members of type CUSTOMER.
+
+The `recursive` option adds two columns, level and subgroup, to the output:
+* `level` - At what level of the expansion does the user appear; level 0 is the top level
+* `subgroup` - The group that contained the user
+
+Displaying membership of multiple groups or recursive expansion may result in multiple instances of the same user being displayed; these multiple instances can be reduced to one entry.
+* `noduplicates` - Reduce multiple instances of the same user to the first instance
+
+The `includederivedmembership` option is an alternative to `recursive`; it causes the API to expand type GROUP and type CUSTOMER
+members to display their constituent members while still displaying the original member.
+The API produces inconsistent results, use with caution.
+
+The options `recursive noduplicates` and `includederivedmembership types user noduplicates` return the same list of users.
+The `includederivedmembership` option makes less API calls but doesn't show level and subgroup information.
+Expanding a member of type CUSTOMER may produce a large volume of data as it will display all users in your domain.
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display group membership in hierarchical format
+```
+gam show group-members
+        [([domain|domains <DomainNameEntity>] ([member|showownedby <EmailItem>]|[(query <QueryGroup>)|(queries <QueryGroupList>)]))|
+         (group|group_ns|group_susp <GroupItem>)|
+         (select <GroupEntity>)]
+        [emailmatchpattern [not] <RegularExpression>] [namematchpattern [not] <RegularExpression>]
+        [descriptionmatchpattern [not] <RegularExpression>]
+        [admincreatedmatch <Boolean>]
+        [roles <GroupRoleList>] [members] [managers] [owners] [depth <Number>]
+        [notsuspended|suspended] [notarchived|archived]
+        [types <GroupTypeList>]
+        [memberemaildisplaypattern|memberemailskippattern <RegularExpression>]
+        [includederivedmembership]
+```
+By default, the group membership of all groups in the account are displayed, these options allow selection of subsets of groups:
+* `domain|domains <DomainNameEntity>` - Limit display to groups in the domains specified by `<DomainNameEntity>`
+  * You can predefine this list with the `print_agu_domains` variable in `gam.cfg`.
+* `member <EmailItem>` - Limit display to groups that contain `<EmailItem>` as a member; mutually exclusive with `query <QueryGroup>`
+* `showownedby <EmailItem>` - Limit display to groups that contain `<EmailItem>` as an owner; mutually exclusive with `query <QueryGroup>`
+* `(query <QueryGroup>)|(queries <QueryGroupList>)` - Limit groups to those that match a query; each query is run against each domain
+* `group <GroupItem>` - Limit display to the single group `<GroupItem>`
+* `group_ns <GroupItem>` - Limit display to the single group `<GroupItem>`, display non-suspended members
+* `group_susp <GroupItem>` - Limit display to the single group `<GroupItem>`, display suspended members
+* `select <GroupEntity>` - Limit display to the groups specified in `<GroupEntity>`
+* `showownedby <UserItem>` - Limit display to groups owned by `<UserItem>`
+
+When using `query <QueryGroup>` with the `name:{PREFIX}*` query, `PREFIX` must contain at least three characters.
+
+You can identify groups with the `All users in the organization` member with:
+* `query "memberKey=<CustomerID>"` - All versions
+* `member id:<CustomerID>` - Version 6.10.06 or later
+
+These options further limit the list of groups selected above:
+* `emailmatchpattern <RegularExpression>` - Limit display to groups whose email address matches `<RegularExpression>`
+* `emailmatchpattern not <RegularExpression>` - Limit display to groups whose email address does not match `<RegularExpression>`
+* `namematchpattern <RegularExpression>` - Limit display to groups whose name matches `<RegularExpression>`
+* `namematchpattern not <RegularExpression>` - Limit display to groups whose name does not match `<RegularExpression>`
+* `descriptionmatchpattern <RegularExpression>` - Limit display to groups whose description matches `<RegularExpression>`
+* `descriptionmatchpattern not <RegularExpression>` - Limit display to groups whose description does not match `<RegularExpression>`
+* `admincreatedmatch True` - Limit display to groups created by administrators
+* `admincreatedmatch False` - Limit display to groups created by users
+
+By default, all members, managers and owners in the group are displayed; these options modify that behavior:
+* `roles <GroupRoleList>` - Display specified roles
+* `members` - Display members
+* `managers` - Display managers
+* `owners` - Display owners
+
+By default, when displaying members from a group, all members, whether suspended/archived or not, are included.
+* `notsuspended` - Display only non-suspended members
+* `suspended` - Display only suspended members
+* `notarchived` - Do not include archived members
+* `archived` - Only include archived members, this is not common but allows creating groups that allow easy identification of archived users
+* `notsuspended notarchived` - Do not include suspended and archived members
+* `suspended archived` - Include only suspended or archived members
+* `notsuspended archived` - Only include archived members, this is not common but allows creating groups that allow easy identification of archived users
+* `suspended notarchived` - Only include suspended members, this is not common but allows creating groups that allow easy identification of suspended users
+
+By default, all types of members (customer, group, user) in the group are displayed; this option modifies that behavior:
+* `types <GroupTypeList>` - Display specified types
+
+Members that have met the above qualifications to be displayed can be further qualifed by their email address.
+* `memberemaildisplaypattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be displayed; others will not be displayed
+* `memberemailskippattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will not be displayed; others will be displayed
+
+By default, members of type GROUP are recursively expanded to show their constituent members. (Members of
+type CUSTOMER are not expanded.) The `depth <Number>` argument controls the depth to which nested groups are displayed.
+* `depth -1` - all groups in the selected group and below are displayed; this is the default.
+* `depth 0` - the groups within a selected group are displayed, no descendants are displayed.
+* `depth N` - the groups within the selected group and those groups N levels below the selected group are displayed.
+
+The `includederivedmembership` option causes the API to expand type GROUP and type CUSTOMER
+members to display their constituent members while still displaying the original member.
+
+The options `types user` and `includederivedmembership types user` return the same list of users.
+The `includederivedmembership` option makes less API calls but doesn't show hierarchy.
+Expanding a member of type CUSTOMER may produce a large volume of data as it will display all users in your domain.
+
+### Display group structure
+To see a group's structure of nested groups use the `type group` option.
+```
+$ gam show group-members group testgroup5 types group
+Group: testgroup5@domain.com
+  MEMBER, GROUP, testgroup1@domain.com, ACTIVE
+    MEMBER, GROUP, testgroup2@domain.com, ACTIVE
+  MEMBER, GROUP, testgroup3@domain.com, ACTIVE
+    MEMBER, GROUP, testgroup2@domain.com, ACTIVE
+    MEMBER, GROUP, testgroup4@domain.com, ACTIVE
+```
+To show the structure of all groups you can do the following; it will be time consuming for a large number of groups.
+```
+gam redirect stdout ./groups.txt show group-members types group
+```
+
+### Examples
+#### Print a CSV of all members of a group regardless of role, all fields
+```
+gam print group-members <GroupEntity>
+```
+#### Print a CSV containing all managers emails
+```
+gam print group-members <GroupEntity> role manager fields email
+```
+#### Print a CSV output of all members and their emails only
+```
+gam print group-members <GroupEntity> role member fields email
+```
+#### Display group owners in your domain, but excluding groups where the email starts with a 4 digit code
+```
+gam print group-members domain <Your Domain> emailmatchpattern not '^1234.*' roles owners
+```

docs/Groups.md

@@ -0,0 +1,594 @@
+# Groups
+- [API documentation](#api-documentation)
+- [Name guidelines](#name-guidelines)
+- [Query documentation](#query-documentation)
+- [Python Regular Expressions](Python-Regular-Expressions) Match function
+- [Cloud Identity Groups](#cloud-identity-groups)
+- [Security Groups](#security-groups)
+- [Transition to new Group Settings](#transition-to-new-Group-settings)
+- [Collaborative Inbox](#collaborative-inbox)
+- [Definitions](#definitions)
+- [GUI API Group settings mapping](#gui-api-group-settings-mapping)
+- [GUI API Group access type settings mapping](#gui-api-group-access-type-settings-mapping)
+- [Manage groups](#manage-groups)
+- [Update a group's settings with JSON data](#update-a-groups-settings-with-json-data)
+- [Display information about specific groups](#display-information-about-specific-groups)
+- [Display information about selected groups](#display-information-about-selected-groups)
+- [Display a group and its parents](#Display-a-group-and-its-parents)
+- [Examples](#Examples)
+- [Display group counts](#display-group-counts)
+
+## API documentation
+* https://developers.google.com/admin-sdk/directory/reference/rest/v1/groups
+* https://developers.google.com/admin-sdk/groups-settings/v1/reference/groups
+* https://cloud.google.com/identity/docs/groups
+* https://cloud.google.com/identity/docs/reference/rest/v1/groups
+
+## Name guidelines
+* https://support.google.com/a/answer/9193374?hl=en
+
+## Query documentation
+* https://developers.google.com/admin-sdk/directory/v1/guides/search-groups
+* https://cloud.google.com/identity/docs/reference/rest/v1/groups#dynamicgroupquery
+
+## Cloud Identity Groups
+* https://gsuiteupdates.googleblog.com/2020/08/new-api-cloud-identity-groups-google.html
+
+## Security Groups
+* https://gsuiteupdates.googleblog.com/2020/09/security-groups-beta.html
+
+## Transition to new Group Settings
+* https://support.google.com/a/answer/9191148
+* https://drive.google.com/file/d/1-ux3z6-hcjsPbhAj_EIwS7cd_emkE-NC/view
+
+## Collaborative Inbox
+* https://support.google.com/a/answer/167430
+
+## Definitions
+See [Collections of Items](Collections-of-Items)
+```
+<DomainName> ::= <String>(.<String>)+
+<DomainNameList> ::= "<DomainName>(,<DomainName>)*"
+<DomainNameEntity> ::=
+        <DomainNameList> | <FileSelector> | <CSVFileSelector>
+<EmailAddress> ::= <String>@<DomainName>
+<UniqueID> ::= id:<String>
+<EmailItem> ::= <EmailAddress>|<UniqueID>|<String>
+<GroupItem> ::= <EmailAddress>|<UniqueID>|<String>
+<GroupList> ::= "<GroupItem>(,<GroupItem>)*"
+<GroupEntity> ::=
+        <GroupList> | <FileSelector> | <CSVkmdSelector> | <CSVDataSelector>
+        See: https://github.com/taers232c/GAMADV-XTD3/wiki/Collections-of-Items
+<GroupRole> ::= owner|manager|member
+<GroupRoleList> ::= "<GroupRole>(,<GroupRole>)*"
+<GroupType> ::= customer|group|user
+<GroupTypeList> ::= "<GroupType>(,<GroupType>)*"
+<QueryGroup> ::= <String>
+        See: https://developers.google.com/admin-sdk/directory/v1/guides/search-groups
+<QueryGroupList> ::= "<QueryGroup>(,<QueryGroup>)*"
+<QueryDynamicGroup> ::= <String>
+        See: https://cloud.google.com/identity/docs/reference/rest/v1/groups#dynamicgroupquery
+
+<JSONData> ::= (json [charset <Charset>] <String>) | (json file <FileName> [charset <Charset>]) |
+
+<GroupSettingsAttribute> ::=
+        (accesstype public|team|announcementonly|restricted)|
+        (allowexternalmembers <Boolean>)|
+        (allowwebposting <Boolean>)|
+        (archiveonly <Boolean>)|
+        (customfootertext <String>)|
+        (customreplyto <EmailAddress>)|
+        (defaultmessagedenynotificationtext <String>)|
+        (defaultsender self|group)|
+        (description <String>)|
+        (enablecollaborativeinbox|collaborative <Boolean>)|
+        (includeinglobaladdresslist|gal <Boolean>)|
+        (includecustomfooter <Boolean>)|
+        (isarchived <Boolean>)|
+        (memberscanpostasthegroup <Boolean>)|
+        (messagemoderationlevel moderate_all_messages|moderate_non_members|moderate_new_members|moderate_none)|
+        (name <String>)|
+        (primarylanguage <Language>)|
+        (replyto reply_to_custom|reply_to_sender|reply_to_list|reply_to_owner|reply_to_ignore|reply_to_managers)|
+        (sendmessagedenynotification <Boolean>)|
+        (spammoderationlevel allow|moderate|silently_moderate|reject)|
+        (whocanadd all_members_can_add|all_managers_can_add|all_owners_can_add|none_can_add)|
+        (whocancontactowner anyone_can_contact|all_in_domain_can_contact|all_members_can_contact|all_managers_can_contact|all_owners_can_contact)|
+        (whocanjoin anyone_can_join|all_in_domain_can_join|invited_can_join|can_request_to_join)|
+        (whocanleavegroup all_members_can_leave|all_managers_can_leave|all_owners_can_leave|none_can_leave)|
+        (whocanpostmessage none_can_post|all_managers_can_post|all_members_can_post|all_owners_can_post|all_in_domain_can_post|anyone_can_post)|
+        (whocanviewgroup anyone_can_view|all_in_domain_can_view|all_members_can_view|all_managers_can_view|all_owners_can_view)|
+        (whocanviewmembership all_in_domain_can_view|all_members_can_view|all_managers_can_view|all_owners_can_view)
+<GroupWhoCanDiscoverGroupDeprecatedAttribute> ::=
+        (showingroupdirectory <Boolean>)
+<GroupWhoCanAssistContentDeprecatedAttribute> ::=
+        (whocanassigntopics all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanenterfreeformtags all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanhideabuse all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmaketopicssticky all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmarkduplicate all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmarkfavoritereplyonanytopic all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmarknoresponseneeded all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmodifytagsandcategories all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocantaketopics all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanunassigntopic all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanunmarkfavoritereplyonanytopic all_members|owners_and_managers|managers_only|owners_only|none)
+<GroupWhoCanModerateContentDeprecatedAttribute> ::=
+        (whocanapprovemessages all_members|owners_and_managers|owners_only|none)|
+        (whocandeleteanypost all_members|owners_and_managers|owners_only|none)|
+        (whocandeletetopics all_members|owners_and_managers|owners_only|none)|
+        (whocanlocktopics all_members|owners_and_managers|owners_only|none)|
+        (whocanmovetopicsin all_members|owners_and_managers|owners_only|none)|
+        (whocanmovetopicsout all_members|owners_and_managers|owners_only|none)|
+        (whocanpostannouncements all_members|owners_and_managers|owners_only|none)
+<GroupWhoCanModerateMembersDeprecatedAttribute> ::=
+        (whocanadd all_members_can_add|all_managers_can_add|none_can_add)|
+        (whocanapprovemembers all_members_can_approve|all_managers_can_approve|all_owners_can_approve|none_can_approve)|
+        (whocanbanusers all_members|owners_and_managers|owners_only|none)|
+        (whocaninvite all_members_can_invite|all_managers_can_invite|all_owners_can_invite|none_can_invite)|
+        (whocanmodifymembers all_members|owners_and_managers|owners_only|none)
+<GroupDeprecatedAttribute> ::=
+        (allowgooglecommunication <Boolean>)|
+        (favoriterepliesontop <Boolean>)|
+        (maxmessagebytes <ByteCount>)|
+        (messagedisplayfont default_font|fixed_width_font)|
+        (whocanaddreferences all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmarkfavoritereplyonowntopic all_members|owners_and_managers|managers_only|owners_only|none)
+<GroupAttribute> ::=
+        <JSONData>|
+        <GroupSettingsAttribute>|
+        (whocandiscovergroup all_members_can_discover|all_in_domain_can_discover|anyone_can_discover)|
+        (whocanassistcontent all_members|owners_and_managers|managers_only|owners_only|none)|
+        (whocanmoderatecontent all_members|owners_and_managers|owners_only|none)|
+        (whocanmoderatemembers all_members|owners_and_managers|owners_only|none)|
+        <GroupWhoCanDiscoverGroupDeprecatedAttribute>|
+        <GroupWhoCanAssistContentDeprecatedAttribute>|
+        <GroupWhoCanModerateContentDeprecatedAttribute>|
+        <GroupWhoCanModerateMembersDeprecatedAttribute>|
+        <GroupDeprecatedAttribute>
+```
+```
+<GroupFieldName> ::=
+        admincreated|
+        aliases|
+        allowexternalmembers|
+        allowgooglecommunication|
+        allowwebposting|
+        archiveonly|
+        customfootertext|
+        customreplyto|
+        customrolesenabledforsettingstobemerged|
+        defaultmessagedenynotificationtext|
+        description|
+        directmemberscount|
+        email|
+        enablecollaborativeinbox|collaborative|
+        favoriterepliesontop|
+        id|
+        includecustomfooter|
+        includeinglobaladdresslist|gal|
+        isarchived|
+        maxmessagebytes|
+        memberscanpostasthegroup|
+        messagedisplayfont|
+        messagemoderationlevel|
+        name|
+        primarylanguage|
+        replyto|
+        sendmessagedenynotification|
+        showingroupdirectory|
+        spammoderationlevel|
+        whocanaddreferences|
+        whocanadd|
+        whocanapprovemessages|
+        whocanassigntopics|
+        whocanassistcontent|
+        whocancontactowner|
+        whocandeleteanypost|
+        whocandeletetopics|
+        whocandiscovergroup|
+        whocanenterfreeformtags|
+        whocanhideabuse|
+        whocaninvite|
+        whocanjoin|
+        whocanleavegroup|
+        whocanlocktopics|
+        whocanmaketopicssticky|
+        whocanmarkduplicate|
+        whocanmarkfavoritereplyonanytopic|
+        whocanmarkfavoritereplyonowntopic|
+        whocanmarknoresponseneeded|
+        whocanmoderatecontent|
+        whocanmodifytagsandcategories|
+        whocanmovetopicsin|
+        whocanmovetopicsout|
+        whocanpostannouncements|
+        whocanpostmessage|
+        whocantaketopics|
+        whocanunassigntopic|
+        whocanunmarkfavoritereplyonanytopic|
+        whocanviewgroup|
+        whocanviewmembership
+<GroupFieldNameList> ::= "<GroupFieldName>(,<GroupFieldName>)*"
+```
+```
+<CIGroupFieldName> ::=
+        additionalgroupkeys|
+        createtime|
+        description|
+        displayname|
+        dynamicgroupmetadata|
+        groupkey|
+        labels|
+        name|
+        parent|
+        updatetime
+<CIGroupFieldNameList> ::= "<CIGroupFieldName>(,<CIGroupFieldName>)*"
+```
+## GUI API Group settings mapping
+The entries appear in the order presented on the GUI Group settings page.
+
+| GUI setting | API setting |
+|------------|------------|
+| Group name | name |
+| Group email | email |
+| Group description | description |
+| Welcome message | Not available |
+| Collaborative Inbox | enableCollaborativeInbox |
+| Enable shared labels for this group | Not available |
+| Who can see group | whoCanDiscoverGroup |
+| Who can join group | whoCanJoin |
+| Allow external members | allowExternalMembers |
+| Who can view conversations | whoCanViewGroup |
+| Who can post | whoCanPostMessage |
+| Who can view members | whoCanViewMembership |
+| Identification required for new members | Not available |
+| Who can contact group owners | whoCanContactOwner |
+| Who can view member email addresses | Not available |
+| Allow Email Posting | Not available |
+| Allow web posting | allowWebPosting |
+| Conversation history | isArchived |
+| Who can reply privately to authors | Not available |
+| Who can attach files | Not available |
+| Who can moderate content | whoCanModerateContent |
+| Who can moderate metadata | whoCanAssistContent |
+| Who can post as group | membersCanPostAsTheGroup |
+| Default sender | defaultSender |
+| Message moderation | messageModerationLevel |
+| New member restrictions | Not available |
+| Spam message handling | spamModerationLevel |
+| Rejected message notification | sendMessageDenyNotification |
+| Include default rejected message response | defaultMessageDenyNotificationText |
+| Subject prefix | Not available |
+| Include the standard Groups footer | Not available |
+| Include a custom footer | includeCustomFooter |
+| Custom footer text | customFooterText |
+| Group email language | primaryLanguage |
+| Auto replies | Not available |
+| Post replies to | replyTo |
+| Custom address for replies | customReplyTo |
+| Conversation mode | Not available |
+| Who can manage members | whoCanModerateMembers |
+| Who can manaage custom roles | Not available |
+
+## GUI API Group access type settings mapping
+You can apply these settings when creating/updating a group with the option:
+```
+accesstype public|team|announcementonly|restricted
+```
+
+```
+Public
+  whoCanJoin ALL_IN_DOMAIN_CAN_JOIN
+  whoCanPostMessage ALL_IN_DOMAIN_CAN_POST
+  whoCanViewGroup ALL_IN_DOMAIN_CAN_VIEW
+  whoCanViewMembership ALL_IN_DOMAIN_CAN_VIEW
+
+Team
+  whoCanJoin CAN_REQUEST_TO_JOIN
+  whoCanPostMessage ALL_IN_DOMAIN_CAN_POST
+  whoCanViewGroup ALL_IN_DOMAIN_CAN_VIEW
+  whoCanViewMembership ALL_IN_DOMAIN_CAN_VIEW
+
+Announcement Only
+  whoCanJoin ALL_IN_DOMAIN_CAN_JOIN
+  whoCanPostMessage ALL_MANAGERS_CAN_POST
+  whoCanViewGroup ALL_IN_DOMAIN_CAN_VIEW
+  whoCanViewMembership ALL_MANAGERS_CAN_VIEW
+
+Restricted
+  whoCanJoin CAN_REQUEST_TO_JOIN
+  whoCanPostMessage ALL_MEMBERS_CAN_POST
+  whoCanViewGroup ALL_MEMBERS_CAN_VIEW
+  whoCanViewMembership ALL_MEMBERS_CAN_VIEW
+```
+
+## Manage groups
+
+These commands allow you to create, update and delete groups.
+```
+gam create group <EmailAddress>
+        [copyfrom <GroupItem>] <GroupAttribute>*
+        [verifynotinvitable]
+gam update group|groups <GroupEntity> [email <EmailAddress>]
+        [copyfrom <GroupItem>] <GroupAttribute>*
+        [makesecuritygroup|security]
+        [admincreated <Boolean>]
+        [verifynotinvitable]
+gam delete group|groups <GroupEntity> [noactionifalias]
+```
+The `copyfrom <GroupItem>` allows copying of group attributes from one group to another.
+The following attributes are not copied: name, description, email, admincreated, aliases, noneditablealiases.
+Any `<GroupAttribute>` specified will override the copied attributes.
+
+You can update a group to a security group with the `makesecuritygroup` option.
+* Warning: A Security Group cannot be changed back to a Google Group.
+
+When deleting and `noactionifalias` is specified, no action is performed if `<GroupEntity>` specifies an alias rather than a primary email address.
+
+## Update a group's settings with JSON data
+You can save group settings in JSON format which can simplify updating multiple settings. Suppose you have
+a set of test groups that you will use to experiment with the new group settings coming in May 2019. You
+want to backup the current settings so you can restore them later after your experiments.
+
+Backup the current settings.
+```
+$ gam redirect csv ./groups.csv print groups query "name:test*" settings formatjson quotechar "'"
+Getting all Groups that match query (query="name:test*"), may take some time on a large Google Workspace Account...
+Got 4 Groups: testgroup1@domain.com - testgroup4@domain.com
+Getting Group Settings for testgroup1@domain.com (1/4)
+Getting Group Settings for testgroup2@domain.com (2/4)
+Getting Group Settings for testgroup3@domain.com (3/4)
+Getting Group Settings for testgroup4@domain.com (4/4)
+```
+Perform your experiments and then restore the original settings.
+```
+$ gam csv ./groups.csv quotechar "'" gam update group ~email json ~JSON-settings
+Using 4 processes...
+Group: testgroup1@domain.com, Updated
+Group: testgroup2@domain.com, Updated
+Group: testgroup3@domain.com, Updated
+Group: testgroup4@domain.com, Updated
+```
+
+## Display information about specific groups
+The info command displays information as an indented list of keys and values.
+```
+gam info group|groups <GroupEntity>
+        [nousers] [quick] [noaliases] [groups]
+        [basic] <GroupFieldName>* [fields <GroupFieldNameList>] [nodeprecated]
+        [ciallfields|(cifields <CIGroupFieldNameList>)]
+        [roles <GroupRoleList>] [members] [managers] [owners]
+        [notsuspended|suspended] [notarchived|archived]
+        [types <GroupTypeList>]
+        [memberemaildisplaypattern|memberemailskippattern <RegularExpression>]
+        [formatjson]
+```
+By default, all members, managers and owners in the group are displayed; these options modify that behavior:
+* `members` - Display members
+* `managers` - Display managers
+* `owners` - Display owners
+* `nousers` or `quick` - Do not display any members, managers or owners
+* `roles <GroupRoleList>` - Display specified roles
+
+
+By default, when displaying members from a group, all members, whether suspended or not, are included.
+* `notsuspended` - Display only non-suspended members
+* `suspended` - Display only suspended members
+* `notarchived` - Display only non-archived members
+* `archived` - Display only archived members
+* `notsuspended notarchived` - Display only non-suspended and non-archived members
+* `suspended archived` - Display only suspended or archived members
+* `notsuspended archived` - Display only archived members
+* `suspended notarchived` - Display only suspended members
+
+By default, when displaying members from a group, all types of members (customer, group, user) in the group are displayed; this option modifies that behavior:
+* `types <GroupTypeList>` - Display specified types
+
+Members that have met the above qualifications to be displayed can be further qualifed by their email address.
+* `memberemaildisplaypattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be displayed; others will not be displayed
+* `memberemailskippattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will not be displayed; others will be displayed
+
+By default, all group aliases are displayed, these options modify that behavior:
+* `noaliases` or `quick` - Do not display group aliases
+
+By default, the groups of which this group is a member are not displayed, this option enables that display
+* `groups` - Display groups of which this group is a member
+
+These options specify what group fields to display:
+* `basic` - These fields `id,name,description,directMembersCount,aliases,nonEditableAliases,adminCreated` are displayed
+* `<GroupFieldName>*` - Individual fields to display
+* `fields <GroupFieldNameList>` - A comma separated list of fields to display
+* `ciallfields` - All Cloud Identity Group fields
+* `cifields <CIGroupFieldNameList>` - A comma separated list of Cloud Identity Groups fields to display
+* `nodeprecated` - Do not display deprecated fields
+
+By default, Gam displays the information as an indented list of keys and values.
+* `formatjson` - Display the output in JSON notation
+
+## Display information about selected groups
+This command displays information in CSV format.
+```
+gam print groups [todrive <ToDriveAttribute>*]
+        [([domain|domains <DomainNameEntity>] ([member|showownedby <EmailItem>]|[(query <QueryGroup>)|(queries <QueryGroupList>)]))|
+         (select <GroupEntity>)]
+        [emailmatchpattern [not] <RegularExpression>] [namematchpattern [not] <RegularExpression>]
+        [descriptionmatchpattern [not] <RegularExpression>] (matchsetting [not] <GroupAttribute>)*
+        [admincreatedmatch <Boolean>]
+        [maxresults <Number>]
+        [allfields|([basic] [settings] <GroupFieldName>* [fields <GroupFieldNameList>])]
+        [ciallfields|(cifields <CIGroupFieldNameList>)]
+        [nodeprecated]
+        [roles <GroupRoleList>]
+        [members|memberscount] [managers|managerscount] [owners|ownerscount] [totalcount] [countsonly]
+        [includederivedmembership]
+        [notsuspended|suspended] [notarchived|archived]
+        [types <GroupTypeList>]
+        [memberemaildisplaypattern|memberemailskippattern <RegularExpression>]
+        [convertcrnl] [delimiter <Character>] [sortheaders]
+        [formatjson [quotechar <Character>]]
+```
+By default, all groups in the account are displayed, these options allow selection of subsets of groups:
+* `domain|domains <DomainNameEntity>` - Limit display to groups in the domains specified by `<DomainNameEntity>`
+  * You can predefine this list with the `print_agu_domains` variable in `gam.cfg`.
+* `member <EmailItem>` - Limit display to groups that contain `<EmailItem>` as a member; mutually exclusive with `query <QueryGroup>`
+* `showownedby <EmailItem>` - Limit display to groups that contain `<EmailItem>` as an owner; mutually exclusive with `query <QueryGroup>`
+* `(query <QueryGroup>)|(queries <QueryGroupList>)` - Limit groups to those that match a query; each query is run against each domain
+* `select <GroupEntity>` - Limit display to the groups specified in `<GroupEntity>`
+
+When using `query <QueryGroup>` with the `name:{PREFIX}*` query, `PREFIX` must contain at least three characters.
+
+You can identify groups with the `All users in the organization` member with:
+* `query "memberKey=<CustomerID>"` - All versiona
+* `member id:<CustomerID>` - Version 6.10.06 or later
+
+These options further limit the list of groups selected above:
+* `emailmatchpattern <RegularExpression>` - Limit display to groups whose email address matches `<RegularExpression>`
+* `emailmatchpattern not <RegularExpression>` - Limit display to groups whose email address does not match `<RegularExpression>`
+* `namematchpattern <RegularExpression>` - Limit display to groups whose name matches `<RegularExpression>`
+* `namematchpattern not <RegularExpression>` - Limit display to groups whose name does not match `<RegularExpression>`
+* `descriptionmatchpattern <RegularExpression>` - Limit display to groups whose description matches `<RegularExpression>`
+* `descriptionmatchpattern not <RegularExpression>` - Limit display to groups whose description does not match `<RegularExpression>`
+* `admincreatedmatch True` - Limit display to groups created by administrators
+* `admincreatedmatch False` - Limit display to groups created by users
+* `matchsetting <GroupAttribute>` - Limit display to groups that have `<GroupAttribute>`
+* `matchsetting not <GroupAttribute>` - Limit display to groups that do not have `<GroupAttribute>`
+
+* You can specify multiple `matchsetting` options, all of them must match for the group to be displayed.
+* You can specify multiple `matchsetting` options for the same `<GroupAttribute>`, it is a match if the group has any of the `<GroupAttribute>` values.
+* You can specify multiple `matchsetting not` options for the same `<GroupAttribute>`, it is a match if the group has none of the `<GroupAttribute>` values.
+
+When retrieving lists of Google Groups from API, how many should be retrieved in each API call.
+* `maxresults <Number>` - How many groups to retrieve in each API call; default is 200.
+
+By default, only the group email address is displayed, these options specify what group fields to display:
+* `basic` - These fields `id,name,description,directMembersCount,aliases,nonEditableAliases,adminCreated` are displayed
+* `allfields` - All group fields are displayed
+* `settings` - All group settings fields are displayed
+* `<GroupFieldName>*` - Individual fields to display
+* `fields <GroupFieldNameList>` - A comma separated list of fields to display
+* `ciallfields` - All Cloud Identity Group fields
+* `cifields <CIGroupFieldNameList>` - A comma separated list of Cloud Identity Groups fields to display
+* `nodeprecated` - Do not display deprecated fields
+
+Some text fields may contain carriage returns or line feeds, displaying fields containing these characters will make processing the CSV file with a script hard; this option converts those characters to a text form.
+The default value is `csv_output_convert_cr_nl` from `gam.cfg`
+* `convertcrnl` - Convert carriage return to \r and line feed to \n
+
+When lists of items are displayed, the delimiter between items defaults to the `csv_output_column_delimiter` value in gam.cfg; you can specify a different delimiter:
+* `delimiter <Character>` - Use `<Character>` as the list item delimiter, `<Character>` must be a single character after processing any escape character
+
+By default, no members, managers or owners in the group are displayed; these options modify that behavior:
+* `members` - Display list of members
+* `memberscount` - Display count of members but not individual members
+* `managers` - Display list of managers
+* `managerscount` - Display count of managers but not individual managers
+* `owners` - Display list of owners
+* `roles <GroupRoleList>` - Display lists of the specified roles
+* `ownerscount` - Display count of owners but not individual owners
+* `countsonly` - Change any `members`, `managers`, `owners` or `roles` options to `memberscount`, `managerscount` or `ownerscount`
+* `totalcount` - Display sum of counts of members, managers, owners.
+
+The `includederivedmembership` option causes the API to expand type GROUP and type CUSTOMER
+members to display their constituent members while still displaying the original member.
+The API produces inconsistent results, use with caution.
+
+By default, when displaying members from a group, all members, whether suspended or not, are included.
+* `notsuspended` - Display only non-suspended members
+* `suspended` - Display only suspended members
+* `notarchived` - Display only non-archived members
+* `archived` - Display only archived members
+* `notsuspended notarchived` - Display only non-suspended and non-archived members
+* `suspended archived` - Display only suspended or archived members
+* `notsuspended archived` - Display only archived members
+* `suspended notarchived` - Display only suspended members
+
+By default, when displaying members from a group, all types of members (customer, group, user) in the group are displayed; this option modifies that behavior:
+* `types <GroupTypeList>` - Display specified types
+
+Members that have met the above qualifications to be displayed can be further qualifed by their email address.
+* `memberemaildisplaypattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will be displayed; others will not be displayed
+* `memberemailskippattern <RegularExpression>` - Members with email addresses that match `<RegularExpression>` will not be displayed; others will be displayed
+
+By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format,
+* `formatjson` - Display the fields in JSON format.
+
+By default, when writing CSV files, Gam uses a quote character of double quote `"`. The quote character is used to enclose columns that contain
+the quote character itself, the column delimiter (comma by default) and new-line characters. Any quote characters within the column are doubled.
+When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
+The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
+`quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Examples
+### Some simple use cases.
+#### Output can be either redirected to a file on the command line using ">file.ouput", or the csv redirect gam option
+#### Print a list of all your groups
+```
+gam print groups
+```
+#### Display groups with no members.
+```
+gam config csv_output_row_filter "directMembersCount:count=0" print groups directmemberscount
+```
+
+## Display a group and its parents
+Display a group and its parents as an indented list.
+```
+gam show grouptree <GroupEntity>
+```
+Display a group and its parents in CSV format.
+```
+gam print grouptree <GroupEntity> [todrive <ToDriveAttribute>*]
+        [showparentsaslist [<Boolean>]] [delimiter <Character>]
+```
+By default, the group parent emails and names are displayed in multiple indexed columns.
+Use options `showparentsaslist [<Boolean>]` and `delimiter <Character>` to display
+the group parent emails and names in two columns as delimited lists .
+
+#### Examples
+```
+$ gam show grouptree testgroup2@domain.com
+Show 1 Group Tree
+testgroup2@domain.com: Test - Group 2
+  testgroup1@domain.com: Test Group1
+    testgroup@domain.com: Test Group Org
+  testgroup@domain.net: Test Group Net
+
+$ gam print grouptree testgroup2@domain.com
+Group,Name,parents,parents.0.email,parents.0.name,parents.1.email,parents.1.name
+testgroup2@domain.com,Test - Group 2,2,testgroup1@domain.com,Test Group1,testgroup@domain.com,Test Group Org
+testgroup2@domain.com,Test - Group 2,1,testgroup@domain.net,Test Group Net,,
+
+$ gam print grouptree testgroup2@domain.com showparentsaslist delimiter "|"
+Group,Name,ParentsCount,Parents,ParentsName
+testgroup2@domain.com,Test - Group 2,2,testgroup1@domain.com|testgroup@domain.com,Test Group1|Test Group Org
+testgroup2@domain.com,Test - Group 2,1,testgroup@domain.net,Test Group Net
+```
+## Display group counts
+Display the number of groups.
+```
+gam print groups
+        [([domain|domains <DomainNameEntity>] ([member|showownedby <EmailItem>]|[(query <QueryGroup>)|(queries <QueryGroupList>)]))|
+         (select <GroupEntity>)]
+        [emailmatchpattern [not] <RegularExpression>] [namematchpattern [not] <RegularExpression>]
+        [descriptionmatchpattern [not] <RegularExpression>] (matchsetting [not] <GroupAttribute>)*
+        [admincreatedmatch <Boolean>]
+        showitemcountonly
+```
+Example
+```
+$ gam print groups showitemcountonly
+Getting all Groups, may take some time on a large Google Workspace Account...
+Got 200 Groups: 1aparents@domain.com - students-genderfood@domain.com
+Got 238 Groups: students-worldculture@domain.com - xcarestaff@domain.com
+238
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print groups showitemcountonly)
+Windows PowerShell
+count = & gam print groups showitemcountonly
+```

docs/HTTPS-Proxy.md

@@ -0,0 +1,33 @@
+# HTTPS Proxy
+
+GAM should be run on a server with direct access to talk to Google servers via the Internet.
+However, if you must push GAM traffic through an HTTPS proxy this can be done by setting the HTTPS_PROXY environment variable.
+
+## Linux and MacOS and Google Cloud Shell
+
+Add the following line (use the actual proxy IP address and port number):
+```
+export HTTPS_PROXY="http://192.168.1.1:3128"
+```
+to one of these files based on your shell:
+```
+~/.bash_profile
+~/.bashrc
+~/.zshrc
+~/.profile
+```
+## Windows
+
+Set a system environment variable (use the actual proxy IP address and port number):
+```
+Start Control Panel
+Click System
+Click Advanced system settings
+Click Environment Variables...
+Set Variable name: HTTPS_PROXY
+Set Variable value: http://192.168.1.1:3128
+Click OK
+Click OK
+Click OK
+Exit Control Panel
+```