Updated gam delete|update chromepolicy

#1720, #1721, #1722

.github/ISSUE_TEMPLATE.txt

@@ -1,8 +1,8 @@
 The issue tracker is for reporting product deficiencies. "How do I?" questions should be posted to the discussion forum at https://groups.google.com/group/google-apps-manager. When in doubt, start at the discussion forum and return here only when instructed to do so.
 
 Please confirm the following:
-* I have upgraded to the latest GAM release from https://git.io/gamreleases and I still have this issue.
-* I am typing the command as described in the GAM Wiki at https://github.com/jay0lee/gam/wiki
+* I have upgraded to the latest GAM release from https://github.com/GAM-team/GAM/releases and I still have this issue.
+* I am typing the command as described in the GAM Wiki at https://github.com/GAM-team/GAM/wiki
 
 Full steps to reproduce the issue:
 1.

.github/ISSUE_TEMPLATE/aa-question.md

@@ -0,0 +1,14 @@
+---
+name: Question about using GAM
+about: Help with using GAM or running it for the first time
+title: Please use the GAM discussion group
+labels: invalid
+assignees: ''
+
+---
+
+If you need help with GAM, please do not file an issue here, it will be closed and ignored.
+
+Please post your question to the GAM discussion group where other admins are ready and willing to help:
+
+https://groups.google.com/g/google-apps-manager

.github/ISSUE_TEMPLATE/za-bug-report.md

@@ -0,0 +1,23 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: jay0lee
+
+---
+
+The issue tracker is for reporting product deficiencies. "How do I?" questions should be posted to the discussion forum at https://groups.google.com/group/google-apps-manager. When in doubt, start at the discussion forum and return here only when instructed to do so.
+
+Please confirm the following:
+* I have upgraded to the latest GAM release from https://github.com/GAM-team/GAM/releases and I still have this issue.
+* I am typing the command as described in the GAM Wiki at https://github.com/jay0lee/gam/wiki
+
+Full steps to reproduce the issue:
+1.
+2.
+3.
+
+Expected outcome (what are you trying to do?):
+
+Actual outcome (what errors or bad behavior do you see instead?):

.github/ISSUE_TEMPLATE/zz-feature-request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for GAM
+title: ''
+labels: enhancement
+assignees: jay0lee
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/actions/decrypt.sh

@@ -0,0 +1,38 @@
+#!/bin/sh
+credspath="$3"
+if [ ! -d "$credspath" ]; then
+    echo "creating ${credspath}"
+    mkdir -p "$credspath"
+fi
+gpgfile="$1"
+if [ -f "$gpgfile" ]; then
+    echo "source file is ${gpgfile}"
+else
+    echo "ERROR: ${gpgfile} does not exist"
+    exit 1
+fi
+credsfile="$2"
+echo "target file is ${credsfile}"
+if [ -z ${PASSCODE+x} ]; then
+  echo "ERROR: PASSCODE is unset";
+  exit 2
+else
+  echo "PASSCODE is set";
+fi
+
+gpg --batch \
+    --yes \
+    --decrypt \
+    --passphrase="$PASSCODE" \
+    --output "$credsfile" \
+    "$gpgfile"
+
+if [[ "$RUNNER_OS" == "macOS" ]]; then
+  tar="gtar"
+else
+  tar="tar"
+fi
+
+"$tar" xlvvf "$credsfile" --directory "$credspath"
+rm -rvf "$gpgfile"
+rm -rvf "$credsfile"

.github/actions/entitlements.plist

@@ -0,0 +1,13 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+        <!-- These are required for binaries built by PyInstaller -->
+        <key>com.apple.security.cs.allow-jit</key>
+        <true/>
+        <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
+        <true/>
+        <key>com.apple.security.cs.disable-library-validation</key>
+        <true/>
+</dict>
+</plist>

.github/actions/package_exclusions.txt

@@ -0,0 +1,6 @@
+oauth2.txt
+nobrowser.txt
+enabledasa.txt
+lastupdatecheck.txt
+*.lck
+*.csv

.github/workflows/codeql-analysis.yml

@@ -0,0 +1,70 @@
+# For most projects, this workflow file will not need changing; you simply need
+# to commit it to your repository.
+#
+# You may wish to alter this file to override the set of languages analyzed,
+# or to provide custom queries or build logic.
+#
+# ******** NOTE ********
+# We have attempted to detect the languages in your repository. Please check
+# the `language` matrix defined below to confirm you have the correct set of
+# supported CodeQL languages.
+#
+name: "CodeQL"
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    # The branches below must be a subset of the branches above
+    branches: [ main ]
+  schedule:
+    - cron: '25 10 * * 1'
+
+jobs:
+  analyze:
+    name: Analyze
+    runs-on: ubuntu-latest
+    permissions:
+      actions: read
+      contents: read
+      security-events: write
+
+    strategy:
+      fail-fast: false
+      matrix:
+        language: [ 'python' ]
+        # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
+        # Learn more about CodeQL language support at https://git.io/codeql-language-support
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v4
+
+    # Initializes the CodeQL tools for scanning.
+    - name: Initialize CodeQL
+      uses: github/codeql-action/init@v3
+      with:
+        languages: ${{ matrix.language }}
+        # If you wish to specify custom queries, you can do so here or in a config file.
+        # By default, queries listed here will override any specified in a config file.
+        # Prefix the list here with "+" to use these queries and those in the config file.
+        # queries: ./path/to/local/query, your-org/your-repo/queries@main
+
+    # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
+    # If this step fails, then you should remove it and run the build manually (see below)
+    - name: Autobuild
+      uses: github/codeql-action/autobuild@v3
+
+    # ℹ️ Command-line programs to run using the OS shell.
+    # 📚 https://git.io/JvXDl
+
+    # ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
+    #    and modify them (or add more) to build your code if your project
+    #    uses a compiled language
+
+    #- run: |
+    #   make bootstrap
+    #   make release
+
+    - name: Perform CodeQL Analysis
+      uses: github/codeql-action/analyze@v3

.github/workflows/get-cacerts.yml

@@ -0,0 +1,52 @@
+name: Check for Google Root CA Updates
+
+on:
+  push:
+  pull_request:
+  schedule:
+    - cron: '23 23 * * *'
+
+defaults:
+  run:
+    shell: bash
+    working-directory: src
+
+jobs:
+  check-apis:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@master
+        with:
+          persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal token
+          fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
+
+      - name: Get Current cacerts.pem hash
+        run: |
+          export CURRENT_HASH=$(sha256sum ./cacerts.pem)
+          echo "Current hash is: ${CURRENT_HASH}"
+          echo "CURRENT_HASH=${CURRENT_HASH}" >> $GITHUB_ENV
+
+      - name: Get latest cacerts.pem file from Google
+        run: |
+          curl -o ./cacerts.pem -vvvv https://pki.goog/roots.pem
+
+      - name: Compare hashes
+        run: |
+          export NEW_HASH=$(sha256sum ./cacerts.pem)
+          if [ "$NEW_HASH" == "$CURRENT_HASH" ]; then
+            echo "Same file."
+          else
+            echo "New file content. Was ${CURRENT_HASH} and now is ${NEW_HASH}"
+          fi
+
+      - name: Commit file
+        run: |
+          git config --local user.email "action@github.com"
+          git config --local user.name "GitHub Action"
+          git add cacerts.pem
+          git diff --quiet && git diff --staged --quiet || git commit -am '[ci skip] Updated cacerts.pem'
+
+      - name: Push changes
+        uses: ad-m/github-push-action@master
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }} 

.pre-commit-config.yaml

@@ -1,7 +1,4 @@
 # See https://pre-commit.com for more information
-# See https://pre-commit.com/hooks.html for more hooks
-default_language_version:
-    python: python3.7
 
 repos:
 -   repo: https://github.com/pre-commit/pre-commit-hooks
@@ -27,3 +24,9 @@ repos:
     hooks:
     -   id: pylint
         args: [--output-format=colorized]
+
+-   repo: https://github.com/asottile/pyupgrade
+    rev: v2.31.0
+    hooks:
+    -   id: pyupgrade
+        args: [--py37-plus]

.travis.yml

@@ -1,249 +0,0 @@
-if: tag IS blank
-os: linux
-language: python
-dist: focal
-
-env:
-  global:
-    - BUILD_PYTHON_VERSION=3.8.6
-    - MIN_PYTHON_VERSION=3.8.6
-    - BUILD_OPENSSL_VERSION=1.1.1h
-    - MIN_OPENSSL_VERSION=1.1.1h
-    - PATCHELF_VERSION=0.11
-    - PYINSTALLER_COMMIT=7aa19839c171d898b5cf957739083c4bb901607e
-cache:
-  directories:
-    - $HOME/.cache/pip
-    - $HOME/python
-    - $HOME/ssl
-
-jobs:
-  allow_failures:
-    - python: nightly
-  fast_finish: true
-  include:
-    - os: linux
-      name: "Linux 64-bit Focal"
-      dist: focal
-      language: shell
-    - os: linux
-      name: "Linux 64-bit Bionic"
-      dist: bionic
-      language: shell
-    - os: linux
-      name: "Linux 64-bit Xenial"
-      dist: xenial
-      language: shell
-    - os: linux
-      name: "Linux ARM64 Focal"
-      dist: focal
-      language: shell
-      arch: arm64
-      filter_secrets: false
-    - os: linux
-      dist: bionic
-      arch: arm64
-      name: "Linux ARM64 Bionic"
-      language: shell
-      filter_secrets: false
-    - os: linux
-      dist: xenial
-      arch: arm64
-      name: "Linux ARM64 Xenial"
-      language: shell
-      filter_secrets: false
-    - os: linux
-      name: "Python 3.6 Source Testing"
-      language: python
-      python: 3.6
-    - os: linux
-      name: "Python 3.7 Source Testing"
-      language: python
-      python: 3.7
-    - os: linux
-      name: "Python 3.9 dev Source Testing"
-      language: python
-      python: 3.9-dev
-#    - os: linux
-#      name: "Python trunk nightly Source Testing"
-#      language: python
-#      python: nightly
-#    - os: linux
-#      name: "Python PyPi Source Testing"
-#      language: python
-#      python: pypy3
-    - os: osx
-      name: "MacOS 10.13"
-      language: generic
-      osx_image: xcode10.1
-    - os: osx
-      name: "MacOS 10.14"
-      language: generic
-      osx_image: xcode11.3
-    - os: osx
-      name: "MacOS 10.15"
-      language: generic
-      osx_image: xcode11.7
-    - os: osx
-      name: "MacOS 10.15 Universal Testing"
-      language: generic
-      osx_image: xcode12u
-    - os: windows
-      name: "Windows 64-bit"
-      language: shell
-    - os: windows
-      name: "Windows 32-bit"
-      language: shell
-
-before_install:
-- if [ "${TRAVIS_OS_NAME}" == "osx" ]; then
-    export GAMOS="macos";
-  else
-    export GAMOS="${TRAVIS_OS_NAME}";
-  fi
-- if [ "${TRAVIS_JOB_NAME}" == "Windows 32-bit" ]; then
-    export PLATFORM="x86";
-  elif [ "${TRAVIS_CPU_ARCH}" == "amd64" ]; then
-    export PLATFORM="x86_64";
-  else
-    export PLATFORM="${TRAVIS_CPU_ARCH}";
-  fi
-- source src/travis/${TRAVIS_OS_NAME}-before-install.sh
-
-install:
-- source src/travis/${TRAVIS_OS_NAME}-install.sh
-
-script:
-# Discover and run all Python unit tests. Buffer output so that it's not sent to the build log.
-- $python -m unittest discover --start-directory ./ --pattern "*_test.py" --buffer
-- touch $gampath/nobrowser.txt
-- $gam version extended
-- $gam version | grep travis # travis should be part of the path (not /tmp or such)
-# determine which Python version GAM is built with and ensure it's at least build version from above.
-- if [[ "$TRAVIS_JOB_NAME" != *"Testing" ]];  then vline=$($gam version | grep "Python "); python_line=($vline); this_python=${python_line[1]}; $python tools/a_atleast_b.py $this_python $MIN_PYTHON_VERSION; fi
-# determine which OpenSSL version GAM is built with and ensure it's at least build version from above.
-- if [[ "$TRAVIS_JOB_NAME" != *"Testing" ]]; then vline=$($gam version extended | grep "OpenSSL "); openssl_line=($vline); this_openssl=${openssl_line[1]}; $python tools/a_atleast_b.py $this_openssl $MIN_OPENSSL_VERSION; fi
-- if [[ "$TRAVIS_JOB_NAME" != *"Testing" ]]; then $gam version extended | grep TLSv1\.[23]; fi # Builds should default TLS 1.2 or 1.3 to Google
-- if [[ "$TRAVIS_JOB_NAME" != *"Testing" ]]; then GAM_TLS_MIN_VERSION=TLSv1_2 $gam version extended location tls-v1-0.badssl.com:1010; [[ $? == 3 ]]; fi # expect fail since server doesn't support our TLS version
-- export jid="$(cut -d'.' -f2 <<<"$TRAVIS_JOB_NUMBER")"
-- if [ "$TRAVIS_EVENT_TYPE" != "pull_request" ]; then export e2e=true; fi
-- if [ "$e2e" = true ]; then export gam_user=gam-travis-$jid@pdl.jaylee.us; fi
-- if [ "$e2e" = true ]; then openssl aes-256-cbc -K $encrypted_6294a53f809d_key -iv $encrypted_6294a53f809d_iv -in travis/creds.tar.enc -out travis/creds.tar -d; fi
-- if [ "$e2e" = true ]; then tar xvf travis/creds.tar -C $gampath; fi
-- if [ "$e2e" = true ]; then export OAUTHFILE=oauth2.txt-gam-travis-$jid; fi
-- if [ "$e2e" = true ]; then $gam oauth info; fi
-- if [ "$e2e" = true ]; then $gam info domain; fi
-- if [ "$e2e" = true ]; then $gam oauth refresh; fi
-- if [ "$e2e" = true ]; then $gam info user; fi
-- if [ "$e2e" = true ]; then export tstamp=$(date +%s%3N);
-  export newbase=travis-test-$jid-$tstamp;
-  export newuser=$newbase@pdl.jaylee.us;
-  export newgroup=$newbase-group@pdl.jaylee.us;
-  export newalias=$newbase-alias@pdl.jaylee.us;
-  export newbuilding=$newbase-building;
-  export newresource=$newbase-resource;
-  export GAM_THREADS=5; fi
-- if [ "$e2e" = true ]; then echo email > sample.csv;
-  for i in {01..20};
-    do echo $newbase-bulkuser-$i >> sample.csv;
-  done; fi
-- if [ "$e2e" = true ]; then $gam create user $newuser firstname Travis lastname $jid password random recoveryphone 12125121110 recoveryemail jay0lee@gmail.com travis.jid $jid; fi
-- if [ "$e2e" = true ]; then $gam user $gam_user sendemail recipient $newuser subject "test message $newbase" message "Travis test message"; fi
-- if [ "$e2e" = true ]; then $gam user $gam_user sendemail recipient exchange@pdl.jaylee.us subject "test ${tstamp}" message "test message"; fi
-- if [ "$e2e" = true ]; then $gam create group $newgroup name "Travis $jid group" description "This is a description" isarchived true; fi
-- if [ "$e2e" = true ]; then $gam user $newuser add license gsuitebusiness; fi
-- if [ "$e2e" = true ]; then $gam update group $newgroup add owner $gam_user; fi
-- if [ "$e2e" = true ]; then $gam update group $newgroup add member $newuser; fi
-- if [ "$e2e" = true ]; then $gam csv sample.csv gam create user ~~email~~ firstname "Travis Bulk" lastname ~~email~~ travis.jid $jid; fi
-- if [ "$e2e" = true ]; then $gam csv sample.csv gam update user ~~email~~ recoveryphone 12125121110 recoveryemail jay0lee@gmail.com password random; fi
-- if [ "$e2e" = true ]; then $gam csv sample.csv gam update user ~~email~~ recoveryphone "" recoveryemail ""; fi
-- if [ "$e2e" = true ]; then $gam csv sample.csv gam user ~email add license gsuitebusiness; fi
-- if [ "$e2e" = true ]; then $gam csv sample.csv gam user $gam_user sendemail recipient ~~email~~@pdl.jaylee.us subject "test message $newbase" message "Travis test message"; fi
-- if [ "$e2e" = true ]; then $gam csv sample.csv gam update group $newgroup add member ~email; fi
-- if [ "$e2e" = true ]; then $gam info group $newgroup; fi
-- if [ "$e2e" = true ]; then $gam user $gam_user check serviceaccount; fi
-- if [ "$e2e" = true ]; then $gam user $newuser imap on; fi
-- if [ "$e2e" = true ]; then $gam user $newuser show imap; fi
-- if [ "$e2e" = true ]; then $gam csv sample.csv gam user $newuser delegate to ~email; fi
-- if [ "$e2e" = true ]; then $gam user $newuser show delegates; fi
-- if [ "$e2d" = true ]; then export biohazard=$(echo -e '\xe2\x98\xa3'); fi
-- if [ "$e2e" = true ]; then $gam user $newuser label "$biohazard unicode biohazard $biohazard"; fi
-- if [ "$e2e" = true ]; then $gam user $newuser show labels; fi
-- if [ "$e2e" = true ]; then $gam user $newuser show labels > labels.txt; fi
-- if [ "$e2e" = true ]; then $gam user $gam_user importemail subject "Travis import $newbase" message "This is a test import" labels IMPORTANT,UNREAD,INBOX,STARRED; fi
-- if [ "$e2e" = true ]; then $gam user $gam_user insertemail subject "Travis insert $newbase" file gam.py labels INBOX,UNREAD; fi # yep body is gam code
-- if [ "$e2e" = true ]; then $gam user $gam_user sendemail subject "Travis send $gam_user $newbase" file gam.py recipient admin@pdl.jaylee.us; fi
-- if [ "$e2e" = true ]; then $gam user $gam_user draftemail subject "Travis draft $newbase" message "Draft message test"; fi
-- if [ "$e2e" = true ]; then $gam users "$gam_user $newbase-bulkuser-01 $newbase-bulkuser-02 $newbase-bulkuser-03" delete messages query in:anywhere maxtodelete 99999 doit; fi
-- if [ "$e2e" = true ]; then $gam users "$newbase-bulkuser-04 $newbase-bulkuser-05 $newbase-bulkuser-06" trash messages query in:anywhere maxtotrash 99999 doit; fi
-- if [ "$e2e" = true ]; then $gam users "$newbase-bulkuser-07 $newbase-bulkuser-08 $newbase-bulkuser-09" modify messages query in:anywhere maxtomodify 99999 addlabel IMPORTANT addlabel STARRED doit; fi
-- if [ "$e2e" = true ]; then $gam user $newuser delete label --ALL_LABELS--; fi
-- if [ "$e2e" = true ]; then $gam create feature name Whiteboard-$newbase; fi
-- if [ "$e2e" = true ]; then $gam create feature name VC-$newbase; fi
-- if [ "$e2e" = true ]; then $gam create building "My Building - $newbase" id $newbuilding floors 1,2,3,4,5,6,7,8,9,10,11,12,14,15 description "No 13th floor here..."; fi
-- if [ "$e2e" = true ]; then $gam create resource $newresource "Resource Calendar $tstamp" capacity 25 features Whiteboard-$newbase,VC-$newbase building $newbuilding floor 15 type Room; fi
-- if [ "$e2e" = true ]; then $gam info resource $newresource; fi
-- if [ "$e2e" = true ]; then $gam user $newuser show filelist; fi
-- if [ "$e2e" = true ]; then $gam calendar $gam_user printacl | $gam csv - gam calendar $gam_user delete id ~id; fi # clear ACLs
-- if [ "$e2e" = true ]; then $gam calendar $gam_user update read domain; fi
-- if [ "$e2e" = true ]; then $gam calendar $gam_user update freebusy default; fi
-- if [ "$e2e" = true ]; then $gam calendar $gam_user add editor $newuser; fi
-- if [ "$e2e" = true ]; then $gam calendar $gam_user showacl; fi
-- if [ "$e2e" = true ]; then $gam calendar $gam_user printacl | $gam csv - gam calendar $gam_user delete id ~id; fi
-- if [ "$e2e" = true ]; then $gam calendar $gam_user addevent summary "Travis test event" start $(date '+%FT%T.%N%:z' -d "now + 1 hour") end $(date '+%FT%T.%N%:z' -d "now + 2 hours") attendee $newgroup hangoutsmeet guestscanmodify true sendupdates all; fi
-- if [ "$e2e" = true ]; then $gam calendar $gam_user printevents after -0d; fi
-- if [ "$e2e" = true ]; then matterid=uid:$($gam create vaultmatter name "Travis matter $newbase" description "test matter" collaborators $newuser | head -1 | cut -d ' ' -f 3); fi
-- if [ "$e2e" = true ]; then $gam create vaulthold matter $matterid name "Travis hold $newbase" corpus mail accounts $newuser; fi
-- if [ "$e2e" = true ]; then $gam print vaultmatters matterstate open; fi
-- if [ "$e2e" = true ]; then $gam print vaultholds matter $matterid; fi
-- if [ "$e2e" = true ]; then $gam create vaultexport matter $matterid name "Travis export $newbase" corpus mail accounts $newuser; fi
-- if [ "$e2e" = true ]; then $gam print exports matter $matterid | $gam csv - gam info export $matterid id:~~id~~; fi
-- if [ "$e2e" = true ]; then $gam csv sample.csv gam user ~email add calendar id:$newresource; fi
-- if [ "$e2e" = true ]; then $gam delete resource $newresource; fi
-- if [ "$e2e" = true ]; then $gam delete feature Whiteboard-$newbase; fi
-- if [ "$e2e" = true ]; then $gam delete feature VC-$newbase; fi
-- if [ "$e2e" = true ]; then $gam delete building $newbuilding; fi
-- if [ "$e2e" = true ]; then $gam delete group $newgroup; fi
-- if [ "$e2e" = true ]; then $gam create alias $newalias user $newuser; fi
-- if [ "$e2e" = true ]; then $gam whatis $newuser; fi
-- if [ "$e2e" = true ]; then $gam user $gam_user show tokens; fi
-- if [ "$e2e" = true ]; then $gam print exports matter $matterid | $gam csv - gam download export $matterid id:~~id~~; fi
-- if [ "$e2e" = true ]; then $gam delete hold "Travis hold $newbase" matter $matterid; fi
-- if [ "$e2e" = true ]; then $gam update matter $matterid action close; fi
-- if [ "$e2e" = true ]; then $gam update matter $matterid action delete; fi
-- if [ "$e2e" = true ]; then $gam delete user $newuser; fi
-- if [ "$e2e" = true ]; then $gam print users query "travis.jid=$jid" | $gam csv - gam delete user ~primaryEmail; fi
-- if [ "$e2e" = true ]; then $gam print mobile; fi
-- if [ "$e2e" = true ]; then $gam print devices; fi
-- if [ "$e2e" = true ]; then export sn="$jid$jid$jid$jid$jid-$(openssl rand -base64 32 | sed 's/[^a-zA-Z0-9]//g')"; fi
-- if [ "$e2e" = true ]; then $gam create device serialnumber $sn devicetype android; fi
-- if [ "$e2e" = true ]; then $gam print cros allfields nolists; fi
-- if [ "$e2e" = true ]; then $gam report usageparameters customer; fi
-- if [ "$e2e" = true ]; then $gam report usage customer parameters gmail:num_emails_sent,accounts:num_1day_logins; fi
-- if [ "$e2e" = true ]; then $gam report customer todrive; fi
-- if [ "$e2e" = true ]; then $gam report users fields accounts:is_less_secure_apps_access_allowed,gmail:last_imap_time,gmail:last_pop_time filters "accounts:last_login_time>2019-01-01T00:00:00.000Z" todrive; fi
-- if [ "$e2e" = true ]; then $gam report admin start -3d todrive; fi
-- if [ "$e2e" = true ]; then $gam print devices nopersonaldevices nodeviceusers filter "serial:$jid$jid$jid$jid$jid-" | $gam csv - gam delete device id ~name; fi
-- if ([ "$e2e" = true ] && [[ "$TRAVIS_JOB_NAME" != *"Testing" ]]); then
-    for gamfile in gam-$GAMVERSION-*; do
-      fileid=$($gam user $gam_user add drivefile localfile $gamfile drivefilename $GAMVERSION-${TRAVIS_COMMIT:0:7}-$gamfile parentid 1N2zbO33qzUQFsGM49-m9AQC1ijzd_ru1 returnidonly);
-      $gam user $gam_user add drivefileacl $fileid anyone role reader withlink;
-    done;
-  fi
-
-before_deploy:
-- export TRAVIS_TAG="preview"
-- unset LD_LIBRARY_PATH
-
-deploy:
-  provider: releases
-  token:
-    secure: bzambMcQwyv/o5c5GrKGCsZHgE5R85tg8sNFvPfpISz3+uosCjnBXas7wvCKzT75XUFi2ztfbYak6HdKf4sGnNHk0saEicB3slH+ghPyZbYzp76yvvduhFO2nWW3/F01tL+Yfqqt4/q8wFaWGjrC5km+6GLVyB4lWA/Uyu49qKnz02uSwyhBD/VFbO7DOQ65a1iWk9HngyMsu0Oi7HIbSjSLtxTHedNfOf3waW0NivTTxYXiYGX/MCu3GWhgIGj47a+H3A6FcQ/9QWvnKgnoixdgPBUz7kDb7ktsWwQsILPGStgH7iMuG49ZlXdEFmqwifBri2wvzmFEevBGZjHcupy1IGrNFRG+IUGKMotio+OkLHlLjuv7ZJtqCz/Vf5SNFgNyMSanx6jKEUJuYvndVg99IRXmYVwHFwPu5BAcJACpU6C0AfyGmmSqqwxCd46uXL62ynxNFpHuRfOqlDnmCTfZgjOciJSlDDpf+Xz9fF7+oCoeCi3mrcZVFjhd3tT6Oxw5HrsDtm0ZNld1cdLidaq8H6vOFgHMd0A9yNYZzTzXTvpmxzkXT4Zc7s+PYKN6z5fRZ+pJeckUjRXblvVEfs5HFSymavcOc5AkRwxpvOsTQMNmlnaJCBo5UNs0K/rVmRi5cFmaiwTcBCY0kTllOBJ4zWsfq8seiokWwNUNK2g=
-  file_glob: true
-  overwrite: true
-  file: gam-$GAMVERSION-*
-  skip_cleanup: true
-  draft: true
-  on:
-    repo: jay0lee/GAM
-    condition: $TRAVIS_JOB_NAME != *"Testing"

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -1,24 +1,41 @@
-GAM is a command line tool for Google G Suite Administrators to manage domain and user settings quickly and easily. [![Build Status](https://travis-ci.org/jay0lee/GAM.svg?branch=master)](https://travis-ci.org/jay0lee/GAM)
-# Quick Start
-## Linux / MacOS
-Open a terminal and run:
-```
-bash <(curl -s -S -L https://git.io/install-gam)
-```
-this will download GAM, install it and start setup.
-## Windows
-Download the MSI Installer from the [GitHub Releases] page. Install the MSI and you'll be prompted to setup GAM.
-# Documentation
-The GAM documentation is hosted in the [GitHub Wiki]
-# Mailing List / Discussion group
-The GAM mailing list / discussion group is hosted on [Google Groups].  You can join the list and interact via email, or just post from the web itself.
-# IM Room
-[![Join the chat at https://gitter.im/jay0lee-GAM/community](https://badges.gitter.im/jay0lee-GAM/community.svg)](https://gitter.im/jay0lee-GAM/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
-# Author
-GAM is maintained by <a href="mailto:jay0lee@gmail.com">Jay Lee</a>. Please direct "how do I?" questions to [Google Groups].
+GAM is a command line tool for Google Workspace admins to manage domain and user settings quickly and easily.
 
-[GAM release]: https://git.io/gamreleases
-[GitHub Releases]: https://github.com/jay0lee/GAM/releases
-[GitHub]: https://github.com/jay0lee/GAM/tree/master
-[GitHub Wiki]: https://github.com/jay0lee/GAM/wiki/
+[![Build StatusM](https://github.com/GAM-team/GAM/actions/workflows/build.yml/badge.svg)](https://github.com/GAM-team/GAM/actions/workflows/build.yml)
+
+# Quick Start
+
+## Linux / MacOS
+
+Open a terminal and run:
+
+```sh
+bash <(curl -s -S -L https://gam-shortn.appspot.com/gam-install)
+```
+
+this will download GAM, install it and start setup.
+
+## Windows
+
+Download the MSI Installer from the [GitHub Releases] page. Install the MSI and you'll be prompted to setup GAM.
+
+# Documentation
+
+The GAM documentation is hosted in the [GitHub Wiki]
+
+# Mailing List / Discussion group
+
+The GAM mailing list / discussion group is hosted on [Google Groups].  You can join the list and interact via email, or just post from the web itself.
+
+# Chat Room
+
+There is a public chat room hosted in Google Chat. [Instructions to join](https://github.com/GAM-team/GAM/wiki/GAM-Public-Chat-Room).
+
+# Author
+
+GAM is maintained by [Jay (James) Lee](mailto:jay0lee@gmail.com) and [Ross Scroggs](mailto:ross.scroggs@gmail.com). Please direct "how do I?" questions to [Google Groups].
+
+[GAM release]: https://github.com/GAM-team/GAM/releases
+[GitHub Releases]: https://github.com/GAM-team/GAM/releases
+[GitHub]: https://github.com/GAM-team/GAM/tree/master
+[GitHub Wiki]: https://github.com/GAM-team/GAM/wiki/
 [Google Groups]: http://groups.google.com/group/google-apps-manager

src/LICENSE

@@ -1,547 +0,0 @@
-                                 Apache License
-                           Version 2.0, January 2004
-                        http://www.apache.org/licenses/
-
-   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
-   1. Definitions.
-
-      "License" shall mean the terms and conditions for use, reproduction,
-      and distribution as defined by Sections 1 through 9 of this document.
-
-      "Licensor" shall mean the copyright owner or entity authorized by
-      the copyright owner that is granting the License.
-
-      "Legal Entity" shall mean the union of the acting entity and all
-      other entities that control, are controlled by, or are under common
-      control with that entity. For the purposes of this definition,
-      "control" means (i) the power, direct or indirect, to cause the
-      direction or management of such entity, whether by contract or
-      otherwise, or (ii) ownership of fifty percent (50%) or more of the
-      outstanding shares, or (iii) beneficial ownership of such entity.
-
-      "You" (or "Your") shall mean an individual or Legal Entity
-      exercising permissions granted by this License.
-
-      "Source" form shall mean the preferred form for making modifications,
-      including but not limited to software source code, documentation
-      source, and configuration files.
-
-      "Object" form shall mean any form resulting from mechanical
-      transformation or translation of a Source form, including but
-      not limited to compiled object code, generated documentation,
-      and conversions to other media types.
-
-      "Work" shall mean the work of authorship, whether in Source or
-      Object form, made available under the License, as indicated by a
-      copyright notice that is included in or attached to the work
-      (an example is provided in the Appendix below).
-
-      "Derivative Works" shall mean any work, whether in Source or Object
-      form, that is based on (or derived from) the Work and for which the
-      editorial revisions, annotations, elaborations, or other modifications
-      represent, as a whole, an original work of authorship. For the purposes
-      of this License, Derivative Works shall not include works that remain
-      separable from, or merely link (or bind by name) to the interfaces of,
-      the Work and Derivative Works thereof.
-
-      "Contribution" shall mean any work of authorship, including
-      the original version of the Work and any modifications or additions
-      to that Work or Derivative Works thereof, that is intentionally
-      submitted to Licensor for inclusion in the Work by the copyright owner
-      or by an individual or Legal Entity authorized to submit on behalf of
-      the copyright owner. For the purposes of this definition, "submitted"
-      means any form of electronic, verbal, or written communication sent
-      to the Licensor or its representatives, including but not limited to
-      communication on electronic mailing lists, source code control systems,
-      and issue tracking systems that are managed by, or on behalf of, the
-      Licensor for the purpose of discussing and improving the Work, but
-      excluding communication that is conspicuously marked or otherwise
-      designated in writing by the copyright owner as "Not a Contribution."
-
-      "Contributor" shall mean Licensor and any individual or Legal Entity
-      on behalf of whom a Contribution has been received by Licensor and
-      subsequently incorporated within the Work.
-
-   2. Grant of Copyright License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      copyright license to reproduce, prepare Derivative Works of,
-      publicly display, publicly perform, sublicense, and distribute the
-      Work and such Derivative Works in Source or Object form.
-
-   3. Grant of Patent License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      (except as stated in this section) patent license to make, have made,
-      use, offer to sell, sell, import, and otherwise transfer the Work,
-      where such license applies only to those patent claims licensable
-      by such Contributor that are necessarily infringed by their
-      Contribution(s) alone or by combination of their Contribution(s)
-      with the Work to which such Contribution(s) was submitted. If You
-      institute patent litigation against any entity (including a
-      cross-claim or counterclaim in a lawsuit) alleging that the Work
-      or a Contribution incorporated within the Work constitutes direct
-      or contributory patent infringement, then any patent licenses
-      granted to You under this License for that Work shall terminate
-      as of the date such litigation is filed.
-
-   4. Redistribution. You may reproduce and distribute copies of the
-      Work or Derivative Works thereof in any medium, with or without
-      modifications, and in Source or Object form, provided that You
-      meet the following conditions:
-
-      (a) You must give any other recipients of the Work or
-          Derivative Works a copy of this License; and
-
-      (b) You must cause any modified files to carry prominent notices
-          stating that You changed the files; and
-
-      (c) You must retain, in the Source form of any Derivative Works
-          that You distribute, all copyright, patent, trademark, and
-          attribution notices from the Source form of the Work,
-          excluding those notices that do not pertain to any part of
-          the Derivative Works; and
-
-      (d) If the Work includes a "NOTICE" text file as part of its
-          distribution, then any Derivative Works that You distribute must
-          include a readable copy of the attribution notices contained
-          within such NOTICE file, excluding those notices that do not
-          pertain to any part of the Derivative Works, in at least one
-          of the following places: within a NOTICE text file distributed
-          as part of the Derivative Works; within the Source form or
-          documentation, if provided along with the Derivative Works; or,
-          within a display generated by the Derivative Works, if and
-          wherever such third-party notices normally appear. The contents
-          of the NOTICE file are for informational purposes only and
-          do not modify the License. You may add Your own attribution
-          notices within Derivative Works that You distribute, alongside
-          or as an addendum to the NOTICE text from the Work, provided
-          that such additional attribution notices cannot be construed
-          as modifying the License.
-
-      You may add Your own copyright statement to Your modifications and
-      may provide additional or different license terms and conditions
-      for use, reproduction, or distribution of Your modifications, or
-      for any such Derivative Works as a whole, provided Your use,
-      reproduction, and distribution of the Work otherwise complies with
-      the conditions stated in this License.
-
-   5. Submission of Contributions. Unless You explicitly state otherwise,
-      any Contribution intentionally submitted for inclusion in the Work
-      by You to the Licensor shall be under the terms and conditions of
-      this License, without any additional terms or conditions.
-      Notwithstanding the above, nothing herein shall supersede or modify
-      the terms of any separate license agreement you may have executed
-      with Licensor regarding such Contributions.
-
-   6. Trademarks. This License does not grant permission to use the trade
-      names, trademarks, service marks, or product names of the Licensor,
-      except as required for reasonable and customary use in describing the
-      origin of the Work and reproducing the content of the NOTICE file.
-
-   7. Disclaimer of Warranty. Unless required by applicable law or
-      agreed to in writing, Licensor provides the Work (and each
-      Contributor provides its Contributions) on an "AS IS" BASIS,
-      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-      implied, including, without limitation, any warranties or conditions
-      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
-      PARTICULAR PURPOSE. You are solely responsible for determining the
-      appropriateness of using or redistributing the Work and assume any
-      risks associated with Your exercise of permissions under this License.
-
-   8. Limitation of Liability. In no event and under no legal theory,
-      whether in tort (including negligence), contract, or otherwise,
-      unless required by applicable law (such as deliberate and grossly
-      negligent acts) or agreed to in writing, shall any Contributor be
-      liable to You for damages, including any direct, indirect, special,
-      incidental, or consequential damages of any character arising as a
-      result of this License or out of the use or inability to use the
-      Work (including but not limited to damages for loss of goodwill,
-      work stoppage, computer failure or malfunction, or any and all
-      other commercial damages or losses), even if such Contributor
-      has been advised of the possibility of such damages.
-
-   9. Accepting Warranty or Additional Liability. While redistributing
-      the Work or Derivative Works thereof, You may choose to offer,
-      and charge a fee for, acceptance of support, warranty, indemnity,
-      or other liability obligations and/or rights consistent with this
-      License. However, in accepting such obligations, You may act only
-      on Your own behalf and on Your sole responsibility, not on behalf
-      of any other Contributor, and only if You agree to indemnify,
-      defend, and hold each Contributor harmless for any liability
-      incurred by, or claims asserted against, such Contributor by reason
-      of your accepting any such warranty or additional liability.
-
-   END OF TERMS AND CONDITIONS
-
-   APPENDIX: How to apply the Apache License to your work.
-
-      To apply the Apache License to your work, attach the following
-      boilerplate notice, with the fields enclosed by brackets "[]"
-      replaced with your own identifying information. (Don't include
-      the brackets!)  The text should be enclosed in the appropriate
-      comment syntax for the file format. We also recommend that a
-      file or class name and description of purpose be included on the
-      same "printed page" as the copyright notice for easier
-      identification within third-party archives.
-
-   Copyright [yyyy] [name of copyright owner]
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
-
-
-
-APACHE HTTP SERVER SUBCOMPONENTS:
-
-The Apache HTTP Server includes a number of subcomponents with
-separate copyright notices and license terms. Your use of the source
-code for the these subcomponents is subject to the terms and
-conditions of the following licenses.
-
-For the mod_mime_magic component:
-
-/*
- * mod_mime_magic: MIME type lookup via file magic numbers
- * Copyright (c) 1996-1997 Cisco Systems, Inc.
- *
- * This software was submitted by Cisco Systems to the Apache Group in July
- * 1997.  Future revisions and derivatives of this source code must
- * acknowledge Cisco Systems as the original contributor of this module.
- * All other licensing and usage conditions are those of the Apache Group.
- *
- * Some of this code is derived from the free version of the file command
- * originally posted to comp.sources.unix.  Copyright info for that program
- * is included below as required.
- * ---------------------------------------------------------------------------
- * - Copyright (c) Ian F. Darwin, 1987. Written by Ian F. Darwin.
- *
- * This software is not subject to any license of the American Telephone and
- * Telegraph Company or of the Regents of the University of California.
- *
- * Permission is granted to anyone to use this software for any purpose on any
- * computer system, and to alter it and redistribute it freely, subject to
- * the following restrictions:
- *
- * 1. The author is not responsible for the consequences of use of this
- * software, no matter how awful, even if they arise from flaws in it.
- *
- * 2. The origin of this software must not be misrepresented, either by
- * explicit claim or by omission.  Since few users ever read sources, credits
- * must appear in the documentation.
- *
- * 3. Altered versions must be plainly marked as such, and must not be
- * misrepresented as being the original software.  Since few users ever read
- * sources, credits must appear in the documentation.
- *
- * 4. This notice may not be removed or altered.
- * -------------------------------------------------------------------------
- *
- */
-
-
-For the  modules\mappers\mod_imagemap.c component:
-
-  "macmartinized" polygon code copyright 1992 by Eric Haines, erich@eye.com
-
-For the  server\util_md5.c component:
-
-/************************************************************************
- * NCSA HTTPd Server
- * Software Development Group
- * National Center for Supercomputing Applications
- * University of Illinois at Urbana-Champaign
- * 605 E. Springfield, Champaign, IL 61820
- * httpd@ncsa.uiuc.edu
- *
- * Copyright  (C)  1995, Board of Trustees of the University of Illinois
- *
- ************************************************************************
- *
- * md5.c: NCSA HTTPd code which uses the md5c.c RSA Code
- *
- *  Original Code Copyright (C) 1994, Jeff Hostetler, Spyglass, Inc.
- *  Portions of Content-MD5 code Copyright (C) 1993, 1994 by Carnegie Mellon
- *     University (see Copyright below).
- *  Portions of Content-MD5 code Copyright (C) 1991 Bell Communications
- *     Research, Inc. (Bellcore) (see Copyright below).
- *  Portions extracted from mpack, John G. Myers - jgm+@cmu.edu
- *  Content-MD5 Code contributed by Martin Hamilton (martin@net.lut.ac.uk)
- *
- */
-
-
-/* these portions extracted from mpack, John G. Myers - jgm+@cmu.edu */
-/* (C) Copyright 1993,1994 by Carnegie Mellon University
- * All Rights Reserved.
- *
- * Permission to use, copy, modify, distribute, and sell this software
- * and its documentation for any purpose is hereby granted without
- * fee, provided that the above copyright notice appear in all copies
- * and that both that copyright notice and this permission notice
- * appear in supporting documentation, and that the name of Carnegie
- * Mellon University not be used in advertising or publicity
- * pertaining to distribution of the software without specific,
- * written prior permission.  Carnegie Mellon University makes no
- * representations about the suitability of this software for any
- * purpose.  It is provided "as is" without express or implied
- * warranty.
- *
- * CARNEGIE MELLON UNIVERSITY DISCLAIMS ALL WARRANTIES WITH REGARD TO
- * THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
- * AND FITNESS, IN NO EVENT SHALL CARNEGIE MELLON UNIVERSITY BE LIABLE
- * FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
- * AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING
- * OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS
- * SOFTWARE.
- */
-
-/*
- * Copyright (c) 1991 Bell Communications Research, Inc. (Bellcore)
- *
- * Permission to use, copy, modify, and distribute this material
- * for any purpose and without fee is hereby granted, provided
- * that the above copyright notice and this permission notice
- * appear in all copies, and that the name of Bellcore not be
- * used in advertising or publicity pertaining to this
- * material without the specific, prior written permission
- * of an authorized representative of Bellcore.  BELLCORE
- * MAKES NO REPRESENTATIONS ABOUT THE ACCURACY OR SUITABILITY
- * OF THIS MATERIAL FOR ANY PURPOSE.  IT IS PROVIDED "AS IS",
- * WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES.
- */
-
-For the  srclib\apr\include\apr_md5.h component:
-/*
- * This is work is derived from material Copyright RSA Data Security, Inc.
- *
- * The RSA copyright statement and Licence for that original material is
- * included below. This is followed by the Apache copyright statement and
- * licence for the modifications made to that material.
- */
-
-/* Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
-   rights reserved.
-
-   License to copy and use this software is granted provided that it
-   is identified as the "RSA Data Security, Inc. MD5 Message-Digest
-   Algorithm" in all material mentioning or referencing this software
-   or this function.
-
-   License is also granted to make and use derivative works provided
-   that such works are identified as "derived from the RSA Data
-   Security, Inc. MD5 Message-Digest Algorithm" in all material
-   mentioning or referencing the derived work.
-
-   RSA Data Security, Inc. makes no representations concerning either
-   the merchantability of this software or the suitability of this
-   software for any particular purpose. It is provided "as is"
-   without express or implied warranty of any kind.
-
-   These notices must be retained in any copies of any part of this
-   documentation and/or software.
- */
-
-For the  srclib\apr\passwd\apr_md5.c component:
-
-/*
- * This is work is derived from material Copyright RSA Data Security, Inc.
- *
- * The RSA copyright statement and Licence for that original material is
- * included below. This is followed by the Apache copyright statement and
- * licence for the modifications made to that material.
- */
-
-/* MD5C.C - RSA Data Security, Inc., MD5 message-digest algorithm
- */
-
-/* Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
-   rights reserved.
-
-   License to copy and use this software is granted provided that it
-   is identified as the "RSA Data Security, Inc. MD5 Message-Digest
-   Algorithm" in all material mentioning or referencing this software
-   or this function.
-
-   License is also granted to make and use derivative works provided
-   that such works are identified as "derived from the RSA Data
-   Security, Inc. MD5 Message-Digest Algorithm" in all material
-   mentioning or referencing the derived work.
-
-   RSA Data Security, Inc. makes no representations concerning either
-   the merchantability of this software or the suitability of this
-   software for any particular purpose. It is provided "as is"
-   without express or implied warranty of any kind.
-
-   These notices must be retained in any copies of any part of this
-   documentation and/or software.
- */
-/*
- * The apr_md5_encode() routine uses much code obtained from the FreeBSD 3.0
- * MD5 crypt() function, which is licenced as follows:
- * ----------------------------------------------------------------------------
- * "THE BEER-WARE LICENSE" (Revision 42):
- * <phk@login.dknet.dk> wrote this file.  As long as you retain this notice you
- * can do whatever you want with this stuff. If we meet some day, and you think
- * this stuff is worth it, you can buy me a beer in return.  Poul-Henning Kamp
- * ----------------------------------------------------------------------------
- */
-
-For the srclib\apr-util\crypto\apr_md4.c component:
-
- * This is derived from material copyright RSA Data Security, Inc.
- * Their notice is reproduced below in its entirety.
- *
- * Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
- * rights reserved.
- *
- * License to copy and use this software is granted provided that it
- * is identified as the "RSA Data Security, Inc. MD4 Message-Digest
- * Algorithm" in all material mentioning or referencing this software
- * or this function.
- *
- * License is also granted to make and use derivative works provided
- * that such works are identified as "derived from the RSA Data
- * Security, Inc. MD4 Message-Digest Algorithm" in all material
- * mentioning or referencing the derived work.
- *
- * RSA Data Security, Inc. makes no representations concerning either
- * the merchantability of this software or the suitability of this
- * software for any particular purpose. It is provided "as is"
- * without express or implied warranty of any kind.
- *
- * These notices must be retained in any copies of any part of this
- * documentation and/or software.
- */
-
-For the srclib\apr-util\include\apr_md4.h component:
-
- *
- * This is derived from material copyright RSA Data Security, Inc.
- * Their notice is reproduced below in its entirety.
- *
- * Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
- * rights reserved.
- *
- * License to copy and use this software is granted provided that it
- * is identified as the "RSA Data Security, Inc. MD4 Message-Digest
- * Algorithm" in all material mentioning or referencing this software
- * or this function.
- *
- * License is also granted to make and use derivative works provided
- * that such works are identified as "derived from the RSA Data
- * Security, Inc. MD4 Message-Digest Algorithm" in all material
- * mentioning or referencing the derived work.
- *
- * RSA Data Security, Inc. makes no representations concerning either
- * the merchantability of this software or the suitability of this
- * software for any particular purpose. It is provided "as is"
- * without express or implied warranty of any kind.
- *
- * These notices must be retained in any copies of any part of this
- * documentation and/or software.
- */
-
-
-For the srclib\apr-util\test\testmd4.c component:
-
- *
- * This is derived from material copyright RSA Data Security, Inc.
- * Their notice is reproduced below in its entirety.
- *
- * Copyright (C) 1990-2, RSA Data Security, Inc. Created 1990. All
- * rights reserved.
- *
- * RSA Data Security, Inc. makes no representations concerning either
- * the merchantability of this software or the suitability of this
- * software for any particular purpose. It is provided "as is"
- * without express or implied warranty of any kind.
- *
- * These notices must be retained in any copies of any part of this
- * documentation and/or software.
- */
-
-For the srclib\apr-util\xml\expat\conftools\install-sh component:
-
-#
-# install - install a program, script, or datafile
-# This comes from X11R5 (mit/util/scripts/install.sh).
-#
-# Copyright 1991 by the Massachusetts Institute of Technology
-#
-# Permission to use, copy, modify, distribute, and sell this software and its
-# documentation for any purpose is hereby granted without fee, provided that
-# the above copyright notice appear in all copies and that both that
-# copyright notice and this permission notice appear in supporting
-# documentation, and that the name of M.I.T. not be used in advertising or
-# publicity pertaining to distribution of the software without specific,
-# written prior permission.  M.I.T. makes no representations about the
-# suitability of this software for any purpose.  It is provided "as is"
-# without express or implied warranty.
-#
-
-For the test\zb.c component:
-
-/*                          ZeusBench V1.01
-			    ===============
-
-This program is Copyright (C) Zeus Technology Limited 1996.
-
-This program may be used and copied freely providing this copyright notice
-is not removed.
-
-This software is provided "as is" and any express or implied waranties,
-including but not limited to, the implied warranties of merchantability and
-fitness for a particular purpose are disclaimed.  In no event shall
-Zeus Technology Ltd. be liable for any direct, indirect, incidental, special,
-exemplary, or consequential damaged (including, but not limited to,
-procurement of substitute good or services; loss of use, data, or profits;
-or business interruption) however caused and on theory of liability.  Whether
-in contract, strict liability or tort (including negligence or otherwise)
-arising in any way out of the use of this software, even if advised of the
-possibility of such damage.
-
-     Written by Adam Twiss (adam@zeus.co.uk).  March 1996
-
-Thanks to the following people for their input:
-  Mike Belshe (mbelshe@netscape.com)
-  Michael Campanella (campanella@stevms.enet.dec.com)
-
-*/
-
-For the expat xml parser component:
-
-Copyright (c) 1998, 1999, 2000 Thai Open Source Software Center Ltd
-                               and Clark Cooper
-
-Permission is hereby granted, free of charge, to any person obtaining
-a copy of this software and associated documentation files (the
-"Software"), to deal in the Software without restriction, including
-without limitation the rights to use, copy, modify, merge, publish,
-distribute, sublicense, and/or sell copies of the Software, and to
-permit persons to whom the Software is furnished to do so, subject to
-the following conditions:
-
-The above copyright notice and this permission notice shall be included
-in all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
-IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
-CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
-TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
-SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-
-====================================================================

src/LICENSE

@@ -0,0 +1 @@
+../LICENSE

src/callgam.py

@@ -0,0 +1,21 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+""" Sample Python script to call GAM"""
+
+import multiprocessing
+import platform
+
+from gam import initializeLogging, CallGAMCommand
+
+if __name__ == '__main__':
+# One time initialization
+  if platform.system() != 'Linux':
+    multiprocessing.freeze_support()
+    multiprocessing.set_start_method('spawn')
+  initializeLogging()
+#
+  CallGAMCommand(['gam', 'version'])
+  # Issue command, output goes to stdout/stderr
+  rc = CallGAMCommand(['gam', 'info', 'domain'])
+  # Issue command, redirect stdout/stderr
+  rc = CallGAMCommand(['gam', 'redirect', 'stdout', 'domain.txt', 'redirect', 'stderr', 'stdout', 'info', 'domain'])

src/gam-install.sh

@@ -16,10 +16,12 @@ OPTIONS:
    -u      Admin user email address to use with GAM. Default is to prompt.
    -r      Regular user email address. Used to test service account access to user data. Default is to prompt.
    -v      Version to install (latest, prerelease, draft, 3.8, etc). Default is latest.
+   -s      Strip gam component from extracted files, files will be downloaded directly to $target_dir
 EOF
 }
 
 target_dir="$HOME/bin"
+target_gam="gam7/gam"
 gamarch=$(uname -m)
 gamos=$(uname -s)
 osversion=""
@@ -28,10 +30,9 @@ upgrade_only=false
 gamversion="latest"
 adminuser=""
 regularuser=""
-gam_glibc_vers="2.27 2.23"
-gam_macos_vers="10.15.6 10.14.6 10.13.6"
+strip_gam="--strip-components 0"
 
-while getopts "hd:a:o:b:lp:u:r:v:" OPTION
+while getopts "hd:a:o:b:lp:u:r:v:s" OPTION
 do
      case $OPTION in
          h) usage; exit;;
@@ -44,6 +45,7 @@ do
          u) adminuser="$OPTARG";;
          r) regularuser="$OPTARG";;
          v) gamversion="$OPTARG";;
+         s) strip_gam="--strip-components 1"; target_gam="gam";;
          ?) usage; exit;;
      esac
 done
@@ -52,15 +54,15 @@ done
 target_dir=${target_dir%/}
 
 update_profile() {
-	[ $2 -eq 1 ] || [ -f "$1" ] || return 1
+        [ "$2" -eq 1 ] || [ -f "$1" ] || return 1
 
-	grep -F "$alias_line" "$1" > /dev/null 2>&1
-	if [ $? -ne 0 ]; then
+        grep -F "$alias_line" "$1" > /dev/null 2>&1
+        if [ $? -ne 0 ]; then
                 echo_yellow "Adding gam alias to profile file $1."
-		echo -e "\n$alias_line" >> "$1"
+                echo -e "\n$alias_line" >> "$1"
         else
           echo_yellow "gam alias already exists in profile file $1. Skipping add."
-	fi
+        fi
 }
 
 echo_red()
@@ -95,69 +97,38 @@ else
 fi
 }
 
-case $gamos in
-  [lL]inux)
-    gamos="linux"
-    if [ "$osversion" == "" ]; then
-      this_glibc_ver=$(ldd --version | awk '/ldd/{print $NF}')
-    else
-      this_glibc_ver=$osversion
-    fi
-    echo "This Linux distribution uses glibc $this_glibc_ver"
-    useglibc="legacy"
-    for gam_glibc_ver in $gam_glibc_vers; do
-      if version_gt $this_glibc_ver $gam_glibc_ver; then
-        useglibc="glibc$gam_glibc_ver"
-        echo_green "Using GAM compiled against $useglibc"
-        break
-      fi
-    done
-    case $gamarch in
-      x86_64) gamfile="linux-x86_64-$useglibc.tar.xz";;
-      arm64|aarch64) gamfile="linux-arm64-$useglibc.tar.xz";;
-      *)
-        echo_red "ERROR: this installer currently only supports x86_64 and arm64 Linux. Looks like you're running on $gamarch. Exiting."
-        exit
-    esac
-    ;;
-  [Mm]ac[Oo][sS]|[Dd]arwin)
-    gamos="macos"
-    if [ "$osversion" == "" ]; then
-      this_macos_ver=$(sw_vers -productVersion)
-    else
-      this_macos_ver=$osversion
-    fi
-    echo "You are running MacOS $this_macos_ver"
-    use_macos_ver=""
-    for gam_macos_ver in $gam_macos_vers; do
-      if version_gt $this_macos_ver $gam_macos_ver; then
-        use_macos_ver="MacOS$gam_macos_ver"
-        echo_green "Using GAM compiled on $use_macos_ver"
-        break
-      fi
-    done
-    if [ "$use_macos_ver" == "" ]; then
-      echo_red "Sorry, you need to be running at least MacOS $gam_macos_ver to run GAM"
-      exit
-    fi
-    gamfile="macos-x86_64-$use_macos_ver.tar.xz"
-    ;;
-  *)
-    echo_red "Sorry, this installer currently only supports Linux and MacOS. Looks like you're runnning on $gamos. Exiting."
-    exit
-    ;;
-esac
-
-if [ "$gamversion" == "latest" -o "$gamversion" == "prerelease" -o "$gamversion" == "draft" ]; then
-  release_url="https://api.github.com/repos/jay0lee/GAM/releases"
+if [ "$gamversion" == "latest" ]; then
+  release_url="https://api.github.com/repos/GAM-team/GAM/releases/latest"
+elif [ "$gamversion" == "prerelease" -o "$gamversion" == "draft" ]; then
+  release_url="https://api.github.com/repos/GAM-team/GAM/releases"
 else
-  release_url="https://api.github.com/repos/jay0lee/GAM/releases/tags/v$gamversion"
+  release_url="https://api.github.com/repos/GAM-team/GAM/releases/tags/v$gamversion"
 fi
 
-echo_yellow "Checking GitHub URL $release_url for $gamversion GAM release..."
-release_json=$(curl -s $release_url 2>&1 /dev/null)
+if [ -z ${GHCLIENT+x} ]; then
+  check_type="unauthenticated"
+  curl_opts=( )
+else
+  check_type="authenticated"
+  curl_opts=( "$GHCLIENT" )
+fi
+echo_yellow "Checking GitHub URL $release_url for $gamversion GAM release ($check_type)..."
+release_json=$(curl \
+	--silent \
+	"${curl_opts[@]}" \
+	-H "Accept: application/vnd.github+json" \
+	-H "X-GitHub-Api-Version: 2022-11-28" \
+	"$release_url" \
+	--fail-with-body)
+curl_exit_code=$?
+if [ $curl_exit_code -ne 0 ]; then
+  echo_red "ERROR retrieving URL: ${release_json}"
+  exit
+else
+  echo_green "done"
+fi
 
-echo_yellow "Getting file and download URL..."
+echo_yellow "Calculating download URL for this device..."
 # Python is sadly the nearest to universal way to safely handle JSON with Bash
 # At least this code should be compatible with just about any Python version ever
 # unlike GAM itself. If some users don't have Python we can try grep / sed / etc
@@ -179,11 +150,9 @@ if type(release) is list:
     break
 try:
   for asset in release['assets']:
-    if asset[attrib].endswith('$gamfile'):
-      print(asset[attrib])
-      break
-  else:
-    print('ERROR: Attribute: {0} for $gamfile version {1} not found'.format(attrib, gamversion))
+    print(asset[attrib])
+  #else:
+  #  print('ERROR: Attribute: {0} for version {1} not found'.format(attrib, gamversion))
 except KeyError:
   print('ERROR: assets value not found in JSON value of:\n\n%s' % release)"
 
@@ -195,6 +164,11 @@ if (( $rc != 0 )); then
 fi
 $pycmd -V >/dev/null 2>&1
 rc=$?
+if (( $rc != 0 )); then
+  pycmd="/usr/bin/python3"
+fi
+$pycmd -V >/dev/null 2>&1
+rc=$?
 if (( $rc != 0 )); then
   pycmd="python2"
 fi
@@ -204,32 +178,163 @@ if (( $rc != 0 )); then
   echo_red "ERROR: No version of python installed."
   exit
 fi
+# also sort the URLs once so we're evaluating newest OS version first
+download_urls=$(echo "$release_json" | \
+	        $pycmd -c "$pycode" browser_download_url "$gamversion" | \
+	        sort --version-sort --reverse)
+if [[ ${download_urls:0:5} = "ERROR" ]]; then
+  echo_red "${download_urls}"
+  exit
+fi
+
+case $gamos in
+  [lL]inux)
+    gamos="linux"
+    download_urls=$(echo -e "$download_urls" | grep "\-linux-")
+    if [ "$osversion" == "" ]; then
+      this_glibc_ver=$(ldd --version | awk '/ldd/{print $NF}')
+    else
+      this_glibc_ver=$osversion
+    fi
+    echo "This Linux distribution uses glibc $this_glibc_ver"
+    case $gamarch in
+      x86_64)
+	download_urls=$(echo -e "$download_urls" | grep "\-x86_64-")
+	gam_x86_64_glibc_vers=$(echo -e "$download_urls" | \
+		grep --only-matching 'glibc[0-9\.]*\.tar\.xz$' \
+	        | cut -c 6-9 )
+	useglibc="legacy"
+        for gam_glibc_ver in $gam_x86_64_glibc_vers; do
+	  if version_gt $this_glibc_ver $gam_glibc_ver; then
+            useglibc="glibc$gam_glibc_ver"
+            echo_green "Using GAM compiled against $useglibc"
+            break
+          fi
+        done
+        download_url=$(echo -e "$download_urls" | grep "$useglibc")
+	;;
+      arm|arm64|aarch64)
+        download_urls=$(echo -e "$download_urls" | grep "\-aarch64-")
+        gam_arm64_glibc_vers=$(echo -e "$download_urls" | \
+                grep --only-matching 'glibc[0-9\.]*\.tar\.xz$' | \
+                cut -c 6-9)
+        useglibc="legacy"
+        for gam_glibc_ver in $gam_arm64_glibc_vers; do
+          if version_gt $this_glibc_ver $gam_glibc_ver; then
+            useglibc="glibc$gam_glibc_ver"
+            echo_green "Using GAM compiled against $useglibc"
+            break
+          fi
+        done
+        download_url=$(echo -e "$download_urls" | grep "$useglibc")
+	;;
+      *)
+        echo_red "ERROR: this installer currently only supports x86_64 and arm64 Linux. Looks like you're running on $gamarch. Exiting."
+        exit
+    esac
+    ;;
+  [Mm]ac[Oo][sS]|[Dd]arwin)
+    gamos="macos"
+    currentversion=$(sw_vers -productVersion | awk -F '.' '{print $1 "." $2}')
+    # override osversion only if it wasn't set by cli arguments
+    osversion=${osversion:-${currentversion}}
+    # override osversion only if it wasn't set by cli arguments
+    download_urls=$(echo -e "$download_urls" | grep "\-macos")
+    case $gamarch in
+      x86_64)
+        archgrep="\-x86_64"
+	;;
+      arm|arm64|aarch64)
+        archgrep="\-aarch64"
+        ;;
+      *)
+        echo_red "ERROR: this installer currently only supports x86_64 and arm64 MacOS. Looks like you're running on ${gamarch}. Exiting."
+        exit
+	;;
+    esac
+    gam_macos_urls=$(echo -e "$download_urls" | \
+                     grep "$archgrep")
+    versionless_urls=$(echo -e "$gam_macos_urls" | \
+                       grep "\-macos-")
+    if [ "$versionless_urls" == "" ]; then
+        # versions after 7.00.38 include MacOS version info
+        gam_macos_vers=$(echo -e "$gam_macos_urls" | \
+                         grep --only-matching '\-macos[0-9\.]*' | \
+                         cut -c 7-10)
+        for gam_mac_ver in $gam_macos_vers; do
+            if version_gt $currentversion $gam_mac_ver; then
+                download_url=$(echo -e "$gam_macos_urls" | grep "$gam_mac_ver")
+                echo_green "You are running MacOS ${currentversion} Using GAM compiled against ${gam_mac_ver}"
+                break
+            fi
+            done
+        if [ -z ${download_url+x} ]; then
+	    echo_red "Sorry, you are running MacOS ${osversion} but GAM on ${gamarch} requires MacOS ${gam_mac_ver} or newer. Exiting."
+            exit
+	fi
+    else
+        # versions 7.00.38 and older don't include version info
+        case $gamarch in
+            x86_64)
+	        minimum_version=13
+                download_url=$(echo -e "$download_urls" | grep "\-x86_64")
+                ;;
+            arm|arm64|aarch64)
+                download_url=$(echo -e "$download_urls" | grep "\-aarch64")
+	        minimum_version=14
+                ;;
+        esac
+        if version_gt "$osversion" "$minimum_version"; then
+            echo_green "You are running MacOS ${osversion}, good. Downloading GAM from ${download_url}."
+        else
+          echo_red "Sorry, you are running MacOS ${osversion} but GAM on ${gamarch} requires MacOS ${minimum_version}. Exiting."
+          exit
+        fi 
+        if [ -z ${download_url+x} ]; then
+          echo_red "Sorry, you are running MacOS ${currentversion} but GAM on ${gamarch} requires MacOS ${minimum_version}. Exiting."
+          exit
+        fi
+    fi
+    ;;
+  MINGW64_NT*)
+    gamos="windows"
+    echo "You are running Windows"
+    download_url=$(echo -e "$download_urls" | \
+                   grep "\-windows-" | \
+                   grep ".zip")
+    ;;
+  *)
+    echo_red "Sorry, this installer currently only supports Linux and MacOS. Looks like you're running on ${gamos}. Exiting."
+    exit
+    ;;
+esac
 
-browser_download_url=$(echo "$release_json" | $pycmd -c "$pycode" browser_download_url $gamversion)
-if [[ ${browser_download_url:0:5} = "ERROR" ]]; then
-  echo_red "${browser_download_url}"
-  exit
-fi
-name=$(echo "$release_json" | $pycmd -c "$pycode" name $gamversion)
-if [[ ${name:0:5} = "ERROR" ]]; then
-  echo_red "${name}"
-  exit
-fi
 # Temp dir for archive
-#temp_archive_dir=$(mktemp -d)
 temp_archive_dir=$(mktemp -d 2>/dev/null || mktemp -d -t 'mytmpdir')
 
 # Clean up after ourselves even if we are killed with CTRL-C
 trap "rm -rf $temp_archive_dir" EXIT
 
-echo_yellow "Downloading file $name from $browser_download_url to $temp_archive_dir."
+# hack to grab the end of the URL which should be the filename.
+name=$(echo -e "$download_url" | rev | cut -f1 -d "/" | rev)
+
+echo_yellow "Downloading ${download_url} to $temp_archive_dir ($check_type)..."
 # Save archive to temp w/o losing our path
-(cd $temp_archive_dir && curl -O -L $browser_download_url)
+(cd "$temp_archive_dir" && curl -O -L -s "${curl_opts[@]}" "$download_url")
 
 mkdir -p "$target_dir"
+echo_yellow "Deleting contents of $target_dir/gam7/lib"
+rm -frv "$target_dir/gam7/lib"
 
 echo_yellow "Extracting archive to $target_dir"
-tar xf $temp_archive_dir/$name -C "$target_dir"
+if [[ "$name" =~ tar.xz|tar.gz|tar ]]; then
+  tar $strip_gam -xf "$temp_archive_dir"/"$name" -C "$target_dir"
+elif [[ "$name" == *.zip ]]; then
+  unzip -o "${temp_archive_dir}/${name}" -d "${target_dir}"
+else
+  echo "I don't know what to do with files like ${name}. Giving up."
+  exit 1
+fi
 rc=$?
 if (( $rc != 0 )); then
   echo_red "ERROR: extracting the GAM archive with tar failed with error $rc. Exiting."
@@ -240,7 +345,7 @@ fi
 
 # Update profile to add gam command
 if [ "$update_profile" = true ]; then
-  alias_line="function gam() { \"$target_dir/gam/gam\" \"\$@\" ; }"
+  alias_line="alias gam=\"${target_dir// /\\ }/$target_gam\""
   if [ "$gamos" == "linux" ]; then
     update_profile "$HOME/.bash_aliases" 0 || update_profile "$HOME/.bash_profile" 0 || update_profile "$HOME/.bashrc" 0
     update_profile "$HOME/.zshrc" 0
@@ -254,10 +359,10 @@ fi
 
 if [ "$upgrade_only" = true ]; then
   echo_green "Here's information about your GAM upgrade:"
-  "$target_dir/gam/gam" version extended
+  "$target_dir/$target_gam" version extended
   rc=$?
   if (( $rc != 0 )); then
-    echo_red "ERROR: Failed running GAM for the first time with $rc. Please report this error to GAM mailing list. Exiting."
+    echo_red "ERROR: Failed running GAM for the first time with return code $rc. Please report this error to GAM mailing list. Exiting."
     exit
   fi
 
@@ -265,6 +370,9 @@ if [ "$upgrade_only" = true ]; then
   exit
 fi
 
+# Set config command
+#config_cmd="config no_browser false"
+
 while true; do
   read -p "Can you run a full browser on this machine? (usually Y for MacOS, N for Linux if you SSH into this machine) " yn
   case $yn in
@@ -272,7 +380,8 @@ while true; do
       break
       ;;
     [Nn]*)
-      touch "$target_dir/gam/nobrowser.txt" > /dev/null 2>&1
+#      config_cmd="config no_browser true"
+      touch "$target_dir/gam7/nobrowser.txt" > /dev/null 2>&1
       break
       ;;
     *)
@@ -288,9 +397,10 @@ while true; do
   case $yn in
     [Yy]*)
       if [ "$adminuser" == "" ]; then
-        read -p "Please enter your G Suite admin email address: " adminuser
+        read -p "Please enter your Google Workspace admin email address: " adminuser
       fi
-      "$target_dir/gam/gam" create project $adminuser
+#      "$target_dir/$target_gam" $config_cmd create project $adminuser
+      "$target_dir/$target_gam" create project $adminuser
       rc=$?
       if (( $rc == 0 )); then
         echo_green "Project creation complete."
@@ -312,10 +422,11 @@ done
 
 admin_authorized=false
 while $project_created; do
-  read -p "Are you ready to authorize GAM to perform G Suite management operations as your admin account? (yes or no) " yn
+  read -p "Are you ready to authorize GAM to perform Google Workspace management operations as your admin account? (yes or no) " yn
   case $yn in
     [Yy]*)
-      "$target_dir/gam/gam" oauth create $adminuser
+#      "$target_dir/$target_gam" $config_cmd oauth create $adminuser
+      "$target_dir/$target_gam" oauth create $adminuser
       rc=$?
       if (( $rc == 0 )); then
         echo_green "Admin authorization complete."
@@ -336,15 +447,16 @@ while $project_created; do
 done
 
 service_account_authorized=false
-while $project_created; do
-  read -p "Are you ready to authorize GAM to manage G Suite user data and settings? (yes or no) " yn
+while $admin_authorized; do
+  read -p "Are you ready to authorize GAM to manage Google Workspace user data and settings? (yes or no) " yn
   case $yn in
     [Yy]*)
       if [ "$regularuser" == "" ]; then
-        read -p "Please enter the email address of a regular G Suite user: " regularuser
+        read -p "Please enter the email address of a regular Google Workspace user: " regularuser
       fi
       echo_yellow "Great! Checking service account scopes.This will fail the first time. Follow the steps to authorize and retry. It can take a few minutes for scopes to PASS after they've been authorized in the admin console."
-      "$target_dir/gam/gam" user $adminuser check serviceaccount
+#      "$target_dir/$target_gam" $config_cmd user $regularuser check serviceaccount
+      "$target_dir/$target_gam" user $regularuser check serviceaccount
       rc=$?
       if (( $rc == 0 )); then
         echo_green "Service account authorization complete."
@@ -355,7 +467,7 @@ while $project_created; do
       fi
       ;;
      [Nn]*)
-       echo -e "\nYou can authorize a service account later by running:\n\ngam check serviceaccount\n"
+       echo -e "\nYou can authorize a service account later by running:\n\ngam user $adminuser check serviceaccount\n"
        break
        ;;
      *)
@@ -365,7 +477,8 @@ while $project_created; do
 done
 
 echo_green "Here's information about your new GAM installation:"
-"$target_dir/gam/gam" version extended
+#"$target_dir/$target_gam" $config_cmd save version extended
+"$target_dir/$target_gam" version extended
 rc=$?
 if (( $rc != 0 )); then
   echo_red "ERROR: Failed running GAM for the first time with $rc. Please report this error to GAM mailing list. Exiting."

src/gam-setup.bat

@@ -1,83 +1,84 @@
 :neworupgrade
-@echo(
-@set /p nu= "Is this a new install or an upgrade? [n or u] "
+@echo.
+@set /p nu= "If you have installed any version of GAM on any computer for your domain, enter u to upgrade, otherwise enter n? [u or n] "
 @if /I "%nu%"=="u" (
 @  echo GAM installation and setup complete!
 @  goto alldone
    )
 @if /I not "%nu%"=="n" (
-@  echo(
+@  echo.
 @  echo Please answer n or u.
 @  goto neworupgrade
    )
 
 :createproject
-@echo(
+@echo.
 @set /p yn= "Are you ready to set up a Google API project for GAM? [y or n] "
 @if /I "%yn%"=="n" (
-@  echo(
+@  echo.
 @  echo You can create an API project later by running:
-@  echo(
+@  echo.
 @  echo gam create project
 @  goto alldone
    )
 @if /I not "%yn%"=="y" (
-@  echo(
+@  echo.
 @  echo Please answer y or n.
 @  goto createproject
    )
-@echo(
-@set /p adminemail= "Please enter your G Suite admin email address: "
+
+@set /p adminemail= "Please enter your Google Workspace admin email address: "
+
 @gam create project %adminemail%
 @if not ERRORLEVEL 1 goto projectdone
-@echo(
+@echo.
 @echo Project creation failed. Trying again. Say n to skip project creation.
 @goto createproject
 :projectdone
 
 :adminauth
-@echo(
-@set /p yn= "Are you ready to authorize GAM to perform G Suite management operations as your admin account? [y or n] "
+@echo.
+@set /p yn= "Are you ready to authorize GAM to perform Google Workspace management operations as your admin account? [y or n] "
 @if /I "%yn%"=="n" (
-@  echo(
+@  echo.
 @  echo You can authorize an admin later by running:
-@  echo(
+@  echo.
 @  echo gam oauth create %adminemail%
 @  goto admindone
    )
 @if /I not "%yn%"=="y" (
-@  echo(
+@  echo.
 @  echo Please answer y or n.
 @  goto adminauth
    )
 @gam oauth create %adminemail%
 @if not ERRORLEVEL 1 goto admindone
-@echo(
+@echo.
 @echo Admin authorization failed. Trying again. Say n to skip admin authorization.
 @goto adminauth
 :admindone
 
 :saauth
-@echo(
-@set /p yn= "Are you ready to authorize GAM to manage G Suite user data and settings? [y or n] "
+@echo.
+@set /p yn= "Are you ready to authorize GAM to manage Google Workspace user data and settings? [y or n] "
 @if /I "%yn%"=="n" (
-@  echo(
+@  echo.
 @  echo You can authorize a service account later by running:
-@  echo(
+@  echo.
 @  echo gam user %adminemail% check serviceaccount
 @  goto sadone
    )
 @if /I not "%yn%"=="y" (
-@  echo(
+@  echo.
 @  echo Please answer y or n.
 @  goto saauth
    )
-@echo(
-@set /p regularuser= "Please enter the email address of a regular G Suite user: "
+@echo.
+@set /p regularuser= "Please enter the email address of a regular Google Workspace user: "
 @echo Great! Checking service account scopes. This will fail the first time. Follow the steps to authorize and retry. It can take a few minutes for scopes to PASS after they've been authorized in the admin console.
 @gam user %regularuser% check serviceaccount
 @if not ERRORLEVEL 1 goto sadone
-@echo(
+@echo.
 @echo Service account authorization failed. Confirm you entered the scopes correctly in the admin console. It can take a few minutes for scopes to PASS after they are entered in the admin console so if you're sure you entered them correctly, go grab a coffee and then hit Y to try again. Say N to skip admin authorization.
 @goto saauth
 :sadone

src/gam.exe.manifest

@@ -0,0 +1,11 @@
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0"> 
+<compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1"> 
+    <application> 
+        <!-- Windows 8.1 / Server 2012 R2 -->
+        <supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}" />
+        <!-- Windows 10+ / Server 2016+ -->
+        <supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}" />
+    </application> 
+</compatibility>
+</assembly>

src/gam.py

@@ -2,10 +2,14 @@
 # -*- coding: utf-8 -*-
 """Provides backwards compatibility for calling gam as a single .py file"""
 
-import sys
+import multiprocessing
+import platform
 
 from gam.__main__ import main
 
 # Run from command line
 if __name__ == '__main__':
-    main(sys.argv)
+  if platform.system() != 'Linux':
+    multiprocessing.freeze_support()
+    multiprocessing.set_start_method('spawn')
+  main()

src/gam.spec

@@ -1,39 +1,151 @@
-# -*- mode: python -*-
+# -*- mode: python ; coding: utf-8 -*-
+from os import getenv
+import re
+from sys import platform
 
-import sys
-
-import importlib
 from PyInstaller.utils.hooks import copy_metadata
 
-sys.modules['FixTk'] = None
+from gam.gamlib.glverlibs import GAM_VER_LIBS
 
-# dynamically determine where httplib2/cacerts.txt lives
-proot = os.path.dirname(importlib.import_module('httplib2').__file__)
-extra_files = [(os.path.join(proot, 'cacerts.txt'), 'httplib2')]
 
-extra_files += copy_metadata('google-api-python-client')
+with open("gam/__init__.py") as f:
+    version_file = f.read()
+version = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]", version_file, re.M).group(1)
+version_list = [int(i) for i in version.split('.')]
+while len(version_list) < 4:
+  version_list.append(0)
+version_tuple = tuple(version_list)
+version_str = str(version_tuple)
+with open("version_info.txt.in") as f:
+    version_info = f.read()
+version_info = version_info.replace("{VERSION}", version).replace(
+    "{VERSION_TUPLE}", version_str
+)
+with open("version_info.txt", "w") as f:
+    f.write(version_info)
+print(version_info)
 
-a = Analysis(['gam/__main__.py'],
-             hiddenimports=[],
-             hookspath=None,
-             excludes=['FixTk', 'tcl', 'tk', '_tkinter', 'tkinter', 'Tkinter'],
-             datas=extra_files,
-             runtime_hooks=None)
+datas = []
+for pkg in GAM_VER_LIBS:
+    datas += copy_metadata(pkg, recursive=True)
+datas += [('gam/cbcm-v1.1beta1.json', '.')]
+datas += [('gam/contactdelegation-v1.json', '.')]
+datas += [('gam/datastudio-v1.json', '.')]
+datas += [('gam/serviceaccountlookup-v1.json', '.')]
+datas += [('cacerts.pem', '.')]
+hiddenimports = [
+     'gam.gamlib.yubikey',
+     ]
 
+runtime_hooks = []
+a = Analysis(
+    ['gam/__main__.py'],
+    pathex=[],
+    binaries=[],
+    datas=datas,
+    hiddenimports=hiddenimports,
+    hookspath=[],
+    hooksconfig={},
+    runtime_hooks=runtime_hooks,
+    excludes=[],
+    win_no_prefer_redirects=False,
+    win_private_assemblies=False,
+    cipher=None,
+    noarchive=False,
+    )
+#print(f"datas from analysis:\n{a.datas}")
 for d in a.datas:
     if 'pyconfig' in d[0]:
         a.datas.remove(d)
         break
+#print(f"datas after pyconfig cleanup:\n{a.datas}")
+pyz = PYZ(a.pure,
+          a.zipped_data,
+          cipher=None)
+# requires Python 3.10+ but no one should be compiling
+# GAM with older versions anyway
+target_arch = None
+codesign_identity = None
+entitlements_file = None
+manifest = None
+version = 'version_info.txt'
+match platform:
+    case "darwin":
+        if getenv('arch') == 'universal2':
+            target_arch = "universal2"
 
+        codesign_identity = getenv('codesign_identity')
+        if codesign_identity:
+            entitlements_file = '../.github/actions/entitlements.plist'
+        strip = True
+    case "win32":
+        target_arch = None
+        strip = False
+        manifest = 'gam.exe.manifest'
+    case _:
+        target_arch = None
+        strip = True
+name = 'gam'
+debug = False
+bootloader_ignore_signals = False
+upx = False
+console = True
+disable_windowed_traceback = False
+argv_emulation = False
+if getenv('PYINSTALLER_BUILD_ONEDIR') == 'yes':
+    # Build one directory
+    exe = EXE(
+              pyz,
+              a.scripts,
+              [],
+              exclude_binaries=True,
+              name=name,
+              debug=debug,
+              bootloader_ignore_signals=bootloader_ignore_signals,
+              strip=strip,
+              manifest=manifest,
+              upx=upx,
+              console=console,
+              # put most everyting under a lib/ subfolder
+              contents_directory='lib',
+              disable_windowed_traceback=disable_windowed_traceback,
+              argv_emulation=argv_emulation,
+              target_arch=target_arch,
+              codesign_identity=codesign_identity,
+              entitlements_file=entitlements_file,
+              version=version,
+              )
+    coll = COLLECT(
+        exe,
+        a.binaries,
+        a.zipfiles,
+        a.datas,
+        strip=strip,
+        upx=upx,
+        upx_exclude=[],
+        name=name,
+        )
+else:
+    # Build one file
+    exe = EXE(
+              pyz,
+              a.scripts,
+              a.binaries,
+              a.zipfiles,
+              a.datas,
+              [],
+              name=name,
+              debug=debug,
+              bootloader_ignore_signals=bootloader_ignore_signals,
+              manifest=manifest,
+              strip=strip,
+              upx=upx,
+              console=console,
+              disable_windowed_traceback=disable_windowed_traceback,
+              argv_emulation=argv_emulation,
+              target_arch=target_arch,
+              codesign_identity=codesign_identity,
+              entitlements_file=entitlements_file,
+              version=version,
+              )
 
-pyz = PYZ(a.pure)
-exe = EXE(pyz,
-          a.scripts,
-          a.binaries,
-          a.zipfiles,
-          a.datas,
-          name='gam',
-          debug=False,
-          strip=None,
-          upx=False,
-          console=True )

src/gam.wxs

@@ -2,11 +2,11 @@
 <Wix xmlns="http://schemas.microsoft.com/wix/2006/wi" >
   <Product
       Id="*"
-      Name="GAM"
+      Name="GAM7"
       Language="1033"
       Version="$(env.GAMVERSION)"
-      Manufacturer="Jay Lee - jay0lee@gmail.com"
-      UpgradeCode="15C5FD61-B04C-4E04-A26D-CD8424C19D9F">
+      Manufacturer="GAM Team - google-apps-manager@googlegroups.com"
+      UpgradeCode="D86B52B2-EFE9-4F9D-8BA3-9D84B9B2D319">
     <Package
         InstallerVersion="200" Compressed="yes" InstallScope="perMachine" />
 
@@ -22,17 +22,21 @@
 
     <Feature
         Id="gam"
-        Title="GAM"
+        Title="GAM7"
         Level="1">
       <ComponentGroupRef Id="ProductComponents" />
     </Feature>
   </Product>
 
   <Fragment>
+    <SetDirectory Id="WINDOWSVOLUME" Value="[WindowsVolume]"/>
     <Directory Id="TARGETDIR" Name="SourceDir">
-      <Directory Id="ROOTDRIVE">
-          <Directory Id="INSTALLFOLDER" Name="GAM" />
-        </Directory>
+      <Directory Id="WINDOWSVOLUME">
+        <Directory Id="INSTALLFOLDER" Name="GAM7">
+          <Directory Id="lib" Name="lib">
+	  </Directory>
+	</Directory>
+      </Directory>
     </Directory>
   </Fragment>
 
@@ -41,20 +45,27 @@
     <ComponentGroup
         Id="ProductComponents"
         Directory="INSTALLFOLDER"
-	Source="dist/gam">
-      <Component Id="gam_exe" Guid="886abc07-73c5-4acc-9f71-58daf62aabc1">
+	Source="dist/gam/gam7">
+      <Component Id="gam_exe" Guid="d046ea24-c9f8-40ca-84db-70b0119933ff">
         <File Name="gam.exe" KeyPath="yes" />
         <Environment Id="PATH" Name="PATH" Value="[INSTALLFOLDER]" Permanent="yes" Part="last" Action="set" System="yes" />
       </Component>
-      <Component Id="license" Guid="7a15de2e-fb91-4d0a-b8bf-c8b19c68f569">
+      <Component Id="license" Guid="c76864c5-d005-44d5-bb7c-a27e5923792d">
         <File Name="LICENSE" KeyPath="yes" />
       </Component>
-      <Component Id="gam_setup_bat" Guid="ef01f93a-4b50-488a-9c04-ec5e13e66218">
+      <Component Id="gam_setup_bat" Guid="5e6bbacb-d86f-4d80-a10b-89b81ee63fcb">
         <File Name="gam-setup.bat" KeyPath="yes" />
       </Component>
-      <Component Id="gamcommands_txt" Guid="58ff9c45-a7c9-4e22-8845-a9a92610c1f3">
-        <File Name="gamcommands.txt" KeyPath="yes" />
+      <Component Id="GamCommands_txt" Guid="a2dca862-b222-469e-a637-95ea2a1c53e7">
+        <File Name="GamCommands.txt" KeyPath="yes" />
       </Component>
+      <Component Id="GamUpdate_txt" Guid="1b7cdd48-0fff-4943-a219-102fcd14c755">
+        <File Name="GamUpdate.txt" KeyPath="yes" />
+      </Component>
+      <Component Id="cacerts_pem" Guid="61fe2b2d-1646-4bed-b844-193965e97727">
+        <File Name="cacerts.pem" KeyPath="yes" />
+      </Component>
+      <ComponentGroupRef Id="Lib" />
     </ComponentGroup>
   </Fragment>
 
@@ -63,9 +74,5 @@
       <ExecuteAction />
       <Show Dialog="WelcomeDlg" Before="ProgressDlg" />
     </InstallUISequence>
-    <CustomAction Id="setup_gam" ExeCommand="[INSTALLFOLDER]gam-setup.bat" Directory="INSTALLFOLDER" Execute="commit" Impersonate="yes" Return="asyncWait"/>
-    <InstallExecuteSequence>
-     <Custom Action="setup_gam" After="InstallFiles" >NOT Installed AND NOT UPGRADINGPRODUCTCODE AND NOT WIX_UPGRADE_DETECTED</Custom>
-    </InstallExecuteSequence>
   </Fragment>
 </Wix>

src/gam/__main__.py

@@ -3,7 +3,7 @@
 #
 # GAM
 #
-# Copyright 2019, LLC All Rights Reserved.
+# Copyright 2023, All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -16,35 +16,25 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-"""GAM is a command line tool which allows Administrators to control their G Suite domain and accounts.
-
-With GAM you can programmatically create users, turn on/off services for users like POP and Forwarding and much more.
-For more information, see https://git.io/gam
-"""
 
+import multiprocessing
+import platform
 import sys
-from multiprocessing import freeze_support
-from multiprocessing import set_start_method
 
-from gam import controlflow
 import gam
 
-
-def main(argv):
-    freeze_support()
-    if sys.platform == 'darwin':
-        # https://bugs.python.org/issue33725 in Python 3.8.0 seems
-        # to break parallel operations with errors about extra -b
-        # command line arguments
-        set_start_method('fork')
-    if sys.version_info[0] < 3 or sys.version_info[1] < 6:
-        controlflow.system_error_exit(
-            5,
-            f'GAM requires Python 3.6 or newer. You are running %s.%s.%s. Please upgrade your Python version or use one of the binary GAM downloads.'
-            % sys.version_info[:3])
-    sys.exit(gam.ProcessGAMCommand(sys.argv))
-
+def main():
+  gam.initializeLogging()
+  rc = gam.ProcessGAMCommand(sys.argv)
+  try:
+    sys.stdout.flush()
+  except (IOError, ValueError):
+    pass
+  sys.exit(rc)
 
 # Run from command line
 if __name__ == '__main__':
-    main(sys.argv)
+  if platform.system() != 'Linux':
+    multiprocessing.freeze_support()
+    multiprocessing.set_start_method('spawn')
+  main()

src/gam/atom/auth.py

@@ -0,0 +1,41 @@
+#!/usr/bin/env python
+#
+#    Copyright (C) 2009 Google Inc.
+#
+#   Licensed under the Apache License 2.0;
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+
+
+# This module is used for version 2 of the Google Data APIs.
+
+
+# __author__ = 'j.s@google.com (Jeff Scudder)'
+
+import base64
+
+
+class BasicAuth(object):
+    """Sets the Authorization header as defined in RFC1945"""
+
+    def __init__(self, user_id, password):
+        self.basic_cookie = base64.encodestring(
+            '%s:%s' % (user_id, password)).strip()
+
+    def modify_request(self, http_request):
+        http_request.headers['Authorization'] = 'Basic %s' % self.basic_cookie
+
+    ModifyRequest = modify_request
+
+
+class NoAuth(object):
+    def modify_request(self, http_request):
+        pass

src/gam/atom/client.py

@@ -0,0 +1,214 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2009 Google Inc.
+#
+# Licensed under the Apache License 2.0;
+
+
+
+"""AtomPubClient provides CRUD ops. in line with the Atom Publishing Protocol.
+
+"""
+
+# __author__ = 'j.s@google.com (Jeff Scudder)'
+
+import atom.http_core
+
+
+class Error(Exception):
+    pass
+
+
+class MissingHost(Error):
+    pass
+
+
+class AtomPubClient(object):
+    host = None
+    auth_token = None
+    ssl = False  # Whether to force all requests over https
+    xoauth_requestor_id = None
+
+    def __init__(self, http_client=None, host=None, auth_token=None, source=None,
+                 xoauth_requestor_id=None, **kwargs):
+        """Creates a new AtomPubClient instance.
+
+        Args:
+          source: The name of your application.
+          http_client: An object capable of performing HTTP requests through a
+                       request method. This object is used to perform the request
+                       when the AtomPubClient's request method is called. Used to
+                       allow HTTP requests to be directed to a mock server, or use
+                       an alternate library instead of the default of httplib to
+                       make HTTP requests.
+          host: str The default host name to use if a host is not specified in the
+                requested URI.
+          auth_token: An object which sets the HTTP Authorization header when its
+                      modify_request method is called.
+        """
+        self.http_client = http_client or atom.http_core.ProxiedHttpClient()
+        if host is not None:
+            self.host = host
+        if auth_token is not None:
+            self.auth_token = auth_token
+        self.xoauth_requestor_id = xoauth_requestor_id
+        self.source = source
+
+    def request(self, method=None, uri=None, auth_token=None,
+                http_request=None, **kwargs):
+        """Performs an HTTP request to the server indicated.
+
+        Uses the http_client instance to make the request.
+
+        Args:
+          method: The HTTP method as a string, usually one of 'GET', 'POST',
+                  'PUT', or 'DELETE'
+          uri: The URI desired as a string or atom.http_core.Uri.
+          http_request:
+          auth_token: An authorization token object whose modify_request method
+                      sets the HTTP Authorization header.
+
+        Returns:
+          The results of calling self.http_client.request. With the default
+          http_client, this is an HTTP response object.
+        """
+        # Modify the request based on the AtomPubClient settings and parameters
+        # passed in to the request.
+        http_request = self.modify_request(http_request)
+        if isinstance(uri, str):
+            uri = atom.http_core.Uri.parse_uri(uri)
+        if uri is not None:
+            uri.modify_request(http_request)
+        if isinstance(method, str):
+            http_request.method = method
+        # Any unrecognized arguments are assumed to be capable of modifying the
+        # HTTP request.
+        for name, value in kwargs.items():
+            if value is not None:
+                if hasattr(value, 'modify_request'):
+                    value.modify_request(http_request)
+                else:
+                    http_request.uri.query[name] = str(value)
+        # Default to an http request if the protocol scheme is not set.
+        if http_request.uri.scheme is None:
+            http_request.uri.scheme = 'http'
+        # Override scheme. Force requests over https.
+        if self.ssl:
+            http_request.uri.scheme = 'https'
+        if http_request.uri.path is None:
+            http_request.uri.path = '/'
+        # Add the Authorization header at the very end. The Authorization header
+        # value may need to be calculated using information in the request.
+        if auth_token:
+            auth_token.modify_request(http_request)
+        elif self.auth_token:
+            self.auth_token.modify_request(http_request)
+        # Check to make sure there is a host in the http_request.
+        if http_request.uri.host is None:
+            raise MissingHost('No host provided in request %s %s' % (
+                http_request.method, str(http_request.uri)))
+        # Perform the fully specified request using the http_client instance.
+        # Sends the request to the server and returns the server's response.
+        return self.http_client.request(http_request)
+
+    Request = request
+
+    def get(self, uri=None, auth_token=None, http_request=None, **kwargs):
+        """Performs a request using the GET method, returns an HTTP response."""
+        return self.request(method='GET', uri=uri, auth_token=auth_token,
+                            http_request=http_request, **kwargs)
+
+    Get = get
+
+    def post(self, uri=None, data=None, auth_token=None, http_request=None,
+             **kwargs):
+        """Sends data using the POST method, returns an HTTP response."""
+        return self.request(method='POST', uri=uri, auth_token=auth_token,
+                            http_request=http_request, data=data, **kwargs)
+
+    Post = post
+
+    def put(self, uri=None, data=None, auth_token=None, http_request=None,
+            **kwargs):
+        """Sends data using the PUT method, returns an HTTP response."""
+        return self.request(method='PUT', uri=uri, auth_token=auth_token,
+                            http_request=http_request, data=data, **kwargs)
+
+    Put = put
+
+    def delete(self, uri=None, auth_token=None, http_request=None, **kwargs):
+        """Performs a request using the DELETE method, returns an HTTP response."""
+        return self.request(method='DELETE', uri=uri, auth_token=auth_token,
+                            http_request=http_request, **kwargs)
+
+    Delete = delete
+
+    def modify_request(self, http_request):
+        """Changes the HTTP request before sending it to the server.
+
+        Sets the User-Agent HTTP header and fills in the HTTP host portion
+        of the URL if one was not included in the request (for this it uses
+        the self.host member if one is set). This method is called in
+        self.request.
+
+        Args:
+          http_request: An atom.http_core.HttpRequest() (optional) If one is
+                        not provided, a new HttpRequest is instantiated.
+
+        Returns:
+          An atom.http_core.HttpRequest() with the User-Agent header set and
+          if this client has a value in its host member, the host in the request
+          URL is set.
+        """
+        if http_request is None:
+            http_request = atom.http_core.HttpRequest()
+
+        if self.host is not None and http_request.uri.host is None:
+            http_request.uri.host = self.host
+
+        if self.xoauth_requestor_id is not None:
+            http_request.uri.query['xoauth_requestor_id'] = self.xoauth_requestor_id
+
+        # Set the user agent header for logging purposes.
+        if self.source:
+            http_request.headers['User-Agent'] = '%s gdata-py/2.0.18' % self.source
+        else:
+            http_request.headers['User-Agent'] = 'gdata-py/2.0.17'
+
+        return http_request
+
+    ModifyRequest = modify_request
+
+
+class CustomHeaders(object):
+    """Add custom headers to an http_request.
+
+    Usage:
+      >>> custom_headers = atom.client.CustomHeaders(header1='value1',
+              header2='value2')
+      >>> client.get(uri, custom_headers=custom_headers)
+    """
+
+    def __init__(self, **kwargs):
+        """Creates a CustomHeaders instance.
+
+        Initialize the headers dictionary with the arguments list.
+        """
+        self.headers = kwargs
+
+    def modify_request(self, http_request):
+        """Changes the HTTP request before sending it to the server.
+
+        Adds the custom headers to the HTTP request.
+
+        Args:
+          http_request: An atom.http_core.HttpRequest().
+
+        Returns:
+          An atom.http_core.HttpRequest() with the added custom headers.
+        """
+
+        for name, value in self.headers.items():
+            if value is not None:
+                http_request.headers[name] = value
+        return http_request

src/gam/atom/core.py

@@ -0,0 +1,535 @@
+#!/usr/bin/env python
+#
+#    Copyright (C) 2008 Google Inc.
+#
+#   Licensed under the Apache License 2.0;
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+
+
+# This module is used for version 2 of the Google Data APIs.
+
+
+# __author__ = 'j.s@google.com (Jeff Scudder)'
+
+import inspect
+
+import lxml.etree as ElementTree
+
+try:
+    from xml.dom.minidom import parseString as xmlString
+except ImportError:
+    xmlString = None
+
+STRING_ENCODING = 'utf-8'
+
+
+class XmlElement(object):
+    """Represents an element node in an XML document.
+
+    The text member is a UTF-8 encoded str or unicode.
+    """
+    _qname = None
+    _other_elements = None
+    _other_attributes = None
+    # The rule set contains mappings for XML qnames to child members and the
+    # appropriate member classes.
+    _rule_set = None
+    _members = None
+    text = None
+
+    def __init__(self, text=None, *args, **kwargs):
+        if ('_members' not in self.__class__.__dict__
+            or self.__class__._members is None):
+            self.__class__._members = tuple(self.__class__._list_xml_members())
+        for member_name, member_type in self.__class__._members:
+            if member_name in kwargs:
+                setattr(self, member_name, kwargs[member_name])
+            else:
+                if isinstance(member_type, list):
+                    setattr(self, member_name, [])
+                else:
+                    setattr(self, member_name, None)
+        self._other_elements = []
+        self._other_attributes = {}
+        if text is not None:
+            self.text = text
+
+    def _list_xml_members(cls):
+        """Generator listing all members which are XML elements or attributes.
+
+        The following members would be considered XML members:
+        foo = 'abc' - indicates an XML attribute with the qname abc
+        foo = SomeElement - indicates an XML child element
+        foo = [AnElement] - indicates a repeating XML child element, each instance
+            will be stored in a list in this member
+        foo = ('att1', '{http://example.com/namespace}att2') - indicates an XML
+            attribute which has different parsing rules in different versions of
+            the protocol. Version 1 of the XML parsing rules will look for an
+            attribute with the qname 'att1' but verion 2 of the parsing rules will
+            look for a namespaced attribute with the local name of 'att2' and an
+            XML namespace of 'http://example.com/namespace'.
+        """
+        members = []
+        for pair in inspect.getmembers(cls):
+            if not pair[0].startswith('_') and pair[0] != 'text':
+                member_type = pair[1]
+                if (isinstance(member_type, tuple) or isinstance(member_type, list)
+                    or isinstance(member_type, str)
+                    or (inspect.isclass(member_type)
+                        and issubclass(member_type, XmlElement))):
+                    members.append(pair)
+        return members
+
+    _list_xml_members = classmethod(_list_xml_members)
+
+    def _get_rules(cls, version):
+        """Initializes the _rule_set for the class which is used when parsing XML.
+
+        This method is used internally for parsing and generating XML for an
+        XmlElement. It is not recommended that you call this method directly.
+
+        Returns:
+          A tuple containing the XML parsing rules for the appropriate version.
+
+          The tuple looks like:
+          (qname, {sub_element_qname: (member_name, member_class, repeating), ..},
+           {attribute_qname: member_name})
+
+          To give a couple of concrete example, the atom.data.Control _get_rules
+          with version of 2 will return:
+          ('{http://www.w3.org/2007/app}control',
+           {'{http://www.w3.org/2007/app}draft': ('draft',
+                                                  <class 'atom.data.Draft'>,
+                                                  False)},
+           {})
+          Calling _get_rules with version 1 on gdata.data.FeedLink will produce:
+          ('{http://schemas.google.com/g/2005}feedLink',
+           {'{http://www.w3.org/2005/Atom}feed': ('feed',
+                                                  <class 'gdata.data.GDFeed'>,
+                                                  False)},
+           {'href': 'href', 'readOnly': 'read_only', 'countHint': 'count_hint',
+            'rel': 'rel'})
+        """
+        # Initialize the _rule_set to make sure there is a slot available to store
+        # the parsing rules for this version of the XML schema.
+        # Look for rule set in the class __dict__ proxy so that only the
+        # _rule_set for this class will be found. By using the dict proxy
+        # we avoid finding rule_sets defined in superclasses.
+        # The four lines below provide support for any number of versions, but it
+        # runs a bit slower then hard coding slots for two versions, so I'm using
+        # the below two lines.
+        # if '_rule_set' not in cls.__dict__ or cls._rule_set is None:
+        #  cls._rule_set = []
+        # while len(cls.__dict__['_rule_set']) < version:
+        #  cls._rule_set.append(None)
+        # If there is no rule set cache in the class, provide slots for two XML
+        # versions. If and when there is a version 3, this list will need to be
+        # expanded.
+        if '_rule_set' not in cls.__dict__ or cls._rule_set is None:
+            cls._rule_set = [None, None]
+        # If a version higher than 2 is requested, fall back to version 2 because
+        # 2 is currently the highest supported version.
+        if version > 2:
+            return cls._get_rules(2)
+        # Check the dict proxy for the rule set to avoid finding any rule sets
+        # which belong to the superclass. We only want rule sets for this class.
+        if cls._rule_set[version - 1] is None:
+            # The rule set for each version consists of the qname for this element
+            # ('{namespace}tag'), a dictionary (elements) for looking up the
+            # corresponding class member when given a child element's qname, and a
+            # dictionary (attributes) for looking up the corresponding class member
+            # when given an XML attribute's qname.
+            elements = {}
+            attributes = {}
+            if ('_members' not in cls.__dict__ or cls._members is None):
+                cls._members = tuple(cls._list_xml_members())
+            for member_name, target in cls._members:
+                if isinstance(target, list):
+                    # This member points to a repeating element.
+                    elements[_get_qname(target[0], version)] = (member_name, target[0],
+                                                                True)
+                elif isinstance(target, tuple):
+                    # This member points to a versioned XML attribute.
+                    if version <= len(target):
+                        attributes[target[version - 1]] = member_name
+                    else:
+                        attributes[target[-1]] = member_name
+                elif isinstance(target, str):
+                    # This member points to an XML attribute.
+                    attributes[target] = member_name
+                elif issubclass(target, XmlElement):
+                    # This member points to a single occurance element.
+                    elements[_get_qname(target, version)] = (member_name, target, False)
+            version_rules = (_get_qname(cls, version), elements, attributes)
+            cls._rule_set[version - 1] = version_rules
+            return version_rules
+        else:
+            return cls._rule_set[version - 1]
+
+    _get_rules = classmethod(_get_rules)
+
+    def get_elements(self, tag=None, namespace=None, version=1):
+        """Find all sub elements which match the tag and namespace.
+
+        To find all elements in this object, call get_elements with the tag and
+        namespace both set to None (the default). This method searches through
+        the object's members and the elements stored in _other_elements which
+        did not match any of the XML parsing rules for this class.
+
+        Args:
+          tag: str
+          namespace: str
+          version: int Specifies the version of the XML rules to be used when
+                   searching for matching elements.
+
+        Returns:
+          A list of the matching XmlElements.
+        """
+        matches = []
+        ignored1, elements, ignored2 = self.__class__._get_rules(version)
+        if elements:
+            for qname, element_def in elements.items():
+                member = getattr(self, element_def[0])
+                if member:
+                    if _qname_matches(tag, namespace, qname):
+                        if element_def[2]:
+                            # If this is a repeating element, copy all instances into the
+                            # result list.
+                            matches.extend(member)
+                        else:
+                            matches.append(member)
+        for element in self._other_elements:
+            if _qname_matches(tag, namespace, element._qname):
+                matches.append(element)
+        return matches
+
+    GetElements = get_elements
+    # FindExtensions and FindChildren are provided for backwards compatibility
+    # to the atom.AtomBase class.
+    # However, FindExtensions may return more results than the v1 atom.AtomBase
+    # method does, because get_elements searches both the expected children
+    # and the unexpected "other elements". The old AtomBase.FindExtensions
+    # method searched only "other elements" AKA extension_elements.
+    FindExtensions = get_elements
+    FindChildren = get_elements
+
+    def get_attributes(self, tag=None, namespace=None, version=1):
+        """Find all attributes which match the tag and namespace.
+
+        To find all attributes in this object, call get_attributes with the tag
+        and namespace both set to None (the default). This method searches
+        through the object's members and the attributes stored in
+        _other_attributes which did not fit any of the XML parsing rules for this
+        class.
+
+        Args:
+          tag: str
+          namespace: str
+          version: int Specifies the version of the XML rules to be used when
+                   searching for matching attributes.
+
+        Returns:
+          A list of XmlAttribute objects for the matching attributes.
+        """
+        matches = []
+        ignored1, ignored2, attributes = self.__class__._get_rules(version)
+        if attributes:
+            for qname, attribute_def in attributes.items():
+                if isinstance(attribute_def, (list, tuple)):
+                    attribute_def = attribute_def[0]
+                member = getattr(self, attribute_def)
+                # TODO: ensure this hasn't broken existing behavior.
+                # member = getattr(self, attribute_def[0])
+                if member:
+                    if _qname_matches(tag, namespace, qname):
+                        matches.append(XmlAttribute(qname, member))
+        for qname, value in self._other_attributes.items():
+            if _qname_matches(tag, namespace, qname):
+                matches.append(XmlAttribute(qname, value))
+        return matches
+
+    GetAttributes = get_attributes
+
+    def _harvest_tree(self, tree, version=1):
+        """Populates object members from the data in the tree Element."""
+        qname, elements, attributes = self.__class__._get_rules(version)
+        for element in tree:
+            if elements and element.tag in elements:
+                definition = elements[element.tag]
+                # If this is a repeating element, make sure the member is set to a
+                # list.
+                if definition[2]:
+                    if getattr(self, definition[0]) is None:
+                        setattr(self, definition[0], [])
+                    getattr(self, definition[0]).append(_xml_element_from_tree(element,
+                                                                               definition[1], version))
+                else:
+                    setattr(self, definition[0], _xml_element_from_tree(element,
+                                                                        definition[1], version))
+            else:
+                self._other_elements.append(_xml_element_from_tree(element, XmlElement,
+                                                                   version))
+        for attrib, value in tree.attrib.items():
+            if attributes and attrib in attributes:
+                setattr(self, attributes[attrib], value)
+            else:
+                self._other_attributes[attrib] = value
+        if tree.text:
+            self.text = tree.text
+
+    def _to_tree(self, version=1):
+        new_tree = ElementTree.Element(_get_qname(self, version))
+        self._attach_members(new_tree, version)
+        return new_tree
+
+    def _attach_members(self, tree, version=1):
+        """Convert members to XML elements/attributes and add them to the tree.
+
+        Args:
+          tree: An ElementTree.Element which will be modified. The members of
+                this object will be added as child elements or attributes
+                according to the rules described in _expected_elements and
+                _expected_attributes. The elements and attributes stored in
+                other_attributes and other_elements are also added a children
+                of this tree.
+          version: int Ingnored in this method but used by VersionedElement.
+          encoding: str (optional)
+        """
+        qname, elements, attributes = self.__class__._get_rules(version)
+        encoding = STRING_ENCODING
+        # Add the expected elements and attributes to the tree.
+        if elements:
+            for tag, element_def in elements.items():
+                member = getattr(self, element_def[0])
+                # If this is a repeating element and there are members in the list.
+                if member and element_def[2]:
+                    for instance in member:
+                        instance._become_child(tree, version)
+                elif member:
+                    member._become_child(tree, version)
+        if attributes:
+            for attribute_tag, member_name in attributes.items():
+                value = getattr(self, member_name)
+                if value:
+                    tree.attrib[attribute_tag] = value
+        # Add the unexpected (other) elements and attributes to the tree.
+        for element in self._other_elements:
+            element._become_child(tree, version)
+        for key, value in self._other_attributes.items():
+            # I'm not sure if unicode can be used in the attribute name, so for now
+            # we assume the encoding is correct for the attribute name.
+            if not isinstance(value, str):
+                value = value.decode(encoding)
+            tree.attrib[key] = value
+        if self.text:
+            if isinstance(self.text, str):
+                tree.text = self.text
+            else:
+                tree.text = self.text.decode(encoding)
+
+    def to_string(self, version=1, encoding=None, pretty_print=None):
+        """Converts this object to XML."""
+
+        tree_string = ElementTree.tostring(self._to_tree(version))
+
+        if pretty_print and xmlString is not None:
+            return xmlString(tree_string).toprettyxml()
+
+        return tree_string
+
+    ToString = to_string
+
+    def __str__(self):
+        return self.to_string()
+
+    def _become_child(self, tree, version=1):
+        """Adds a child element to tree with the XML data in self."""
+        new_child = ElementTree.Element(_get_qname(self, version))
+        tree.append(new_child)
+        new_child.tag = _get_qname(self, version)
+        self._attach_members(new_child, version)
+
+    def __get_extension_elements(self):
+        return self._other_elements
+
+    def __set_extension_elements(self, elements):
+        self._other_elements = elements
+
+    extension_elements = property(__get_extension_elements,
+                                  __set_extension_elements,
+                                  """Provides backwards compatibility for v1 atom.AtomBase classes.""")
+
+    def __get_extension_attributes(self):
+        return self._other_attributes
+
+    def __set_extension_attributes(self, attributes):
+        self._other_attributes = attributes
+
+    extension_attributes = property(__get_extension_attributes,
+                                    __set_extension_attributes,
+                                    """Provides backwards compatibility for v1 atom.AtomBase classes.""")
+
+    def _get_tag(self, version=1):
+        qname = _get_qname(self, version)
+        if qname:
+            return qname[qname.find('}') + 1:]
+        return None
+
+    def _get_namespace(self, version=1):
+        qname = _get_qname(self, version)
+        if qname.startswith('{'):
+            return qname[1:qname.find('}')]
+        else:
+            return None
+
+    def _set_tag(self, tag):
+        if isinstance(self._qname, tuple):
+            self._qname = self._qname.copy()
+            if self._qname[0].startswith('{'):
+                self._qname[0] = '{%s}%s' % (self._get_namespace(1), tag)
+            else:
+                self._qname[0] = tag
+        else:
+            if self._qname is not None and self._qname.startswith('{'):
+                self._qname = '{%s}%s' % (self._get_namespace(), tag)
+            else:
+                self._qname = tag
+
+    def _set_namespace(self, namespace):
+        tag = self._get_tag(1)
+        if tag is None:
+            tag = ''
+        if isinstance(self._qname, tuple):
+            self._qname = self._qname.copy()
+            if namespace:
+                self._qname[0] = '{%s}%s' % (namespace, tag)
+            else:
+                self._qname[0] = tag
+        else:
+            if namespace:
+                self._qname = '{%s}%s' % (namespace, tag)
+            else:
+                self._qname = tag
+
+    tag = property(_get_tag, _set_tag,
+                   """Provides backwards compatibility for v1 atom.AtomBase classes.""")
+
+    namespace = property(_get_namespace, _set_namespace,
+                         """Provides backwards compatibility for v1 atom.AtomBase classes.""")
+
+    # Provided for backwards compatibility to atom.ExtensionElement
+    children = extension_elements
+    attributes = extension_attributes
+
+
+def _get_qname(element, version):
+    if isinstance(element._qname, tuple):
+        if version <= len(element._qname):
+            return element._qname[version - 1]
+        else:
+            return element._qname[-1]
+    else:
+        return element._qname
+
+
+def _qname_matches(tag, namespace, qname):
+    """Logic determines if a QName matches the desired local tag and namespace.
+
+    This is used in XmlElement.get_elements and XmlElement.get_attributes to
+    find matches in the element's members (among all expected-and-unexpected
+    elements-and-attributes).
+
+    Args:
+      expected_tag: string
+      expected_namespace: string
+      qname: string in the form '{xml_namespace}localtag' or 'tag' if there is
+             no namespace.
+
+    Returns:
+      boolean True if the member's tag and namespace fit the expected tag and
+      namespace.
+    """
+    # If there is no expected namespace or tag, then everything will match.
+    if qname is None:
+        member_tag = None
+        member_namespace = None
+    else:
+        if qname.startswith('{'):
+            member_namespace = qname[1:qname.index('}')]
+            member_tag = qname[qname.index('}') + 1:]
+        else:
+            member_namespace = None
+            member_tag = qname
+    return ((tag is None and namespace is None)
+            # If there is a tag, but no namespace, see if the local tag matches.
+            or (namespace is None and member_tag == tag)
+            # There was no tag, but there was a namespace so see if the namespaces
+            # match.
+            or (tag is None and member_namespace == namespace)
+            # There was no tag, and the desired elements have no namespace, so check
+            # to see that the member's namespace is None.
+            or (tag is None and namespace == ''
+                and member_namespace is None)
+            # The tag and the namespace both match.
+            or (tag == member_tag
+                and namespace == member_namespace)
+            # The tag matches, and the expected namespace is the empty namespace,
+            # check to make sure the member's namespace is None.
+            or (tag == member_tag and namespace == ''
+                and member_namespace is None))
+
+
+def parse(xml_string, target_class=None, version=1):
+    """Parses the XML string according to the rules for the target_class.
+
+    Args:
+      xml_string: bytes
+      target_class: XmlElement or a subclass. If None is specified, the
+          XmlElement class is used.
+      version: int (optional) The version of the schema which should be used when
+          converting the XML into an object. The default is 1.
+      encoding: str (optional) The character encoding of the bytes in the
+          xml_string. Default is 'UTF-8'.
+    """
+    if target_class is None:
+        target_class = XmlElement
+    if not isinstance(xml_string, bytes):
+        raise Exception("This function only accepts bytes")
+    tree = ElementTree.fromstring(xml_string)
+    return _xml_element_from_tree(tree, target_class, version)
+
+
+Parse = parse
+xml_element_from_string = parse
+XmlElementFromString = xml_element_from_string
+
+
+def _xml_element_from_tree(tree, target_class, version=1):
+    if target_class._qname is None:
+        instance = target_class()
+        instance._qname = tree.tag
+        instance._harvest_tree(tree, version)
+        return instance
+    # TODO handle the namespace-only case
+    # Namespace only will be used with Google Spreadsheets rows and
+    # Google Base item attributes.
+    elif tree.tag == _get_qname(target_class, version):
+        instance = target_class()
+        instance._harvest_tree(tree, version)
+        return instance
+    return None
+
+
+class XmlAttribute(object):
+    def __init__(self, qname, value):
+        self._qname = qname
+        self.value = value

src/gam/atom/data.py

@@ -0,0 +1,327 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2009 Google Inc.
+#
+# Licensed under the Apache License 2.0;
+
+
+
+# This module is used for version 2 of the Google Data APIs.
+
+
+# __author__ = 'j.s@google.com (Jeff Scudder)'
+
+import atom.core
+
+XML_TEMPLATE = '{http://www.w3.org/XML/1998/namespace}%s'
+ATOM_TEMPLATE = '{http://www.w3.org/2005/Atom}%s'
+APP_TEMPLATE_V1 = '{http://purl.org/atom/app#}%s'
+APP_TEMPLATE_V2 = '{http://www.w3.org/2007/app}%s'
+
+
+class Name(atom.core.XmlElement):
+    """The atom:name element."""
+    _qname = ATOM_TEMPLATE % 'name'
+
+
+class Email(atom.core.XmlElement):
+    """The atom:email element."""
+    _qname = ATOM_TEMPLATE % 'email'
+
+
+class Uri(atom.core.XmlElement):
+    """The atom:uri element."""
+    _qname = ATOM_TEMPLATE % 'uri'
+
+
+class Person(atom.core.XmlElement):
+    """A foundation class which atom:author and atom:contributor extend.
+
+    A person contains information like name, email address, and web page URI for
+    an author or contributor to an Atom feed.
+    """
+    name = Name
+    email = Email
+    uri = Uri
+
+
+class Author(Person):
+    """The atom:author element.
+
+    An author is a required element in Feed unless each Entry contains an Author.
+    """
+    _qname = ATOM_TEMPLATE % 'author'
+
+
+class Contributor(Person):
+    """The atom:contributor element."""
+    _qname = ATOM_TEMPLATE % 'contributor'
+
+
+class Link(atom.core.XmlElement):
+    """The atom:link element."""
+    _qname = ATOM_TEMPLATE % 'link'
+    href = 'href'
+    rel = 'rel'
+    type = 'type'
+    hreflang = 'hreflang'
+    title = 'title'
+    length = 'length'
+
+
+class Generator(atom.core.XmlElement):
+    """The atom:generator element."""
+    _qname = ATOM_TEMPLATE % 'generator'
+    uri = 'uri'
+    version = 'version'
+
+
+class Text(atom.core.XmlElement):
+    """A foundation class from which atom:title, summary, etc. extend.
+
+    This class should never be instantiated.
+    """
+    type = 'type'
+
+
+class Title(Text):
+    """The atom:title element."""
+    _qname = ATOM_TEMPLATE % 'title'
+
+
+class Subtitle(Text):
+    """The atom:subtitle element."""
+    _qname = ATOM_TEMPLATE % 'subtitle'
+
+
+class Rights(Text):
+    """The atom:rights element."""
+    _qname = ATOM_TEMPLATE % 'rights'
+
+
+class Summary(Text):
+    """The atom:summary element."""
+    _qname = ATOM_TEMPLATE % 'summary'
+
+
+class Content(Text):
+    """The atom:content element."""
+    _qname = ATOM_TEMPLATE % 'content'
+    src = 'src'
+
+
+class Category(atom.core.XmlElement):
+    """The atom:category element."""
+    _qname = ATOM_TEMPLATE % 'category'
+    term = 'term'
+    scheme = 'scheme'
+    label = 'label'
+
+
+class Id(atom.core.XmlElement):
+    """The atom:id element."""
+    _qname = ATOM_TEMPLATE % 'id'
+
+
+class Icon(atom.core.XmlElement):
+    """The atom:icon element."""
+    _qname = ATOM_TEMPLATE % 'icon'
+
+
+class Logo(atom.core.XmlElement):
+    """The atom:logo element."""
+    _qname = ATOM_TEMPLATE % 'logo'
+
+
+class Draft(atom.core.XmlElement):
+    """The app:draft element which indicates if this entry should be public."""
+    _qname = (APP_TEMPLATE_V1 % 'draft', APP_TEMPLATE_V2 % 'draft')
+
+
+class Control(atom.core.XmlElement):
+    """The app:control element indicating restrictions on publication.
+
+    The APP control element may contain a draft element indicating whether or
+    not this entry should be publicly available.
+    """
+    _qname = (APP_TEMPLATE_V1 % 'control', APP_TEMPLATE_V2 % 'control')
+    draft = Draft
+
+
+class Date(atom.core.XmlElement):
+    """A parent class for atom:updated, published, etc."""
+
+
+class Updated(Date):
+    """The atom:updated element."""
+    _qname = ATOM_TEMPLATE % 'updated'
+
+
+class Published(Date):
+    """The atom:published element."""
+    _qname = ATOM_TEMPLATE % 'published'
+
+
+class LinkFinder(object):
+    """An "interface" providing methods to find link elements
+
+    Entry elements often contain multiple links which differ in the rel
+    attribute or content type. Often, developers are interested in a specific
+    type of link so this class provides methods to find specific classes of
+    links.
+
+    This class is used as a mixin in Atom entries and feeds.
+    """
+
+    def find_url(self, rel):
+        """Returns the URL (as a string) in a link with the desired rel value."""
+        for link in self.link:
+            if link.rel == rel and link.href:
+                return link.href
+        return None
+
+    FindUrl = find_url
+
+    def get_link(self, rel):
+        """Returns a link object which has the desired rel value.
+
+        If you are interested in the URL instead of the link object,
+        consider using find_url instead.
+        """
+        for link in self.link:
+            if link.rel == rel and link.href:
+                return link
+        return None
+
+    GetLink = get_link
+
+    def find_self_link(self):
+        """Find the first link with rel set to 'self'
+
+        Returns:
+          A str containing the link's href or None if none of the links had rel
+          equal to 'self'
+        """
+        return self.find_url('self')
+
+    FindSelfLink = find_self_link
+
+    def get_self_link(self):
+        return self.get_link('self')
+
+    GetSelfLink = get_self_link
+
+    def find_edit_link(self):
+        return self.find_url('edit')
+
+    FindEditLink = find_edit_link
+
+    def get_edit_link(self):
+        return self.get_link('edit')
+
+    GetEditLink = get_edit_link
+
+    def find_edit_media_link(self):
+        link = self.find_url('edit-media')
+        # Search for media-edit as well since Picasa API used media-edit instead.
+        if link is None:
+            return self.find_url('media-edit')
+        return link
+
+    FindEditMediaLink = find_edit_media_link
+
+    def get_edit_media_link(self):
+        link = self.get_link('edit-media')
+        if link is None:
+            return self.get_link('media-edit')
+        return link
+
+    GetEditMediaLink = get_edit_media_link
+
+    def find_next_link(self):
+        return self.find_url('next')
+
+    FindNextLink = find_next_link
+
+    def get_next_link(self):
+        return self.get_link('next')
+
+    GetNextLink = get_next_link
+
+    def find_license_link(self):
+        return self.find_url('license')
+
+    FindLicenseLink = find_license_link
+
+    def get_license_link(self):
+        return self.get_link('license')
+
+    GetLicenseLink = get_license_link
+
+    def find_alternate_link(self):
+        return self.find_url('alternate')
+
+    FindAlternateLink = find_alternate_link
+
+    def get_alternate_link(self):
+        return self.get_link('alternate')
+
+    GetAlternateLink = get_alternate_link
+
+
+class FeedEntryParent(atom.core.XmlElement, LinkFinder):
+    """A super class for atom:feed and entry, contains shared attributes"""
+    author = [Author]
+    category = [Category]
+    contributor = [Contributor]
+    id = Id
+    link = [Link]
+    rights = Rights
+    title = Title
+    updated = Updated
+
+    def __init__(self, atom_id=None, text=None, *args, **kwargs):
+        if atom_id is not None:
+            self.id = atom_id
+        atom.core.XmlElement.__init__(self, text=text, *args, **kwargs)
+
+
+class Source(FeedEntryParent):
+    """The atom:source element."""
+    _qname = ATOM_TEMPLATE % 'source'
+    generator = Generator
+    icon = Icon
+    logo = Logo
+    subtitle = Subtitle
+
+
+class Entry(FeedEntryParent):
+    """The atom:entry element."""
+    _qname = ATOM_TEMPLATE % 'entry'
+    content = Content
+    published = Published
+    source = Source
+    summary = Summary
+    control = Control
+
+
+class Feed(Source):
+    """The atom:feed element which contains entries."""
+    _qname = ATOM_TEMPLATE % 'feed'
+    entry = [Entry]
+
+
+class ExtensionElement(atom.core.XmlElement):
+    """Provided for backwards compatibility to the v1 atom.ExtensionElement."""
+
+    def __init__(self, tag=None, namespace=None, attributes=None,
+                 children=None, text=None, *args, **kwargs):
+        if namespace:
+            self._qname = '{%s}%s' % (namespace, tag)
+        else:
+            self._qname = tag
+        self.children = children or []
+        self.attributes = attributes or {}
+        self.text = text
+
+    _BecomeChildElement = atom.core.XmlElement._become_child

src/gam/atom/http.py

@@ -0,0 +1,354 @@
+#
+# Copyright (C) 2008 Google Inc.
+#
+# Licensed under the Apache License 2.0;
+
+
+
+"""HttpClients in this module use httplib to make HTTP requests.
+
+This module make HTTP requests based on httplib, but there are environments
+in which an httplib based approach will not work (if running in Google App
+Engine for example). In those cases, higher level classes (like AtomService
+and GDataService) can swap out the HttpClient to transparently use a 
+different mechanism for making HTTP requests.
+
+  HttpClient: Contains a request method which performs an HTTP call to the 
+      server.
+      
+  ProxiedHttpClient: Contains a request method which connects to a proxy using
+      settings stored in operating system environment variables then 
+      performs an HTTP call to the endpoint server.
+"""
+
+# __author__ = 'api.jscudder (Jeff Scudder)'
+
+import base64
+import http.client
+import os
+import socket
+
+import atom.http_core
+import atom.http_interface
+import atom.url
+
+ssl_imported = False
+ssl = None
+try:
+    import ssl
+
+    ssl_imported = True
+except ImportError:
+    pass
+
+
+class ProxyError(atom.http_interface.Error):
+    pass
+
+
+class TestConfigurationError(Exception):
+    pass
+
+
+DEFAULT_CONTENT_TYPE = 'application/atom+xml'
+
+
+class HttpClient(atom.http_interface.GenericHttpClient):
+    # Added to allow old v1 HttpClient objects to use the new
+    # http_code.HttpClient. Used in unit tests to inject a mock client.
+    v2_http_client = None
+
+    def __init__(self, headers=None):
+        self.debug = False
+        self.headers = headers or {}
+
+    def request(self, operation, url, data=None, headers=None):
+        """Performs an HTTP call to the server, supports GET, POST, PUT, and
+        DELETE.
+
+        Usage example, perform and HTTP GET on http://www.google.com/:
+          import atom.http
+          client = atom.http.HttpClient()
+          http_response = client.request('GET', 'http://www.google.com/')
+
+        Args:
+          operation: str The HTTP operation to be performed. This is usually one
+              of 'GET', 'POST', 'PUT', or 'DELETE'
+          data: filestream, list of parts, or other object which can be converted
+              to a string. Should be set to None when performing a GET or DELETE.
+              If data is a file-like object which can be read, this method will
+              read a chunk of 100K bytes at a time and send them.
+              If the data is a list of parts to be sent, each part will be
+              evaluated and sent.
+          url: The full URL to which the request should be sent. Can be a string
+              or atom.url.Url.
+          headers: dict of strings. HTTP headers which should be sent
+              in the request.
+        """
+        all_headers = self.headers.copy()
+        if headers:
+            all_headers.update(headers)
+
+        # If the list of headers does not include a Content-Length, attempt to
+        # calculate it based on the data object.
+        if data and 'Content-Length' not in all_headers:
+            if isinstance(data, (str,)):
+                all_headers['Content-Length'] = str(len(data))
+            else:
+                raise atom.http_interface.ContentLengthRequired('Unable to calculate '
+                                                                'the length of the data parameter. Specify a value for '
+                                                                'Content-Length')
+
+        # Set the content type to the default value if none was set.
+        if 'Content-Type' not in all_headers:
+            all_headers['Content-Type'] = DEFAULT_CONTENT_TYPE
+
+        if self.v2_http_client is not None:
+            http_request = atom.http_core.HttpRequest(method=operation)
+            atom.http_core.Uri.parse_uri(str(url)).modify_request(http_request)
+            http_request.headers = all_headers
+            if data:
+                http_request._body_parts.append(data)
+            return self.v2_http_client.request(http_request=http_request)
+
+        if not isinstance(url, atom.url.Url):
+            if isinstance(url, str):
+                url = atom.url.parse_url(url)
+            else:
+                raise atom.http_interface.UnparsableUrlObject('Unable to parse url parameter because it was not a string or atom.url.Url')
+
+        connection = self._prepare_connection(url, all_headers)
+
+        if self.debug:
+            connection.debuglevel = 1
+
+        connection.putrequest(operation, self._get_access_url(url), skip_host=True)
+
+        if url.port is not None:
+            connection.putheader('Host', '%s:%s' % (url.host, url.port))
+        else:
+            connection.putheader('Host', url.host)
+
+        # Overcome a bug in Python 2.4 and 2.5
+        # httplib.HTTPConnection.putrequest adding
+        # HTTP request header 'Host: www.google.com:443' instead of
+        # 'Host: www.google.com', and thus resulting the error message
+        # 'Token invalid - AuthSub token has wrong scope' in the HTTP response.
+        if (url.protocol == 'https' and int(url.port or 443) == 443 and
+                hasattr(connection, '_buffer') and
+                isinstance(connection._buffer, list)):
+            header_line = 'Host: %s:443' % url.host
+            replacement_header_line = 'Host: %s' % url.host
+            try:
+                connection._buffer[connection._buffer.index(header_line)] = (
+                    replacement_header_line)
+            except ValueError:  # header_line missing from connection._buffer
+                pass
+
+        # Send the HTTP headers.
+        for header_name in all_headers:
+            connection.putheader(header_name, all_headers[header_name])
+        connection.endheaders()
+
+        # If there is data, send it in the request.
+        if data:
+            if isinstance(data, list):
+                for data_part in data:
+                    _send_data_part(data_part, connection)
+            else:
+                _send_data_part(data, connection)
+
+        # Return the HTTP Response from the server.
+        return connection.getresponse()
+
+    def _prepare_connection(self, url, headers):
+        if not isinstance(url, atom.url.Url):
+            if isinstance(url, (str,)):
+                url = atom.url.parse_url(url)
+            else:
+                raise atom.http_interface.UnparsableUrlObject('Unable to parse url '
+                                                              'parameter because it was not a string or atom.url.Url')
+        if url.protocol == 'https':
+            if not url.port:
+                return http.client.HTTPSConnection(url.host)
+            return http.client.HTTPSConnection(url.host, int(url.port))
+        else:
+            if not url.port:
+                return http.client.HTTPConnection(url.host)
+            return http.client.HTTPConnection(url.host, int(url.port))
+
+    def _get_access_url(self, url):
+        return url.to_string()
+
+
+class ProxiedHttpClient(HttpClient):
+    """Performs an HTTP request through a proxy.
+
+    The proxy settings are obtained from enviroment variables. The URL of the
+    proxy server is assumed to be stored in the environment variables
+    'https_proxy' and 'http_proxy' respectively. If the proxy server requires
+    a Basic Auth authorization header, the username and password are expected to
+    be in the 'proxy-username' or 'proxy_username' variable and the
+    'proxy-password' or 'proxy_password' variable, or in 'http_proxy' or
+    'https_proxy' as "protocol://[username:password@]host:port".
+
+    After connecting to the proxy server, the request is completed as in
+    HttpClient.request.
+    """
+
+    def _prepare_connection(self, url, headers):
+        proxy_settings = os.environ.get('%s_proxy' % url.protocol)
+        if not proxy_settings:
+            # The request was HTTP or HTTPS, but there was no appropriate proxy set.
+            return HttpClient._prepare_connection(self, url, headers)
+        else:
+            proxy_auth = _get_proxy_auth(proxy_settings)
+            proxy_netloc = _get_proxy_net_location(proxy_settings)
+            if url.protocol == 'https':
+                # Set any proxy auth headers
+                if proxy_auth:
+                    proxy_auth = 'Proxy-Authorization: %s' % proxy_auth
+
+                # Construct the proxy connect command.
+                port = url.port
+                if not port:
+                    port = '443'
+                proxy_connect = 'CONNECT %s:%s HTTP/1.0\r\n' % (url.host, port)
+
+                # Set the user agent to send to the proxy
+                if headers and 'User-Agent' in headers:
+                    user_agent = 'User-Agent: %s\r\n' % (headers['User-Agent'])
+                else:
+                    user_agent = 'User-Agent: python\r\n'
+
+                proxy_pieces = '%s%s%s\r\n' % (proxy_connect, proxy_auth, user_agent)
+
+                # Find the proxy host and port.
+                proxy_url = atom.url.parse_url(proxy_netloc)
+                if not proxy_url.port:
+                    proxy_url.port = '80'
+
+                # Connect to the proxy server, very simple recv and error checking
+                p_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+                p_sock.connect((proxy_url.host, int(proxy_url.port)))
+                #p_sock.sendall(proxy_pieces)
+                p_sock.sendall(proxy_pieces.encode('utf-8'))
+                response = ''
+
+                # Wait for the full response.
+                while response.find("\r\n\r\n") == -1:
+                    #response += p_sock.recv(8192)
+                    response += p_sock.recv(8192).decode('utf-8')
+
+                p_status = response.split()[1]
+                if p_status != str(200):
+                    raise ProxyError('Error status=%s' % str(p_status))
+
+                # Trivial setup for ssl socket.
+                sslobj = None
+                if ssl_imported:
+                    context = ssl.SSLContext(ssl.PROTOCOL_TLS)
+                    context.minimum_version = ssl.TLSVersion.TLSv1_2
+                    sslobj = context.wrap_socket(p_sock, server_hostname=url.host)
+                else:
+                    sock_ssl = socket.ssl(p_sock, None, None)
+                    sslobj = http.client.FakeSocket(p_sock, sock_ssl)
+
+                # Initalize httplib and replace with the proxy socket.
+                connection = http.client.HTTPConnection(proxy_url.host)
+                connection.sock = sslobj
+                return connection
+            else:
+                # If protocol was not https.
+                # Find the proxy host and port.
+                proxy_url = atom.url.parse_url(proxy_netloc)
+                if not proxy_url.port:
+                    proxy_url.port = '80'
+
+                if proxy_auth:
+                    headers['Proxy-Authorization'] = proxy_auth.strip()
+
+                return http.client.HTTPConnection(proxy_url.host, int(proxy_url.port))
+
+    def _get_access_url(self, url):
+        return url.to_string()
+
+
+def _get_proxy_auth(proxy_settings):
+    """Returns proxy authentication string for header.
+
+    Will check environment variables for proxy authentication info, starting with
+    proxy(_/-)username and proxy(_/-)password before checking the given
+    proxy_settings for a [protocol://]username:password@host[:port] string.
+
+    Args:
+      proxy_settings: String from http_proxy or https_proxy environment variable.
+
+    Returns:
+      Authentication string for proxy, or empty string if no proxy username was
+      found.
+    """
+    proxy_username = None
+    proxy_password = None
+
+    proxy_username = os.environ.get('proxy-username')
+    if not proxy_username:
+        proxy_username = os.environ.get('proxy_username')
+    proxy_password = os.environ.get('proxy-password')
+    if not proxy_password:
+        proxy_password = os.environ.get('proxy_password')
+
+    if not proxy_username:
+        if '@' in proxy_settings:
+            protocol_and_proxy_auth = proxy_settings.split('@')[0].split(':')
+            if len(protocol_and_proxy_auth) == 3:
+                # 3 elements means we have [<protocol>, //<user>, <password>]
+                proxy_username = protocol_and_proxy_auth[1].lstrip('/')
+                proxy_password = protocol_and_proxy_auth[2]
+            elif len(protocol_and_proxy_auth) == 2:
+                # 2 elements means we have [<user>, <password>]
+                proxy_username = protocol_and_proxy_auth[0]
+                proxy_password = protocol_and_proxy_auth[1]
+    if proxy_username:
+        user_auth = base64.b64encode(('%s:%s' % (proxy_username, proxy_password)).encode('utf-8'))
+        return 'Basic %s\r\n' % (user_auth.strip().decode('utf-8'))
+    else:
+        return ''
+
+
+def _get_proxy_net_location(proxy_settings):
+    """Returns proxy host and port.
+
+    Args:
+      proxy_settings: String from http_proxy or https_proxy environment variable.
+          Must be in the form of protocol://[username:password@]host:port
+
+    Returns:
+      String in the form of protocol://host:port
+    """
+    if '@' in proxy_settings:
+        protocol = proxy_settings.split(':')[0]
+        netloc = proxy_settings.split('@')[1]
+        return '%s://%s' % (protocol, netloc)
+    else:
+        return proxy_settings
+
+
+def _send_data_part(data, connection):
+    if isinstance(data, (str,)):
+        connection.send(data)
+        return
+    # Check to see if data is a file-like object that has a read method.
+    elif hasattr(data, 'read'):
+        # Read the file and send it a chunk at a time.
+        while 1:
+            binarydata = data.read(100000)
+            if binarydata == b'': break
+            connection.send(binarydata)
+        return
+    else:
+        # The data object was not a file.
+        # Try to convert to a string and send the data.
+        #connection.send(str(data))
+        connection.send(str(data).encode('utf-8'))
+        return

src/gam/atom/http_core.py

@@ -0,0 +1,599 @@
+#!/usr/bin/env python
+#
+#    Copyright (C) 2009 Google Inc.
+#
+#   Licensed under the Apache License 2.0;
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+
+
+# This module is used for version 2 of the Google Data APIs.
+# TODO: add proxy handling.
+
+
+# __author__ = 'j.s@google.com (Jeff Scudder)'
+
+import http.client
+import io
+import os
+import urllib.error
+import urllib.parse
+import urllib.request
+
+ssl = None
+try:
+    import ssl
+except ImportError:
+    pass
+
+
+class Error(Exception):
+    pass
+
+
+class UnknownSize(Error):
+    pass
+
+
+class ProxyError(Error):
+    pass
+
+
+MIME_BOUNDARY = 'END_OF_PART'
+
+
+def get_headers(http_response):
+    """Retrieves all HTTP headers from an HTTP response from the server.
+
+    This method is provided for backwards compatibility for Python2.2 and 2.3.
+    The httplib.HTTPResponse object in 2.2 and 2.3 does not have a getheaders
+    method so this function will use getheaders if available, but if not it
+    will retrieve a few using getheader.
+    """
+    if hasattr(http_response, 'getheaders'):
+        return http_response.getheaders()
+    else:
+        headers = []
+        for header in (
+                'location', 'content-type', 'content-length', 'age', 'allow',
+                'cache-control', 'content-location', 'content-encoding', 'date',
+                'etag', 'expires', 'last-modified', 'pragma', 'server',
+                'set-cookie', 'transfer-encoding', 'vary', 'via', 'warning',
+                'www-authenticate', 'gdata-version'):
+            value = http_response.getheader(header, None)
+            if value is not None:
+                headers.append((header, value))
+        return headers
+
+
+class HttpRequest(object):
+    """Contains all of the parameters for an HTTP 1.1 request.
+
+    The HTTP headers are represented by a dictionary, and it is the
+    responsibility of the user to ensure that duplicate field names are combined
+    into one header value according to the rules in section 4.2 of RFC 2616.
+    """
+    method = None
+    uri = None
+
+    def __init__(self, uri=None, method=None, headers=None):
+        """Construct an HTTP request.
+
+        Args:
+          uri: The full path or partial path as a Uri object or a string.
+          method: The HTTP method for the request, examples include 'GET', 'POST',
+                  etc.
+          headers: dict of strings The HTTP headers to include in the request.
+        """
+        self.headers = headers or {}
+        self._body_parts = []
+        if method is not None:
+            self.method = method
+        if isinstance(uri, str):
+            uri = Uri.parse_uri(uri)
+        self.uri = uri or Uri()
+
+    def add_body_part(self, data, mime_type, size=None):
+        """Adds data to the HTTP request body.
+
+        If more than one part is added, this is assumed to be a mime-multipart
+        request. This method is designed to create MIME 1.0 requests as specified
+        in RFC 1341.
+
+        Args:
+          data: str or a file-like object containing a part of the request body.
+          mime_type: str The MIME type describing the data
+          size: int Required if the data is a file like object. If the data is a
+                string, the size is calculated so this parameter is ignored.
+        """
+        if hasattr(data, '__len__'):
+            size = len(data)
+        if size is None:
+            # TODO: support chunked transfer if some of the body is of unknown size.
+            raise UnknownSize('Each part of the body must have a known size.')
+        if 'Content-Length' in self.headers:
+            content_length = int(self.headers['Content-Length'])
+        else:
+            content_length = 0
+        # If this is the first part added to the body, then this is not a multipart
+        # request.
+        if len(self._body_parts) == 0:
+            self.headers['Content-Type'] = mime_type
+            content_length = size
+            self._body_parts.append(data)
+        elif len(self._body_parts) == 1:
+            # This is the first member in a mime-multipart request, so change the
+            # _body_parts list to indicate a multipart payload.
+            self._body_parts.insert(0, 'Media multipart posting')
+            boundary_string = '\r\n--%s\r\n' % (MIME_BOUNDARY,)
+            content_length += len(boundary_string) + size
+            self._body_parts.insert(1, boundary_string)
+            content_length += len('Media multipart posting')
+            # Put the content type of the first part of the body into the multipart
+            # payload.
+            original_type_string = 'Content-Type: %s\r\n\r\n' % (
+                self.headers['Content-Type'],)
+            self._body_parts.insert(2, original_type_string)
+            content_length += len(original_type_string)
+            boundary_string = '\r\n--%s\r\n' % (MIME_BOUNDARY,)
+            self._body_parts.append(boundary_string)
+            content_length += len(boundary_string)
+            # Change the headers to indicate this is now a mime multipart request.
+            self.headers['Content-Type'] = 'multipart/related; boundary="%s"' % (
+                MIME_BOUNDARY,)
+            self.headers['MIME-version'] = '1.0'
+            # Include the mime type of this part.
+            type_string = 'Content-Type: %s\r\n\r\n' % (mime_type)
+            self._body_parts.append(type_string)
+            content_length += len(type_string)
+            self._body_parts.append(data)
+            ending_boundary_string = '\r\n--%s--' % (MIME_BOUNDARY,)
+            self._body_parts.append(ending_boundary_string)
+            content_length += len(ending_boundary_string)
+        else:
+            # This is a mime multipart request.
+            boundary_string = '\r\n--%s\r\n' % (MIME_BOUNDARY,)
+            self._body_parts.insert(-1, boundary_string)
+            content_length += len(boundary_string) + size
+            # Include the mime type of this part.
+            type_string = 'Content-Type: %s\r\n\r\n' % (mime_type)
+            self._body_parts.insert(-1, type_string)
+            content_length += len(type_string)
+            self._body_parts.insert(-1, data)
+        self.headers['Content-Length'] = str(content_length)
+
+    # I could add an "append_to_body_part" method as well.
+
+    AddBodyPart = add_body_part
+
+    def add_form_inputs(self, form_data,
+                        mime_type='application/x-www-form-urlencoded'):
+        """Form-encodes and adds data to the request body.
+
+        Args:
+          form_data: dict or sequnce or two member tuples which contains the
+                     form keys and values.
+          mime_type: str The MIME type of the form data being sent. Defaults
+                     to 'application/x-www-form-urlencoded'.
+        """
+        body = urllib.parse.urlencode(form_data)
+        self.add_body_part(body, bytes(mime_type, 'ascii'))
+
+    AddFormInputs = add_form_inputs
+
+    def _copy(self):
+        """Creates a deep copy of this request."""
+        copied_uri = Uri(self.uri.scheme, self.uri.host, self.uri.port,
+                         self.uri.path, self.uri.query.copy())
+        new_request = HttpRequest(uri=copied_uri, method=self.method,
+                                  headers=self.headers.copy())
+        new_request._body_parts = self._body_parts[:]
+        return new_request
+
+    def _dump(self):
+        """Converts to a printable string for debugging purposes.
+
+        In order to preserve the request, it does not read from file-like objects
+        in the body.
+        """
+        output = 'HTTP Request\n  method: %s\n  url: %s\n  headers:\n' % (
+            self.method, str(self.uri))
+        for header, value in self.headers.items():
+            output += '    %s: %s\n' % (header, value)
+        output += '  body sections:\n'
+        i = 0
+        for part in self._body_parts:
+            if isinstance(part, str):
+                output += '    %s: %s\n' % (i, part)
+            else:
+                output += '    %s: <file like object>\n' % i
+            i += 1
+        return output
+
+
+def _apply_defaults(http_request):
+    if http_request.uri.scheme is None:
+        if http_request.uri.port == 443:
+            http_request.uri.scheme = 'https'
+        else:
+            http_request.uri.scheme = 'http'
+
+
+class Uri(object):
+    """A URI as used in HTTP 1.1"""
+    scheme = None
+    host = None
+    port = None
+    path = None
+
+    def __init__(self, scheme=None, host=None, port=None, path=None, query=None):
+        """Constructor for a URI.
+
+        Args:
+          scheme: str This is usually 'http' or 'https'.
+          host: str The host name or IP address of the desired server.
+          post: int The server's port number.
+          path: str The path of the resource following the host. This begins with
+                a /, example: '/calendar/feeds/default/allcalendars/full'
+          query: dict of strings The URL query parameters. The keys and values are
+                 both escaped so this dict should contain the unescaped values.
+                 For example {'my key': 'val', 'second': '!!!'} will become
+                 '?my+key=val&second=%21%21%21' which is appended to the path.
+        """
+        self.query = query or {}
+        if scheme is not None:
+            self.scheme = scheme
+        if host is not None:
+            self.host = host
+        if port is not None:
+            self.port = port
+        if path:
+            self.path = path
+
+    def _get_query_string(self):
+        param_pairs = []
+        for key, value in self.query.items():
+            quoted_key = urllib.parse.quote_plus(str(key))
+            if value is None:
+                param_pairs.append(quoted_key)
+            else:
+                quoted_value = urllib.parse.quote_plus(str(value))
+                param_pairs.append('%s=%s' % (quoted_key, quoted_value))
+        return '&'.join(param_pairs)
+
+    def _get_relative_path(self):
+        """Returns the path with the query parameters escaped and appended."""
+        param_string = self._get_query_string()
+        if self.path is None:
+            path = '/'
+        else:
+            path = self.path
+        if param_string:
+            return '?'.join([path, param_string])
+        else:
+            return path
+
+    def _to_string(self):
+        if self.scheme is None and self.port == 443:
+            scheme = 'https'
+        elif self.scheme is None:
+            scheme = 'http'
+        else:
+            scheme = self.scheme
+        if self.path is None:
+            path = '/'
+        else:
+            path = self.path
+        if self.port is None:
+            return '%s://%s%s' % (scheme, self.host, self._get_relative_path())
+        else:
+            return '%s://%s:%s%s' % (scheme, self.host, str(self.port),
+                                     self._get_relative_path())
+
+    def __str__(self):
+        return self._to_string()
+
+    def modify_request(self, http_request=None):
+        """Sets HTTP request components based on the URI."""
+        if http_request is None:
+            http_request = HttpRequest()
+        if http_request.uri is None:
+            http_request.uri = Uri()
+        # Determine the correct scheme.
+        if self.scheme:
+            http_request.uri.scheme = self.scheme
+        if self.port:
+            http_request.uri.port = self.port
+        if self.host:
+            http_request.uri.host = self.host
+        # Set the relative uri path
+        if self.path:
+            http_request.uri.path = self.path
+        if self.query:
+            http_request.uri.query = self.query.copy()
+        return http_request
+
+    ModifyRequest = modify_request
+
+    def parse_uri(uri_string):
+        """Creates a Uri object which corresponds to the URI string.
+
+        This method can accept partial URIs, but it will leave missing
+        members of the Uri unset.
+        """
+        parts = urllib.parse.urlparse(uri_string)
+        uri = Uri()
+        if parts[0]:
+            uri.scheme = parts[0]
+        if parts[1]:
+            host_parts = parts[1].split(':')
+            if host_parts[0]:
+                uri.host = host_parts[0]
+            if len(host_parts) > 1:
+                uri.port = int(host_parts[1])
+        if parts[2]:
+            uri.path = parts[2]
+        if parts[4]:
+            param_pairs = parts[4].split('&')
+            for pair in param_pairs:
+                pair_parts = pair.split('=')
+                if len(pair_parts) > 1:
+                    uri.query[urllib.parse.unquote_plus(pair_parts[0])] = (
+                        urllib.parse.unquote_plus(pair_parts[1]))
+                elif len(pair_parts) == 1:
+                    uri.query[urllib.parse.unquote_plus(pair_parts[0])] = None
+        return uri
+
+    parse_uri = staticmethod(parse_uri)
+
+    ParseUri = parse_uri
+
+
+parse_uri = Uri.parse_uri
+
+ParseUri = Uri.parse_uri
+
+
+class HttpResponse(object):
+    status = None
+    reason = None
+    _body = None
+
+    def __init__(self, status=None, reason=None, headers=None, body=None):
+        self._headers = headers or {}
+        if status is not None:
+            self.status = status
+        if reason is not None:
+            self.reason = reason
+        if body is not None:
+            if hasattr(body, 'read'):
+                self._body = body
+            else:
+                self._body = io.StringIO(body)
+
+    def getheader(self, name, default=None):
+        if name in self._headers:
+            return self._headers[name]
+        else:
+            return default
+
+    def getheaders(self):
+        return self._headers
+
+    def read(self, amt=None):
+        if self._body is None:
+            return None
+        if not amt:
+            return self._body.read()
+        else:
+            return self._body.read(amt)
+
+
+def _dump_response(http_response):
+    """Converts to a string for printing debug messages.
+
+    Does not read the body since that may consume the content.
+    """
+    output = 'HttpResponse\n  status: %s\n  reason: %s\n  headers:' % (
+        http_response.status, http_response.reason)
+    headers = get_headers(http_response)
+    if isinstance(headers, dict):
+        for header, value in headers.items():
+            output += '    %s: %s\n' % (header, value)
+    else:
+        for pair in headers:
+            output += '    %s: %s\n' % (pair[0], pair[1])
+    return output
+
+
+class HttpClient(object):
+    """Performs HTTP requests using httplib."""
+    debug = None
+
+    def request(self, http_request):
+        return self._http_request(http_request.method, http_request.uri,
+                                  http_request.headers, http_request._body_parts)
+
+    Request = request
+
+    def _get_connection(self, uri, headers=None):
+        """Opens a socket connection to the server to set up an HTTP request.
+
+        Args:
+          uri: The full URL for the request as a Uri object.
+          headers: A dict of string pairs containing the HTTP headers for the
+              request.
+        """
+        connection = None
+        if uri.scheme == 'https':
+            if not uri.port:
+                connection = http.client.HTTPSConnection(uri.host)
+            else:
+                connection = http.client.HTTPSConnection(uri.host, int(uri.port))
+        else:
+            if not uri.port:
+                connection = http.client.HTTPConnection(uri.host)
+            else:
+                connection = http.client.HTTPConnection(uri.host, int(uri.port))
+        return connection
+
+    def _http_request(self, method, uri, headers=None, body_parts=None):
+        """Makes an HTTP request using httplib.
+
+        Args:
+          method: str example: 'GET', 'POST', 'PUT', 'DELETE', etc.
+          uri: str or atom.http_core.Uri
+          headers: dict of strings mapping to strings which will be sent as HTTP
+                   headers in the request.
+          body_parts: list of strings, objects with a read method, or objects
+                      which can be converted to strings using str. Each of these
+                      will be sent in order as the body of the HTTP request.
+        """
+        if isinstance(uri, str):
+            uri = Uri.parse_uri(uri)
+
+        connection = self._get_connection(uri, headers=headers)
+
+        if self.debug:
+            connection.debuglevel = 1
+
+        if connection.host != uri.host:
+            connection.putrequest(method, str(uri))
+        else:
+            connection.putrequest(method, uri._get_relative_path())
+
+        # Overcome a bug in Python 2.4 and 2.5
+        # httplib.HTTPConnection.putrequest adding
+        # HTTP request header 'Host: www.google.com:443' instead of
+        # 'Host: www.google.com', and thus resulting the error message
+        # 'Token invalid - AuthSub token has wrong scope' in the HTTP response.
+        if (uri.scheme == 'https' and int(uri.port or 443) == 443 and
+                hasattr(connection, '_buffer') and
+                isinstance(connection._buffer, list)):
+            header_line = 'Host: %s:443' % uri.host
+            replacement_header_line = 'Host: %s' % uri.host
+            try:
+                connection._buffer[connection._buffer.index(header_line)] = (
+                    replacement_header_line)
+            except ValueError:  # header_line missing from connection._buffer
+                pass
+
+        # Send the HTTP headers.
+        for header_name, value in headers.items():
+            connection.putheader(header_name, value)
+        connection.endheaders()
+
+        # If there is data, send it in the request.
+        if body_parts and [x for x in body_parts if x != '']:
+            for part in body_parts:
+                _send_data_part(part, connection)
+
+        # Return the HTTP Response from the server.
+        return connection.getresponse()
+
+
+def _send_data_part(data, connection):
+    if isinstance(data, str):
+        # I might want to just allow str, not unicode.
+        connection.send(data.encode())
+    # Check to see if data is a file-like object that has a read method.
+    elif hasattr(data, 'read'):
+        # Read the file and send it a chunk at a time.
+        while 1:
+            binarydata = data.read(100000)
+            if binarydata == '': break
+            connection.send(binarydata)
+    else:
+        # The data object was not a file.
+        # Try to convert to a string and send the data.
+        connection.send(data)
+
+
+class ProxiedHttpClient(HttpClient):
+    def _get_connection(self, uri, headers=None):
+        # Check to see if there are proxy settings required for this request.
+        proxy = None
+        if uri.scheme == 'https':
+            proxy = os.environ.get('https_proxy')
+        elif uri.scheme == 'http':
+            proxy = os.environ.get('http_proxy')
+        if not proxy:
+            return HttpClient._get_connection(self, uri, headers=headers)
+        # Now we have the URL of the appropriate proxy server.
+        # Get a username and password for the proxy if required.
+        proxy_auth = _get_proxy_auth()
+        if uri.scheme == 'https':
+            import socket
+            if proxy_auth:
+                proxy_auth = 'Proxy-Authorization: %s' % proxy_auth
+            # Construct the proxy connect command.
+            port = uri.port
+            if not port:
+                port = 443
+            proxy_connect = 'CONNECT %s:%s HTTP/1.0\r\n' % (uri.host, port)
+            # Set the user agent to send to the proxy
+            user_agent = ''
+            if headers and 'User-Agent' in headers:
+                user_agent = 'User-Agent: %s\r\n' % (headers['User-Agent'])
+            proxy_pieces = '%s%s%s\r\n' % (proxy_connect, proxy_auth, user_agent)
+            # Find the proxy host and port.
+            proxy_uri = Uri.parse_uri(proxy)
+            if not proxy_uri.port:
+                proxy_uri.port = '80'
+            # Connect to the proxy server, very simple recv and error checking
+            p_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            p_sock.connect((proxy_uri.host, int(proxy_uri.port)))
+            #p_sock.sendall(proxy_pieces)
+            p_sock.sendall(proxy_pieces.encode('utf-8'))
+            response = ''
+            # Wait for the full response.
+            while response.find("\r\n\r\n") == -1:
+                #response += p_sock.recv(8192)
+                response += p_sock.recv(8192).decode('utf-8')
+            p_status = response.split()[1]
+            if p_status != str(200):
+                raise ProxyError('Error status=%s' % str(p_status))
+            # Trivial setup for ssl socket.
+            sslobj = None
+            if ssl is not None:
+                context = ssl.SSLContext(ssl.PROTOCOL_TLS)
+                context.minimum_version = ssl.TLSVersion.TLSv1_2
+                sslobj = context.wrap_socket(p_sock, server_hostname=uri.host)
+            else:
+                sock_ssl = socket.ssl(p_sock, None, None)
+                sslobj = http.client.FakeSocket(p_sock, sock_ssl)
+            # Initalize httplib and replace with the proxy socket.
+            connection = http.client.HTTPConnection(proxy_uri.host)
+            connection.sock = sslobj
+            return connection
+        elif uri.scheme == 'http':
+            proxy_uri = Uri.parse_uri(proxy)
+            if not proxy_uri.port:
+                proxy_uri.port = '80'
+            if proxy_auth:
+                headers['Proxy-Authorization'] = proxy_auth.strip()
+            return http.client.HTTPConnection(proxy_uri.host, int(proxy_uri.port))
+        return None
+
+
+def _get_proxy_auth():
+    import base64
+    proxy_username = os.environ.get('proxy-username')
+    if not proxy_username:
+        proxy_username = os.environ.get('proxy_username')
+    proxy_password = os.environ.get('proxy-password')
+    if not proxy_password:
+        proxy_password = os.environ.get('proxy_password')
+    if proxy_username:
+        user_auth = base64.b64encode(('%s:%s' % (proxy_username, proxy_password)).encode('utf-8'))
+        return 'Basic %s\r\n' % (user_auth.strip().decode('utf-8'))
+    else:
+        return ''

src/gam/atom/http_interface.py

@@ -0,0 +1,144 @@
+#
+# Copyright (C) 2008 Google Inc.
+#
+# Licensed under the Apache License 2.0;
+
+
+"""This module provides a common interface for all HTTP requests.
+
+  HttpResponse: Represents the server's response to an HTTP request. Provides
+      an interface identical to httplib.HTTPResponse which is the response
+      expected from higher level classes which use HttpClient.request.
+
+  GenericHttpClient: Provides an interface (superclass) for an object 
+      responsible for making HTTP requests. Subclasses of this object are
+      used in AtomService and GDataService to make requests to the server. By
+      changing the http_client member object, the AtomService is able to make
+      HTTP requests using different logic (for example, when running on 
+      Google App Engine, the http_client makes requests using the App Engine
+      urlfetch API). 
+"""
+
+# __author__ = 'api.jscudder (Jeff Scudder)'
+
+import io
+
+USER_AGENT = '%s GData-Python/2.0.18'
+
+
+class Error(Exception):
+    pass
+
+
+class UnparsableUrlObject(Error):
+    pass
+
+
+class ContentLengthRequired(Error):
+    pass
+
+
+class HttpResponse(object):
+    def __init__(self, body=None, status=None, reason=None, headers=None):
+        """Constructor for an HttpResponse object.
+
+        HttpResponse represents the server's response to an HTTP request from
+        the client. The HttpClient.request method returns a httplib.HTTPResponse
+        object and this HttpResponse class is designed to mirror the interface
+        exposed by httplib.HTTPResponse.
+
+        Args:
+          body: A file like object, with a read() method. The body could also
+              be a string, and the constructor will wrap it so that
+              HttpResponse.read(self) will return the full string.
+          status: The HTTP status code as an int. Example: 200, 201, 404.
+          reason: The HTTP status message which follows the code. Example:
+              OK, Created, Not Found
+          headers: A dictionary containing the HTTP headers in the server's
+              response. A common header in the response is Content-Length.
+        """
+        if body:
+            if hasattr(body, 'read'):
+                self._body = body
+            else:
+                self._body = io.StringIO(body)
+        else:
+            self._body = None
+        if status is not None:
+            self.status = int(status)
+        else:
+            self.status = None
+        self.reason = reason
+        self._headers = headers or {}
+
+    def getheader(self, name, default=None):
+        if name in self._headers:
+            return self._headers[name]
+        else:
+            return default
+
+    def read(self, amt=None):
+        if not amt:
+            return self._body.read()
+        else:
+            return self._body.read(amt)
+
+
+class GenericHttpClient(object):
+    debug = False
+
+    def __init__(self, http_client, headers=None):
+        """
+
+        Args:
+          http_client: An object which provides a request method to make an HTTP
+              request. The request method in GenericHttpClient performs a
+              call-through to the contained HTTP client object.
+          headers: A dictionary containing HTTP headers which should be included
+              in every HTTP request. Common persistent headers include
+              'User-Agent'.
+        """
+        self.http_client = http_client
+        self.headers = headers or {}
+
+    def request(self, operation, url, data=None, headers=None):
+        all_headers = self.headers.copy()
+        if headers:
+            all_headers.update(headers)
+        return self.http_client.request(operation, url, data=data,
+                                        headers=all_headers)
+
+    def get(self, url, headers=None):
+        return self.request('GET', url, headers=headers)
+
+    def post(self, url, data, headers=None):
+        return self.request('POST', url, data=data, headers=headers)
+
+    def put(self, url, data, headers=None):
+        return self.request('PUT', url, data=data, headers=headers)
+
+    def delete(self, url, headers=None):
+        return self.request('DELETE', url, headers=headers)
+
+
+class GenericToken(object):
+    """Represents an Authorization token to be added to HTTP requests.
+
+    Some Authorization headers included calculated fields (digital
+    signatures for example) which are based on the parameters of the HTTP
+    request. Therefore the token is responsible for signing the request
+    and adding the Authorization header.
+    """
+
+    def perform_request(self, http_client, operation, url, data=None,
+                        headers=None):
+        """For the GenericToken, no Authorization token is set."""
+        return http_client.request(operation, url, data=data, headers=headers)
+
+    def valid_for_scope(self, url):
+        """Tells the caller if the token authorizes access to the desired URL.
+
+        Since the generic token doesn't add an auth header, it is not valid for
+        any scope.
+        """
+        return False

src/gam/atom/mock_http.py

@@ -0,0 +1,123 @@
+#
+# Copyright (C) 2008 Google Inc.
+#
+# Licensed under the Apache License 2.0;
+
+
+
+# __author__ = 'api.jscudder (Jeff Scudder)'
+
+import atom.http_interface
+import atom.url
+
+
+class Error(Exception):
+    pass
+
+
+class NoRecordingFound(Error):
+    pass
+
+
+class MockRequest(object):
+    """Holds parameters of an HTTP request for matching against future requests.
+    """
+
+    def __init__(self, operation, url, data=None, headers=None):
+        self.operation = operation
+        if isinstance(url, str):
+            url = atom.url.parse_url(url)
+        self.url = url
+        self.data = data
+        self.headers = headers
+
+
+class MockResponse(atom.http_interface.HttpResponse):
+    """Simulates an httplib.HTTPResponse object."""
+
+    def __init__(self, body=None, status=None, reason=None, headers=None):
+        if body and hasattr(body, 'read'):
+            self.body = body.read()
+        else:
+            self.body = body
+        if status is not None:
+            self.status = int(status)
+        else:
+            self.status = None
+        self.reason = reason
+        self._headers = headers or {}
+
+    def read(self):
+        return self.body
+
+
+class MockHttpClient(atom.http_interface.GenericHttpClient):
+    def __init__(self, headers=None, recordings=None, real_client=None):
+        """An HttpClient which responds to request with stored data.
+
+        The request-response pairs are stored as tuples in a member list named
+        recordings.
+
+        The MockHttpClient can be switched from replay mode to record mode by
+        setting the real_client member to an instance of an HttpClient which will
+        make real HTTP requests and store the server's response in list of
+        recordings.
+
+        Args:
+          headers: dict containing HTTP headers which should be included in all
+              HTTP requests.
+          recordings: The initial recordings to be used for responses. This list
+              contains tuples in the form: (MockRequest, MockResponse)
+          real_client: An HttpClient which will make a real HTTP request. The
+              response will be converted into a MockResponse and stored in
+              recordings.
+        """
+        self.recordings = recordings or []
+        self.real_client = real_client
+        self.headers = headers or {}
+
+    def add_response(self, response, operation, url, data=None, headers=None):
+        """Adds a request-response pair to the recordings list.
+
+        After the recording is added, future matching requests will receive the
+        response.
+
+        Args:
+          response: MockResponse
+          operation: str
+          url: str
+          data: str, Currently the data is ignored when looking for matching
+              requests.
+          headers: dict of strings: Currently the headers are ignored when
+              looking for matching requests.
+        """
+        request = MockRequest(operation, url, data=data, headers=headers)
+        self.recordings.append((request, response))
+
+    def request(self, operation, url, data=None, headers=None):
+        """Returns a matching MockResponse from the recordings.
+
+        If the real_client is set, the request will be passed along and the
+        server's response will be added to the recordings and also returned.
+
+        If there is no match, a NoRecordingFound error will be raised.
+        """
+        if self.real_client is None:
+            if isinstance(url, str):
+                url = atom.url.parse_url(url)
+            for recording in self.recordings:
+                if recording[0].operation == operation and recording[0].url == url:
+                    return recording[1]
+            raise NoRecordingFound('No recodings found for %s %s' % (
+                operation, url))
+        else:
+            # There is a real HTTP client, so make the request, and record the
+            # response.
+            response = self.real_client.request(operation, url, data=data,
+                                                headers=headers)
+            # TODO: copy the headers
+            stored_response = MockResponse(body=response, status=response.status,
+                                           reason=response.reason)
+            self.add_response(stored_response, operation, url, data=data,
+                              headers=headers)
+            return stored_response

src/gam/atom/mock_http_core.py

@@ -0,0 +1,313 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2009 Google Inc.
+#
+# Licensed under the Apache License 2.0;
+
+
+
+# This module is used for version 2 of the Google Data APIs.
+
+
+# __author__ = 'j.s@google.com (Jeff Scudder)'
+
+import io
+import os.path
+import pickle
+import tempfile
+
+import atom.http_core
+
+
+class Error(Exception):
+    pass
+
+
+class NoRecordingFound(Error):
+    pass
+
+
+class MockHttpClient(object):
+    debug = None
+    real_client = None
+    last_request_was_live = False
+
+    # The following members are used to construct the session cache temp file
+    # name.
+    # These are combined to form the file name
+    # /tmp/cache_prefix.cache_case_name.cache_test_name
+    cache_name_prefix = 'gdata_live_test'
+    cache_case_name = ''
+    cache_test_name = ''
+
+    def __init__(self, recordings=None, real_client=None):
+        self._recordings = recordings or []
+        if real_client is not None:
+            self.real_client = real_client
+
+    def add_response(self, http_request, status, reason, headers=None,
+                     body=None):
+        response = MockHttpResponse(status, reason, headers, body)
+        # TODO Scrub the request and the response.
+        self._recordings.append((http_request._copy(), response))
+
+    AddResponse = add_response
+
+    def request(self, http_request):
+        """Provide a recorded response, or record a response for replay.
+
+        If the real_client is set, the request will be made using the
+        real_client, and the response from the server will be recorded.
+        If the real_client is None (the default), this method will examine
+        the recordings and find the first which matches.
+        """
+        request = http_request._copy()
+        _scrub_request(request)
+        if self.real_client is None:
+            self.last_request_was_live = False
+            for recording in self._recordings:
+                if _match_request(recording[0], request):
+                    return recording[1]
+        else:
+            # Pass along the debug settings to the real client.
+            self.real_client.debug = self.debug
+            # Make an actual request since we can use the real HTTP client.
+            self.last_request_was_live = True
+            response = self.real_client.request(http_request)
+            scrubbed_response = _scrub_response(response)
+            self.add_response(request, scrubbed_response.status,
+                              scrubbed_response.reason,
+                              dict(atom.http_core.get_headers(scrubbed_response)),
+                              scrubbed_response.read())
+            # Return the recording which we just added.
+            return self._recordings[-1][1]
+        raise NoRecordingFound('No recoding was found for request: %s %s' % (
+            request.method, str(request.uri)))
+
+    Request = request
+
+    def _save_recordings(self, filename):
+        recording_file = open(os.path.join(tempfile.gettempdir(), filename),
+                              'wb')
+        pickle.dump(self._recordings, recording_file)
+        recording_file.close()
+
+    def _load_recordings(self, filename):
+        recording_file = open(os.path.join(tempfile.gettempdir(), filename),
+                              'rb')
+        self._recordings = pickle.load(recording_file)
+        recording_file.close()
+
+    def _delete_recordings(self, filename):
+        full_path = os.path.join(tempfile.gettempdir(), filename)
+        if os.path.exists(full_path):
+            os.remove(full_path)
+
+    def _load_or_use_client(self, filename, http_client):
+        if os.path.exists(os.path.join(tempfile.gettempdir(), filename)):
+            self._load_recordings(filename)
+        else:
+            self.real_client = http_client
+
+    def use_cached_session(self, name=None, real_http_client=None):
+        """Attempts to load recordings from a previous live request.
+
+        If a temp file with the recordings exists, then it is used to fulfill
+        requests. If the file does not exist, then a real client is used to
+        actually make the desired HTTP requests. Requests and responses are
+        recorded and will be written to the desired temprary cache file when
+        close_session is called.
+
+        Args:
+          name: str (optional) The file name of session file to be used. The file
+                is loaded from the temporary directory of this machine. If no name
+                is passed in, a default name will be constructed using the
+                cache_name_prefix, cache_case_name, and cache_test_name of this
+                object.
+          real_http_client: atom.http_core.HttpClient the real client to be used
+                            if the cached recordings are not found. If the default
+                            value is used, this will be an
+                            atom.http_core.HttpClient.
+        """
+        if real_http_client is None:
+            real_http_client = atom.http_core.HttpClient()
+        if name is None:
+            self._recordings_cache_name = self.get_cache_file_name()
+        else:
+            self._recordings_cache_name = name
+        self._load_or_use_client(self._recordings_cache_name, real_http_client)
+
+    def close_session(self):
+        """Saves recordings in the temporary file named in use_cached_session."""
+        if self.real_client is not None:
+            self._save_recordings(self._recordings_cache_name)
+
+    def delete_session(self, name=None):
+        """Removes recordings from a previous live request."""
+        if name is None:
+            self._delete_recordings(self._recordings_cache_name)
+        else:
+            self._delete_recordings(name)
+
+    def get_cache_file_name(self):
+        return '%s.%s.%s' % (self.cache_name_prefix, self.cache_case_name,
+                             self.cache_test_name)
+
+    def _dump(self):
+        """Provides debug information in a string."""
+        output = 'MockHttpClient\n  real_client: %s\n  cache file name: %s\n' % (
+            self.real_client, self.get_cache_file_name())
+        output += '  recordings:\n'
+        i = 0
+        for recording in self._recordings:
+            output += '    recording %i is for: %s %s\n' % (
+                i, recording[0].method, str(recording[0].uri))
+            i += 1
+        return output
+
+
+def _match_request(http_request, stored_request):
+    """Determines whether a request is similar enough to a stored request
+       to cause the stored response to be returned."""
+    # Check to see if the host names match.
+    if (http_request.uri.host is not None
+        and http_request.uri.host != stored_request.uri.host):
+        return False
+    # Check the request path in the URL (/feeds/private/full/x)
+    elif http_request.uri.path != stored_request.uri.path:
+        return False
+    # Check the method used in the request (GET, POST, etc.)
+    elif http_request.method != stored_request.method:
+        return False
+    # If there is a gsession ID in either request, make sure that it is matched
+    # exactly.
+    elif ('gsessionid' in http_request.uri.query
+          or 'gsessionid' in stored_request.uri.query):
+        if 'gsessionid' not in stored_request.uri.query:
+            return False
+        elif 'gsessionid' not in http_request.uri.query:
+            return False
+        elif (http_request.uri.query['gsessionid']
+                  != stored_request.uri.query['gsessionid']):
+            return False
+    # Ignores differences in the query params (?start-index=5&max-results=20),
+    # the body of the request, the port number, HTTP headers, just to name a
+    # few.
+    return True
+
+
+def _scrub_request(http_request):
+    """ Removes email address and password from a client login request.
+
+    Since the mock server saves the request and response in plantext, sensitive
+    information like the password should be removed before saving the
+    recordings. At the moment only requests sent to a ClientLogin url are
+    scrubbed.
+    """
+    if (http_request and http_request.uri and http_request.uri.path and
+            http_request.uri.path.endswith('ClientLogin')):
+        # Remove the email and password from a ClientLogin request.
+        http_request._body_parts = []
+        http_request.add_form_inputs(
+            {'form_data': 'client login request has been scrubbed'})
+    else:
+        # We can remove the body of the post from the recorded request, since
+        # the request body is not used when finding a matching recording.
+        http_request._body_parts = []
+    return http_request
+
+
+def _scrub_response(http_response):
+    return http_response
+
+
+class EchoHttpClient(object):
+    """Sends the request data back in the response.
+
+    Used to check the formatting of the request as it was sent. Always responds
+    with a 200 OK, and some information from the HTTP request is returned in
+    special Echo-X headers in the response. The following headers are added
+    in the response:
+    'Echo-Host': The host name and port number to which the HTTP connection is
+                 made. If no port was passed in, the header will contain
+                 host:None.
+    'Echo-Uri': The path portion of the URL being requested. /example?x=1&y=2
+    'Echo-Scheme': The beginning of the URL, usually 'http' or 'https'
+    'Echo-Method': The HTTP method being used, 'GET', 'POST', 'PUT', etc.
+    """
+
+    def request(self, http_request):
+        return self._http_request(http_request.uri, http_request.method,
+                                  http_request.headers, http_request._body_parts)
+
+    def _http_request(self, uri, method, headers=None, body_parts=None):
+        body = io.StringIO()
+        response = atom.http_core.HttpResponse(status=200, reason='OK', body=body)
+        if headers is None:
+            response._headers = {}
+        else:
+            # Copy headers from the request to the response but convert values to
+            # strings. Server response headers always come in as strings, so an int
+            # should be converted to a corresponding string when echoing.
+            for header, value in headers.items():
+                response._headers[header] = str(value)
+        response._headers['Echo-Host'] = '%s:%s' % (uri.host, str(uri.port))
+        response._headers['Echo-Uri'] = uri._get_relative_path()
+        response._headers['Echo-Scheme'] = uri.scheme
+        response._headers['Echo-Method'] = method
+        for part in body_parts:
+            if isinstance(part, str):
+                body.write(part)
+            elif hasattr(part, 'read'):
+                body.write(part.read())
+        body.seek(0)
+        return response
+
+
+class SettableHttpClient(object):
+    """An HTTP Client which responds with the data given in set_response."""
+
+    def __init__(self, status, reason, body, headers):
+        """Configures the response for the server.
+
+        See set_response for details on the arguments to the constructor.
+        """
+        self.set_response(status, reason, body, headers)
+        self.last_request = None
+
+    def set_response(self, status, reason, body, headers):
+        """Determines the response which will be sent for each request.
+
+        Args:
+          status: An int for the HTTP status code, example: 200, 404, etc.
+          reason: String for the HTTP reason, example: OK, NOT FOUND, etc.
+          body: The body of the HTTP response as a string or a file-like
+                object (something with a read method).
+          headers: dict of strings containing the HTTP headers in the response.
+        """
+        self.response = atom.http_core.HttpResponse(status=status, reason=reason,
+                                                    body=body)
+        self.response._headers = headers.copy()
+
+    def request(self, http_request):
+        self.last_request = http_request
+        return self.response
+
+
+class MockHttpResponse(atom.http_core.HttpResponse):
+    def __init__(self, status=None, reason=None, headers=None, body=None):
+        self._headers = headers or {}
+        if status is not None:
+            self.status = status
+        if reason is not None:
+            self.reason = reason
+        if body is not None:
+            # Instead of using a file-like object for the body, store as a string
+            # so that reads can be repeated.
+            if hasattr(body, 'read'):
+                self._body = body.read()
+            else:
+                self._body = body
+
+    def read(self):
+        return self._body

src/gam/atom/mock_service.py

@@ -0,0 +1,235 @@
+#
+# Copyright (C) 2008 Google Inc.
+#
+# Licensed under the Apache License 2.0;
+
+
+
+"""MockService provides CRUD ops. for mocking calls to AtomPub services.
+
+  MockService: Exposes the publicly used methods of AtomService to provide
+      a mock interface which can be used in unit tests.
+"""
+
+import pickle
+
+import atom.service
+
+# __author__ = 'api.jscudder (Jeffrey Scudder)'
+
+# Recordings contains pairings of HTTP MockRequest objects with MockHttpResponse objects.
+recordings = []
+# If set, the mock service HttpRequest are actually made through this object.
+real_request_handler = None
+
+
+def ConcealValueWithSha(source):
+    import sha
+    return sha.new(source[:-5]).hexdigest()
+
+
+def DumpRecordings(conceal_func=ConcealValueWithSha):
+    if conceal_func:
+        for recording_pair in recordings:
+            recording_pair[0].ConcealSecrets(conceal_func)
+    return pickle.dumps(recordings)
+
+
+def LoadRecordings(recordings_file_or_string):
+    if isinstance(recordings_file_or_string, str):
+        atom.mock_service.recordings = pickle.loads(recordings_file_or_string)
+    elif hasattr(recordings_file_or_string, 'read'):
+        atom.mock_service.recordings = pickle.loads(
+            recordings_file_or_string.read())
+
+
+def HttpRequest(service, operation, data, uri, extra_headers=None,
+                url_params=None, escape_params=True, content_type='application/atom+xml'):
+    """Simulates an HTTP call to the server, makes an actual HTTP request if
+    real_request_handler is set.
+
+    This function operates in two different modes depending on if
+    real_request_handler is set or not. If real_request_handler is not set,
+    HttpRequest will look in this module's recordings list to find a response
+    which matches the parameters in the function call. If real_request_handler
+    is set, this function will call real_request_handler.HttpRequest, add the
+    response to the recordings list, and respond with the actual response.
+
+    Args:
+      service: atom.AtomService object which contains some of the parameters
+          needed to make the request. The following members are used to
+          construct the HTTP call: server (str), additional_headers (dict),
+          port (int), and ssl (bool).
+      operation: str The HTTP operation to be performed. This is usually one of
+          'GET', 'POST', 'PUT', or 'DELETE'
+      data: ElementTree, filestream, list of parts, or other object which can be
+          converted to a string.
+          Should be set to None when performing a GET or PUT.
+          If data is a file-like object which can be read, this method will read
+          a chunk of 100K bytes at a time and send them.
+          If the data is a list of parts to be sent, each part will be evaluated
+          and sent.
+      uri: The beginning of the URL to which the request should be sent.
+          Examples: '/', '/base/feeds/snippets',
+          '/m8/feeds/contacts/default/base'
+      extra_headers: dict of strings. HTTP headers which should be sent
+          in the request. These headers are in addition to those stored in
+          service.additional_headers.
+      url_params: dict of strings. Key value pairs to be added to the URL as
+          URL parameters. For example {'foo':'bar', 'test':'param'} will
+          become ?foo=bar&test=param.
+      escape_params: bool default True. If true, the keys and values in
+          url_params will be URL escaped when the form is constructed
+          (Special characters converted to %XX form.)
+      content_type: str The MIME type for the data being sent. Defaults to
+          'application/atom+xml', this is only used if data is set.
+    """
+    full_uri = atom.service.BuildUri(uri, url_params, escape_params)
+    (server, port, ssl, uri) = atom.service.ProcessUrl(service, uri)
+    current_request = MockRequest(operation, full_uri, host=server, ssl=ssl,
+                                  data=data, extra_headers=extra_headers, url_params=url_params,
+                                  escape_params=escape_params, content_type=content_type)
+    # If the request handler is set, we should actually make the request using
+    # the request handler and record the response to replay later.
+    if real_request_handler:
+        response = real_request_handler.HttpRequest(service, operation, data, uri,
+                                                    extra_headers=extra_headers, url_params=url_params,
+                                                    escape_params=escape_params, content_type=content_type)
+        # TODO: need to copy the HTTP headers from the real response into the
+        # recorded_response.
+        recorded_response = MockHttpResponse(body=response.read(),
+                                             status=response.status, reason=response.reason)
+        # Insert a tuple which maps the request to the response object returned
+        # when making an HTTP call using the real_request_handler.
+        recordings.append((current_request, recorded_response))
+        return recorded_response
+    else:
+        # Look through available recordings to see if one matches the current
+        # request.
+        for request_response_pair in recordings:
+            if request_response_pair[0].IsMatch(current_request):
+                return request_response_pair[1]
+    return None
+
+
+class MockRequest(object):
+    """Represents a request made to an AtomPub server.
+
+    These objects are used to determine if a client request matches a recorded
+    HTTP request to determine what the mock server's response will be.
+    """
+
+    def __init__(self, operation, uri, host=None, ssl=False, port=None,
+                 data=None, extra_headers=None, url_params=None, escape_params=True,
+                 content_type='application/atom+xml'):
+        """Constructor for a MockRequest
+
+        Args:
+          operation: str One of 'GET', 'POST', 'PUT', or 'DELETE' this is the
+              HTTP operation requested on the resource.
+          uri: str The URL describing the resource to be modified or feed to be
+              retrieved. This should include the protocol (http/https) and the host
+              (aka domain). For example, these are some valud full_uris:
+              'http://example.com', 'https://www.google.com/accounts/ClientLogin'
+          host: str (optional) The server name which will be placed at the
+              beginning of the URL if the uri parameter does not begin with 'http'.
+              Examples include 'example.com', 'www.google.com', 'www.blogger.com'.
+          ssl: boolean (optional) If true, the request URL will begin with https
+              instead of http.
+          data: ElementTree, filestream, list of parts, or other object which can be
+              converted to a string. (optional)
+              Should be set to None when performing a GET or PUT.
+              If data is a file-like object which can be read, the constructor
+              will read the entire file into memory. If the data is a list of
+              parts to be sent, each part will be evaluated and stored.
+          extra_headers: dict (optional) HTTP headers included in the request.
+          url_params: dict (optional) Key value pairs which should be added to
+              the URL as URL parameters in the request. For example uri='/',
+              url_parameters={'foo':'1','bar':'2'} could become '/?foo=1&bar=2'.
+          escape_params: boolean (optional) Perform URL escaping on the keys and
+              values specified in url_params. Defaults to True.
+          content_type: str (optional) Provides the MIME type of the data being
+              sent.
+        """
+        self.operation = operation
+        self.uri = _ConstructFullUrlBase(uri, host=host, ssl=ssl)
+        self.data = data
+        self.extra_headers = extra_headers
+        self.url_params = url_params or {}
+        self.escape_params = escape_params
+        self.content_type = content_type
+
+    def ConcealSecrets(self, conceal_func):
+        """Conceal secret data in this request."""
+        if 'Authorization' in self.extra_headers:
+            self.extra_headers['Authorization'] = conceal_func(
+                self.extra_headers['Authorization'])
+
+    def IsMatch(self, other_request):
+        """Check to see if the other_request is equivalent to this request.
+
+        Used to determine if a recording matches an incoming request so that a
+        recorded response should be sent to the client.
+
+        The matching is not exact, only the operation and URL are examined
+        currently.
+
+        Args:
+          other_request: MockRequest The request which we want to check this
+              (self) MockRequest against to see if they are equivalent.
+        """
+        # More accurate matching logic will likely be required.
+        return (self.operation == other_request.operation and self.uri ==
+                other_request.uri)
+
+
+def _ConstructFullUrlBase(uri, host=None, ssl=False):
+    """Puts URL components into the form http(s)://full.host.strinf/uri/path
+
+    Used to construct a roughly canonical URL so that URLs which begin with
+    'http://example.com/' can be compared to a uri of '/' when the host is
+    set to 'example.com'
+
+    If the uri contains 'http://host' already, the host and ssl parameters
+    are ignored.
+
+    Args:
+      uri: str The path component of the URL, examples include '/'
+      host: str (optional) The host name which should prepend the URL. Example:
+          'example.com'
+      ssl: boolean (optional) If true, the returned URL will begin with https
+          instead of http.
+
+    Returns:
+      String which has the form http(s)://example.com/uri/string/contents
+    """
+    if uri.startswith('http'):
+        return uri
+    if ssl:
+        return 'https://%s%s' % (host, uri)
+    else:
+        return 'http://%s%s' % (host, uri)
+
+
+class MockHttpResponse(object):
+    """Returned from MockService crud methods as the server's response."""
+
+    def __init__(self, body=None, status=None, reason=None, headers=None):
+        """Construct a mock HTTPResponse and set members.
+
+        Args:
+          body: str (optional) The HTTP body of the server's response.
+          status: int (optional)
+          reason: str (optional)
+          headers: dict (optional)
+        """
+        self.body = body
+        self.status = status
+        self.reason = reason
+        self.headers = headers or {}
+
+    def read(self):
+        return self.body
+
+    def getheader(self, header_name):
+        return self.headers[header_name]

src/gam/atom/service.py

@@ -0,0 +1,723 @@
+#
+# Copyright (C) 2006, 2007, 2008 Google Inc.
+#
+# Licensed under the Apache License 2.0;
+
+
+
+"""AtomService provides CRUD ops. in line with the Atom Publishing Protocol.
+
+  AtomService: Encapsulates the ability to perform insert, update and delete
+               operations with the Atom Publishing Protocol on which GData is
+               based. An instance can perform query, insertion, deletion, and
+               update.
+
+  HttpRequest: Function that performs a GET, POST, PUT, or DELETE HTTP request
+       to the specified end point. An AtomService object or a subclass can be
+       used to specify information about the request.
+"""
+
+# __author__ = 'api.jscudder (Jeff Scudder)'
+
+import base64
+import http.client
+import os
+import socket
+import urllib.error
+import urllib.parse
+import urllib.request
+import warnings
+
+import atom.http
+import atom.http_interface
+import atom.token_store
+import atom.url
+
+import lxml.etree as ElementTree
+import atom
+
+
+class AtomService(object):
+    """Performs Atom Publishing Protocol CRUD operations.
+
+    The AtomService contains methods to perform HTTP CRUD operations.
+    """
+
+    # Default values for members
+    port = 80
+    ssl = False
+    # Set the current_token to force the AtomService to use this token
+    # instead of searching for an appropriate token in the token_store.
+    current_token = None
+    auto_store_tokens = True
+    auto_set_current_token = True
+
+    def _get_override_token(self):
+        return self.current_token
+
+    def _set_override_token(self, token):
+        self.current_token = token
+
+    override_token = property(_get_override_token, _set_override_token)
+
+    # @atom.v1_deprecated('Please use atom.client.AtomPubClient instead.')
+    def __init__(self, server=None, additional_headers=None,
+                 application_name='', http_client=None, token_store=None):
+        """Creates a new AtomService client.
+
+        Args:
+          server: string (optional) The start of a URL for the server
+                  to which all operations should be directed. Example:
+                  'www.google.com'
+          additional_headers: dict (optional) Any additional HTTP headers which
+                              should be included with CRUD operations.
+          http_client: An object responsible for making HTTP requests using a
+                       request method. If none is provided, a new instance of
+                       atom.http.ProxiedHttpClient will be used.
+          token_store: Keeps a collection of authorization tokens which can be
+                       applied to requests for a specific URLs. Critical methods are
+                       find_token based on a URL (atom.url.Url or a string), add_token,
+                       and remove_token.
+        """
+        self.http_client = http_client or atom.http.ProxiedHttpClient()
+        self.token_store = token_store or atom.token_store.TokenStore()
+        self.server = server
+        self.additional_headers = additional_headers or {}
+        self.additional_headers['User-Agent'] = atom.http_interface.USER_AGENT % (
+            application_name,)
+        # If debug is True, the HTTPConnection will display debug information
+        self._set_debug(False)
+
+    __init__ = atom.v1_deprecated(
+        'Please use atom.client.AtomPubClient instead.')(
+        __init__)
+
+    def _get_debug(self):
+        return self.http_client.debug
+
+    def _set_debug(self, value):
+        self.http_client.debug = value
+
+    debug = property(_get_debug, _set_debug,
+                     doc='If True, HTTP debug information is printed.')
+
+    def use_basic_auth(self, username, password, scopes=None):
+        if username is not None and password is not None:
+            if scopes is None:
+                scopes = [atom.token_store.SCOPE_ALL]
+            base_64_string = base64.encodestring('%s:%s' % (username, password))
+            token = BasicAuthToken('Basic %s' % base_64_string.strip(),
+                                   scopes=[atom.token_store.SCOPE_ALL])
+            if self.auto_set_current_token:
+                self.current_token = token
+            if self.auto_store_tokens:
+                return self.token_store.add_token(token)
+            return True
+        return False
+
+    def UseBasicAuth(self, username, password, for_proxy=False):
+        """Sets an Authenticaiton: Basic HTTP header containing plaintext.
+
+        Deprecated, use use_basic_auth instead.
+
+        The username and password are base64 encoded and added to an HTTP header
+        which will be included in each request. Note that your username and
+        password are sent in plaintext.
+
+        Args:
+          username: str
+          password: str
+        """
+        self.use_basic_auth(username, password)
+
+    # @atom.v1_deprecated('Please use atom.client.AtomPubClient for requests.')
+    def request(self, operation, url, data=None, headers=None,
+                url_params=None):
+        if isinstance(url, str):
+            if url.startswith('http:') and self.ssl:
+                # Force all requests to be https if self.ssl is True.
+                url = atom.url.parse_url('https:' + url[5:])
+            elif not url.startswith('http') and self.ssl:
+                url = atom.url.parse_url('https://%s%s' % (self.server, url))
+            elif not url.startswith('http'):
+                url = atom.url.parse_url('http://%s%s' % (self.server, url))
+            else:
+                url = atom.url.parse_url(url)
+
+        if url_params:
+            for name, value in url_params.items():
+                url.params[name] = value
+
+        all_headers = self.additional_headers.copy()
+        if headers:
+            all_headers.update(headers)
+
+        # If the list of headers does not include a Content-Length, attempt to
+        # calculate it based on the data object.
+        if data and 'Content-Length' not in all_headers:
+            content_length = CalculateDataLength(data)
+            if content_length:
+                all_headers['Content-Length'] = str(content_length)
+
+        # Find an Authorization token for this URL if one is available.
+        if self.override_token:
+            auth_token = self.override_token
+        else:
+            auth_token = self.token_store.find_token(url)
+        return auth_token.perform_request(self.http_client, operation, url,
+                                          data=data, headers=all_headers)
+
+    request = atom.v1_deprecated(
+        'Please use atom.client.AtomPubClient for requests.')(
+        request)
+
+    # CRUD operations
+    def Get(self, uri, extra_headers=None, url_params=None, escape_params=True):
+        """Query the APP server with the given URI
+
+        The uri is the portion of the URI after the server value
+        (server example: 'www.google.com').
+
+        Example use:
+        To perform a query against Google Base, set the server to
+        'base.google.com' and set the uri to '/base/feeds/...', where ... is
+        your query. For example, to find snippets for all digital cameras uri
+        should be set to: '/base/feeds/snippets?bq=digital+camera'
+
+        Args:
+          uri: string The query in the form of a URI. Example:
+               '/base/feeds/snippets?bq=digital+camera'.
+          extra_headers: dicty (optional) Extra HTTP headers to be included
+                         in the GET request. These headers are in addition to
+                         those stored in the client's additional_headers property.
+                         The client automatically sets the Content-Type and
+                         Authorization headers.
+          url_params: dict (optional) Additional URL parameters to be included
+                      in the query. These are translated into query arguments
+                      in the form '&dict_key=value&...'.
+                      Example: {'max-results': '250'} becomes &max-results=250
+          escape_params: boolean (optional) If false, the calling code has already
+                         ensured that the query will form a valid URL (all
+                         reserved characters have been escaped). If true, this
+                         method will escape the query and any URL parameters
+                         provided.
+
+        Returns:
+          httplib.HTTPResponse The server's response to the GET request.
+        """
+        return self.request('GET', uri, data=None, headers=extra_headers,
+                            url_params=url_params)
+
+    def Post(self, data, uri, extra_headers=None, url_params=None,
+             escape_params=True, content_type='application/atom+xml'):
+        """Insert data into an APP server at the given URI.
+
+        Args:
+          data: string, ElementTree._Element, or something with a __str__ method
+                The XML to be sent to the uri.
+          uri: string The location (feed) to which the data should be inserted.
+               Example: '/base/feeds/items'.
+          extra_headers: dict (optional) HTTP headers which are to be included.
+                         The client automatically sets the Content-Type,
+                         Authorization, and Content-Length headers.
+          url_params: dict (optional) Additional URL parameters to be included
+                      in the URI. These are translated into query arguments
+                      in the form '&dict_key=value&...'.
+                      Example: {'max-results': '250'} becomes &max-results=250
+          escape_params: boolean (optional) If false, the calling code has already
+                         ensured that the query will form a valid URL (all
+                         reserved characters have been escaped). If true, this
+                         method will escape the query and any URL parameters
+                         provided.
+
+        Returns:
+          httplib.HTTPResponse Server's response to the POST request.
+        """
+        if extra_headers is None:
+            extra_headers = {}
+        if content_type:
+            extra_headers['Content-Type'] = content_type
+        return self.request('POST', uri, data=data, headers=extra_headers,
+                            url_params=url_params)
+
+    def Put(self, data, uri, extra_headers=None, url_params=None,
+            escape_params=True, content_type='application/atom+xml'):
+        """Updates an entry at the given URI.
+
+        Args:
+          data: string, ElementTree._Element, or xml_wrapper.ElementWrapper The
+                XML containing the updated data.
+          uri: string A URI indicating entry to which the update will be applied.
+               Example: '/base/feeds/items/ITEM-ID'
+          extra_headers: dict (optional) HTTP headers which are to be included.
+                         The client automatically sets the Content-Type,
+                         Authorization, and Content-Length headers.
+          url_params: dict (optional) Additional URL parameters to be included
+                      in the URI. These are translated into query arguments
+                      in the form '&dict_key=value&...'.
+                      Example: {'max-results': '250'} becomes &max-results=250
+          escape_params: boolean (optional) If false, the calling code has already
+                         ensured that the query will form a valid URL (all
+                         reserved characters have been escaped). If true, this
+                         method will escape the query and any URL parameters
+                         provided.
+
+        Returns:
+          httplib.HTTPResponse Server's response to the PUT request.
+        """
+        if extra_headers is None:
+            extra_headers = {}
+        if content_type:
+            extra_headers['Content-Type'] = content_type
+        return self.request('PUT', uri, data=data, headers=extra_headers,
+                            url_params=url_params)
+
+    def Delete(self, uri, extra_headers=None, url_params=None,
+               escape_params=True):
+        """Deletes the entry at the given URI.
+
+        Args:
+          uri: string The URI of the entry to be deleted. Example:
+               '/base/feeds/items/ITEM-ID'
+          extra_headers: dict (optional) HTTP headers which are to be included.
+                         The client automatically sets the Content-Type and
+                         Authorization headers.
+          url_params: dict (optional) Additional URL parameters to be included
+                      in the URI. These are translated into query arguments
+                      in the form '&dict_key=value&...'.
+                      Example: {'max-results': '250'} becomes &max-results=250
+          escape_params: boolean (optional) If false, the calling code has already
+                         ensured that the query will form a valid URL (all
+                         reserved characters have been escaped). If true, this
+                         method will escape the query and any URL parameters
+                         provided.
+
+        Returns:
+          httplib.HTTPResponse Server's response to the DELETE request.
+        """
+        return self.request('DELETE', uri, data=None, headers=extra_headers,
+                            url_params=url_params)
+
+
+class BasicAuthToken(atom.http_interface.GenericToken):
+    def __init__(self, auth_header, scopes=None):
+        """Creates a token used to add Basic Auth headers to HTTP requests.
+
+        Args:
+          auth_header: str The value for the Authorization header.
+          scopes: list of str or atom.url.Url specifying the beginnings of URLs
+              for which this token can be used. For example, if scopes contains
+              'http://example.com/foo', then this token can be used for a request to
+              'http://example.com/foo/bar' but it cannot be used for a request to
+              'http://example.com/baz'
+        """
+        self.auth_header = auth_header
+        self.scopes = scopes or []
+
+    def perform_request(self, http_client, operation, url, data=None,
+                        headers=None):
+        """Sets the Authorization header to the basic auth string."""
+        if headers is None:
+            headers = {'Authorization': self.auth_header}
+        else:
+            headers['Authorization'] = self.auth_header
+        return http_client.request(operation, url, data=data, headers=headers)
+
+    def __str__(self):
+        return self.auth_header
+
+    def valid_for_scope(self, url):
+        """Tells the caller if the token authorizes access to the desired URL.
+        """
+        if isinstance(url, str):
+            url = atom.url.parse_url(url)
+        for scope in self.scopes:
+            if scope == atom.token_store.SCOPE_ALL:
+                return True
+            if isinstance(scope, str):
+                scope = atom.url.parse_url(scope)
+            if scope == url:
+                return True
+            # Check the host and the path, but ignore the port and protocol.
+            elif scope.host == url.host and not scope.path:
+                return True
+            elif scope.host == url.host and scope.path and not url.path:
+                continue
+            elif scope.host == url.host and url.path.startswith(scope.path):
+                return True
+        return False
+
+
+def PrepareConnection(service, full_uri):
+    """Opens a connection to the server based on the full URI.
+
+    This method is deprecated, instead use atom.http.HttpClient.request.
+
+    Examines the target URI and the proxy settings, which are set as
+    environment variables, to open a connection with the server. This
+    connection is used to make an HTTP request.
+
+    Args:
+      service: atom.AtomService or a subclass. It must have a server string which
+        represents the server host to which the request should be made. It may also
+        have a dictionary of additional_headers to send in the HTTP request.
+      full_uri: str Which is the target relative (lacks protocol and host) or
+      absolute URL to be opened. Example:
+      'https://www.google.com/accounts/ClientLogin' or
+      'base/feeds/snippets' where the server is set to www.google.com.
+
+    Returns:
+      A tuple containing the httplib.HTTPConnection and the full_uri for the
+      request.
+    """
+    deprecation('calling deprecated function PrepareConnection')
+    (server, port, ssl, partial_uri) = ProcessUrl(service, full_uri)
+    if ssl:
+        # destination is https
+        proxy = os.environ.get('https_proxy')
+        if proxy:
+            (p_server, p_port, p_ssl, p_uri) = ProcessUrl(service, proxy, True)
+            proxy_username = os.environ.get('proxy-username')
+            if not proxy_username:
+                proxy_username = os.environ.get('proxy_username')
+            proxy_password = os.environ.get('proxy-password')
+            if not proxy_password:
+                proxy_password = os.environ.get('proxy_password')
+            if proxy_username:
+                user_auth = base64.encodestring('%s:%s' % (proxy_username,
+                                                           proxy_password))
+                proxy_authorization = ('Proxy-authorization: Basic %s\r\n' % (
+                    user_auth.strip()))
+            else:
+                proxy_authorization = ''
+            proxy_connect = 'CONNECT %s:%s HTTP/1.0\r\n' % (server, port)
+            user_agent = 'User-Agent: %s\r\n' % (
+                service.additional_headers['User-Agent'])
+            proxy_pieces = (proxy_connect + proxy_authorization + user_agent
+                            + '\r\n')
+
+            # now connect, very simple recv and error checking
+            p_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            p_sock.connect((p_server, p_port))
+            p_sock.sendall(proxy_pieces)
+            response = ''
+
+            # Wait for the full response.
+            while response.find("\r\n\r\n") == -1:
+                response += p_sock.recv(8192)
+
+            p_status = response.split()[1]
+            if p_status != str(200):
+                raise atom.http.ProxyError('Error status=%s' % p_status)
+
+            # Trivial setup for ssl socket.
+            ssl = socket.ssl(p_sock, None, None)
+            fake_sock = http.client.FakeSocket(p_sock, ssl)
+
+            # Initalize httplib and replace with the proxy socket.
+            connection = http.client.HTTPConnection(server)
+            connection.sock = fake_sock
+            full_uri = partial_uri
+
+        else:
+            connection = http.client.HTTPSConnection(server, port)
+            full_uri = partial_uri
+
+    else:
+        # destination is http
+        proxy = os.environ.get('http_proxy')
+        if proxy:
+            (p_server, p_port, p_ssl, p_uri) = ProcessUrl(service.server, proxy, True)
+            proxy_username = os.environ.get('proxy-username')
+            if not proxy_username:
+                proxy_username = os.environ.get('proxy_username')
+            proxy_password = os.environ.get('proxy-password')
+            if not proxy_password:
+                proxy_password = os.environ.get('proxy_password')
+            if proxy_username:
+                UseBasicAuth(service, proxy_username, proxy_password, True)
+            connection = http.client.HTTPConnection(p_server, p_port)
+            if not full_uri.startswith("http://"):
+                if full_uri.startswith("/"):
+                    full_uri = "http://%s%s" % (service.server, full_uri)
+                else:
+                    full_uri = "http://%s/%s" % (service.server, full_uri)
+        else:
+            connection = http.client.HTTPConnection(server, port)
+            full_uri = partial_uri
+
+    return (connection, full_uri)
+
+
+def UseBasicAuth(service, username, password, for_proxy=False):
+    """Sets an Authenticaiton: Basic HTTP header containing plaintext.
+
+    Deprecated, use AtomService.use_basic_auth insread.
+
+    The username and password are base64 encoded and added to an HTTP header
+    which will be included in each request. Note that your username and
+    password are sent in plaintext. The auth header is added to the
+    additional_headers dictionary in the service object.
+
+    Args:
+      service: atom.AtomService or a subclass which has an
+          additional_headers dict as a member.
+      username: str
+      password: str
+    """
+    deprecation('calling deprecated function UseBasicAuth')
+    base_64_string = base64.encodestring('%s:%s' % (username, password))
+    base_64_string = base_64_string.strip()
+    if for_proxy:
+        header_name = 'Proxy-Authorization'
+    else:
+        header_name = 'Authorization'
+    service.additional_headers[header_name] = 'Basic %s' % (base_64_string,)
+
+
+def ProcessUrl(service, url, for_proxy=False):
+    """Processes a passed URL.  If the URL does not begin with https?, then
+    the default value for server is used
+
+    This method is deprecated, use atom.url.parse_url instead.
+    """
+    if not isinstance(url, atom.url.Url):
+        url = atom.url.parse_url(url)
+
+    server = url.host
+    ssl = False
+    port = 80
+
+    if not server:
+        if hasattr(service, 'server'):
+            server = service.server
+        else:
+            server = service
+        if not url.protocol and hasattr(service, 'ssl'):
+            ssl = service.ssl
+        if hasattr(service, 'port'):
+            port = service.port
+    else:
+        if url.protocol == 'https':
+            ssl = True
+        elif url.protocol == 'http':
+            ssl = False
+        if url.port:
+            port = int(url.port)
+        elif port == 80 and ssl:
+            port = 443
+
+    return (server, port, ssl, url.get_request_uri())
+
+
+def DictionaryToParamList(url_parameters, escape_params=True):
+    """Convert a dictionary of URL arguments into a URL parameter string.
+
+    This function is deprcated, use atom.url.Url instead.
+
+    Args:
+      url_parameters: The dictionaty of key-value pairs which will be converted
+                      into URL parameters. For example,
+                      {'dry-run': 'true', 'foo': 'bar'}
+                      will become ['dry-run=true', 'foo=bar'].
+
+    Returns:
+      A list which contains a string for each key-value pair. The strings are
+      ready to be incorporated into a URL by using '&'.join([] + parameter_list)
+    """
+    # Choose which function to use when modifying the query and parameters.
+    # Use quote_plus when escape_params is true.
+    transform_op = [str, urllib.parse.quote_plus][bool(escape_params)]
+    # Create a list of tuples containing the escaped version of the
+    # parameter-value pairs.
+    parameter_tuples = [(transform_op(param), transform_op(value))
+                        for param, value in list((url_parameters or {}).items())]
+    # Turn parameter-value tuples into a list of strings in the form
+    # 'PARAMETER=VALUE'.
+    return ['='.join(x) for x in parameter_tuples]
+
+
+def BuildUri(uri, url_params=None, escape_params=True):
+    """Converts a uri string and a collection of parameters into a URI.
+
+    This function is deprcated, use atom.url.Url instead.
+
+    Args:
+      uri: string
+      url_params: dict (optional)
+      escape_params: boolean (optional)
+      uri: string The start of the desired URI. This string can alrady contain
+           URL parameters. Examples: '/base/feeds/snippets',
+           '/base/feeds/snippets?bq=digital+camera'
+      url_parameters: dict (optional) Additional URL parameters to be included
+                      in the query. These are translated into query arguments
+                      in the form '&dict_key=value&...'.
+                      Example: {'max-results': '250'} becomes &max-results=250
+      escape_params: boolean (optional) If false, the calling code has already
+                     ensured that the query will form a valid URL (all
+                     reserved characters have been escaped). If true, this
+                     method will escape the query and any URL parameters
+                     provided.
+
+    Returns:
+      string The URI consisting of the escaped URL parameters appended to the
+      initial uri string.
+    """
+    # Prepare URL parameters for inclusion into the GET request.
+    parameter_list = DictionaryToParamList(url_params, escape_params)
+
+    # Append the URL parameters to the URL.
+    if parameter_list:
+        if uri.find('?') != -1:
+            # If there are already URL parameters in the uri string, add the
+            # parameters after a new & character.
+            full_uri = '&'.join([uri] + parameter_list)
+        else:
+            # The uri string did not have any URL parameters (no ? character)
+            # so put a ? between the uri and URL parameters.
+            full_uri = '%s%s' % (uri, '?%s' % ('&'.join([] + parameter_list)))
+    else:
+        full_uri = uri
+
+    return full_uri
+
+
+def HttpRequest(service, operation, data, uri, extra_headers=None,
+                url_params=None, escape_params=True, content_type='application/atom+xml'):
+    """Performs an HTTP call to the server, supports GET, POST, PUT, and DELETE.
+
+    This method is deprecated, use atom.http.HttpClient.request instead.
+
+    Usage example, perform and HTTP GET on http://www.google.com/:
+      import atom.service
+      client = atom.service.AtomService()
+      http_response = client.Get('http://www.google.com/')
+    or you could set the client.server to 'www.google.com' and use the
+    following:
+      client.server = 'www.google.com'
+      http_response = client.Get('/')
+
+    Args:
+      service: atom.AtomService object which contains some of the parameters
+          needed to make the request. The following members are used to
+          construct the HTTP call: server (str), additional_headers (dict),
+          port (int), and ssl (bool).
+      operation: str The HTTP operation to be performed. This is usually one of
+          'GET', 'POST', 'PUT', or 'DELETE'
+      data: ElementTree, filestream, list of parts, or other object which can be
+          converted to a string.
+          Should be set to None when performing a GET or PUT.
+          If data is a file-like object which can be read, this method will read
+          a chunk of 100K bytes at a time and send them.
+          If the data is a list of parts to be sent, each part will be evaluated
+          and sent.
+      uri: The beginning of the URL to which the request should be sent.
+          Examples: '/', '/base/feeds/snippets',
+          '/m8/feeds/contacts/default/base'
+      extra_headers: dict of strings. HTTP headers which should be sent
+          in the request. These headers are in addition to those stored in
+          service.additional_headers.
+      url_params: dict of strings. Key value pairs to be added to the URL as
+          URL parameters. For example {'foo':'bar', 'test':'param'} will
+          become ?foo=bar&test=param.
+      escape_params: bool default True. If true, the keys and values in
+          url_params will be URL escaped when the form is constructed
+          (Special characters converted to %XX form.)
+      content_type: str The MIME type for the data being sent. Defaults to
+          'application/atom+xml', this is only used if data is set.
+    """
+    deprecation('call to deprecated function HttpRequest')
+    full_uri = BuildUri(uri, url_params, escape_params)
+    (connection, full_uri) = PrepareConnection(service, full_uri)
+
+    if extra_headers is None:
+        extra_headers = {}
+
+    # Turn on debug mode if the debug member is set.
+    if service.debug:
+        connection.debuglevel = 1
+
+    connection.putrequest(operation, full_uri)
+
+    # If the list of headers does not include a Content-Length, attempt to
+    # calculate it based on the data object.
+    if (data and 'Content-Length' not in service.additional_headers and
+                'Content-Length' not in extra_headers):
+        content_length = CalculateDataLength(data)
+        if content_length:
+            extra_headers['Content-Length'] = str(content_length)
+
+    if content_type:
+        extra_headers['Content-Type'] = content_type
+
+        # Send the HTTP headers.
+    if isinstance(service.additional_headers, dict):
+        for header in service.additional_headers:
+            connection.putheader(header, service.additional_headers[header])
+    if isinstance(extra_headers, dict):
+        for header in extra_headers:
+            connection.putheader(header, extra_headers[header])
+    connection.endheaders()
+
+    # If there is data, send it in the request.
+    if data:
+        if isinstance(data, list):
+            for data_part in data:
+                __SendDataPart(data_part, connection)
+        else:
+            __SendDataPart(data, connection)
+
+    # Return the HTTP Response from the server.
+    return connection.getresponse()
+
+
+def __SendDataPart(data, connection):
+    """This method is deprecated, use atom.http._send_data_part"""
+    deprecated('call to deprecated function __SendDataPart')
+    if isinstance(data, str):
+        # TODO add handling for unicode.
+        connection.send(data)
+        return
+    elif ElementTree.iselement(data):
+        connection.send(ElementTree.tostring(data))
+        return
+    # Check to see if data is a file-like object that has a read method.
+    elif hasattr(data, 'read'):
+        # Read the file and send it a chunk at a time.
+        while 1:
+            binarydata = data.read(100000)
+            if binarydata == '': break
+            connection.send(binarydata)
+        return
+    else:
+        # The data object was not a file.
+        # Try to convert to a string and send the data.
+        connection.send(str(data))
+        return
+
+
+def CalculateDataLength(data):
+    """Attempts to determine the length of the data to send.
+
+    This method will respond with a length only if the data is a string or
+    and ElementTree element.
+
+    Args:
+      data: object If this is not a string or ElementTree element this funtion
+          will return None.
+    """
+    if isinstance(data, str):
+        return len(data)
+    elif isinstance(data, list):
+        return None
+    elif ElementTree.iselement(data):
+        return len(ElementTree.tostring(data))
+    elif hasattr(data, 'read'):
+        # If this is a file-like object, don't try to guess the length.
+        return None
+    else:
+        return len(str(data).encode('utf-8'))
+
+
+def deprecation(message):
+    warnings.warn(message, DeprecationWarning, stacklevel=2)

src/gam/atom/token_store.py

@@ -0,0 +1,105 @@
+#
+# Copyright (C) 2008 Google Inc.
+#
+# Licensed under the Apache License 2.0;
+
+
+"""This module provides a TokenStore class which is designed to manage
+auth tokens required for different services.
+
+Each token is valid for a set of scopes which is the start of a URL. An HTTP
+client will use a token store to find a valid Authorization header to send
+in requests to the specified URL. If the HTTP client determines that a token
+has expired or been revoked, it can remove the token from the store so that
+it will not be used in future requests.
+"""
+
+# __author__ = 'api.jscudder (Jeff Scudder)'
+
+import atom.http_interface
+import atom.url
+
+SCOPE_ALL = 'http'
+
+
+class TokenStore(object):
+    """Manages Authorization tokens which will be sent in HTTP headers."""
+
+    def __init__(self, scoped_tokens=None):
+        self._tokens = scoped_tokens or {}
+
+    def add_token(self, token):
+        """Adds a new token to the store (replaces tokens with the same scope).
+
+        Args:
+          token: A subclass of http_interface.GenericToken. The token object is
+              responsible for adding the Authorization header to the HTTP request.
+              The scopes defined in the token are used to determine if the token
+              is valid for a requested scope when find_token is called.
+
+        Returns:
+          True if the token was added, False if the token was not added becase
+          no scopes were provided.
+        """
+        if not hasattr(token, 'scopes') or not token.scopes:
+            return False
+
+        for scope in token.scopes:
+            self._tokens[str(scope)] = token
+        return True
+
+    def find_token(self, url):
+        """Selects an Authorization header token which can be used for the URL.
+
+        Args:
+          url: str or atom.url.Url or a list containing the same.
+              The URL which is going to be requested. All
+              tokens are examined to see if any scopes begin match the beginning
+              of the URL. The first match found is returned.
+
+        Returns:
+          The token object which should execute the HTTP request. If there was
+          no token for the url (the url did not begin with any of the token
+          scopes available), then the atom.http_interface.GenericToken will be
+          returned because the GenericToken calls through to the http client
+          without adding an Authorization header.
+        """
+        if url is None:
+            return None
+        if isinstance(url, str):
+            url = atom.url.parse_url(url)
+        if url in self._tokens:
+            token = self._tokens[url]
+            if token.valid_for_scope(url):
+                return token
+            else:
+                del self._tokens[url]
+        for scope, token in self._tokens.items():
+            if token.valid_for_scope(url):
+                return token
+        return atom.http_interface.GenericToken()
+
+    def remove_token(self, token):
+        """Removes the token from the token_store.
+
+        This method is used when a token is determined to be invalid. If the
+        token was found by find_token, but resulted in a 401 or 403 error stating
+        that the token was invlid, then the token should be removed to prevent
+        future use.
+
+        Returns:
+          True if a token was found and then removed from the token
+          store. False if the token was not in the TokenStore.
+        """
+        token_found = False
+        scopes_to_delete = []
+        for scope, stored_token in self._tokens.items():
+            if stored_token == token:
+                scopes_to_delete.append(scope)
+                token_found = True
+        for scope in scopes_to_delete:
+            del self._tokens[scope]
+        return token_found
+
+    def remove_all_tokens(self):
+        self._tokens = {}

src/gam/atom/url.py

@@ -0,0 +1,130 @@
+#
+# Copyright (C) 2008 Google Inc.
+#
+# Licensed under the Apache License 2.0;
+
+
+
+# __author__ = 'api.jscudder (Jeff Scudder)'
+
+import urllib.error
+import urllib.parse
+import urllib.parse
+import urllib.request
+
+DEFAULT_PROTOCOL = 'http'
+DEFAULT_PORT = 80
+
+
+def parse_url(url_string):
+    """Creates a Url object which corresponds to the URL string.
+
+    This method can accept partial URLs, but it will leave missing
+    members of the Url unset.
+    """
+    parts = urllib.parse.urlparse(url_string)
+    url = Url()
+    if parts[0]:
+        url.protocol = parts[0]
+    if parts[1]:
+        host_parts = parts[1].split(':')
+        if host_parts[0]:
+            url.host = host_parts[0]
+        if len(host_parts) > 1:
+            url.port = host_parts[1]
+    if parts[2]:
+        url.path = parts[2]
+    if parts[4]:
+        param_pairs = parts[4].split('&')
+        for pair in param_pairs:
+            pair_parts = pair.split('=')
+            if len(pair_parts) > 1:
+                url.params[urllib.parse.unquote_plus(pair_parts[0])] = (
+                    urllib.parse.unquote_plus(pair_parts[1]))
+            elif len(pair_parts) == 1:
+                url.params[urllib.parse.unquote_plus(pair_parts[0])] = None
+    return url
+
+
+class Url(object):
+    """Represents a URL and implements comparison logic.
+
+    URL strings which are not identical can still be equivalent, so this object
+    provides a better interface for comparing and manipulating URLs than
+    strings. URL parameters are represented as a dictionary of strings, and
+    defaults are used for the protocol (http) and port (80) if not provided.
+    """
+
+    def __init__(self, protocol=None, host=None, port=None, path=None,
+                 params=None):
+        self.protocol = protocol
+        self.host = host
+        self.port = port
+        self.path = path
+        self.params = params or {}
+
+    def to_string(self):
+        url_parts = ['', '', '', '', '', '']
+        if self.protocol:
+            url_parts[0] = self.protocol
+        if self.host:
+            if self.port:
+                url_parts[1] = ':'.join((self.host, str(self.port)))
+            else:
+                url_parts[1] = self.host
+        if self.path:
+            url_parts[2] = self.path
+        if self.params:
+            url_parts[4] = self.get_param_string()
+        return urllib.parse.urlunparse(url_parts)
+
+    def get_param_string(self):
+        param_pairs = []
+        for key, value in self.params.items():
+            param_pairs.append('='.join((urllib.parse.quote_plus(key),
+                                         urllib.parse.quote_plus(str(value)))))
+        return '&'.join(param_pairs)
+
+    def get_request_uri(self):
+        """Returns the path with the parameters escaped and appended."""
+        param_string = self.get_param_string()
+        if param_string:
+            return '?'.join([self.path, param_string])
+        else:
+            return self.path
+
+    def __cmp__(self, other):
+        if not isinstance(other, Url):
+            return cmp(self.to_string(), str(other))
+        difference = 0
+        # Compare the protocol
+        if self.protocol and other.protocol:
+            difference = cmp(self.protocol, other.protocol)
+        elif self.protocol and not other.protocol:
+            difference = cmp(self.protocol, DEFAULT_PROTOCOL)
+        elif not self.protocol and other.protocol:
+            difference = cmp(DEFAULT_PROTOCOL, other.protocol)
+        if difference != 0:
+            return difference
+        # Compare the host
+        difference = cmp(self.host, other.host)
+        if difference != 0:
+            return difference
+        # Compare the port
+        if self.port and other.port:
+            difference = cmp(self.port, other.port)
+        elif self.port and not other.port:
+            difference = cmp(self.port, DEFAULT_PORT)
+        elif not self.port and other.port:
+            difference = cmp(DEFAULT_PORT, other.port)
+        if difference != 0:
+            return difference
+        # Compare the path
+        difference = cmp(self.path, other.path)
+        if difference != 0:
+            return difference
+        # Compare the parameters
+        return cmp(self.params, other.params)
+
+    def __str__(self):
+        return self.to_string()

src/gam/auth/__init__.py

@@ -1,46 +0,0 @@
-"""Authentication/Credentials general purpose and convenience methods."""
-
-import json
-import os
-
-from google.auth.jwt import Credentials as JWTCredentials
-
-from gam.auth import oauth
-from gam.var import _FN_OAUTH2_TXT
-from gam.var import _FN_OAUTH2SERVICE_JSON
-from gam.var import GC_OAUTH2_TXT
-from gam.var import GC_OAUTH2SERVICE_JSON
-from gam.var import GC_ENABLE_DASA
-from gam.var import GC_Values
-
-# TODO: Move logic that determines file name into this module. We should be able
-# to discover the file location without accessing a private member or waiting
-# for a global initialization.
-
-
-def get_admin_credentials_filename():
-    """Gets the name of the file that stores the admin account credentials."""
-    # If the environment globals are loaded, use the set global value. It may have
-    # some custom name in it. Otherwise, just use the default name.
-    if GC_Values[GC_ENABLE_DASA]:
-        return GC_Values[GC_OAUTH2SERVICE_JSON] if GC_Values[GC_OAUTH2SERVICE_JSON] else _FN_OAUTH2SERVICE_JSON
-    else:
-        return GC_Values[GC_OAUTH2_TXT] if GC_Values[GC_OAUTH2_TXT] else _FN_OAUTH2_TXT
-
-
-def get_admin_credentials(api=None):
-    """Gets oauth.Credentials that are authenticated as the domain's admin user."""
-    credential_file = get_admin_credentials_filename()
-    if not os.path.isfile(credential_file):
-        raise oauth.InvalidCredentialsFileError
-    with open(credential_file, 'r') as f:
-        creds_data = json.load(f)
-    # Validate that enable DASA matches content of authorization file
-    if GC_Values[GC_ENABLE_DASA] and 'private_key' in creds_data:
-        audience = f'https://{api}.googleapis.com/'
-        return JWTCredentials.from_service_account_info(creds_data,
-                                                        audience=audience)
-    elif not GC_Values[GC_ENABLE_DASA] and 'token' in creds_data:
-        return oauth.Credentials.from_credentials_file(credential_file)
-    else:
-        raise oauth.InvalidCredentialsFileError

src/gam/auth/oauth.py

@@ -1,560 +0,0 @@
-"""OAuth2.0 user credentials."""
-
-import datetime
-import json
-import os
-import re
-import threading
-from urllib.parse import urlencode
-
-from filelock import FileLock
-import google_auth_oauthlib.flow
-import google.oauth2.credentials
-import google.oauth2.id_token
-
-from gam import fileutils
-from gam import transport
-from gam.var import GM_Globals
-from gam.var import GM_WINDOWS
-from gam import utils
-
-MESSAGE_CONSOLE_AUTHORIZATION_PROMPT = ('\nGo to the following link in your '
-                                        'browser:\n\n\t{url}\n')
-MESSAGE_CONSOLE_AUTHORIZATION_CODE = 'Enter verification code: '
-MESSAGE_LOCAL_SERVER_AUTHORIZATION_PROMPT = ('\nYour browser has been opened to'
-                                             ' visit:\n\n\t{url}\n\nIf your '
-                                             'browser is on a different machine'
-                                             ' then press CTRL+C and create a '
-                                             'file called nobrowser.txt in the '
-                                             'same folder as GAM.\n')
-MESSAGE_LOCAL_SERVER_SUCCESS = ('The authentication flow has completed. You may'
-                                ' close this browser window and return to GAM.')
-
-
-class CredentialsError(Exception):
-    """Base error class."""
-    pass
-
-
-class InvalidCredentialsFileError(CredentialsError):
-    """Error raised when a file cannot be opened into a credentials object."""
-    pass
-
-
-class EmptyCredentialsFileError(InvalidCredentialsFileError):
-    """Error raised when a credentials file contains no content."""
-    pass
-
-
-class InvalidClientSecretsFileFormatError(CredentialsError):
-    """Error raised when a client secrets file format is invalid."""
-    pass
-
-
-class InvalidClientSecretsFileError(CredentialsError):
-    """Error raised when client secrets file cannot be read."""
-    pass
-
-
-class Credentials(google.oauth2.credentials.Credentials):
-    """Google OAuth2.0 Credentials with GAM-specific properties and methods."""
-
-    DATETIME_FORMAT = '%Y-%m-%dT%H:%M:%SZ'
-
-    def __init__(self,
-                 token,
-                 refresh_token=None,
-                 id_token=None,
-                 token_uri=None,
-                 client_id=None,
-                 client_secret=None,
-                 scopes=None,
-                 quota_project_id=None,
-                 expiry=None,
-                 id_token_data=None,
-                 filename=None):
-        """A thread-safe OAuth2.0 credentials object.
-
-    Credentials adds additional utility properties and methods to a
-    standard OAuth2.0 credentials object. When used to store credentials on
-    disk, it implements a file lock to avoid collision during writes.
-
-    Args:
-      token: Optional String, The OAuth 2.0 access token. Can be None if refresh
-        information is provided.
-      refresh_token: String, The OAuth 2.0 refresh token. If specified,
-        credentials can be refreshed.
-      id_token: String, The Open ID Connect ID Token.
-      token_uri: String, The OAuth 2.0 authorization server's token endpoint
-        URI. Must be specified for refresh, can be left as None if the token can
-        not be refreshed.
-      client_id: String, The OAuth 2.0 client ID. Must be specified for refresh,
-        can be left as None if the token can not be refreshed.
-      client_secret: String, The OAuth 2.0 client secret. Must be specified for
-        refresh, can be left as None if the token can not be refreshed.
-      scopes: Sequence[str], The scopes used to obtain authorization.
-        This parameter is used by :meth:`has_scopes`. OAuth 2.0 credentials can
-          not request additional scopes after authorization. The scopes must be
-          derivable from the refresh token if refresh information is provided
-          (e.g. The refresh token scopes are a superset of this or contain a
-          wild card scope like
-            'https://www.googleapis.com/auth/any-api').
-      quota_project_id: String, The project ID used for quota and billing. This
-        project may be different from the project used to create the
-        credentials.
-      expiry: datetime.datetime, The time at which the provided token will
-        expire.
-      id_token_data: Oauth2.0 ID Token data which was previously fetched for
-        this access token against the google.oauth2.id_token library.
-      filename: String, Path to a file that will be used to store the
-        credentials. If provided, a lock file of the same name and a ".lock"
-          extension will be created for concurrency controls. Note: New
-            credentials are not saved to disk until write() or refresh() are
-            called.
-
-    Raises:
-      TypeError: If id_token_data is not the required dict type.
-    """
-        super(Credentials, self).__init__(token=token,
-                                          refresh_token=refresh_token,
-                                          id_token=id_token,
-                                          token_uri=token_uri,
-                                          client_id=client_id,
-                                          client_secret=client_secret,
-                                          scopes=scopes,
-                                          quota_project_id=quota_project_id)
-
-        # Load data not restored by the super class
-        self.expiry = expiry
-        if id_token_data and not isinstance(id_token_data, dict):
-            raise TypeError(f'Expected type id_token_data dict but received '
-                            f'{type(id_token_data)}')
-        self._id_token_data = id_token_data.copy() if id_token_data else None
-
-        # If a filename is provided, use a lock file to control concurrent access
-        # to the resource. If no filename is provided, use a thread lock that has
-        # the same interface as FileLock in order to simplify the implementation.
-        if filename:
-            # Convert relative paths into absolute
-            self._filename = os.path.abspath(filename)
-            lock_file = os.path.abspath(f'{self._filename}.lock')
-            self._lock = FileLock(lock_file)
-        else:
-            self._filename = None
-            self._lock = _FileLikeThreadLock()
-
-    # Use a property to prevent external mutation of the filename.
-    @property
-    def filename(self):
-        return self._filename
-
-    @classmethod
-    def from_authorized_user_info(cls, info, filename=None):
-        """Generates Credentials from JSON containing authorized user info.
-
-    Args:
-      info: Dict, authorized user info in Google format.
-      filename: String, the filename used to store these credentials on disk. If
-        no filename is provided, the credentials will not be saved to disk.
-
-    Raises:
-      ValueError: If missing fields are detected in the info.
-    """
-        # We need all of these keys
-        keys_needed = set(('client_id', 'client_secret'))
-        # We need 1 or more of these keys
-        keys_need_one_of = set(('refresh_token', 'auth_token', 'token'))
-        missing = keys_needed.difference(info.keys())
-        has_one_of = set(info) & keys_need_one_of
-        if missing or not has_one_of:
-            raise ValueError(
-                'Authorized user info was not in the expected format, missing '
-                f'fields {", ".join(missing)} and one of '
-                f'{", ".join(keys_need_one_of)}.')
-
-        expiry = info.get('token_expiry')
-        if expiry:
-            # Convert the raw expiry to datetime
-            expiry = datetime.datetime.strptime(expiry,
-                                                Credentials.DATETIME_FORMAT)
-        id_token_data = info.get('decoded_id_token')
-
-        # Provide backwards compatibility with field names when loading from JSON.
-        # Some field names may be different, depending on when/how the credentials
-        # were pickled.
-        return cls(token=info.get('token', info.get('auth_token', '')),
-                   refresh_token=info.get('refresh_token', ''),
-                   id_token=info.get('id_token_jwt', info.get('id_token')),
-                   token_uri=info.get('token_uri'),
-                   client_id=info['client_id'],
-                   client_secret=info['client_secret'],
-                   scopes=info.get('scopes'),
-                   quota_project_id=info.get('quota_project_id'),
-                   expiry=expiry,
-                   id_token_data=id_token_data,
-                   filename=filename)
-
-    @classmethod
-    def from_google_oauth2_credentials(cls, credentials, filename=None):
-        """Generates Credentials from a google.oauth2.Credentials object."""
-        info = json.loads(credentials.to_json())
-        # Add properties which are not exported with the native to_json() output.
-        info['id_token'] = credentials.id_token
-        if credentials.expiry:
-            info['token_expiry'] = credentials.expiry.strftime(
-                Credentials.DATETIME_FORMAT)
-        info['quota_project_id'] = credentials.quota_project_id
-
-        return cls.from_authorized_user_info(info, filename=filename)
-
-    @classmethod
-    def from_credentials_file(cls, filename):
-        """Generates Credentials from a stored Credentials file.
-
-    The same file will be used to save the credentials when the access token is
-    refreshed.
-
-    Args:
-      filename: String, the name of a file containing JSON credentials to load.
-        The same filename will be used to save credentials back to disk.
-
-    Returns:
-      The credentials loaded from disk.
-
-    Raises:
-      InvalidCredentialsFileError: When the credentials file cannot be opened.
-      EmptyCredentialsFileError: When the provided file contains no credentials.
-    """
-        file_content = fileutils.read_file(filename,
-                                           continue_on_error=True,
-                                           display_errors=False)
-        if file_content is None:
-            raise InvalidCredentialsFileError(
-                f'File {filename} could not be opened')
-        info = json.loads(file_content)
-        if not info:
-            raise EmptyCredentialsFileError(
-                f'File {filename} contains no credential data')
-
-        try:
-            # We read the existing data from the passed in file, but we also want to
-            # save future data/tokens in the same place.
-            return cls.from_authorized_user_info(info, filename=filename)
-        except ValueError as e:
-            raise InvalidCredentialsFileError(str(e))
-
-    @classmethod
-    def from_client_secrets(cls,
-                            client_id,
-                            client_secret,
-                            scopes,
-                            access_type='offline',
-                            login_hint=None,
-                            filename=None,
-                            use_console_flow=False):
-        """Runs an OAuth Flow from client secrets to generate credentials.
-
-    Args:
-      client_id: String, The OAuth2.0 Client ID.
-      client_secret: String, The OAuth2.0 Client Secret.
-      scopes: Sequence[str], A list of scopes to include in the credentials.
-      access_type: String, 'offline' or 'online'.  Indicates whether your
-        application can refresh access tokens when the user is not present at
-        the browser. Valid parameter values are online, which is the default
-        value, and offline.  Set the value to offline if your application needs
-        to refresh access tokens when the user is not present at the browser.
-        This is the method of refreshing access tokens described later in this
-        document. This value instructs the Google authorization server to return
-        a refresh token and an access token the first time that your application
-        exchanges an authorization code for tokens.
-      login_hint: String, The email address that will be displayed on the Google
-        login page as a hint for the user to login to the correct account.
-      filename: String, the path to a file to use to save the credentials.
-      use_console_flow: Boolean, True if the authentication flow should be run
-        strictly from a console; False to launch a browser for authentication.
-
-    Returns:
-      Credentials
-    """
-        client_config = {
-            'installed': {
-                'client_id': client_id,
-                'client_secret': client_secret,
-                'redirect_uris': [
-                    'http://localhost', 'urn:ietf:wg:oauth:2.0:oob'
-                ],
-                'auth_uri': 'https://accounts.google.com/o/oauth2/v2/auth',
-                'token_uri': 'https://oauth2.googleapis.com/token',
-            }
-        }
-
-        flow = _ShortURLFlow.from_client_config(client_config,
-                                                scopes,
-                                                autogenerate_code_verifier=True)
-        flow_kwargs = {'access_type': access_type}
-        if login_hint:
-            flow_kwargs['login_hint'] = login_hint
-
-        # TODO: Move code for browser detection somewhere in this file so that the
-        # messaging about `nobrowser.txt` is co-located with the logic that uses it.
-        if use_console_flow:
-            flow.run_console(
-                authorization_prompt_message=
-                MESSAGE_CONSOLE_AUTHORIZATION_PROMPT,
-                authorization_code_message=MESSAGE_CONSOLE_AUTHORIZATION_CODE,
-                **flow_kwargs)
-        else:
-            flow.run_local_server(authorization_prompt_message=
-                                  MESSAGE_LOCAL_SERVER_AUTHORIZATION_PROMPT,
-                                  success_message=MESSAGE_LOCAL_SERVER_SUCCESS,
-                                  **flow_kwargs)
-        return cls.from_google_oauth2_credentials(flow.credentials,
-                                                  filename=filename)
-
-    @classmethod
-    def from_client_secrets_file(cls,
-                                 client_secrets_file,
-                                 scopes,
-                                 access_type='offline',
-                                 login_hint=None,
-                                 credentials_file=None,
-                                 use_console_flow=False):
-        """Runs an OAuth Flow from secrets stored on disk to generate credentials.
-
-    Args:
-      client_secrets_file: String, path to a file containing a client ID and
-        secret.
-      scopes: Sequence[str], A list of scopes to include in the credentials.
-      access_type: String, 'offline' or 'online'.  Indicates whether your
-        application can refresh access tokens when the user is not present at
-        the browser. Valid parameter values are online, which is the default
-        value, and offline.  Set the value to offline if your application needs
-        to refresh access tokens when the user is not present at the browser.
-        This is the method of refreshing access tokens described later in this
-        document. This value instructs the Google authorization server to return
-        a refresh token and an access token the first time that your application
-        exchanges an authorization code for tokens.
-      login_hint: String, The email address that will be displayed on the Google
-        login page as a hint for the user to login to the correct account.
-      credentials_file: String, the path to a file to use to save the
-        credentials.
-      use_console_flow: Boolean, True if the authentication flow should be run
-        strictly from a console; False to launch a browser for authentication.
-
-    Raises:
-      InvalidClientSecretsFileError: If the client secrets file cannot be
-        opened.
-      InvalidClientSecretsFileFormatError: If the client secrets file does not
-        contain the required data or the data is malformed.
-
-    Returns:
-      Credentials
-    """
-        cs_data = fileutils.read_file(client_secrets_file,
-                                      continue_on_error=True,
-                                      display_errors=False)
-        if not cs_data:
-            raise InvalidClientSecretsFileError(
-                f'File {client_secrets_file} could not be opened')
-        try:
-            cs_json = json.loads(cs_data)
-            client_id = cs_json['installed']['client_id']
-            # Chop off .apps.googleusercontent.com suffix as it's not needed
-            # and we need to keep things short for the Auth URL.
-            client_id = re.sub(r'\.apps\.googleusercontent\.com$', '',
-                               client_id)
-            client_secret = cs_json['installed']['client_secret']
-        except (ValueError, IndexError, KeyError):
-            raise InvalidClientSecretsFileFormatError(
-                f'Could not extract Client ID or Client Secret from file {client_secrets_file}'
-            )
-
-        return cls.from_client_secrets(client_id,
-                                       client_secret,
-                                       scopes,
-                                       access_type=access_type,
-                                       login_hint=login_hint,
-                                       filename=credentials_file,
-                                       use_console_flow=use_console_flow)
-
-    def _fetch_id_token_data(self):
-        """Fetches verification details from Google for the OAuth2.0 token.
-
-    See more: https://developers.google.com/identity/sign-in/web/backend-auth
-
-    Raises:
-      CredentialsError: If no id_token is present.
-    """
-        if not self.id_token:
-            raise CredentialsError(
-                'Failed to fetch token data. No id_token present.')
-
-        request = transport.create_request()
-        if self.expired:
-            # The id_token needs to be unexpired, in order to request data about it.
-            self.refresh(request)
-
-        self._id_token_data = google.oauth2.id_token.verify_oauth2_token(
-            self.id_token, request)
-
-    def get_token_value(self, field):
-        """Retrieves data from the OAuth ID token.
-
-    See more: https://developers.google.com/identity/sign-in/web/backend-auth
-
-    Args:
-      field: The name of the key/field to fetch
-
-    Returns:
-      The value associated with the given key or 'Unknown' if the key data can
-      not be found in the access token data.
-    """
-        if not self._id_token_data:
-            self._fetch_id_token_data()
-        # Maintain legacy GAM behavior here to return "Unknown" if the field is
-        # otherwise unpopulated.
-        return self._id_token_data.get(field, 'Unknown')
-
-    def to_json(self, strip=None):
-        """Creates a JSON representation of a Credentials.
-
-    Args:
-        strip: Sequence[str], Optional list of members to exclude from the
-          generated JSON.
-
-    Returns:
-        str: A JSON representation of this instance, suitable to pass to
-             from_json().
-    """
-        expiry = self.expiry.strftime(
-            Credentials.DATETIME_FORMAT) if self.expiry else None
-        prep = {
-            'token': self.token,
-            'refresh_token': self.refresh_token,
-            'token_uri': self.token_uri,
-            'client_id': self.client_id,
-            'client_secret': self.client_secret,
-            'id_token': self.id_token,
-            # Google auth doesn't currently give us scopes back on refresh.
-            # 'scopes': sorted(self.scopes),
-            'token_expiry': expiry,
-            'decoded_id_token': self._id_token_data,
-        }
-
-        # Remove empty entries
-        prep = {k: v for k, v in prep.items() if v is not None}
-
-        # Remove entries that explicitly need to be removed
-        if strip is not None:
-            prep = {k: v for k, v in prep.items() if k not in strip}
-
-        return json.dumps(prep, indent=2, sort_keys=True)
-
-    def refresh(self, request=None):
-        """Refreshes the credential's access token.
-
-    Args:
-      request: google.auth.transport.Request, The object used to make HTTP
-        requests. If not provided, a default request will be used.
-
-    Raises:
-      google.auth.exceptions.RefreshError: If the credentials could not be
-      refreshed.
-    """
-        with self._lock:
-            if request is None:
-                request = transport.create_request()
-            self._locked_refresh(request)
-            # Save the new tokens back to disk, if these credentials are disk-backed.
-            if self._filename:
-                self._locked_write()
-
-    def _locked_refresh(self, request):
-        """Refreshes the credential's access token while the file lock is held."""
-        assert self._lock.is_locked
-        super(Credentials, self).refresh(request)
-
-    def write(self):
-        """Writes credentials to disk."""
-        with self._lock:
-            self._locked_write()
-
-    def _locked_write(self):
-        """Writes credentials to disk while the file lock is held."""
-        assert self._lock.is_locked
-        if not self.filename:
-            # If no filename was provided to the constructor, these credentials cannot
-            # be saved to disk.
-            raise CredentialsError(
-                'The credentials have no associated filename and cannot be saved '
-                'to disk.')
-        fileutils.write_file(self._filename, self.to_json())
-
-    def delete(self):
-        """Deletes all files on disk related to these credentials."""
-        with self._lock:
-            # Only attempt to remove the file if the lock we're using is a FileLock.
-            if isinstance(self._lock, FileLock):
-                os.remove(self._filename)
-                if self._lock.lock_file and not GM_Globals[GM_WINDOWS]:
-                    os.remove(self._lock.lock_file)
-
-    _REVOKE_TOKEN_BASE_URI = 'https://accounts.google.com/o/oauth2/revoke'
-
-    def revoke(self, http=None):
-        """Revokes this credential's access token with the server.
-
-    Args:
-      http: httplib2.Http compatible object for use as a transport. If no http
-        is provided, a default will be used.
-    """
-        with self._lock:
-            if http is None:
-                http = transport.create_http()
-            params = urlencode({'token': self.refresh_token})
-            revoke_uri = f'{Credentials._REVOKE_TOKEN_BASE_URI}?{params}'
-            http.request(revoke_uri, 'GET')
-
-
-class _ShortURLFlow(google_auth_oauthlib.flow.InstalledAppFlow):
-    """InstalledAppFlow which utilizes a URL shortener for authorization URLs."""
-
-    URL_SHORTENER_ENDPOINT = 'https://gam-shortn.appspot.com/create'
-
-    def authorization_url(self, http=None, **kwargs):
-        """Gets a shortened authorization URL."""
-        long_url, state = super(_ShortURLFlow, self).authorization_url(**kwargs)
-        short_url = utils.shorten_url(long_url)
-        return short_url, state
-
-
-class _FileLikeThreadLock(object):
-    """A threading.lock which has the same interface as filelock.Filelock."""
-
-    def __init__(self):
-        """A shell object that holds a threading.Lock.
-
-    Since we cannot inherit from built-in classes such as threading.Lock, we
-    just use a shell object and maintain a lock inside of it.
-    """
-        self._lock = threading.Lock()
-
-    def __enter__(self, *args, **kwargs):
-        return self._lock.__enter__(*args, **kwargs)
-
-    def __exit__(self, *args, **kwargs):
-        return self._lock.__exit__(*args, **kwargs)
-
-    def acquire(self, **kwargs):
-        return self._lock.acquire(**kwargs)
-
-    def release(self):
-        return self._lock.release()
-
-    @property
-    def is_locked(self):
-        return self._lock.locked()
-
-    @property
-    def lock_file(self):
-        return None

src/gam/auth/oauth_test.py

@@ -1,697 +0,0 @@
-"""Tests for oauth."""
-
-import datetime
-import json
-import os
-import platform
-import unittest
-from unittest.mock import MagicMock
-from unittest.mock import patch
-
-import google.oauth2.credentials
-
-from gam.auth import oauth
-
-
-class CredentialsTest(unittest.TestCase):
-
-    def setUp(self):
-        self.fake_token = 'fake_token'
-        self.fake_refresh_token = 'fake_refresh_token'
-        self.fake_id_token = 'fake_id_token'
-        self.fake_token_uri = 'https://fake.token.uri'
-        self.fake_client_id = 'fake_client_id'
-        self.fake_client_secret = 'fake_client_secret'
-        self.fake_scopes = [
-            'fake_api.readonly',
-            'fake_other_api.write',
-        ]
-        self.fake_quota_project_id = 'fake_quota_project_id'
-        self.fake_token_expiry = datetime.datetime(2020, 1, 1, 10)
-        self.fake_filename = 'fake_filename'
-        self.fake_token_data = {
-            'field': 'value',
-            'another-field': 'another-value',
-        }
-        self.info_with_only_required_fields = {
-            'refresh_token': self.fake_refresh_token,
-            'client_id': self.fake_client_id,
-            'client_secret': self.fake_client_secret,
-        }
-        super(CredentialsTest, self).setUp()
-
-    def tearDown(self):
-        # Remove any credential files that may have been created.
-        if os.path.exists(self.fake_filename):
-            os.remove(self.fake_filename)
-        if os.path.exists('%s.lock' % self.fake_filename):
-            os.remove('%s.lock' % self.fake_filename)
-        super(CredentialsTest, self).tearDown()
-
-    def test_from_authorized_user_info_only_required_info(self):
-        creds = oauth.Credentials.from_authorized_user_info(
-            self.info_with_only_required_fields)
-        self.assertEqual(self.fake_refresh_token, creds.refresh_token)
-        self.assertEqual(self.fake_client_id, creds.client_id)
-        self.assertEqual(self.fake_client_secret, creds.client_secret)
-        self.assertIsNone(creds.id_token)
-        self.assertIsNone(creds.expiry)
-        self.assertIsNone(creds.filename)
-
-    def test_from_authorized_user_info_all_info_provided(self):
-        info = {
-            'token':
-                self.fake_token,
-            'refresh_token':
-                self.fake_refresh_token,
-            'id_token':
-                self.fake_id_token,
-            'token_uri':
-                self.fake_token_uri,
-            'client_id':
-                self.fake_client_id,
-            'client_secret':
-                self.fake_client_secret,
-            'token_expiry':
-                self.fake_token_expiry.strftime(
-                    oauth.Credentials.DATETIME_FORMAT),
-            'id_token_data':
-                self.fake_token_data,
-        }
-        creds = oauth.Credentials.from_authorized_user_info(info)
-        self.assertEqual(self.fake_refresh_token, creds.refresh_token)
-        self.assertEqual(self.fake_client_id, creds.client_id)
-        self.assertEqual(self.fake_client_secret, creds.client_secret)
-        self.assertEqual(self.fake_id_token, creds.id_token)
-        self.assertEqual(self.fake_token_uri, creds.token_uri)
-        self.assertEqual(self.fake_token_expiry, creds.expiry)
-        self.assertIsNone(creds.filename)
-
-    def test_from_authorized_user_info_missing_required_info(self):
-        info_with_missing_fields = {'token': self.fake_token}
-        with self.assertRaises(ValueError):
-            oauth.Credentials.from_authorized_user_info(
-                info_with_missing_fields)
-
-    def test_from_authorized_user_info_no_expiry_in_info(self):
-        info_with_no_token_expiry = self.info_with_only_required_fields.copy()
-        self.assertIsNone(info_with_no_token_expiry.get('expiry'))
-        creds = oauth.Credentials.from_authorized_user_info(
-            info_with_no_token_expiry)
-        self.assertIsNone(creds.expiry)
-
-    def test_init_saves_filename(self):
-        creds = oauth.Credentials(token=self.fake_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  filename=self.fake_filename)
-        self.assertEqual(os.path.abspath(self.fake_filename), creds.filename)
-
-    @patch.object(oauth.google.oauth2.id_token, 'verify_oauth2_token')
-    def test_init_loads_decoded_id_token_data(self, mock_verify_token):
-        creds = oauth.Credentials(token=self.fake_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  id_token=self.fake_id_token,
-                                  id_token_data=self.fake_token_data)
-        self.assertEqual(self.fake_token_data.get('field'),
-                         creds.get_token_value('field'))
-        # Verify the fetching method was not called, since the token
-        # data was supposed to be loaded from the passed in info.
-        self.assertEqual(mock_verify_token.call_count, 0)
-
-    def test_credentials_uses_file_lock_when_filename_provided(self):
-        creds = oauth.Credentials(token=self.fake_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  filename=self.fake_filename)
-        self.assertIsInstance(creds._lock, oauth.FileLock)
-        self.assertEqual(creds._lock.lock_file, '%s.lock' % creds.filename)
-
-    def test_credentials_uses_thread_lock_when_filename_not_provided(self):
-        creds = oauth.Credentials(token=self.fake_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  filename=None)
-        self.assertIsInstance(creds._lock, oauth._FileLikeThreadLock)
-        self.assertIsNone(creds.filename)
-
-    def test_from_oauth2credentials(self):
-        google_creds = google.oauth2.credentials.Credentials(
-            token=self.fake_token,
-            refresh_token=self.fake_refresh_token,
-            client_id=self.fake_client_id,
-            client_secret=self.fake_client_secret,
-            id_token=self.fake_id_token)
-        creds = oauth.Credentials.from_google_oauth2_credentials(
-            google_creds, filename=self.fake_filename)
-        self.assertEqual(google_creds.token, creds.token)
-        self.assertEqual(google_creds.refresh_token, creds.refresh_token)
-        self.assertEqual(google_creds.client_id, creds.client_id)
-        self.assertEqual(google_creds.client_secret, creds.client_secret)
-        self.assertEqual(google_creds.id_token, creds.id_token)
-        self.assertEqual(google_creds.expiry, creds.expiry)
-        self.assertEqual(google_creds.quota_project_id, creds.quota_project_id)
-        self.assertEqual(os.path.abspath(self.fake_filename), creds.filename)
-
-    def test_from_credentials_file_corrupt_or_missing_file_raises_error(self):
-        self.assertFalse(os.path.exists(self.fake_filename))
-        with self.assertRaises(oauth.InvalidCredentialsFileError) as e:
-            oauth.Credentials.from_credentials_file(self.fake_filename)
-        self.assertIn('could not be opened', str(e.exception))
-
-    @patch.object(oauth.fileutils, 'read_file')
-    def test_from_credentials_file_no_serialized_data_in_file_raises_error(
-        self, mock_read_file):
-        mock_read_file.return_value = json.dumps({})
-        with self.assertRaises(oauth.EmptyCredentialsFileError):
-            oauth.Credentials.from_credentials_file(self.fake_filename)
-
-    @patch.object(oauth.fileutils, 'read_file')
-    def test_from_credentials_file_missing_any_token_raises_error(
-        self, mock_read_file):
-        mock_read_file.return_value = json.dumps({
-            # This data is missing a token key/value pair
-            'client_id': self.fake_client_id,
-            'client_secret': self.fake_client_secret,
-        })
-        with self.assertRaises(oauth.InvalidCredentialsFileError):
-            oauth.Credentials.from_credentials_file(self.fake_filename)
-
-    @patch.object(oauth.fileutils, 'read_file')
-    def test_from_credentials_file_missing_required_raises_error(
-        self, mock_read_file):
-        mock_read_file.return_value = json.dumps({
-            # This data is missing a client_secret key/value pair
-            'client_id': self.fake_client_id,
-            'refresh_token': self.fake_refresh_token,
-        })
-        with self.assertRaises(oauth.InvalidCredentialsFileError):
-            oauth.Credentials.from_credentials_file(self.fake_filename)
-
-    @patch.object(oauth._ShortURLFlow, 'from_client_config')
-    def test_from_client_secrets_console_flow(self, mock_flow):
-        flow_creds = google.oauth2.credentials.Credentials(
-            token=self.fake_token,
-            refresh_token=self.fake_refresh_token,
-            client_id=self.fake_client_id,
-            client_secret=self.fake_client_secret,
-            id_token=self.fake_id_token)
-        mock_flow.return_value.credentials = flow_creds
-
-        creds = oauth.Credentials.from_client_secrets(self.fake_client_id,
-                                                      self.fake_client_secret,
-                                                      self.fake_scopes,
-                                                      use_console_flow=True)
-        self.assertTrue(mock_flow.return_value.run_console.called)
-        self.assertFalse(mock_flow.return_value.run_local_server.called)
-        self.assertEqual(flow_creds.token, creds.token)
-        self.assertEqual(flow_creds.refresh_token, creds.refresh_token)
-        self.assertEqual(flow_creds.client_id, creds.client_id)
-        self.assertEqual(flow_creds.client_secret, creds.client_secret)
-        self.assertEqual(flow_creds.id_token, creds.id_token)
-
-    @patch.object(oauth._ShortURLFlow, 'from_client_config')
-    def test_from_client_secrets_local_server_flow(self, mock_flow):
-        flow_creds = google.oauth2.credentials.Credentials(
-            token=self.fake_token,
-            refresh_token=self.fake_refresh_token,
-            client_id=self.fake_client_id,
-            client_secret=self.fake_client_secret,
-            id_token=self.fake_id_token)
-        mock_flow.return_value.credentials = flow_creds
-
-        creds = oauth.Credentials.from_client_secrets(self.fake_client_id,
-                                                      self.fake_client_secret,
-                                                      self.fake_scopes,
-                                                      use_console_flow=False)
-        self.assertFalse(mock_flow.return_value.run_console.called)
-        self.assertTrue(mock_flow.return_value.run_local_server.called)
-        self.assertEqual(flow_creds.token, creds.token)
-        self.assertEqual(flow_creds.refresh_token, creds.refresh_token)
-        self.assertEqual(flow_creds.client_id, creds.client_id)
-        self.assertEqual(flow_creds.client_secret, creds.client_secret)
-        self.assertEqual(flow_creds.id_token, creds.id_token)
-
-    @patch.object(oauth._ShortURLFlow, 'from_client_config')
-    def test_from_client_secrets_uses_login_hint(self, mock_flow):
-        flow_creds = google.oauth2.credentials.Credentials(
-            token=self.fake_token,
-            refresh_token=self.fake_refresh_token,
-            client_id=self.fake_client_id,
-            client_secret=self.fake_client_secret,
-            id_token=self.fake_id_token)
-        mock_flow.return_value.credentials = flow_creds
-
-        oauth.Credentials.from_client_secrets(self.fake_client_id,
-                                              self.fake_client_secret,
-                                              self.fake_scopes,
-                                              login_hint='someone@domain.com')
-
-        run_flow_args = mock_flow.return_value.run_local_server.call_args[1]
-        self.assertEqual('someone@domain.com', run_flow_args.get('login_hint'))
-
-    def test_from_client_secrets_uses_shortened_url_flow(self):
-        with patch.object(oauth._ShortURLFlow,
-                          'from_client_config') as mock_flow:
-            flow_creds = google.oauth2.credentials.Credentials(
-                token=self.fake_token,
-                refresh_token=self.fake_refresh_token,
-                client_id=self.fake_client_id,
-                client_secret=self.fake_client_secret,
-                id_token=self.fake_id_token)
-            mock_flow.return_value.credentials = flow_creds
-            oauth.Credentials.from_client_secrets(self.fake_client_id,
-                                                  self.fake_client_secret,
-                                                  self.fake_scopes)
-        self.assertTrue(mock_flow.called)
-
-    @patch.object(oauth._ShortURLFlow, 'from_client_config')
-    def test_from_client_secrets_passes_credentials_filename(self, mock_flow):
-        flow_creds = google.oauth2.credentials.Credentials(
-            token=self.fake_token,
-            refresh_token=self.fake_refresh_token,
-            client_id=self.fake_client_id,
-            client_secret=self.fake_client_secret,
-            id_token=self.fake_id_token)
-        mock_flow.return_value.credentials = flow_creds
-
-        creds = oauth.Credentials.from_client_secrets(
-            self.fake_client_id,
-            self.fake_client_secret,
-            self.fake_scopes,
-            filename=self.fake_filename)
-        self.assertEqual(os.path.abspath(self.fake_filename), creds.filename)
-
-    def test_from_client_secrets_file_corrupt_or_missing_file_raises_error(
-        self):
-        self.assertFalse(os.path.exists(self.fake_filename))
-        with self.assertRaises(oauth.InvalidClientSecretsFileError):
-            oauth.Credentials.from_client_secrets_file(self.fake_filename,
-                                                       self.fake_scopes)
-
-    @patch.object(oauth.fileutils, 'read_file')
-    def test_from_client_secrets_file_missing_required_json_raises_error(
-        self, mock_read_file):
-        mock_read_file.return_value = json.dumps({})
-        with self.assertRaises(oauth.InvalidClientSecretsFileFormatError) as e:
-            oauth.Credentials.from_client_secrets_file(self.fake_filename,
-                                                       self.fake_scopes)
-        self.assertIn('Could not extract Client ID or Client Secret',
-                      str(e.exception))
-
-    @patch.object(oauth.Credentials, 'from_client_secrets')
-    @patch.object(oauth.fileutils, 'read_file')
-    def test_from_client_secrets_file_strips_domain_from_client_id(
-        self, mock_read_file, mock_creds_from_client_secrets):
-        mock_read_file.return_value = json.dumps({
-            'installed': {
-                'client_id':
-                    self.fake_client_id + '.apps.googleusercontent.com',
-                'client_secret':
-                    self.fake_client_secret,
-            }
-        })
-
-        oauth.Credentials.from_client_secrets_file(self.fake_filename,
-                                                   self.fake_scopes)
-        self.assertEqual(self.fake_client_id,
-                         mock_creds_from_client_secrets.call_args[0][0])
-
-    def test_get_token_value_known_token_field(self):
-        token_data = {'known-field': 'known-value'}
-        creds = oauth.Credentials(token=self.fake_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  id_token_data=token_data)
-        self.assertEqual('known-value', creds.get_token_value('known-field'))
-
-    def test_get_token_value_unknown_field_returns_unknown(self):
-        creds = oauth.Credentials(token=self.fake_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  id_token_data=self.fake_token_data)
-        self.assertEqual('Unknown', creds.get_token_value('unknown-field'))
-
-    @patch.object(oauth.google.oauth2.id_token, 'verify_oauth2_token')
-    def test_get_token_value_credentials_expired(self,
-                                                 mock_verify_oauth2_token):
-        mock_verify_oauth2_token.return_value = {
-            'fetched-field': 'fetched-value'
-        }
-        time_earlier_than_now = datetime.datetime.now() - datetime.timedelta(
-            minutes=5)
-        creds = oauth.Credentials(token=self.fake_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  expiry=time_earlier_than_now,
-                                  id_token=self.fake_id_token,
-                                  id_token_data=None)
-        self.assertTrue(creds.expired)
-        creds.refresh = MagicMock()
-
-        token_value = creds.get_token_value('fetched-field')
-
-        self.assertEqual('fetched-value', token_value)
-        self.assertTrue(creds.refresh.called)
-
-    def test_to_json_contains_all_required_fields(self):
-        creds = oauth.Credentials(token=self.fake_token,
-                                  refresh_token=self.fake_refresh_token,
-                                  id_token=self.fake_id_token,
-                                  id_token_data=self.fake_token_data,
-                                  token_uri=self.fake_token_uri,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  scopes=self.fake_scopes,
-                                  quota_project_id=self.fake_quota_project_id,
-                                  expiry=self.fake_token_expiry)
-        json_string = creds.to_json()
-        json_data = json.loads(json_string)
-        keys = json_data.keys()
-        self.assertIn('token', keys)
-        self.assertEqual(self.fake_token, json_data['token'])
-        self.assertIn('refresh_token', keys)
-        self.assertEqual(self.fake_refresh_token, json_data['refresh_token'])
-        self.assertIn('id_token', keys)
-        self.assertEqual(self.fake_id_token, json_data['id_token'])
-        self.assertIn('token_uri', keys)
-        self.assertEqual(self.fake_token_uri, json_data['token_uri'])
-        self.assertIn('client_id', keys)
-        self.assertEqual(self.fake_client_id, json_data['client_id'])
-        self.assertIn('client_secret', keys)
-        self.assertEqual(self.fake_client_secret, json_data['client_secret'])
-        self.assertNotIn('scopes', keys)  # Scopes are not currently saved
-        self.assertIn('token_expiry', keys)
-        self.assertEqual(
-            self.fake_token_expiry.strftime(oauth.Credentials.DATETIME_FORMAT),
-            json_data['token_expiry'])
-        self.assertIn('decoded_id_token', keys)
-        self.assertEqual(self.fake_token_data, json_data['decoded_id_token'])
-
-    def test_credentials_to_json_and_back(self):
-        original_creds = oauth.Credentials(
-            token=self.fake_token,
-            refresh_token=self.fake_refresh_token,
-            id_token=self.fake_id_token,
-            id_token_data=self.fake_token_data,
-            token_uri=self.fake_token_uri,
-            client_id=self.fake_client_id,
-            client_secret=self.fake_client_secret,
-            scopes=self.fake_scopes,
-            quota_project_id=self.fake_quota_project_id,
-            expiry=self.fake_token_expiry)
-        pickled_creds = original_creds.to_json()
-        serialized_json = json.loads(pickled_creds)
-        unpickled_creds = oauth.Credentials.from_authorized_user_info(
-            serialized_json)
-        self.assertEqual(original_creds.token, unpickled_creds.token)
-        self.assertEqual(original_creds.refresh_token,
-                         unpickled_creds.refresh_token)
-        self.assertEqual(original_creds.id_token, unpickled_creds.id_token)
-        self.assertEqual(original_creds.token_uri, unpickled_creds.token_uri)
-        self.assertEqual(original_creds.client_id, unpickled_creds.client_id)
-        self.assertEqual(original_creds.client_secret,
-                         unpickled_creds.client_secret)
-        self.assertEqual(original_creds.expiry, unpickled_creds.expiry)
-
-    @patch.object(oauth.google.oauth2.credentials.Credentials, 'refresh')
-    def test_refresh_calls_super_refresh(self, mock_super_refresh):
-        creds = oauth.Credentials(token=None,
-                                  refresh_token=self.fake_refresh_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret)
-        request = MagicMock()
-
-        creds.refresh(request)
-        self.assertTrue(mock_super_refresh.called)
-        self.assertEqual(request, mock_super_refresh.call_args[0][0])
-
-    def test_refresh_locks_resource_during_refresh(self):
-        creds = oauth.Credentials(token=None,
-                                  refresh_token=self.fake_refresh_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret)
-        lock = creds._lock
-
-        def check_lock_is_locked(*unused_args, **unused_kwargs):
-            self.assertTrue(lock.is_locked)
-
-        # We need to mock the superclass refresh so it doesn't actually try to
-        # refresh our fake token.
-        # At the same time, we'll make sure the lock is held during the refresh.
-        with patch.object(oauth.google.oauth2.credentials.Credentials,
-                          'refresh') as mock_refresh:
-            mock_refresh.side_effect = check_lock_is_locked
-            creds.refresh(request=MagicMock())
-
-        # Make sure our side effect was actually performed.
-        self.assertTrue(mock_refresh.called)
-        # The lock should be released after refresh
-        self.assertFalse(lock.is_locked)
-
-    @patch.object(oauth.google.oauth2.credentials.Credentials, 'refresh')
-    @patch.object(oauth.fileutils, 'write_file')
-    def test_refresh_writes_new_credentials_to_disk_after_refresh(
-        self, mock_write_file, mock_super_refresh):
-        creds = oauth.Credentials(token=None,
-                                  refresh_token=self.fake_refresh_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  filename=self.fake_filename)
-
-        def update_access_token(unused_request):
-            creds.token = 'refreshed_access_token'
-
-        mock_super_refresh.side_effect = update_access_token
-
-        self.assertIsNone(creds.token)
-        creds.refresh(request=MagicMock())
-        self.assertEqual('refreshed_access_token', creds.token,
-                         'Access token was not refreshed')
-        text_written_to_file = mock_write_file.call_args[0][1]
-        self.assertIsNotNone(text_written_to_file,
-                             'Nothing was written to file')
-        saved_json = json.loads(text_written_to_file)
-        self.assertEqual('refreshed_access_token', saved_json['token'],
-                         'Refreshed access token was not saved to disk')
-
-    def test_write_writes_credentials_to_disk(self):
-        creds = oauth.Credentials(token=None,
-                                  refresh_token=self.fake_refresh_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  filename=self.fake_filename)
-
-        self.assertFalse(os.path.exists(self.fake_filename))
-        creds.write()
-        self.assertTrue(os.path.exists(self.fake_filename))
-
-    def test_write_raises_error_when_no_credentials_file_is_set(self):
-        creds = oauth.Credentials(token=None,
-                                  refresh_token=self.fake_refresh_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret)
-
-        self.assertIsNone(creds.filename)
-        with self.assertRaises(oauth.CredentialsError):
-            creds.write()
-
-    @patch.object(oauth.google.oauth2.credentials.Credentials, 'refresh')
-    @patch.object(oauth.fileutils, 'write_file')
-    def test_write_locks_resource_during_write(self, mock_write_file,
-                                               unused_mock_super_refresh):
-        creds = oauth.Credentials(token=None,
-                                  refresh_token=self.fake_refresh_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  filename=self.fake_filename)
-        lock = creds._lock
-
-        def check_lock_is_locked(*unused_args, **unused_kwargs):
-            self.assertTrue(creds._lock.is_locked)
-
-        mock_write_file.side_effect = check_lock_is_locked
-
-        self.assertFalse(lock.is_locked)
-        creds.refresh(request=MagicMock())
-        self.assertFalse(lock.is_locked)
-        self.assertTrue(mock_write_file.called)
-
-    def test_delete_removes_credentials_file(self):
-        self.assertFalse(os.path.exists(self.fake_filename))
-        creds = oauth.Credentials(token=None,
-                                  refresh_token=self.fake_refresh_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  filename=self.fake_filename)
-        creds.write()
-        self.assertTrue(os.path.exists(self.fake_filename))
-        creds.delete()
-        self.assertFalse(os.path.exists(self.fake_filename))
-
-    @unittest.skipIf(
-        platform.system() == 'Windows',
-        reason=('On Windows, Filelock deletes the lock file each time the lock '
-                'is released. Delete does not remove it.'))
-    def test_delete_removes_lock_file(self):
-        creds = oauth.Credentials(token=None,
-                                  refresh_token=self.fake_refresh_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret,
-                                  filename=self.fake_filename)
-        lock_file = '%s.lock' % creds.filename
-        creds.write()
-        self.assertTrue(os.path.exists(lock_file))
-        creds.delete()
-        self.assertFalse(os.path.exists(lock_file))
-
-    def test_delete_is_noop_when_not_using_filelock(self):
-        creds = oauth.Credentials(token=None,
-                                  refresh_token=self.fake_refresh_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret)
-        self.assertIsNone(creds.filename)
-        creds.delete()  # This should not raise an exception.
-
-    def test_revoke_requests_credential_revoke(self):
-        creds = oauth.Credentials(token=self.fake_token,
-                                  refresh_token=self.fake_refresh_token,
-                                  client_id=self.fake_client_id,
-                                  client_secret=self.fake_client_secret)
-        mock_http = MagicMock()
-
-        creds.revoke(http=mock_http)
-
-        uri = mock_http.request.call_args[0][0]
-        self.assertRegex(uri, '^%s' % oauth.Credentials._REVOKE_TOKEN_BASE_URI)
-        params = uri[uri.index('?'):]
-        self.assertIn('token=%s' % creds.refresh_token, params)
-        self.assertEqual('GET', mock_http.request.call_args[0][1])
-
-
-class ShortUrlFlowTest(unittest.TestCase):
-
-    def setUp(self):
-        self.fake_client_id = 'fake_client_id'
-        self.fake_client_secret = 'fake_client_secret'
-        self.fake_scopes = [
-            'fake_api.readonly',
-            'fake_other_api.write',
-        ]
-        self.fake_client_config = {
-            'installed': {
-                'client_id': self.fake_client_id,
-                'client_secret': self.fake_client_secret,
-                'redirect_uris': [
-                    'http://localhost', 'urn:ietf:wg:oauth:2.0:oob'
-                ],
-                'auth_uri': 'https://accounts.google.com/o/oauth2/v2/auth',
-                'token_uri': 'https://oauth2.googleapis.com/token',
-            }
-        }
-        self.long_url = 'http://example.com/some/long/url'
-        self.short_url = 'http://ex.co/short'
-        super(ShortUrlFlowTest, self).setUp()
-
-    @patch.object(oauth.google_auth_oauthlib.flow.InstalledAppFlow,
-                  'authorization_url')
-    @unittest.skip('disable short url tests temporarily.')
-    def test_shorturlflow_returns_shortened_url(self, mock_super_auth_url):
-        url_flow = oauth._ShortURLFlow.from_client_config(
-            self.fake_client_config, scopes=self.fake_scopes)
-        mock_super_auth_url.return_value = (self.long_url, 'fake_state')
-
-        mock_http = MagicMock()
-        mock_response = MagicMock()
-        mock_response.status = 200
-        content = json.dumps({'short_url': self.short_url})
-        mock_http.request.return_value = (mock_response, content)
-
-        url, state = url_flow.authorization_url(http=mock_http)
-        self.assertEqual(self.short_url, url)
-        self.assertEqual('fake_state', state)
-
-        # Verify request() was called with the expected arguments.
-        self.assertEqual(oauth._ShortURLFlow.URL_SHORTENER_ENDPOINT,
-                         mock_http.request.call_args[0][0])
-        self.assertEqual('POST', mock_http.request.call_args[0][1])
-        self.assertIn(self.long_url, mock_http.request.call_args[0][2])
-
-    @patch.object(oauth.google_auth_oauthlib.flow.InstalledAppFlow,
-                  'authorization_url')
-    @unittest.skip('disable short url tests temporarily.')
-    def test_shorturlflow_falls_back_to_long_url_on_request_error(
-        self, mock_super_auth_url):
-        url_flow = oauth._ShortURLFlow.from_client_config(
-            self.fake_client_config, scopes=self.fake_scopes)
-        mock_super_auth_url.return_value = (self.long_url, 'fake_state')
-
-        mock_http = MagicMock()
-        mock_http.request.side_effect = Exception()
-
-        url, state = url_flow.authorization_url(http=mock_http)
-        self.assertEqual(self.long_url, url)
-        self.assertEqual('fake_state', state)
-
-    @patch.object(oauth.google_auth_oauthlib.flow.InstalledAppFlow,
-                  'authorization_url')
-    @unittest.skip('disable short url tests temporarily.')
-    def test_shorturlflow_falls_back_to_long_url_on_non_200_response_status(
-        self, mock_super_auth_url):
-        url_flow = oauth._ShortURLFlow.from_client_config(
-            self.fake_client_config, scopes=self.fake_scopes)
-        mock_super_auth_url.return_value = (self.long_url, 'fake_state')
-
-        mock_http = MagicMock()
-        mock_response = MagicMock()
-        mock_response.status = 404  # Use a status that is not 200
-        content = json.dumps({'short_url': self.short_url})
-        mock_http.request.return_value = (mock_response, content)
-
-        url, state = url_flow.authorization_url(http=mock_http)
-        self.assertEqual(self.long_url, url)
-        self.assertEqual('fake_state', state)
-
-    @patch.object(oauth.google_auth_oauthlib.flow.InstalledAppFlow,
-                  'authorization_url')
-    @unittest.skip('disable short url tests temporarily.')
-    def test_shorturlflow_falls_back_to_long_url_on_bad_json_response(
-        self, mock_super_auth_url):
-        url_flow = oauth._ShortURLFlow.from_client_config(
-            self.fake_client_config, scopes=self.fake_scopes)
-        mock_super_auth_url.return_value = (self.long_url, 'fake_state')
-
-        mock_http = MagicMock()
-        mock_response = MagicMock()
-        mock_response.status = 200
-        content = None
-        mock_http.request.return_value = (mock_response, content)
-
-        url, state = url_flow.authorization_url(http=mock_http)
-        self.assertEqual(self.long_url, url)
-        self.assertEqual('fake_state', state)
-
-    @patch.object(oauth.google_auth_oauthlib.flow.InstalledAppFlow,
-                  'authorization_url')
-    @unittest.skip('disable short url tests temporarily.')
-    def test_shorturlflow_falls_back_to_long_url_on_empty_short_url_field(
-        self, mock_super_auth_url):
-        url_flow = oauth._ShortURLFlow.from_client_config(
-            self.fake_client_config, scopes=self.fake_scopes)
-        mock_super_auth_url.return_value = (self.long_url, 'fake_state')
-
-        mock_http = MagicMock()
-        mock_response = MagicMock()
-        mock_response.status = 200
-        content = json.dumps(
-            {})  # This json content contains no "short-url" key
-        mock_http.request.return_value = (mock_response, content)
-
-        url, state = url_flow.authorization_url(http=mock_http)
-        self.assertEqual(self.long_url, url)
-        self.assertEqual('fake_state', state)
-
-
-if __name__ == '__main__':
-    unittest.main()

src/gam/cbcm-v1.1beta1.json

@@ -0,0 +1,593 @@
+{
+  "auth": {
+    "oauth2": {
+      "scopes": {
+        "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers": {
+          "description": "View and manage your Chrome browsers registered with Cloud Management"
+        },
+        "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers.readonly": {
+          "description": "View your Chrome browsers registered with Cloud Management"
+        }
+      }
+    }
+  },
+  "basePath": "",
+  "baseUrl": "https://admin.googleapis.com/admin/directory/v1.1beta1/customer/",
+  "batchPath": "batch",
+  "canonicalName": "cbcm",
+  "discoveryVersion": "v1",
+  "documentationLink": "https://support.google.com/chrome/a/answer/9681204",
+  "fullyEncodeReservedExpansion": true,
+  "icons": {
+    "x16": "http://www.google.com/images/icons/product/search-16.gif",
+    "x32": "http://www.google.com/images/icons/product/search-32.gif"
+  },
+  "id": "cbcm:v1.1beta1",
+  "kind": "discovery#restDescription",
+  "mtlsRootUrl": "https://admin.mtls.googleapis.com/",
+  "name": "cbcm",
+  "ownerDomain": "google.com",
+  "ownerName": "Jay Lee",
+  "packagePath": "cbcm",
+  "parameters": {
+    "$.xgafv": {
+      "description": "V1 error format.",
+      "enum": [
+        "1",
+        "2"
+      ],
+      "enumDescriptions": [
+        "v1 error format",
+        "v2 error format"
+      ],
+      "location": "query",
+      "type": "string"
+    },
+    "access_token": {
+      "description": "OAuth access token.",
+      "location": "query",
+      "type": "string"
+    },
+    "alt": {
+      "default": "json",
+      "description": "Data format for response.",
+      "enum": [
+        "json",
+        "media",
+        "proto"
+      ],
+      "enumDescriptions": [
+        "Responses with Content-Type of application/json",
+        "Media download with context-dependent Content-Type",
+        "Responses with Content-Type of application/x-protobuf"
+      ],
+      "location": "query",
+      "type": "string"
+    },
+    "callback": {
+      "description": "JSONP",
+      "location": "query",
+      "type": "string"
+    },
+    "fields": {
+      "description": "Selector specifying which fields to include in a partial response.",
+      "location": "query",
+      "type": "string"
+    },
+    "key": {
+      "description": "API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.",
+      "location": "query",
+      "type": "string"
+    },
+    "oauth_token": {
+      "description": "OAuth 2.0 token for the current user.",
+      "location": "query",
+      "type": "string"
+    },
+    "prettyPrint": {
+      "default": "true",
+      "description": "Returns response with indentations and line breaks.",
+      "location": "query",
+      "type": "boolean"
+    },
+    "quotaUser": {
+      "description": "Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.",
+      "location": "query",
+      "type": "string"
+    },
+    "uploadType": {
+      "description": "Legacy upload protocol for media (e.g. \"media\", \"multipart\").",
+      "location": "query",
+      "type": "string"
+    },
+    "upload_protocol": {
+      "description": "Upload protocol for media (e.g. \"raw\", \"multipart\").",
+      "location": "query",
+      "type": "string"
+    }
+  },
+  "protocol": "rest",
+  "resources": {
+    "chromebrowsers": {
+      "methods": {
+        "delete": {
+          "description": "Deletes a browser.",
+          "flatPath": "{customer}/devices/chromebrowsers/{deviceId}",
+          "httpMethod": "DELETE",
+          "id": "cbcm.chromebrowsers.delete",
+          "parameterOrder": [
+            "customer",
+            "deviceId"
+          ],
+          "parameters": {
+            "customer": {
+              "description": "Immutable ID of the G Suite account.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            },
+            "deviceId": {
+              "description": "Immutable ID of the browser.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            }
+          },
+          "path": "{customer}/devices/chromebrowsers/{deviceId}",
+          "scopes": [
+            "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers"
+          ]
+        },
+        "get": {
+          "description": "Retrieves a browser.",
+          "flatPath": "{customer}/devices/chromebrowsers/{deviceId}",
+          "httpMethod": "GET",
+          "id": "cbcm.chromebrowsers.get",
+          "parameterOrder": [
+            "customer",
+            "deviceId"
+          ],
+          "parameters": {
+            "customer": {
+              "description": "Immutable ID of the G Suite account.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            },
+            "deviceId": {
+              "description": "Immutable ID of the browser.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            },
+            "projection": {
+              "description": "Restrict information returned to a set of selected fields. FULL or BASIC.",
+              "location": "query",
+              "type": "string"
+            }
+          },
+          "path": "{customer}/devices/chromebrowsers/{deviceId}",
+          "response": {
+            "$ref": "ChromeBrowser"
+          },
+          "scopes": [
+            "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers",
+            "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers.readonly"
+          ]
+        },
+        "list": {
+          "description": "Retrieves a paginated list of all the browsers in a domain.",
+          "flatPath": "{customer}/devices/chromebrowsers",
+          "httpMethod": "GET",
+          "id": "cbcm.chromebrowsers.list",
+          "parameterOrder": [
+            "customer"
+          ],
+          "parameters": {
+            "customer": {
+              "description": "Immutable ID of the G Suite account.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            },
+            "maxResults": {
+              "description": "Maximum number of results to return.",
+              "format": "int32",
+              "location": "query",
+              "maximum": "100",
+              "minimum": "1",
+              "type": "integer"
+            },
+            "orderBy": {
+              "description": "property to use for sorting results.",
+              "location": "query",
+              "type": "string"
+            },
+            "orgUnitPath": {
+              "description": "The full path of the organizational unit or its unique ID.",
+              "location": "query",
+              "type": "string"
+            },
+            "pageToken": {
+              "description": "Token to specify the next page in the list.",
+              "location": "query",
+              "type": "string"
+            },
+            "projection": {
+              "description": "Restrict information returned to a set of selected fields. FULL or BASIC.",
+              "location": "query",
+              "type": "string"
+            },
+            "query": {
+              "description": "Search string using the list page query language.",
+              "location": "query",
+              "type": "string"
+            },
+            "sortOrder": {
+              "description": "Whether to return results in ascending or descending order. Must be used with the orderBy parameter.",
+              "location": "query",
+              "type": "string"
+            }
+          },
+          "path": "{customer}/devices/chromebrowsers",
+          "response": {
+            "$ref": "ChromeBrowsers"
+          },
+          "scopes": [
+            "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers",
+            "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers.readonly"
+          ]
+        },
+        "moveChromeBrowsersToOu": {
+          "description": "Move Chrome Browsers Device between Organization Units",
+          "flatPath": "{customer}/devices/chromebrowsers/moveChromeBrowsersToOu",
+          "httpMethod": "POST",
+          "id": "cbcm.chromebrowsers.moveChromeBrowsersToOu",
+          "parameterOrder": [
+            "customer"
+          ],
+          "parameters": {
+            "customer": {
+              "description": "Immutable ID of the G Suite account.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            }
+          },
+          "path": "{customer}/devices/chromebrowsers/moveChromeBrowsersToOu",
+          "request": {
+            "$ref": "MoveChromeBrowsersRequest"
+          },
+          "scopes": [
+            "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers"
+          ]
+        },
+        "update": {
+          "description": "Updates a browser.",
+          "flatPath": "{customer}/devices/chromebrowsers/{deviceId}",
+          "httpMethod": "PUT",
+          "id": "cbcm.chromebrowsers.update",
+          "parameterOrder": [
+            "customer",
+            "deviceId"
+          ],
+          "parameters": {
+            "customer": {
+              "description": "Immutable ID of the G Suite account.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            },
+            "deviceId": {
+              "description": "Immutable ID of the browser.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            },
+            "projection": {
+              "description": "BASIC or FULL",
+              "location": "query",
+              "type": "string"
+            }
+          },
+          "path": "{customer}/devices/chromebrowsers/{deviceId}",
+          "request": {
+            "$ref": "ChromeBrowser"
+          },
+          "response": {
+            "$ref": "ChromeBrowser"
+          },
+          "scopes": [
+            "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers"
+          ]
+        }
+      }
+    },
+    "enrollmentTokens": {
+      "methods": {
+        "list": {
+          "description": "Retrieves a paginated list of all the browser entollment tokens in a domain.",
+          "flatPath": "{customer}/chrome/enrollmentTokens",
+          "httpMethod": "GET",
+          "id": "cbcm.enrollmentTokens.list",
+          "parameterOrder": [
+            "customer"
+          ],
+          "parameters": {
+            "customer": {
+              "description": "Immutable ID of the G Suite account.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            },
+            "pageSize": {
+              "description": "Maximum number of results to return.",
+              "format": "int32",
+              "location": "query",
+              "maximum": "100",
+              "minimum": "1",
+              "type": "integer"
+            },
+            "orgUnitPath": {
+              "description": "The full path of the organizational unit or its unique ID.",
+              "location": "query",
+              "type": "string"
+            },
+            "pageToken": {
+              "description": "Token to specify the next page in the list.",
+              "location": "query",
+              "type": "string"
+            },
+            "query": {
+              "description": "Search string using the list page query language.",
+              "location": "query",
+              "type": "string"
+            }
+          },
+          "path": "{customer}/chrome/enrollmentTokens",
+          "response": {
+            "$ref": "EnrollmentTokens"
+          },
+          "scopes": [
+            "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers",
+            "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers.readonly"
+          ]
+        },
+        "create": {
+          "description": "Creates a browser enrollment token in a domain.",
+          "flatPath": "{customer}/chrome/enrollmentTokens",
+          "httpMethod": "POST",
+          "id": "cbcm.enrollmentTokens.create",
+          "parameterOrder": [
+            "customer"
+          ],
+          "parameters": {
+            "customer": {
+              "description": "Immutable ID of the G Suite account.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            }
+	  },
+          "path": "{customer}/chrome/enrollmentTokens",
+          "request": {
+            "$ref": "CreateEnrollmentTokenRequest"
+          },
+          "response": {
+            "$ref": "EnrollmentToken"
+          },
+          "scopes": [
+            "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers"
+          ]
+	},
+        "revoke": {
+          "description": "Revokes a browser enrollment token in a domain.",
+          "flatPath": "{customer}/chrome/enrollmentTokens/{tokenPermanentId}:revoke",
+          "httpMethod": "POST",
+          "id": "cbcm.enrollmentTokens.revoke",
+          "parameterOrder": [
+            "customer",
+	    "tokenPermanentId"
+          ],
+          "parameters": {
+            "customer": {
+              "description": "Immutable ID of the G Suite account.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            },
+            "tokenPermanentId": {
+              "description": "Unique identifier for an enrollment token.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            }
+	  },
+          "path": "{customer}/chrome/enrollmentTokens/{tokenPermanentId}:revoke",
+          "scopes": [
+            "https://www.googleapis.com/auth/admin.directory.device.chromebrowsers"
+          ]
+	}
+      }
+    }
+  },
+  "revision": "20201203",
+  "rootUrl": "https://admin.googleapis.com/admin/directory/v1.1beta1/customer/",
+  "schemas": {
+    "ChromeBrowser": {
+      "id": "ChromeBrowser",
+      "properties": {
+        "annotatedAssetId": {
+          "description": "Asset identifier as annotated by the administrator or specified during enrollment.",
+          "type": "string"
+        },
+        "annotatedLocation": {
+          "description": "Address or location of the device as annotated by the administrator.",
+          "type": "string"
+        },
+        "annotatedNotes": {
+          "description": "Notes about this device as annotated by the administrator",
+          "type": "string"
+        },
+        "annotatedUser": {
+          "description": "User of the device as annotated by the administrator.",
+          "type": "string"
+        },
+        "deviceId": {
+          "annotations": {
+            "required": [
+              "cbcm.chromebrowsers.update"
+            ]
+          },
+          "description": "The unique ID of the device.",
+          "type": "string"
+        }
+      },
+      "type": "object"
+    },
+    "ChromeBrowsers": {
+      "id": "ChromeBrowsers",
+      "properties": {
+        "browsers": {
+          "description": "List of Chrome browser objects.",
+          "items": {
+            "$ref": "ChromeBrowser"
+          },
+          "type": "array"
+        },
+        "etag": {
+          "description": "ETag of the resource.",
+          "type": "string"
+        },
+        "kind": {
+          "default": "admin#directory#chromeosdevices",
+          "description": "Kind of resource this is.",
+          "type": "string"
+        },
+        "nextPageToken": {
+          "description": "Token used to access the next page of this result. To access the next page, use this token's value in the `pageToken` query string of this request.",
+          "type": "string"
+        }
+      },
+      "type": "object"
+    },
+    "EnrollmentToken": {
+      "id": "EnrollmentToken",
+      "properties": {
+        "kind": {
+          "default": "admin#directory#chromeEnrollmentToken",
+          "description": "Kind of resource this is.",
+          "type": "string"
+        },
+        "tokenId": {
+          "description": "Enrollment Token ID.",
+          "type": "string"
+        },
+        "tokenPermanentId": {
+          "description": "Enrollment Token Permanent ID.",
+          "type": "string"
+        },
+        "customerId": {
+          "description": "Immutable ID of the G Suite account.",
+          "type": "string"
+        },
+	"orgUnitPath": {
+          "description": "The full path of the organizational unit or its unique ID.",
+          "type": "string"
+        },
+	"creatorId": {
+          "description": "Creator ID.",
+          "type": "string"
+        },
+	"createTime": {
+          "description": "Creation Time.",
+          "type": "string"
+        },
+	"revokerId": {
+          "description": "Revoker ID.",
+          "type": "string"
+        },
+	"revokeTime": {
+          "description": "Revoke Time",
+          "type": "string"
+        }
+      },
+      "type": "object"
+    },
+    "EnrollmentTokens": {
+      "id": "EnrollmentTokens",
+      "properties": {
+        "chrome_enrollment_tokens": {
+          "description": "List of Chrome browser enrollment token objects.",
+          "items": {
+            "$ref": "EnrollmentToken"
+          },
+          "type": "array"
+        },
+        "kind": {
+          "default": "admin#directory#chromeEnrollmentTokens",
+          "description": "Kind of resource this is.",
+          "type": "string"
+        },
+        "nextPageToken": {
+          "description": "Token used to access the next page of this result. To access the next page, use this token's value in the `pageToken` query string of this request.",
+          "type": "string"
+        }
+      },
+      "type": "object"
+    },
+    "CreateEnrollmentTokenRequest": {
+      "id": "CreateEnrollmentTokenRequest",
+      "properties": {
+	"org_unit_path": {
+          "description": "The full path of the organizational unit or its unique ID.",
+          "type": "string"
+        },
+	"expire_time": {
+          "description": "Expiration Time.",
+          "type": "string"
+        },
+	"token_type": {
+          "id": "token_type",
+          "annotations": {
+            "required": [
+              "cbcm.enrollmentTokens.create"
+            ]
+          },
+          "description": "CHROME_BROWSER.",
+          "type": "string"
+        }
+      }
+    },
+    "MoveChromeBrowsersRequest": {
+      "id": "MoveChromeBrowsersRequest",
+      "type": "object",
+      "properties": {
+        "org_unit_path": {
+          "annotations": {
+            "required": [
+              "cbcm.chromebrowsers.moveChromeBrowsersToOu"
+            ]
+          },
+          "description": "Destination organization unit to move devices to. Full path of the organizational unit or its ID prefixed with id:",
+          "type": "string"
+        },
+        "resource_ids": {
+          "annotations": {
+            "required": [
+              "cbcm.chromebrowsers.moveChromeBrowsersToOu"
+            ]
+          },
+          "description": "List of unique device IDs of Chrome Browser Devices to move. A maximum of 600 browsers may be moved per request.",
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        }
+      }
+    }
+  },
+  "servicePath": "",
+  "title": "Admin SDK API",
+  "version": "cbcm_v1.1beta1"
+}

src/gam/contactdelegation-v1.json

@@ -0,0 +1,249 @@
+{
+  "auth": {
+    "oauth2": {
+      "scopes": {
+        "https://www.googleapis.com/auth/admin.contact.delegation": {
+          "description": "View and manage your Contact Delegation"
+        }, 
+        "https://www.googleapis.com/auth/admin.contact.delegation.readonly": {
+          "description": "View your Contact Delegation"
+        }
+      }
+    }
+  }, 
+  "basePath": "",
+  "baseUrl": "https://admin.googleapis.com/admin/contacts/v1/",
+  "batchPath": "batch",
+  "canonicalName": "contactdelegation",
+  "description": "The Contact Delegation API allows Admins to delegate access of one user's, called the delegator, contacts to another user, called the delegate.",
+  "discoveryVersion": "v1",
+  "documentationLink": "https://developers.google.com/admin-sdk/contact-delegation",
+  "fullyEncodeReservedExpansion": true,
+  "icons": {
+    "x16": "http://www.google.com/images/icons/product/search-16.gif",
+    "x32": "http://www.google.com/images/icons/product/search-32.gif"
+  },
+  "id": "contactdelegation:v1",
+  "kind": "discovery#restDescription",
+  "name": "contactdelegation",
+  "ownerDomain": "google.com",
+  "ownerName": "Google",
+  "packagePath": "admin",
+  "parameters": {
+    "$.xgafv": {
+      "description": "V1 error format.",
+      "enum": [
+        "1",
+        "2"
+      ],
+      "enumDescriptions": [
+        "v1 error format",
+        "v2 error format"
+      ],
+      "location": "query",
+      "type": "string"
+    },
+    "access_token": {
+      "description": "OAuth access token.",
+      "location": "query",
+      "type": "string"
+    },
+    "alt": {
+      "default": "json",
+      "description": "Data format for response.",
+      "enum": [
+        "json",
+        "media",
+        "proto"
+      ],
+      "enumDescriptions": [
+        "Responses with Content-Type of application/json",
+        "Media download with context-dependent Content-Type",
+        "Responses with Content-Type of application/x-protobuf"
+      ],
+      "location": "query",
+      "type": "string"
+    },
+    "callback": {
+      "description": "JSONP",
+      "location": "query",
+      "type": "string"
+    },
+    "fields": {
+      "description": "Selector specifying which fields to include in a partial response.",
+      "location": "query",
+      "type": "string"
+    },
+    "key": {
+      "description": "API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.",
+      "location": "query",
+      "type": "string"
+    },
+    "oauth_token": {
+      "description": "OAuth 2.0 token for the current user.",
+      "location": "query",
+      "type": "string"
+    },
+    "prettyPrint": {
+      "default": "true",
+      "description": "Returns response with indentations and line breaks.",
+      "location": "query",
+      "type": "boolean"
+    },
+    "quotaUser": {
+      "description": "Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.",
+      "location": "query",
+      "type": "string"
+    },
+    "uploadType": {
+      "description": "Legacy upload protocol for media (e.g. \"media\", \"multipart\").",
+      "location": "query",
+      "type": "string"
+    },
+    "upload_protocol": {
+      "description": "Upload protocol for media (e.g. \"raw\", \"multipart\").",
+      "location": "query",
+      "type": "string"
+    }
+  },
+  "protocol": "rest",
+  "resources": {
+    "delegates": {
+      "methods": {
+        "create": {
+          "description": "Creates a contact delegations",
+          "flatPath": "users/{user}/delegates",
+          "httpMethod": "POST",
+          "id": "contactdelegations.delegates.create",
+          "parameterOrder": [
+            "user"
+          ],
+          "parameters": {
+            "user": {
+              "description": "Email address of the delegator.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            }
+          },
+          "path": "users/{user}/delegates/{delegate}",
+          "request": {
+            "$ref": "Delegate"
+          },
+          "scopes": [
+            "https://www.googleapis.com/auth/admin.contact.delegation"
+          ]
+        },
+        "delete": {
+          "description": "Deletes a contact delegation.",
+          "flatPath": "users/{user}/delegates/{delegate}",
+          "httpMethod": "DELETE",
+          "id": "contactdelegations.delegates.delete",
+          "parameterOrder": [
+            "user",
+            "delegate"
+          ],
+          "parameters": {
+            "delegate": {
+              "description": "Email address of the delegate",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            },
+            "user": {
+              "description": "Email address of the delegator.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            }
+          },
+          "path": "users/{user}/delegates/{delegate}",
+          "scopes": [
+            "https://www.googleapis.com/auth/admin.contact.delegation"
+          ]
+        },
+        "list": {
+          "description": "Lists contact delegates for a user",
+          "flatPath": "users/{user}/delegates",
+          "httpMethod": "GET",
+          "id": "contactdelegations.delegates.list",
+          "parameterOrder": [
+            "user"
+          ],
+          "parameters": {
+            "pageSize": {
+              "description": "Determines how many delegates are returned in each response. ",
+              "format": "int32",
+              "location": "query",
+              "minimum": "1",
+              "type": "integer"
+            },
+            "pageToken": {
+              "description": "Token to specify the next page in the list.",
+              "location": "query",
+              "type": "string"
+            },
+            "user": {
+              "description": "Email address of the delegator.",
+              "location": "path",
+              "required": true,
+              "type": "string"
+            }
+          },
+          "path": "users/{user}/delegates",
+          "response": {
+            "$ref": "Delegates"
+          },
+          "scopes": [
+            "https://www.googleapis.com/auth/admin.contact.delegation",
+            "https://www.googleapis.com/auth/admin.contact.delegation.readonly"
+          ]
+        }
+      }
+    }
+  },
+  "rootUrl": "https://admin.googleapis.com/admin/contacts/v1/",
+  "schemas": {
+    "Delegate": {
+      "description": "JSON template for a delegate.",
+      "id": "Delegate",
+      "properties": {
+        "email": {
+          "description": "Email of the delegate.",
+          "type": "string"
+        }
+      },
+      "type": "object"
+    },
+    "Delegates": {
+      "id": "Delegates",
+      "properties": {
+        "delegates": {
+          "description": "List of delegates.",
+          "items": {
+            "$ref": "Delegate"
+          },
+          "type": "array"
+        },
+        "etag": {
+          "description": "ETag of the resource.",
+          "type": "string"
+        },
+        "kind": {
+          "default": "",
+          "description": "Kind of resource this is.",
+          "type": "string"
+        },
+        "nextPageToken": {
+          "description": "Token used to access the next page of this result. To access the next page, use this token's value in the `pageToken` query string of this request.",
+          "type": "string"
+        }
+      },
+      "type": "object"
+    }
+  },
+  "servicePath": "",
+  "title": "Contact Delegation API",
+  "version": "v1",
+  "version_module": true
+}

src/gam/controlflow.py

@@ -1,96 +0,0 @@
-"""Methods related to the central control flow of an application."""
-import random
-import sys
-import time
-
-from gam import display
-from gam.var import MESSAGE_HEADER_NOT_FOUND_IN_CSV_HEADERS
-from gam.var import MESSAGE_INVALID_JSON
-
-
-def system_error_exit(return_code, message):
-    """Raises a system exit with the given return code and message.
-
-  Args:
-    return_code: Int, the return code to yield when the system exits.
-    message: An error message to print before the system exits.
-  """
-    if message:
-        display.print_error(message)
-    sys.exit(return_code)
-
-
-def invalid_argument_exit(argument, command):
-    """Indicate that the argument is not valid for the command.
-
-  Args:
-    argument: the invalid argument
-    command: the base GAM command
-  """
-    system_error_exit(2, f'{argument} is not a valid argument for "{command}"')
-
-
-def missing_argument_exit(argument, command):
-    """Indicate that the argument is missing for the command.
-
-  Args:
-    argument: the missingagrument
-    command: the base GAM command
-  """
-    system_error_exit(2, f'missing argument {argument} for "{command}"')
-
-
-def expected_argument_exit(name, expected, argument):
-    """Indicate that the argument does not have an expected value for the command.
-
-  Args:
-    name: the field name
-    expected: the expected values
-    argument: the invalid argument
-  """
-    system_error_exit(2, f'{name} must be one of {expected}; got {argument}')
-
-
-def csv_field_error_exit(field_name, field_names):
-    """Raises a system exit when a CSV field is malformed.
-
-  Args:
-    field_name: The CSV field name for which a header does not exist in the
-      existing CSV headers.
-    field_names: The known list of CSV headers.
-  """
-    system_error_exit(
-        2,
-        MESSAGE_HEADER_NOT_FOUND_IN_CSV_HEADERS.format(field_name,
-                                                       ','.join(field_names)))
-
-
-def invalid_json_exit(file_name):
-    """Raises a sysyem exit when invalid JSON content is encountered."""
-    system_error_exit(17, MESSAGE_INVALID_JSON.format(file_name))
-
-
-def wait_on_failure(current_attempt_num,
-                    total_num_retries,
-                    error_message,
-                    error_print_threshold=3):
-    """Executes an exponential backoff-style system sleep.
-
-  Args:
-    current_attempt_num: Int, the current number of retries.
-    total_num_retries: Int, the total number of times the current action will be
-      retried.
-    error_message: String, a message to be displayed that will give more context
-      around why the action is being retried.
-    error_print_threshold: Int, the number of attempts which will have their
-      error messages suppressed. Any current_attempt_num greater than
-      error_print_threshold will print the prescribed error.
-  """
-    wait_on_fail = min(2**current_attempt_num,
-                       60) + float(random.randint(1, 1000)) / 1000
-    if current_attempt_num > error_print_threshold:
-        sys.stderr.write((f'Temporary error: {error_message}, Backing off: '
-                          f'{int(wait_on_fail)} seconds, Retry: '
-                          f'{current_attempt_num}/{total_num_retries}\n'))
-        sys.stderr.flush()
-    time.sleep(wait_on_fail)

src/gam/controlflow_test.py

@@ -1,108 +0,0 @@
-"""Tests for controlflow."""
-
-import unittest
-from unittest.mock import patch
-
-from gam import controlflow
-
-
-class ControlFlowTest(unittest.TestCase):
-
-    def test_system_error_exit_raises_systemexit_error(self):
-        with self.assertRaises(SystemExit):
-            controlflow.system_error_exit(1, 'exit message')
-
-    def test_system_error_exit_raises_systemexit_with_return_code(self):
-        with self.assertRaises(SystemExit) as context_manager:
-            controlflow.system_error_exit(100, 'exit message')
-        self.assertEqual(context_manager.exception.code, 100)
-
-    @patch.object(controlflow.display, 'print_error')
-    def test_system_error_exit_prints_error_before_exiting(
-        self, mock_print_err):
-        with self.assertRaises(SystemExit):
-            controlflow.system_error_exit(100, 'exit message')
-        self.assertIn('exit message', mock_print_err.call_args[0][0])
-
-    def test_csv_field_error_exit_raises_systemexit_error(self):
-        with self.assertRaises(SystemExit):
-            controlflow.csv_field_error_exit('aField',
-                                             ['unusedField1', 'unusedField2'])
-
-    def test_csv_field_error_exit_exits_code_2(self):
-        with self.assertRaises(SystemExit) as context_manager:
-            controlflow.csv_field_error_exit('aField',
-                                             ['unusedField1', 'unusedField2'])
-        self.assertEqual(context_manager.exception.code, 2)
-
-    @patch.object(controlflow.display, 'print_error')
-    def test_csv_field_error_exit_prints_error_details(self, mock_print_err):
-        with self.assertRaises(SystemExit):
-            controlflow.csv_field_error_exit('aField',
-                                             ['unusedField1', 'unusedField2'])
-        printed_message = mock_print_err.call_args[0][0]
-        self.assertIn('aField', printed_message)
-        self.assertIn('unusedField1', printed_message)
-        self.assertIn('unusedField2', printed_message)
-
-    def test_invalid_json_exit_raises_systemexit_error(self):
-        with self.assertRaises(SystemExit):
-            controlflow.invalid_json_exit('filename')
-
-    def test_invalid_json_exit_exit_exits_code_17(self):
-        with self.assertRaises(SystemExit) as context_manager:
-            controlflow.invalid_json_exit('filename')
-        self.assertEqual(context_manager.exception.code, 17)
-
-    @patch.object(controlflow.display, 'print_error')
-    def test_invalid_json_exit_prints_error_details(self, mock_print_err):
-        with self.assertRaises(SystemExit):
-            controlflow.invalid_json_exit('filename')
-        printed_message = mock_print_err.call_args[0][0]
-        self.assertIn('filename', printed_message)
-
-    @patch.object(controlflow.time, 'sleep')
-    def test_wait_on_failure_waits_exponentially(self, mock_sleep):
-        controlflow.wait_on_failure(1, 5, 'Backoff attempt #1')
-        controlflow.wait_on_failure(2, 5, 'Backoff attempt #2')
-        controlflow.wait_on_failure(3, 5, 'Backoff attempt #3')
-
-        sleep_calls = mock_sleep.call_args_list
-        self.assertGreaterEqual(sleep_calls[0][0][0], 2**1)
-        self.assertGreaterEqual(sleep_calls[1][0][0], 2**2)
-        self.assertGreaterEqual(sleep_calls[2][0][0], 2**3)
-
-    @patch.object(controlflow.time, 'sleep')
-    def test_wait_on_failure_does_not_exceed_60_secs_wait(self, mock_sleep):
-        total_attempts = 20
-        for attempt in range(1, total_attempts + 1):
-            controlflow.wait_on_failure(
-                attempt,
-                total_attempts,
-                'Attempt #%s' % attempt,
-                # Suppress messages while we make a lot of attempts.
-                error_print_threshold=total_attempts + 1)
-            # Wait time may be between 60 and 61 secs, due to rand addition.
-            self.assertLessEqual(mock_sleep.call_args[0][0], 61)
-
-    # Prevent the system from actually sleeping and thus slowing down the test.
-    @patch.object(controlflow.time, 'sleep')
-    def test_wait_on_failure_prints_errors(self, unused_mock_sleep):
-        message = 'An error message to display'
-        with patch.object(controlflow.sys.stderr, 'write') as mock_stderr_write:
-            controlflow.wait_on_failure(1, 5, message, error_print_threshold=0)
-        self.assertIn(message, mock_stderr_write.call_args[0][0])
-
-    @patch.object(controlflow.time, 'sleep')
-    def test_wait_on_failure_only_prints_after_threshold(
-        self, unused_mock_sleep):
-        total_attempts = 5
-        threshold = 3
-        with patch.object(controlflow.sys.stderr, 'write') as mock_stderr_write:
-            for attempt in range(1, total_attempts + 1):
-                controlflow.wait_on_failure(attempt,
-                                            total_attempts,
-                                            'Attempt #%s' % attempt,
-                                            error_print_threshold=threshold)
-        self.assertEqual(total_attempts - threshold,
-                         mock_stderr_write.call_count)

src/gam/datastudio-v1.json

@@ -0,0 +1,486 @@
+{
+  "basePath": "",
+  "discoveryVersion": "v1",
+  "documentationLink": "https://support.google.com/datastudio",
+  "canonicalName": "Data Studio",
+  "id": "datastudio:v1",
+  "ownerName": "Google",
+  "description": "Allows programmatic viewing and editing of Data Studio assets.",
+  "rootUrl": "https://datastudio.googleapis.com/",
+  "ownerDomain": "google.com",
+  "mtlsRootUrl": "https://datastudio.mtls.googleapis.com/",
+  "batchPath": "batch",
+  "version_module": true,
+  "version": "v1",
+  "schemas": {
+    "Asset": {
+      "id": "Asset",
+      "properties": {
+        "title": {
+          "description": "The title of the asset.",
+          "type": "string"
+        },
+        "createTime": {
+          "format": "google-datetime",
+          "description": "Date the asset was created.",
+          "type": "string"
+        },
+        "lastViewByMeTime": {
+          "type": "string",
+          "description": "Date the asset was last viewed by me.",
+          "format": "google-datetime"
+        },
+        "owner": {
+          "type": "string",
+          "description": "The owner of the asset."
+        },
+        "name": {
+          "type": "string",
+          "description": "The name of the asset."
+        },
+        "trashed": {
+          "type": "boolean",
+          "description": "Value indicating if the asset is in the trash."
+        },
+        "updateTime": {
+          "format": "google-datetime",
+          "description": "Date the asset was last modified.",
+          "type": "string"
+        },
+        "updateByMeTime": {
+          "description": "Date the asset was last modified by me.",
+          "type": "string",
+          "format": "google-datetime"
+        },
+        "assetType": {
+          "enumDescriptions": [
+            "Asset type not specified.",
+            "A report asset.",
+            "A data Source asset."
+          ],
+          "enum": [
+            "ASSET_TYPE_UNSPECIFIED",
+            "REPORT",
+            "DATA_SOURCE"
+          ],
+          "description": "The type of the asset.",
+          "type": "string"
+        }
+      },
+      "description": "A Data Studio asset.",
+      "type": "object"
+    },
+    "SearchAssetsResponse": {
+      "id": "SearchAssetsResponse",
+      "properties": {
+        "assets": {
+          "items": {
+            "$ref": "Asset"
+          },
+          "type": "array",
+          "description": "The list of assets."
+        },
+        "nextPageToken": {
+          "type": "string",
+          "description": "A token to retrieve next page of results. Pass this value in the SearchAssetsRequest.page_token field in the subsequent call to `SearchAssets` method to retrieve the next page of results."
+        }
+      },
+      "description": "Response message for DataStudioService.SearchAssets",
+      "type": "object"
+    },
+    "UpdatePermissionsRequest": {
+      "description": "Request message for DataStudioService.UpdatePermissions",
+      "properties": {
+        "updateMask": {
+          "description": "The list of fields to be updated. Currently not supported.",
+          "type": "string",
+          "format": "google-fieldmask"
+        },
+        "permissions": {
+          "description": "The permissions object to update.",
+          "$ref": "Permissions"
+        }
+      },
+      "id": "UpdatePermissionsRequest",
+      "type": "object"
+    },
+    "AddMembersRequest": {
+      "properties": {
+        "members": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          },
+          "description": "Required. The members to add to the role. The format of a member is one of - user:alice@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-app@appspot.gserviceaccount.com"
+        },
+        "role": {
+          "type": "string",
+          "enumDescriptions": [
+            "Role not specified.",
+            "A viewer.",
+            "An editor.",
+            "An owner.",
+            "Link shared viewer.",
+            "Link shared editor."
+          ],
+          "enum": [
+            "ROLE_UNSPECIFIED",
+            "VIEWER",
+            "EDITOR",
+            "OWNER",
+            "LINK_VIEWER",
+            "LINK_EDITOR"
+          ],
+          "description": "Required. The role to add members to."
+        }
+      },
+      "type": "object",
+      "id": "AddMembersRequest",
+      "description": "Request message for DataStudioService.AddMembers"
+    },
+    "Permissions": {
+      "type": "object",
+      "id": "Permissions",
+      "description": "A Data Studio asset's Permissions.",
+      "properties": {
+        "permissions": {
+          "description": "A map from a Role to a list of members. Role is a string representation of the Role enum. One of: - OWNER - EDITOR - VIEWER",
+          "additionalProperties": {
+            "$ref": "Members"
+          },
+          "type": "object"
+        },
+        "etag": {
+          "format": "byte",
+          "description": "etag to detect and fail concurrent modifications",
+          "type": "string"
+        }
+      }
+    },
+    "RevokeAllPermissionsRequest": {
+      "description": "Request message for DataStudioService.RevokeAllPermissions",
+      "id": "RevokeAllPermissionsRequest",
+      "type": "object",
+      "properties": {
+        "members": {
+          "description": "Required. The members that are having their access revoked. The format of a member is one of - user:alice@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-app@appspot.gserviceaccount.com",
+          "items": {
+            "type": "string"
+          },
+          "type": "array"
+        }
+      }
+    },
+    "Members": {
+      "description": "A wrapper message for a list of members.",
+      "properties": {
+        "members": {
+          "description": "Format of string is one of - user:alice@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-app@appspot.gserviceaccount.com",
+          "items": {
+            "type": "string"
+          },
+          "type": "array"
+        }
+      },
+      "type": "object",
+      "id": "Members"
+    }
+  },
+  "name": "datastudio",
+  "protocol": "rest",
+  "baseUrl": "https://datastudio.googleapis.com/",
+  "title": "Data Studio API",
+  "revision": "20210412",
+  "fullyEncodeReservedExpansion": true,
+  "icons": {
+    "x32": "http://www.google.com/images/icons/product/search-32.gif",
+    "x16": "http://www.google.com/images/icons/product/search-16.gif"
+  },
+  "parameters": {
+    "quotaUser": {
+      "location": "query",
+      "description": "Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.",
+      "type": "string"
+    },
+    "prettyPrint": {
+      "location": "query",
+      "type": "boolean",
+      "description": "Returns response with indentations and line breaks.",
+      "default": "true"
+    },
+    "callback": {
+      "location": "query",
+      "type": "string",
+      "description": "JSONP"
+    },
+    "uploadType": {
+      "description": "Legacy upload protocol for media (e.g. \"media\", \"multipart\").",
+      "type": "string",
+      "location": "query"
+    },
+    "upload_protocol": {
+      "type": "string",
+      "location": "query",
+      "description": "Upload protocol for media (e.g. \"raw\", \"multipart\")."
+    },
+    "$.xgafv": {
+      "enumDescriptions": [
+        "v1 error format",
+        "v2 error format"
+      ],
+      "description": "V1 error format.",
+      "location": "query",
+      "type": "string",
+      "enum": [
+        "1",
+        "2"
+      ]
+    },
+    "oauth_token": {
+      "type": "string",
+      "location": "query",
+      "description": "OAuth 2.0 token for the current user."
+    },
+    "alt": {
+      "type": "string",
+      "enum": [
+        "json",
+        "media",
+        "proto"
+      ],
+      "location": "query",
+      "description": "Data format for response.",
+      "enumDescriptions": [
+        "Responses with Content-Type of application/json",
+        "Media download with context-dependent Content-Type",
+        "Responses with Content-Type of application/x-protobuf"
+      ],
+      "default": "json"
+    },
+    "fields": {
+      "description": "Selector specifying which fields to include in a partial response.",
+      "location": "query",
+      "type": "string"
+    },
+    "access_token": {
+      "type": "string",
+      "description": "OAuth access token.",
+      "location": "query"
+    },
+    "key": {
+      "type": "string",
+      "location": "query",
+      "description": "API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token."
+    }
+  },
+  "servicePath": "",
+  "kind": "discovery#restDescription",
+  "resources": {
+    "assets": {
+      "methods": {
+        "getPermissions": {
+          "parameters": {
+            "name": {
+              "type": "string",
+              "location": "path",
+              "description": "Required. The name of the asset.",
+              "required": true
+            },
+            "role": {
+              "enumDescriptions": [
+                "Role not specified.",
+                "A viewer.",
+                "An editor.",
+                "An owner.",
+                "Link shared viewer.",
+                "Link shared editor."
+              ],
+              "type": "string",
+              "location": "query",
+              "description": "The role of the permssion.",
+              "enum": [
+                "ROLE_UNSPECIFIED",
+                "VIEWER",
+                "EDITOR",
+                "OWNER",
+                "LINK_VIEWER",
+                "LINK_EDITOR"
+              ]
+            }
+          },
+          "id": "datastudio.assets.getPermissions",
+          "response": {
+            "$ref": "Permissions"
+          },
+          "flatPath": "v1/assets/{name}/permissions",
+          "path": "v1/assets/{name}/permissions",
+          "description": "Gets the asset's permission for a given role.",
+          "parameterOrder": [
+            "name"
+          ],
+          "httpMethod": "GET"
+        },
+        "updatePermissions": {
+          "id": "datastudio.assets.updatePermissions",
+          "parameterOrder": [
+            "name"
+          ],
+          "flatPath": "v1/assets/{name}/permissions",
+          "description": "Updates a permission.",
+          "request": {
+            "$ref": "UpdatePermissionsRequest"
+          },
+          "path": "v1/assets/{name}/permissions",
+          "parameters": {
+            "name": {
+              "description": "Required. The name of the asset.",
+              "location": "path",
+              "type": "string",
+              "required": true
+            }
+          },
+          "response": {
+            "$ref": "Permissions"
+          },
+          "httpMethod": "PATCH"
+        },
+        "get": {
+          "path": "v1/{+name}",
+          "id": "datastudio.assets.get",
+          "parameterOrder": [
+            "name"
+          ],
+          "description": "Gets the asset by name.",
+          "parameters": {
+            "name": {
+              "description": "Required. The name of the asset. Format: assets/{asset}",
+              "location": "path",
+              "required": true,
+              "pattern": "^assets/[^/]+$",
+              "type": "string"
+            }
+          },
+          "flatPath": "v1/assets/{assetsId}",
+          "response": {
+            "$ref": "Asset"
+          },
+          "httpMethod": "GET"
+        },
+        "search": {
+          "response": {
+            "$ref": "SearchAssetsResponse"
+          },
+          "path": "v1/assets:search",
+          "parameters": {
+            "pageToken": {
+              "type": "string",
+              "location": "query",
+              "description": "A token identifying a page of results the server should return. Use the value of SearchAssetsResponse.next_page_token returned from the previous call to `SearchAssets` method."
+            },
+            "assetTypes": {
+              "type": "string",
+              "repeated": true,
+              "description": "Exactly one AssetType must be specified.",
+              "enumDescriptions": [
+                "Asset type not specified.",
+                "A report asset.",
+                "A data Source asset."
+              ],
+              "location": "query",
+              "enum": [
+                "ASSET_TYPE_UNSPECIFIED",
+                "REPORT",
+                "DATA_SOURCE"
+              ]
+            },
+            "title": {
+              "description": "The title of assets to include. Not an exact match, works the same as search from the UI.",
+              "location": "query",
+              "type": "string"
+            },
+            "owner": {
+              "type": "string",
+              "location": "query",
+              "description": "The email of the owner of the asset."
+            },
+            "pageSize": {
+              "description": "Requested page size. If unspecified, server will pick an appropriate default.",
+              "location": "query",
+              "type": "integer",
+              "format": "int32"
+            },
+            "orderBy": {
+              "location": "query",
+              "description": "How the results should be ordered. Valid options are: - title",
+              "type": "string"
+            },
+            "includeTrashed": {
+              "location": "query",
+              "type": "boolean",
+              "description": "Value indicating if assets in trash should be included."
+            }
+          },
+          "flatPath": "v1/assets:search",
+          "httpMethod": "GET",
+          "description": "Searches assets.",
+          "id": "datastudio.assets.search",
+          "parameterOrder": []
+        }
+      },
+      "resources": {
+        "permissions": {
+          "methods": {
+            "revokeAllPermissions": {
+              "path": "v1/assets/{name}/permissions:revokeAllPermissions",
+              "response": {
+                "$ref": "Permissions"
+              },
+              "flatPath": "v1/assets/{name}/permissions:revokeAllPermissions",
+              "id": "datastudio.assets.permissions.revokeAllPermissions",
+              "description": "Revokes one or more members' access to an asset.",
+              "parameters": {
+                "name": {
+                  "required": true,
+                  "type": "string",
+                  "location": "path",
+                  "description": "Required. The name of the asset."
+                }
+              },
+              "parameterOrder": [
+                "name"
+              ],
+              "httpMethod": "POST",
+              "request": {
+                "$ref": "RevokeAllPermissionsRequest"
+              }
+            },
+            "addMembers": {
+              "path": "v1/assets/{name}/permissions:addMembers",
+              "parameters": {
+                "name": {
+                  "required": true,
+                  "location": "path",
+                  "type": "string",
+                  "description": "Required. The name of the asset."
+                }
+              },
+              "httpMethod": "POST",
+              "parameterOrder": [
+                "name"
+              ],
+              "response": {
+                "$ref": "Permissions"
+              },
+              "id": "datastudio.assets.permissions.addMembers",
+              "request": {
+                "$ref": "AddMembersRequest"
+              },
+              "description": "Adds one or more members to a role.",
+              "flatPath": "v1/assets/{name}/permissions:addMembers"
+            }
+          }
+        }
+      }
+    }
+  }
+}

src/gam/display.py

@@ -1,297 +0,0 @@
-"""Methods related to display of information to the user."""
-
-import csv
-import datetime
-import io
-import sys
-import webbrowser
-
-import dateutil
-import googleapiclient.http
-
-#TODO: get rid of these hacks
-import gam
-from gam.var import *
-from gam import controlflow
-from gam import gapi
-
-
-def current_count(i, count):
-    return f' ({i}/{count})' if (count > GC_Values[GC_SHOW_COUNTS_MIN]) else ''
-
-
-def current_count_nl(i, count):
-    return f' ({i}/{count})\n' if (
-        count > GC_Values[GC_SHOW_COUNTS_MIN]) else '\n'
-
-
-def add_field_to_fields_list(fieldName, fieldsChoiceMap, fieldsList):
-    fields = fieldsChoiceMap[fieldName.lower()]
-    if isinstance(fields, list):
-        fieldsList.extend(fields)
-    else:
-        fieldsList.append(fields)
-
-
-# Write a CSV file
-def add_titles_to_csv_file(addTitles, titles):
-    for title in addTitles:
-        if title not in titles:
-            titles.append(title)
-
-
-def add_row_titles_to_csv_file(row, csvRows, titles):
-    csvRows.append(row)
-    for title in row:
-        if title not in titles:
-            titles.append(title)
-
-
-# fieldName is command line argument
-# fieldNameMap maps fieldName to API field names; CSV file header will be API field name
-#ARGUMENT_TO_PROPERTY_MAP = {
-#  u'admincreated': [u'adminCreated'],
-#  u'aliases': [u'aliases', u'nonEditableAliases'],
-#  }
-# fieldsList is the list of API fields
-# fieldsTitles maps the API field name to the CSV file header
-def add_field_to_csv_file(fieldName, fieldNameMap, fieldsList, fieldsTitles,
-                          titles):
-    for ftList in fieldNameMap[fieldName]:
-        if ftList not in fieldsTitles:
-            fieldsList.append(ftList)
-            fieldsTitles[ftList] = ftList
-            add_titles_to_csv_file([ftList], titles)
-
-
-# fieldName is command line argument
-# fieldNameTitleMap maps fieldName to API field name and CSV file header
-#ARGUMENT_TO_PROPERTY_TITLE_MAP = {
-#  u'admincreated': [u'adminCreated', u'Admin_Created'],
-#  u'aliases': [u'aliases', u'Aliases', u'nonEditableAliases', u'NonEditableAliases'],
-#  }
-# fieldsList is the list of API fields
-# fieldsTitles maps the API field name to the CSV file header
-def add_field_title_to_csv_file(fieldName, fieldNameTitleMap, fieldsList,
-                                fieldsTitles, titles):
-    ftList = fieldNameTitleMap[fieldName]
-    for i in range(0, len(ftList), 2):
-        if ftList[i] not in fieldsTitles:
-            fieldsList.append(ftList[i])
-            fieldsTitles[ftList[i]] = ftList[i + 1]
-            add_titles_to_csv_file([ftList[i + 1]], titles)
-
-
-def sort_csv_titles(firstTitle, titles):
-    restoreTitles = []
-    for title in firstTitle:
-        if title in titles:
-            titles.remove(title)
-            restoreTitles.append(title)
-    titles.sort()
-    for title in restoreTitles[::-1]:
-        titles.insert(0, title)
-
-
-def QuotedArgumentList(items):
-    return ' '.join([
-        item if item and (item.find(' ') == -1) and
-        (item.find(',') == -1) else '"' + item + '"' for item in items
-    ])
-
-
-def write_csv_file(csvRows, titles, list_type, todrive):
-
-    def rowDateTimeFilterMatch(dateMode, rowDate, op, filterDate):
-        if not rowDate or not isinstance(rowDate, str):
-            return False
-        try:
-            rowTime = dateutil.parser.parse(rowDate, ignoretz=True)
-            if dateMode:
-                rowDate = datetime.datetime(rowTime.year, rowTime.month,
-                                            rowTime.day).isoformat() + 'Z'
-        except ValueError:
-            rowDate = NEVER_TIME
-        if op == '<':
-            return rowDate < filterDate
-        if op == '<=':
-            return rowDate <= filterDate
-        if op == '>':
-            return rowDate > filterDate
-        if op == '>=':
-            return rowDate >= filterDate
-        if op == '!=':
-            return rowDate != filterDate
-        return rowDate == filterDate
-
-    def rowCountFilterMatch(rowCount, op, filterCount):
-        if isinstance(rowCount, str):
-            if not rowCount.isdigit():
-                return False
-            rowCount = int(rowCount)
-        elif not isinstance(rowCount, int):
-            return False
-        if op == '<':
-            return rowCount < filterCount
-        if op == '<=':
-            return rowCount <= filterCount
-        if op == '>':
-            return rowCount > filterCount
-        if op == '>=':
-            return rowCount >= filterCount
-        if op == '!=':
-            return rowCount != filterCount
-        return rowCount == filterCount
-
-    def rowBooleanFilterMatch(rowBoolean, filterBoolean):
-        if not isinstance(rowBoolean, bool):
-            return False
-        return rowBoolean == filterBoolean
-
-    def headerFilterMatch(filters, title):
-        for filterStr in filters:
-            if filterStr.match(title):
-                return True
-        return False
-
-    if GC_Values[GC_CSV_ROW_FILTER]:
-        for column, filterVal in iter(GC_Values[GC_CSV_ROW_FILTER].items()):
-            if column not in titles:
-                sys.stderr.write(
-                    f'WARNING: Row filter column "{column}" is not in output columns\n'
-                )
-                continue
-            if filterVal[0] == 'regex':
-                csvRows = [
-                    row for row in csvRows
-                    if filterVal[1].search(str(row.get(column, '')))
-                ]
-            elif filterVal[0] == 'notregex':
-                csvRows = [
-                    row for row in csvRows
-                    if not filterVal[1].search(str(row.get(column, '')))
-                ]
-            elif filterVal[0] in ['date', 'time']:
-                csvRows = [
-                    row for row in csvRows if rowDateTimeFilterMatch(
-                        filterVal[0] == 'date', row.get(column, ''),
-                        filterVal[1], filterVal[2])
-                ]
-            elif filterVal[0] == 'count':
-                csvRows = [
-                    row for row in csvRows if rowCountFilterMatch(
-                        row.get(column, 0), filterVal[1], filterVal[2])
-                ]
-            else:  #boolean
-                csvRows = [
-                    row for row in csvRows if rowBooleanFilterMatch(
-                        row.get(column, False), filterVal[1])
-                ]
-    if GC_Values[GC_CSV_HEADER_FILTER] or GC_Values[GC_CSV_HEADER_DROP_FILTER]:
-        if GC_Values[GC_CSV_HEADER_DROP_FILTER]:
-            titles = [
-                t for t in titles if
-                not headerFilterMatch(GC_Values[GC_CSV_HEADER_DROP_FILTER], t)
-            ]
-        if GC_Values[GC_CSV_HEADER_FILTER]:
-            titles = [
-                t for t in titles
-                if headerFilterMatch(GC_Values[GC_CSV_HEADER_FILTER], t)
-            ]
-        if not titles:
-            controlflow.system_error_exit(
-                3,
-                'No columns selected with GAM_CSV_HEADER_FILTER and GAM_CSV_HEADER_DROP_FILTER\n'
-            )
-            return
-    csv.register_dialect('nixstdout', lineterminator='\n')
-    if todrive:
-        write_to = io.StringIO()
-    else:
-        write_to = sys.stdout
-    writer = csv.DictWriter(write_to,
-                            fieldnames=titles,
-                            dialect='nixstdout',
-                            extrasaction='ignore',
-                            quoting=csv.QUOTE_MINIMAL)
-    try:
-        writer.writerow(dict((item, item) for item in writer.fieldnames))
-        writer.writerows(csvRows)
-    except IOError as e:
-        controlflow.system_error_exit(6, e)
-    if todrive:
-        admin_email = gam._get_admin_email()
-        _, drive = gam.buildDrive3GAPIObject(admin_email)
-        if not drive:
-            print(f'''\nGAM is not authorized to create Drive files. Please run:
-gam user {admin_email} check serviceaccount
-and follow recommend steps to authorize GAM for Drive access.''')
-            sys.exit(5)
-        result = gapi.call(drive.about(), 'get', fields='maxImportSizes')
-        columns = len(titles)
-        rows = len(csvRows)
-        cell_count = rows * columns
-        data_size = len(write_to.getvalue())
-        max_sheet_bytes = int(result['maxImportSizes'][MIMETYPE_GA_SPREADSHEET])
-        if cell_count > MAX_GOOGLE_SHEET_CELLS or data_size > max_sheet_bytes:
-            print(
-                f'{WARNING_PREFIX}{MESSAGE_RESULTS_TOO_LARGE_FOR_GOOGLE_SPREADSHEET}'
-            )
-            mimeType = 'text/csv'
-        else:
-            mimeType = MIMETYPE_GA_SPREADSHEET
-        body = {
-            'description': QuotedArgumentList(sys.argv),
-            'name': f'{GC_Values[GC_DOMAIN]} - {list_type}',
-            'mimeType': mimeType
-        }
-        result = gapi.call(drive.files(),
-                           'create',
-                           fields='webViewLink',
-                           body=body,
-                           media_body=googleapiclient.http.MediaInMemoryUpload(
-                               write_to.getvalue().encode(),
-                               mimetype='text/csv'))
-        file_url = result['webViewLink']
-        if GC_Values[GC_NO_BROWSER]:
-            msg_txt = f'Drive file uploaded to:\n {file_url}'
-            msg_subj = f'{GC_Values[GC_DOMAIN]} - {list_type}'
-            gam.send_email(msg_subj, msg_txt)
-            print(msg_txt)
-        else:
-            webbrowser.open(file_url)
-
-
-def print_error(message):
-    """Prints a one-line error message to stderr in a standard format."""
-    sys.stderr.write('\n{0}{1}\n'.format(ERROR_PREFIX, message))
-
-
-def print_warning(message):
-    """Prints a one-line warning message to stderr in a standard format."""
-    sys.stderr.write('\n{0}{1}\n'.format(WARNING_PREFIX, message))
-
-
-def print_json(object_value, spacing=''):
-    """Prints Dict or Array to screen in clean human-readable format.."""
-    if isinstance(object_value, list):
-        if len(object_value) == 1 and isinstance(object_value[0],
-                                                 (str, int, bool)):
-            sys.stdout.write(f'{object_value[0]}\n')
-            return
-        if spacing:
-            sys.stdout.write('\n')
-        for i, a_value in enumerate(object_value):
-            if isinstance(a_value, (str, int, bool)):
-                sys.stdout.write(f' {spacing}{i+1}) {a_value}\n')
-            else:
-                sys.stdout.write(f' {spacing}{i+1}) ')
-                print_json(a_value, f' {spacing}')
-    elif isinstance(object_value, dict):
-        for key in ['kind', 'etag', 'etags']:
-            object_value.pop(key, None)
-        for another_object, another_value in object_value.items():
-            sys.stdout.write(f' {spacing}{another_object}: ')
-            print_json(another_value, f' {spacing}')
-    else:
-        sys.stdout.write(f'{object_value}\n')

src/gam/display_test.py

@@ -1,59 +0,0 @@
-"""Tests for display."""
-
-import unittest
-from unittest.mock import patch
-
-from gam import display
-from gam.var import ERROR_PREFIX
-from gam.var import WARNING_PREFIX
-
-
-class DisplayTest(unittest.TestCase):
-
-    def test_print_error_prints_to_stderr(self):
-        message = 'test error'
-        with patch.object(display.sys.stderr, 'write') as mock_write:
-            display.print_error(message)
-        printed_message = mock_write.call_args[0][0]
-        self.assertIn(message, printed_message)
-
-    def test_print_error_prints_error_prefix(self):
-        message = 'test error'
-        with patch.object(display.sys.stderr, 'write') as mock_write:
-            display.print_error(message)
-        printed_message = mock_write.call_args[0][0]
-        self.assertLess(
-            printed_message.find(ERROR_PREFIX), printed_message.find(message),
-            'The error prefix does not appear before the error message')
-
-    def test_print_error_ends_message_with_newline(self):
-        message = 'test error'
-        with patch.object(display.sys.stderr, 'write') as mock_write:
-            display.print_error(message)
-        printed_message = mock_write.call_args[0][0]
-        self.assertRegex(printed_message, '\n$',
-                         'The error message does not end in a newline.')
-
-    def test_print_warning_prints_to_stderr(self):
-        message = 'test warning'
-        with patch.object(display.sys.stderr, 'write') as mock_write:
-            display.print_error(message)
-        printed_message = mock_write.call_args[0][0]
-        self.assertIn(message, printed_message)
-
-    def test_print_warning_prints_error_prefix(self):
-        message = 'test warning'
-        with patch.object(display.sys.stderr, 'write') as mock_write:
-            display.print_error(message)
-        printed_message = mock_write.call_args[0][0]
-        self.assertLess(
-            printed_message.find(WARNING_PREFIX), printed_message.find(message),
-            'The warning prefix does not appear before the error message')
-
-    def test_print_warning_ends_message_with_newline(self):
-        message = 'test warning'
-        with patch.object(display.sys.stderr, 'write') as mock_write:
-            display.print_error(message)
-        printed_message = mock_write.call_args[0][0]
-        self.assertRegex(printed_message, '\n$',
-                         'The warning message does not end in a newline.')

src/gam/fileutils.py

@@ -1,183 +0,0 @@
-"""Common file operations."""
-
-import io
-import os
-import sys
-
-from gam import controlflow
-from gam import display
-from gam.var import GM_Globals
-from gam.var import GM_SYS_ENCODING
-from gam.var import UTF8_SIG
-
-
-def _open_file(filename, mode, encoding=None, newline=None):
-    """Opens a file with no error handling."""
-    # Determine which encoding to use
-    if 'b' in mode:
-        encoding = None
-    elif not encoding:
-        encoding = GM_Globals[GM_SYS_ENCODING]
-    elif 'r' in mode and encoding.lower().replace('-', '') == 'utf8':
-        encoding = UTF8_SIG
-
-    return open(os.path.expanduser(filename),
-                mode,
-                newline=newline,
-                encoding=encoding)
-
-
-def open_file(filename,
-              mode='r',
-              encoding=None,
-              newline=None,
-              strip_utf_bom=False):
-    """Opens a file.
-
-  Args:
-    filename: String, the name of the file to open, or '-' to use stdin/stdout,
-      to read/write, depending on the mode param, respectively.
-    mode: String, the common file mode to open the file with. Default is read.
-    encoding: String, the name of the encoding used to decode or encode the
-      file. This should only be used in text mode.
-    newline: See param description in
-        https://docs.python.org/3.7/library/functions.html#open
-    strip_utf_bom: Boolean, True if the file being opened should seek past the
-      UTF Byte Order Mark before being returned.
-        See more: https://en.wikipedia.org/wiki/UTF-8#Byte_order_mark
-
-  Returns:
-    The opened file.
-  """
-    try:
-        if filename == '-':
-            # Read from stdin, rather than a file
-            if 'r' in mode:
-                return io.StringIO(str(sys.stdin.read()))
-            return sys.stdout
-
-        # Open a file on disk
-        f = _open_file(filename, mode, newline=newline, encoding=encoding)
-        if strip_utf_bom:
-            utf_bom = u'\ufeff'
-            has_bom = False
-
-            if 'b' in mode:
-                has_bom = f.read(3).decode('UTF-8') == utf_bom
-            elif f.encoding and not f.encoding.lower().startswith('utf'):
-                # Convert UTF BOM into ISO-8859-1 via Bytes
-                utf8_bom_bytes = utf_bom.encode('UTF-8')
-                iso_8859_1_bom = utf8_bom_bytes.decode('iso-8859-1').encode(
-                    'iso-8859-1')
-                has_bom = f.read(3).encode('iso-8859-1',
-                                           'replace') == iso_8859_1_bom
-            else:
-                has_bom = f.read(1) == utf_bom
-
-            if not has_bom:
-                f.seek(0)
-
-        return f
-
-    except IOError as e:
-        controlflow.system_error_exit(6, e)
-
-
-def close_file(f, force_flush=False):
-    """Closes a file.
-
-  Args:
-    f: The file to close
-    force_flush: Flush file to disk emptying Python and OS caches. See:
-       https://stackoverflow.com/a/13762137/1503886
-
-  Returns:
-    Boolean, True if the file was successfully closed. False if an error
-        was encountered while closing.
-  """
-    if force_flush:
-        f.flush()
-        os.fsync(f.fileno())
-    try:
-        f.close()
-        return True
-    except IOError as e:
-        display.print_error(e)
-        return False
-
-
-def read_file(filename,
-              mode='r',
-              encoding=None,
-              newline=None,
-              continue_on_error=False,
-              display_errors=True):
-    """Reads a file from disk.
-
-  Args:
-    filename: String, the path of the file to open from disk, or "-" to read
-      from stdin.
-    mode: String, the mode in which to open the file.
-    encoding: String, the name of the encoding used to decode or encode the
-      file. This should only be used in text mode.
-    newline: See param description in
-        https://docs.python.org/3.7/library/functions.html#open
-    continue_on_error: Boolean, If True, suppresses any IO errors and returns to
-      the caller without any externalities.
-    display_errors: Boolean, If True, prints error messages when errors are
-      encountered and continue_on_error is True.
-
-  Returns:
-    The contents of the file, or stdin if filename == "-". Returns None if
-    an error is encountered and continue_on_errors is True.
-  """
-    try:
-        if filename == '-':
-            # Read from stdin, rather than a file.
-            return str(sys.stdin.read())
-
-        with _open_file(filename, mode, newline=newline,
-                        encoding=encoding) as f:
-            return f.read()
-
-    except IOError as e:
-        if continue_on_error:
-            if display_errors:
-                display.print_warning(e)
-            return None
-        controlflow.system_error_exit(6, e)
-    except (LookupError, UnicodeDecodeError, UnicodeError) as e:
-        controlflow.system_error_exit(2, str(e))
-
-
-def write_file(filename,
-               data,
-               mode='w',
-               continue_on_error=False,
-               display_errors=True):
-    """Writes data to a file.
-
-  Args:
-    filename: String, the path of the file to write to disk.
-    data: Serializable data to write to the file.
-    mode: String, the mode in which to open the file and write to it.
-    continue_on_error: Boolean, If True, suppresses any IO errors and returns to
-      the caller without any externalities.
-    display_errors: Boolean, If True, prints error messages when errors are
-      encountered and continue_on_error is True.
-
-  Returns:
-    Boolean, True if the write operation succeeded, or False if not.
-  """
-    try:
-        with _open_file(filename, mode) as f:
-            f.write(data)
-        return True
-
-    except IOError as e:
-        if continue_on_error:
-            if display_errors:
-                display.print_error(e)
-            return False
-        else:
-            controlflow.system_error_exit(6, e)

src/gam/fileutils_test.py

@@ -1,244 +0,0 @@
-"""Tests for fileutils."""
-
-import io
-import os
-import unittest
-from unittest.mock import MagicMock
-from unittest.mock import patch
-
-from gam import fileutils
-
-
-class FileutilsTest(unittest.TestCase):
-
-    def setUp(self):
-        self.fake_path = '/some/path/to/file'
-        super(FileutilsTest, self).setUp()
-
-    @patch.object(fileutils.sys, 'stdin')
-    def test_open_file_stdin(self, mock_stdin):
-        mock_stdin.read.return_value = 'some stdin content'
-        f = fileutils.open_file('-', mode='r')
-        self.assertIsInstance(f, fileutils.io.StringIO)
-        self.assertEqual(f.getvalue(), mock_stdin.read.return_value)
-
-    def test_open_file_stdout(self):
-        f = fileutils.open_file('-', mode='w')
-        self.assertEqual(fileutils.sys.stdout, f)
-
-    @patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
-    def test_open_file_opens_correct_path(self, mock_open):
-        f = fileutils.open_file(self.fake_path)
-        self.assertEqual(self.fake_path, mock_open.call_args[0][0])
-        self.assertEqual(mock_open.return_value, f)
-
-    @patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
-    def test_open_file_expands_user_file_path(self, mock_open):
-        file_path = '~/some/path/containing/tilde/shortcut/to/home'
-        fileutils.open_file(file_path)
-        opened_path = mock_open.call_args[0][0]
-        home_path = os.environ.get('HOME')
-        self.assertIsNotNone(home_path)
-        self.assertIn(home_path, opened_path)
-
-    @patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
-    def test_open_file_opens_correct_mode(self, mock_open):
-        fileutils.open_file(self.fake_path)
-        self.assertEqual('r', mock_open.call_args[0][1])
-
-    @patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
-    def test_open_file_encoding_for_binary(self, mock_open):
-        fileutils.open_file(self.fake_path, mode='b')
-        self.assertIsNone(mock_open.call_args[1]['encoding'])
-
-    @patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
-    def test_open_file_default_system_encoding(self, mock_open):
-        fileutils.open_file(self.fake_path)
-        self.assertEqual(fileutils.GM_Globals[fileutils.GM_SYS_ENCODING],
-                         mock_open.call_args[1]['encoding'])
-
-    @patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
-    def test_open_file_utf8_encoding_specified(self, mock_open):
-        fileutils.open_file(self.fake_path, encoding='UTF-8')
-        self.assertEqual(fileutils.UTF8_SIG, mock_open.call_args[1]['encoding'])
-
-    def test_open_file_strips_utf_bom_in_utf(self):
-        bom_prefixed_data = u'\ufefffoobar'
-        fake_file = io.StringIO(bom_prefixed_data)
-        mock_open = MagicMock(spec=open, return_value=fake_file)
-        with patch.object(fileutils, 'open', mock_open):
-            f = fileutils.open_file(self.fake_path, strip_utf_bom=True)
-            self.assertEqual('foobar', f.read())
-
-    def test_open_file_strips_utf_bom_in_non_utf(self):
-        bom_prefixed_data = b'\xef\xbb\xbffoobar'.decode('iso-8859-1')
-
-        # We need to trick the method under test into believing that a StringIO
-        # instance is a file with an encoding. Since StringIO does not usually have,
-        # an encoding, we'll mock it and add our own encoding, but send the other
-        # methods in use (read and seek) back to the real StringIO object.
-        real_stringio = io.StringIO(bom_prefixed_data)
-        mock_file = MagicMock(spec=io.StringIO)
-        mock_file.read.side_effect = real_stringio.read
-        mock_file.seek.side_effect = real_stringio.seek
-        mock_file.encoding = 'iso-8859-1'
-
-        mock_open = MagicMock(spec=open, return_value=mock_file)
-        with patch.object(fileutils, 'open', mock_open):
-            f = fileutils.open_file(self.fake_path, strip_utf_bom=True)
-            self.assertEqual('foobar', f.read())
-
-    def test_open_file_strips_utf_bom_in_binary(self):
-        bom_prefixed_data = u'\ufefffoobar'.encode('UTF-8')
-        fake_file = io.BytesIO(bom_prefixed_data)
-        mock_open = MagicMock(spec=open, return_value=fake_file)
-        with patch.object(fileutils, 'open', mock_open):
-            f = fileutils.open_file(self.fake_path,
-                                    mode='rb',
-                                    strip_utf_bom=True)
-            self.assertEqual(b'foobar', f.read())
-
-    def test_open_file_strip_utf_bom_when_no_bom_in_data(self):
-        no_bom_data = 'This data has no BOM'
-        fake_file = io.StringIO(no_bom_data)
-        mock_open = MagicMock(spec=open, return_value=fake_file)
-
-        with patch.object(fileutils, 'open', mock_open):
-            f = fileutils.open_file(self.fake_path, strip_utf_bom=True)
-            # Since there was no opening BOM, we should be back at the beginning of
-            # the file.
-            self.assertEqual(fake_file.tell(), 0)
-            self.assertEqual(f.read(), no_bom_data)
-
-    @patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
-    def test_open_file_exits_on_io_error(self, mock_open):
-        mock_open.side_effect = IOError('Fake IOError')
-        with self.assertRaises(SystemExit) as context:
-            fileutils.open_file(self.fake_path)
-        self.assertEqual(context.exception.code, 6)
-
-    def test_close_file_closes_file_successfully(self):
-        mock_file = MagicMock()
-        self.assertTrue(fileutils.close_file(mock_file))
-        self.assertEqual(mock_file.close.call_count, 1)
-
-    def test_close_file_with_error(self):
-        mock_file = MagicMock()
-        mock_file.close.side_effect = IOError()
-        self.assertFalse(fileutils.close_file(mock_file))
-        self.assertEqual(mock_file.close.call_count, 1)
-
-    @patch.object(fileutils.sys, 'stdin')
-    def test_read_file_from_stdin(self, mock_stdin):
-        mock_stdin.read.return_value = 'some stdin content'
-        self.assertEqual(fileutils.read_file('-'), mock_stdin.read.return_value)
-
-    @patch.object(fileutils, '_open_file')
-    def test_read_file_default_params(self, mock_open_file):
-        fake_content = 'some fake content'
-        mock_open_file.return_value.__enter__().read.return_value = fake_content
-        self.assertEqual(fileutils.read_file(self.fake_path), fake_content)
-        self.assertEqual(mock_open_file.call_args[0][0], self.fake_path)
-        self.assertEqual(mock_open_file.call_args[0][1], 'r')
-        self.assertIsNone(mock_open_file.call_args[1]['newline'])
-
-    @patch.object(fileutils.display, 'print_warning')
-    @patch.object(fileutils, '_open_file')
-    def test_read_file_continues_on_errors_without_displaying(
-        self, mock_open_file, mock_print_warning):
-        mock_open_file.side_effect = IOError()
-        contents = fileutils.read_file(self.fake_path,
-                                       continue_on_error=True,
-                                       display_errors=False)
-        self.assertIsNone(contents)
-        self.assertFalse(mock_print_warning.called)
-
-    @patch.object(fileutils.display, 'print_warning')
-    @patch.object(fileutils, '_open_file')
-    def test_read_file_displays_errors(self, mock_open_file,
-                                       mock_print_warning):
-        mock_open_file.side_effect = IOError()
-        fileutils.read_file(self.fake_path,
-                            continue_on_error=True,
-                            display_errors=True)
-        self.assertTrue(mock_print_warning.called)
-
-    @patch.object(fileutils, '_open_file')
-    def test_read_file_exits_code_6_when_continue_on_error_is_false(
-        self, mock_open_file):
-        mock_open_file.side_effect = IOError()
-        with self.assertRaises(SystemExit) as context:
-            fileutils.read_file(self.fake_path, continue_on_error=False)
-        self.assertEqual(context.exception.code, 6)
-
-    @patch.object(fileutils, '_open_file')
-    def test_read_file_exits_code_2_on_lookuperror(self, mock_open_file):
-        mock_open_file.return_value.__enter__().read.side_effect = LookupError()
-        with self.assertRaises(SystemExit) as context:
-            fileutils.read_file(self.fake_path)
-        self.assertEqual(context.exception.code, 2)
-
-    @patch.object(fileutils, '_open_file')
-    def test_read_file_exits_code_2_on_unicodeerror(self, mock_open_file):
-        mock_open_file.return_value.__enter__().read.side_effect = UnicodeError(
-        )
-        with self.assertRaises(SystemExit) as context:
-            fileutils.read_file(self.fake_path)
-        self.assertEqual(context.exception.code, 2)
-
-    @patch.object(fileutils, '_open_file')
-    def test_read_file_exits_code_2_on_unicodedecodeerror(self, mock_open_file):
-        fake_decode_error = UnicodeDecodeError('fake-encoding', b'fakebytes', 0,
-                                               1, 'testing only')
-        mock_open_file.return_value.__enter__(
-        ).read.side_effect = fake_decode_error
-        with self.assertRaises(SystemExit) as context:
-            fileutils.read_file(self.fake_path)
-        self.assertEqual(context.exception.code, 2)
-
-    @patch.object(fileutils, '_open_file')
-    def test_write_file_writes_data_to_file(self, mock_open_file):
-        fake_data = 'some fake data'
-        fileutils.write_file(self.fake_path, fake_data)
-        self.assertEqual(mock_open_file.call_args[0][0], self.fake_path)
-        self.assertEqual(mock_open_file.call_args[0][1], 'w')
-
-        opened_file = mock_open_file.return_value.__enter__()
-        self.assertTrue(opened_file.write.called)
-        self.assertEqual(opened_file.write.call_args[0][0], fake_data)
-
-    @patch.object(fileutils.display, 'print_error')
-    @patch.object(fileutils, '_open_file')
-    def test_write_file_continues_on_errors_without_displaying(
-        self, mock_open_file, mock_print_error):
-        mock_open_file.side_effect = IOError()
-        status = fileutils.write_file(self.fake_path,
-                                      'foo data',
-                                      continue_on_error=True,
-                                      display_errors=False)
-        self.assertFalse(status)
-        self.assertFalse(mock_print_error.called)
-
-    @patch.object(fileutils.display, 'print_error')
-    @patch.object(fileutils, '_open_file')
-    def test_write_file_displays_errors(self, mock_open_file, mock_print_error):
-        mock_open_file.side_effect = IOError()
-        fileutils.write_file(self.fake_path,
-                             'foo data',
-                             continue_on_error=True,
-                             display_errors=True)
-        self.assertTrue(mock_print_error.called)
-
-    @patch.object(fileutils, '_open_file')
-    def test_write_file_exits_code_6_when_continue_on_error_is_false(
-        self, mock_open_file):
-        mock_open_file.side_effect = IOError()
-        with self.assertRaises(SystemExit) as context:
-            fileutils.write_file(self.fake_path,
-                                 'foo data',
-                                 continue_on_error=False)
-        self.assertEqual(context.exception.code, 6)
-
-
-if __name__ == '__main__':
-    unittest.main()

src/gam/gamlib/__init__.py

@@ -0,0 +1,17 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2023 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.

src/gam/gamlib/glaction.py

@@ -0,0 +1,308 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2024 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""GAM action processing
+
+"""
+
+class GamAction():
+
+# Keys into NAMES; arbitrary values but must be unique
+  ACCEPT = 'acpt'
+  ADD = 'add '
+  ADD_PREVIEW = 'addp'
+  APPEND = 'apnd'
+  APPROVE = 'aprv'
+  ARCHIVE = 'arch'
+  BACKUP = 'back'
+  BLOCK = 'blok'
+  CANCEL = 'canc'
+  CANCEL_WIPE = 'canw'
+  CHECK = 'chek'
+  CLAIM = 'clai'
+  CLAIM_OWNERSHIP = 'clow'
+  CLEAR = 'clea'
+  CLOSE = 'clos'
+  COLLECT = 'coll'
+  COMMENT = 'comm'
+  COPY = 'copy'
+  COPY_MERGE = 'copm'
+  CREATE = 'crea'
+  CREATE_PREVIEW = 'crep'
+  CREATE_SHORTCUT = 'crsc'
+  DEDUP = 'dedu'
+  DELETE = 'dele'
+  DELETE_EMPTY = 'delm'
+  DELETE_PREVIEW = 'delp'
+  DELETE_SHORTCUT = 'desc'
+  DEPROVISION = 'depr'
+  DISABLE = 'disa'
+  DOWNLOAD = 'down'
+  DRAFT = 'draf'
+  EMPTY = 'empt'
+  ENABLE = 'enbl'
+  END = 'end '
+  EXISTS = 'exis'
+  EXPORT = 'expo'
+  EXTRACT = 'extr'
+  GET_COMMAND_RESULT = 'gtcr'
+  FETCH = 'fetc'
+  FORWARD = 'forw'
+  HIDE = 'hide'
+  IMPORT = 'impo'
+  INFO = 'info'
+  INITIALIZE = 'init'
+  INSERT = 'insr'
+  INVALIDATE = 'inva'
+  ISSUE_COMMAND = 'isco'
+  LIST = 'list'
+  LOOKUP = 'look'
+  MERGE = 'merg'
+  MODIFY = 'modi'
+  MOVE = 'move'
+  MOVE_MERGE = 'movm'
+  NOACTION = 'noac'
+  NOACTION_PREVIEW = 'noap'
+  OBLITERATE = 'obli'
+  PERFORM = 'perf'
+  PRE_PROVISIONED_DISABLE ='ppdi'
+  PRE_PROVISIONED_REENABLE ='ppre'
+  PRINT = 'prin'
+  PROCESS = 'proc'
+  PROCESS_PREVIEW = 'prop'
+  PURGE = 'purg'
+  RECREATE = 'recr'
+  REENABLE = 'reen'
+  REFRESH = 'refr'
+  RELABEL = 'rela'
+  REMOVE = 'remo'
+  REMOVE_PREVIEW = 'remp'
+  RENAME = 'rena'
+  REOPEN = 'reop'
+  REPLACE = 'repl'
+  REPLACE_DOMAIN = 'repd'
+  REPORT = 'repo'
+  RESET_YUBIKEY_PIV = 'rpiv'
+  RESPOND = 'resp'
+  RESTORE = 'rest'
+  RESUBMIT = 'res'
+  RETAIN = 'reta'
+  RETRIEVE_DATA = 'retd'
+  REVOKE = 'revo'
+  SAVE = 'save'
+  SEND = 'send'
+  SENDEMAIL = 'snem'
+  SET = 'set '
+  SETUP = 'setu'
+  SHARE = 'shar'
+  SHOW = 'show'
+  SIGNOUT = 'siou'
+  SKIP = 'skip'
+  SPAM = 'spam'
+  SUBMIT = 'subm'
+  SUSPEND = 'susp'
+  SYNC = 'sync'
+  TRANSFER = 'tran'
+  TRANSFER_OWNERSHIP = 'trow'
+  TRASH = 'tras'
+  TURNOFF2SV = 'to2s'
+  UNDELETE = 'unde'
+  UNHIDE = 'unhi'
+  UNSUSPEND = 'unsu'
+  UNTRASH = 'untr'
+  UPDATE = 'upda'
+  UPDATE_MOVE = 'upmo'
+  UPDATE_OWNER = 'updo'
+  UPDATE_PREVIEW = 'updp'
+  UPLOAD = 'uplo'
+  UNZIP = 'unzi'
+  USE = 'use '
+  VERIFY = 'vrfy'
+  WAITFORMAILBOX = 'wamb'
+  WATCH = 'watc'
+  WIPE = 'wipe'
+  WIPE_PREVIEW = 'wipp'
+  # Usage:
+  # ACTION_NAMES[1] n Items - Delete 10 Users
+  # Item xxx ACTION_NAMES[0] - User xxx Deleted
+  # These values can be translated into other languages
+  _NAMES = {
+    ACCEPT: ['Accepted', 'Accept'],
+    ADD: ['Added', 'Add'],
+    ADD_PREVIEW: ['Added (Preview)', 'Add (Preview)'],
+    APPEND: ['Appended', 'Append'],
+    APPROVE: ['Approved', 'Approve'],
+    ARCHIVE: ['Archived', 'Archive'],
+    BACKUP: ['Backed up', 'Backup'],
+    BLOCK: ['Blocked', 'Block'],
+    CANCEL: ['Cancelled', 'Cancel'],
+    CANCEL_WIPE: ['Wipe Cancelled', 'Cancel Wipe'],
+    CHECK: ['Checked', 'Check'],
+    CLAIM: ['Claimed', 'Claim'],
+    CLAIM_OWNERSHIP: ['Ownership Claimed', 'Claim Ownership'],
+    CLEAR: ['Cleared', 'Clear'],
+    CLOSE: ['Closed', 'Close'],
+    COLLECT: ['Collected', 'Collect'],
+    COMMENT: ['Commented', 'Comment'],
+    COPY: ['Copied', 'Copy'],
+    COPY_MERGE: ['Copied(Merge)', 'Copy(Merge)'],
+    CREATE: ['Created', 'Create'],
+    CREATE_PREVIEW: ['Created (Preview)', 'Create (Preview)'],
+    CREATE_SHORTCUT: ['Created Shortcut', 'Create Shortcut'],
+    DEDUP: ['Duplicates Deleted', 'Delete Duplicates'],
+    DELETE: ['Deleted', 'Delete'],
+    DELETE_EMPTY: ['Deleted', 'Delete Empty'],
+    DELETE_PREVIEW: ['Deleted (Preview)', 'Delete (Preview)'],
+    DELETE_SHORTCUT: ['Deleted Shortcut', 'Delete Shortcut'],
+    DEPROVISION: ['Deprovisioned', 'Deprovision'],
+    DISABLE: ['Disabled', 'Disable'],
+    DOWNLOAD: ['Downloaded', 'Download'],
+    DRAFT: ['Drafted', 'Draft'],
+    EMPTY: ['Emptied', 'Empty'],
+    ENABLE: ['Enabled', 'Enable'],
+    END: ['Ended', 'End'],
+    EXISTS: ['Exists', 'Exists'],
+    EXPORT: ['Exported', 'Export'],
+    EXTRACT: ['Extracted', 'Extract'],
+    FORWARD: ['Forwarded', 'Forward'],
+    GET_COMMAND_RESULT: ['Got Command Result', 'Get Command Result'],
+    HIDE: ['Hidden', 'Hide'],
+    IMPORT: ['Imported', 'Import'],
+    INFO: ['Shown', 'Show Info'],
+    INITIALIZE: ['Initialized', 'Initialize'],
+    INSERT: ['Inserted', 'Insert'],
+    INVALIDATE: ['Invalidated', 'Invalidate'],
+    ISSUE_COMMAND: ['Command Issued', 'Issue Command'],
+    LIST: ['Listed', 'List'],
+    LOOKUP: ['Lookedup', 'Lookup'],
+    MERGE: ['Merged', 'Merge'],
+    MODIFY: ['Modified', 'Modify'],
+    MOVE: ['Moved', 'Move'],
+    MOVE_MERGE: ['Moved(Merge)', 'Move(Merge)'],
+    NOACTION: ['No Action', 'No Action'],
+    NOACTION_PREVIEW: ['No Action (Preview)', 'No Action (Preview)'],
+    OBLITERATE: ['Obliterated', 'Obliterate'],
+    PERFORM: ['Action Performed', 'Perform Action'],
+    PRE_PROVISIONED_DISABLE: ['PreProvisioned Disabled', 'PreProvisioned Disable'],
+    PRE_PROVISIONED_REENABLE: ['PreProvisioned Reenabled', 'PreProvisioned Reenable'],
+    PRINT: ['Printed', 'Print'],
+    PROCESS: ['Processed', 'Process'],
+    PROCESS_PREVIEW: ['Processed (Preview)', 'Process (Preview)'],
+    PURGE: ['Purged', 'Purge'],
+    RECREATE: ['Recreated', 'Recreate'],
+    REENABLE: ['Reenabled', 'Reenable'],
+    REFRESH: ['Refreshed', 'Refresh'],
+    RELABEL: ['Relabeled', 'Relabel'],
+    REMOVE: ['Removed', 'Remove'],
+    REMOVE_PREVIEW: ['Removed (Preview)', 'Remove (Preview)'],
+    RENAME: ['Renamed', 'Rename'],
+    REOPEN: ['Reopened', 'Reopen'],
+    REPLACE: ['Replaced', 'Replace'],
+    REPLACE_DOMAIN: ['Domain Replaced', 'Replace Domain'],
+    REPORT: ['Reported', 'Report'],
+    RESET_YUBIKEY_PIV: ['Yubikey PIV Reset', 'Reset Yubikey PIV'],
+    RESPOND: ['Responded', 'Respond'],
+    RESTORE: ['Restored', 'Restore'],
+    RESUBMIT: ['Resubmitted', 'Resubmit'],
+    RETAIN: ['Retained', 'Retain'],
+    RETRIEVE_DATA: ['Data Retrieved', 'Retrieve Data'],
+    REVOKE: ['Revoked', 'Revoke'],
+    SAVE: ['Saved', 'Save'],
+    SEND: ['Sent', 'Send'],
+    SENDEMAIL: ['Email Sent', 'Send Email'],
+    SET: ['Set', 'Set'],
+    SETUP: ['Set Up', 'Set Up'],
+    SHARE: ['Shared', 'Share'],
+    SHOW: ['Shown', 'Show'],
+    SIGNOUT: ['Signed Out', 'Signout'],
+    SKIP: ['Skipped', 'Skip'],
+    SPAM: ['Marked as Spam', 'Mark as Spam'],
+    SUBMIT: ['Submitted', 'Submit'],
+    SUSPEND: ['Suspended', 'Suspend'],
+    SYNC: ['Synced', 'Sync'],
+    TRANSFER: ['Transferred', 'Transfer'],
+    TRANSFER_OWNERSHIP: ['Ownership Transferred', 'Transfer Ownership'],
+    TRASH: ['Trashed', 'Trash'],
+    TURNOFF2SV: ['2-Step Verification Turned Off', 'Turn Off 2-Step Verification'],
+    UNDELETE: ['Undeleted', 'Undelete'],
+    UNHIDE: ['Unhidden', 'Unhide'],
+    UNSUSPEND: ['Unsuspended', 'Unsuspend'],
+    UNTRASH: ['Untrashed', 'Untrash'],
+    UNZIP: ['Unzipped', 'Unzip'],
+    UPDATE: ['Updated', 'Update'],
+    UPDATE_MOVE: ['Updated/Moved', 'Update/Move'],
+    UPDATE_OWNER: ['Updated to Owner', 'Update to Owner'],
+    UPDATE_PREVIEW: ['Updated (Preview)', 'Update (Preview)'],
+    UPLOAD: ['Uploaded', 'Upload'],
+    USE: ['Used', 'Use'],
+    VERIFY: ['Verified', 'Verify'],
+    WAITFORMAILBOX: ['Mailbox is Setup', 'Check Mailbox is Setup'],
+    WATCH: ['Watched', 'Watch'],
+    WIPE: ['Wiped', 'Wipe'],
+    WIPE_PREVIEW: ['Wiped (Preview)', 'Wipe (Preview)'],
+    }
+  #
+  MODIFIER_CONTENTS_WITH = 'contents with'
+  MODIFIER_FOR = 'for'
+  MODIFIER_FROM = 'from'
+  MODIFIER_IN = 'in'
+  MODIFIER_INTO = 'into'
+  MODIFIER_PREVIOUSLY_IN = 'previously in'
+  MODIFIER_TO = 'to'
+  MODIFIER_WITH_COTEACHER_OWNER = 'with co-teacher as owner'
+  MODIFIER_WITH_NEW_TEACHER_OWNER = 'with new teacher as owner'
+  MODIFIER_WITH_CURRENT_OWNER = 'with current owner'
+  MODIFIER_WITH = 'with'
+  MODIFIER_WITH_CONTENT_FROM = 'with content from'
+  PREFIX_NOT = 'Not'
+  PREVIEW = 'Preview'
+  SUCCESS = 'Success'
+  SUFFIX_FAILED = 'Failed'
+
+  def __init__(self):
+    self.action = None
+
+  def Set(self, action):
+    self.action = action
+
+  def Get(self):
+    return self.action
+
+  def ToPerform(self):
+    return self._NAMES[self.action][1]
+
+  def Performed(self):
+    return self._NAMES[self.action][0]
+
+  def Failed(self):
+    return f'{self._NAMES[self.action][1]} {self.SUFFIX_FAILED}'
+
+  def NotPerformed(self):
+    actionWords = self._NAMES[self.action][0].split(' ')
+    if len(actionWords) != 2:
+      return f'{self.PREFIX_NOT} {self._NAMES[self.action][0]}'
+    return f'{actionWords[0]} {self.PREFIX_NOT} {actionWords[1]}'
+
+  def PerformedName(self, action):
+    return self._NAMES[action][0]
+
+  def ToPerformName(self, action):
+    return self._NAMES[action][1]
+
+  def csvFormat(self):
+    return self.action == self.PRINT

src/gam/gamlib/glapi.py

@@ -0,0 +1,840 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2024 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""Google API resources
+
+"""
+# APIs
+ACCESSCONTEXTMANAGER = 'accesscontextmanager'
+ALERTCENTER = 'alertcenter'
+ANALYTICS = 'analytics'
+ANALYTICS_ADMIN = 'analyticsadmin'
+CALENDAR = 'calendar'
+CBCM = 'cbcm'
+CHAT = 'chat'
+CHAT_EVENTS = 'chatevents'
+CHAT_MEMBERSHIPS = 'chatmemberships'
+CHAT_MEMBERSHIPS_ADMIN = 'chatmembershipsadmin'
+CHAT_MESSAGES = 'chatmessages'
+CHAT_SPACES = 'chatspaces'
+CHAT_SPACES_ADMIN = 'chatspacesadmin'
+CHAT_SPACES_DELETE = 'chatspacesdelete'
+CHAT_SPACES_DELETE_ADMIN = 'chatspacesdeleteadmin'
+CHROMEMANAGEMENT = 'chromemanagement'
+CHROMEMANAGEMENT_APPDETAILS = 'chromemanagementappdetails'
+CHROMEMANAGEMENT_CHROMEPROFILES = 'chromemanagementchromeprofiles'
+CHROMEMANAGEMENT_TELEMETRY = 'chromemanagementtelemetry'
+CHROMEPOLICY = 'chromepolicy'
+CHROMEVERSIONHISTORY = 'versionhistory'
+CLASSROOM = 'classroom'
+CLOUDCHANNEL = 'cloudchannel'
+CLOUDIDENTITY_DEVICES = 'cloudidentitydevices'
+CLOUDIDENTITY_GROUPS = 'cloudidentitygroups'
+CLOUDIDENTITY_GROUPS_BETA = 'cloudidentitygroupsbeta'
+CLOUDIDENTITY_INBOUND_SSO = 'cloudidentityinboundsso'
+CLOUDIDENTITY_ORGUNITS = 'cloudidentityorgunits'
+CLOUDIDENTITY_ORGUNITS_BETA = 'cloudidentityorgunitsbeta'
+CLOUDIDENTITY_POLICY = 'cloudidentitypolicy'
+CLOUDIDENTITY_USERINVITATIONS = 'cloudidentityuserinvitations'
+CLOUDRESOURCEMANAGER = 'cloudresourcemanager'
+CONTACTS = 'contacts'
+CONTACTDELEGATION = 'contactdelegation'
+DATATRANSFER = 'datatransfer'
+DIRECTORY = 'directory'
+DOCS = 'docs'
+DRIVE2 = 'drive2'
+DRIVE3 = 'drive3'
+DRIVETD = 'drivetd'
+DRIVEACTIVITY = 'driveactivity'
+DRIVELABELS = 'drivelabels'
+DRIVELABELS_ADMIN = 'drivelabelsadmin'
+DRIVELABELS_USER = 'drivelabelsuser'
+EMAIL_AUDIT = 'email-audit'
+FORMS = 'forms'
+GMAIL = 'gmail'
+GROUPSMIGRATION = 'groupsmigration'
+GROUPSSETTINGS = 'groupssettings'
+IAM = 'iam'
+IAM_CREDENTIALS = 'iamcredentials'
+IAP = 'iap'
+KEEP = 'keep'
+LICENSING = 'licensing'
+LOOKERSTUDIO = 'datastudio'
+MEET = 'meet'
+OAUTH2 = 'oauth2'
+ORGPOLICY = 'orgpolicy'
+PEOPLE = 'people'
+PEOPLE_DIRECTORY = 'peopledirectory'
+PEOPLE_OTHERCONTACTS = 'peopleothercontacts'
+PRINTERS = 'printers'
+PUBSUB = 'pubsub'
+REPORTS = 'reports'
+RESELLER = 'reseller'
+SERVICEACCOUNTLOOKUP = 'serviceaccountlookup'
+SERVICEMANAGEMENT = 'servicemanagement'
+SERVICEUSAGE = 'serviceusage'
+SHEETS = 'sheets'
+SHEETSTD = 'sheetstd'
+SITES = 'sites'
+SITEVERIFICATION = 'siteVerification'
+STORAGE = 'storage'
+STORAGEREAD = 'storageread'
+STORAGEWRITE = 'storagewrite'
+TASKS = 'tasks'
+VAULT = 'vault'
+YOUTUBE = 'youtube'
+#
+CHROMEVERSIONHISTORY_URL = 'https://versionhistory.googleapis.com/v1/chrome/platforms'
+DRIVE_SCOPE = 'https://www.googleapis.com/auth/drive'
+GMAIL_SEND_SCOPE = 'https://www.googleapis.com/auth/gmail.send'
+GOOGLE_AUTH_PROVIDER_X509_CERT_URL = 'https://www.googleapis.com/oauth2/v1/certs'
+GOOGLE_OAUTH2_ENDPOINT = 'https://accounts.google.com/o/oauth2/v2/auth'
+GOOGLE_OAUTH2_TOKEN_ENDPOINT = 'https://oauth2.googleapis.com/token'
+CLOUD_PLATFORM_SCOPE = 'https://www.googleapis.com/auth/cloud-platform'
+IAM_SCOPE = 'https://www.googleapis.com/auth/iam'
+PEOPLE_SCOPE = 'https://www.googleapis.com/auth/contacts'
+STORAGE_READONLY_SCOPE = 'https://www.googleapis.com/auth/devstorage.read_only'
+STORAGE_READWRITE_SCOPE = 'https://www.googleapis.com/auth/devstorage.read_write'
+USERINFO_EMAIL_SCOPE = 'https://www.googleapis.com/auth/userinfo.email' # email
+USERINFO_PROFILE_SCOPE = 'https://www.googleapis.com/auth/userinfo.profile' # profile
+VAULT_SCOPES = ['https://www.googleapis.com/auth/ediscovery', 'https://www.googleapis.com/auth/ediscovery.readonly']
+REQUIRED_SCOPES = [USERINFO_EMAIL_SCOPE, USERINFO_PROFILE_SCOPE]
+REQUIRED_SCOPES_SET = set(REQUIRED_SCOPES)
+#
+JWT_APIS = {
+  ACCESSCONTEXTMANAGER: [CLOUD_PLATFORM_SCOPE],
+  CHAT: ['https://www.googleapis.com/auth/chat.bot'],
+  CLOUDRESOURCEMANAGER: [CLOUD_PLATFORM_SCOPE],
+  ORGPOLICY: [CLOUD_PLATFORM_SCOPE],
+  }
+#
+SCOPELESS_APIS = {
+  CHROMEVERSIONHISTORY,
+  OAUTH2,
+  SERVICEACCOUNTLOOKUP,
+  }
+#
+APIS_NEEDING_ACCESS_TOKEN = {
+  CBCM: ['https://www.googleapis.com/auth/admin.directory.device.chromebrowsers']
+  }
+#
+REFRESH_PERM_ERRORS = [
+  'invalid_grant: reauth related error (rapt_required)', # no way to reauth today
+  'invalid_grant: Token has been expired or revoked',
+  ]
+
+OAUTH2_TOKEN_ERRORS = [
+  'access_denied',
+  'access_denied: Requested client not authorized',
+  'access_denied: Account restricted',
+  'internal_failure: Backend Error',
+  'internal_failure: None',
+  'invalid_grant',
+  'invalid_grant: Bad Request',
+  'invalid_grant: Invalid email or User ID',
+  'invalid_grant: Not a valid email',
+  'invalid_grant: Invalid JWT: No valid verifier found for issuer',
+  'invalid_grant: reauth related error (invalid_rapt)',
+  'invalid_grant: The account has been deleted',
+  'invalid_request: Invalid impersonation prn email address'
+  ]
+OAUTH2_UNAUTHORIZED_ERRORS = [
+  'unauthorized_client: Client is unauthorized to retrieve access tokens using this method',
+  'unauthorized_client: Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested',
+  'unauthorized_client: Unauthorized client or scope in request',
+  ]
+
+PROJECT_APIS = [
+  'accesscontextmanager.googleapis.com',
+  'admin.googleapis.com',
+  'alertcenter.googleapis.com',
+  'analytics.googleapis.com',
+  'analyticsadmin.googleapis.com',
+#  'audit.googleapis.com',
+  'calendar-json.googleapis.com',
+  'chat.googleapis.com',
+  'chromemanagement.googleapis.com',
+  'chromepolicy.googleapis.com',
+  'classroom.googleapis.com',
+  'cloudchannel.googleapis.com',
+  'cloudidentity.googleapis.com',
+  'cloudresourcemanager.googleapis.com',
+  'contacts.googleapis.com',
+  'datastudio.googleapis.com',
+  'docs.googleapis.com',
+  'drive.googleapis.com',
+  'driveactivity.googleapis.com',
+  'drivelabels.googleapis.com',
+  'forms.googleapis.com',
+  'gmail.googleapis.com',
+  'groupsmigration.googleapis.com',
+  'groupssettings.googleapis.com',
+  'iam.googleapis.com',
+  'iap.googleapis.com',
+  'keep.googleapis.com',
+  'licensing.googleapis.com',
+  'meet.googleapis.com',
+  'people.googleapis.com',
+  'pubsub.googleapis.com',
+  'reseller.googleapis.com',
+  'sheets.googleapis.com',
+  'siteverification.googleapis.com',
+  'storage-api.googleapis.com',
+  'tasks.googleapis.com',
+  'vault.googleapis.com',
+  'youtube.googleapis.com',
+  ]
+
+_INFO = {
+  ACCESSCONTEXTMANAGER: {'name': 'Access Context Manager API', 'version': 'v1', 'v2discovery': True},
+  ALERTCENTER: {'name': 'AlertCenter API', 'version': 'v1beta1', 'v2discovery': True},
+  ANALYTICS: {'name': 'Analytics API', 'version': 'v3', 'v2discovery': False},
+  ANALYTICS_ADMIN: {'name': 'Analytics Admin API', 'version': 'v1beta', 'v2discovery': True},
+  CALENDAR: {'name': 'Calendar API', 'version': 'v3', 'v2discovery': True, 'mappedAPI': 'calendar-json'},
+  CBCM: {'name': 'Chrome Browser Cloud Management API', 'version': 'v1.1beta1', 'v2discovery': True, 'localjson': True},
+  CHAT: {'name': 'Chat API', 'version': 'v1', 'v2discovery': True},
+  CHAT_EVENTS: {'name': 'Chat API - Events', 'version': 'v1', 'v2discovery': True, 'mappedAPI': CHAT},
+  CHAT_MEMBERSHIPS: {'name': 'Chat API - Memberships', 'version': 'v1', 'v2discovery': True, 'mappedAPI': CHAT},
+  CHAT_MEMBERSHIPS_ADMIN: {'name': 'Chat API - Memberships Admin', 'version': 'v1', 'v2discovery': True, 'mappedAPI': CHAT},
+  CHAT_MESSAGES: {'name': 'Chat API - Messages', 'version': 'v1', 'v2discovery': True, 'mappedAPI': CHAT},
+  CHAT_SPACES: {'name': 'Chat API - Spaces', 'version': 'v1', 'v2discovery': True, 'mappedAPI': CHAT},
+  CHAT_SPACES_ADMIN: {'name': 'Chat API - Spaces Admin', 'version': 'v1', 'v2discovery': True, 'mappedAPI': CHAT},
+  CHAT_SPACES_DELETE: {'name': 'Chat API - Spaces Delete', 'version': 'v1', 'v2discovery': True, 'mappedAPI': CHAT},
+  CHAT_SPACES_DELETE_ADMIN: {'name': 'Chat API - Spaces Delete Admin', 'version': 'v1', 'v2discovery': True, 'mappedAPI': CHAT},
+  CLASSROOM: {'name': 'Classroom API', 'version': 'v1', 'v2discovery': True},
+  CHROMEMANAGEMENT: {'name': 'Chrome Management API', 'version': 'v1', 'v2discovery': True},
+  CHROMEMANAGEMENT_APPDETAILS: {'name': 'Chrome Management API - AppDetails', 'version': 'v1', 'v2discovery': True, 'mappedAPI': CHROMEMANAGEMENT},
+  CHROMEMANAGEMENT_TELEMETRY: {'name': 'Chrome Management API - Telemetry', 'version': 'v1', 'v2discovery': True, 'mappedAPI': CHROMEMANAGEMENT},
+  CHROMEPOLICY: {'name': 'Chrome Policy API', 'version': 'v1', 'v2discovery': True},
+  CHROMEVERSIONHISTORY: {'name': 'Chrome Version History API', 'version': 'v1', 'v2discovery': True},
+  CLOUDCHANNEL: {'name': 'Channel Channel API', 'version': 'v1', 'v2discovery': True},
+  CLOUDIDENTITY_DEVICES: {'name': 'Cloud Identity Devices API', 'version': 'v1', 'v2discovery': True, 'mappedAPI': 'cloudidentity'},
+  CLOUDIDENTITY_GROUPS: {'name': 'Cloud Identity Groups API', 'version': 'v1', 'v2discovery': True, 'mappedAPI': 'cloudidentity'},
+  CLOUDIDENTITY_GROUPS_BETA: {'name': 'Cloud Identity Groups API', 'version': 'v1beta1', 'v2discovery': True, 'mappedAPI': 'cloudidentity'},
+  CLOUDIDENTITY_INBOUND_SSO: {'name': 'Cloud Identity Inbound SSO API', 'version': 'v1', 'v2discovery': True, 'mappedAPI': 'cloudidentity'},
+  CLOUDIDENTITY_ORGUNITS: {'name': 'Cloud Identity OrgUnits API', 'version': 'v1', 'v2discovery': True, 'mappedAPI': 'cloudidentity'},
+  CLOUDIDENTITY_ORGUNITS_BETA: {'name': 'Cloud Identity OrgUnits API', 'version': 'v1beta1', 'v2discovery': True, 'mappedAPI': 'cloudidentity'},
+  CLOUDIDENTITY_POLICY: {'name': 'Cloud Identity Policy API', 'version': 'v1beta1', 'v2discovery': True, 'mappedAPI': 'cloudidentity'},
+  CLOUDIDENTITY_USERINVITATIONS: {'name': 'Cloud Identity User Invitations API', 'version': 'v1', 'v2discovery': True, 'mappedAPI': 'cloudidentity'},
+  CLOUDRESOURCEMANAGER: {'name': 'Cloud Resource Manager API v3', 'version': 'v3', 'v2discovery': True},
+  CONTACTS: {'name': 'Contacts API', 'version': 'v3', 'v2discovery': False},
+  CONTACTDELEGATION: {'name': 'Contact Delegation API', 'version': 'v1', 'v2discovery': True, 'localjson': True},
+  DATATRANSFER: {'name': 'Data Transfer API', 'version': 'datatransfer_v1', 'v2discovery': True, 'mappedAPI': 'admin'},
+  DIRECTORY: {'name': 'Directory API', 'version': 'directory_v1', 'v2discovery': True, 'mappedAPI': 'admin'},
+  DOCS: {'name': 'Docs API', 'version': 'v1', 'v2discovery': True},
+  DRIVE2: {'name': 'Drive API v2', 'version': 'v2', 'v2discovery': False, 'mappedAPI': 'drive'},
+  DRIVE3: {'name': 'Drive API v3', 'version': 'v3', 'v2discovery': False, 'mappedAPI': 'drive'},
+  DRIVETD: {'name': 'Drive API v3 - todrive', 'version': 'v3', 'v2discovery': False, 'mappedAPI': 'drive'},
+  DRIVEACTIVITY: {'name': 'Drive Activity API v2', 'version': 'v2', 'v2discovery': True},
+  DRIVELABELS_ADMIN: {'name': 'Drive Labels API - Admin', 'version': 'v2', 'v2discovery': True, 'mappedAPI': DRIVELABELS},
+  DRIVELABELS_USER: {'name': 'Drive Labels API - User', 'version': 'v2', 'v2discovery': True, 'mappedAPI': DRIVELABELS},
+  EMAIL_AUDIT: {'name': 'Email Audit API', 'version': 'v1', 'v2discovery': False},
+  FORMS: {'name': 'Forms API', 'version': 'v1', 'v2discovery': True},
+  GMAIL: {'name': 'Gmail API', 'version': 'v1', 'v2discovery': True},
+  GROUPSMIGRATION: {'name': 'Groups Migration API', 'version': 'v1', 'v2discovery': False},
+  GROUPSSETTINGS: {'name': 'Groups Settings API', 'version': 'v1', 'v2discovery': True},
+  IAM: {'name': 'Identity and Access Management API', 'version': 'v1', 'v2discovery': True},
+  IAM_CREDENTIALS: {'name': 'Identity and Access Management Credentials API', 'version': 'v1', 'v2discovery': True},
+  IAP: {'name': 'Cloud Identity-Aware Proxy API', 'version': 'v1', 'v2discovery': True},
+  KEEP: {'name': 'Keep API', 'version': 'v1', 'v2discovery': True},
+  LICENSING: {'name': 'License Manager API', 'version': 'v1', 'v2discovery': True},
+  LOOKERSTUDIO: {'name': 'Looker Studio API', 'version': 'v1', 'v2discovery': True, 'localjson': True},
+  MEET: {'name': 'Meet API', 'version': 'v2', 'v2discovery': True},
+  OAUTH2: {'name': 'OAuth2 API', 'version': 'v2', 'v2discovery': False},
+  ORGPOLICY: {'name': 'Organization Policy API', 'version': 'v2', 'v2discovery': True},
+  PEOPLE: {'name': 'People API', 'version': 'v1', 'v2discovery': True},
+  PEOPLE_DIRECTORY: {'name': 'People Directory API', 'version': 'v1', 'v2discovery': True, 'mappedAPI': PEOPLE},
+  PEOPLE_OTHERCONTACTS: {'name': 'People  API - Other Contacts', 'version': 'v1', 'v2discovery': True, 'mappedAPI': PEOPLE},
+  PRINTERS: {'name': 'Directory API Printers', 'version': 'directory_v1', 'v2discovery': True, 'mappedAPI': 'admin'},
+  PUBSUB: {'name': 'Pub / Sub API', 'version': 'v1', 'v2discovery': True},
+  REPORTS: {'name': 'Reports API', 'version': 'reports_v1', 'v2discovery': True, 'mappedAPI': 'admin'},
+  RESELLER: {'name': 'Reseller API', 'version': 'v1', 'v2discovery': True},
+  SERVICEACCOUNTLOOKUP: {'name': 'Service Account Lookup pseudo-API', 'version': 'v1', 'v2discovery': True, 'localjson': True},
+  SERVICEMANAGEMENT: {'name': 'Service Management API', 'version': 'v1', 'v2discovery': True},
+  SERVICEUSAGE: {'name': 'Service Usage API', 'version': 'v1', 'v2discovery': True},
+  SHEETS: {'name': 'Sheets API', 'version': 'v4', 'v2discovery': True},
+  SHEETSTD: {'name': 'Sheets API - todrive', 'version': 'v4', 'v2discovery': True, 'mappedAPI': SHEETS},
+  SITES: {'name': 'Sites API', 'version': 'v1', 'v2discovery': False},
+  SITEVERIFICATION: {'name': 'Site Verification API', 'version': 'v1', 'v2discovery': True},
+  STORAGE: {'name': 'Cloud Storage API', 'version': 'v1', 'v2discovery': True},
+  STORAGEREAD: {'name': 'Cloud Storage API - Read', 'version': 'v1', 'v2discovery': True, 'mappedAPI': STORAGE},
+  STORAGEWRITE: {'name': 'Cloud Storage API - Write', 'version': 'v1', 'v2discovery': True, 'mappedAPI': STORAGE},
+  TASKS: {'name': 'Tasks API', 'version': 'v1', 'v2discovery': True},
+  VAULT: {'name': 'Vault API', 'version': 'v1', 'v2discovery': True},
+  YOUTUBE: {'name': 'Youtube API', 'version': 'v3', 'v2discovery': True},
+  }
+
+READONLY = ['readonly',]
+
+_CLIENT_SCOPES = [
+  {'name': 'Calendar API',
+   'api': CALENDAR,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/calendar'},
+  {'name': 'Chrome Browser Cloud Management API',
+   'api': CBCM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.directory.device.chromebrowsers'},
+  {'name': 'Chrome Management API - read only',
+   'api': CHROMEMANAGEMENT,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/chrome.management.reports.readonly'},
+  {'name': 'Chrome Management API - AppDetails read only',
+   'api': CHROMEMANAGEMENT_APPDETAILS,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/chrome.management.appdetails.readonly'},
+  {'name': 'Chrome Management API - Profiles',
+   'api': CHROMEMANAGEMENT_CHROMEPROFILES,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/chrome.management.profiles'},
+  {'name': 'Chrome Management API - Telemetry read only',
+   'api': CHROMEMANAGEMENT_TELEMETRY,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/chrome.management.telemetry.readonly'},
+  {'name': 'Chrome Policy API',
+   'api': CHROMEPOLICY,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/chrome.management.policy'},
+  {'name': 'Chrome Printer Management API',
+   'api': PRINTERS,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.chrome.printers'},
+  {'name': 'Chrome Version History API',
+   'api': CHROMEVERSIONHISTORY,
+   'subscopes': [],
+   'scope': ''},
+  {'name': 'Classroom API - Courses',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.courses'},
+  {'name': 'Classroom API - Course Announcements',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.announcements'},
+  {'name': 'Classroom API - Course Topics',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.topics'},
+  {'name': 'Classroom API - Course Work/Materials',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.courseworkmaterials'},
+  {'name': 'Classroom API - Course Work/Submissions',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.coursework.students'},
+  {'name': 'Classroom API - Student Guardians',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.guardianlinks.students'},
+  {'name': 'Classroom API - Profile Emails',
+   'api': CLASSROOM,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/classroom.profile.emails'},
+  {'name': 'Classroom API - Profile Photos',
+   'api': CLASSROOM,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/classroom.profile.photos'},
+  {'name': 'Classroom API - Rosters',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.rosters'},
+  {'name': 'Cloud Channel API',
+   'api': CLOUDCHANNEL,
+   'subscopes': READONLY,
+   'offByDefault': True,
+   'scope': 'https://www.googleapis.com/auth/apps.order'},
+  {'name': 'Cloud Identity Groups API',
+   'api': CLOUDIDENTITY_GROUPS,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/cloud-identity.groups'},
+  {'name': 'Cloud Identity Groups API Beta (Enables group locking/unlocking)',
+   'api': CLOUDIDENTITY_GROUPS_BETA,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/cloud-identity.groups'},
+  {'name': 'Cloud Identity - Inbound SSO Settings',
+   'api': CLOUDIDENTITY_INBOUND_SSO,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/cloud-identity.inboundsso'},
+  {'name': 'Cloud Identity OrgUnits API',
+   'api': CLOUDIDENTITY_ORGUNITS_BETA,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/cloud-identity.orgunits'},
+  {'name': 'Cloud Identity - Policy',
+   'api': CLOUDIDENTITY_POLICY,
+   'subscopes': READONLY,
+   'roByDefault': True,
+   'scope': 'https://www.googleapis.com/auth/cloud-identity.policies'
+  },
+  {'name': 'Cloud Identity User Invitations API',
+   'api': CLOUDIDENTITY_USERINVITATIONS,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/cloud-identity.userinvitations'},
+  {'name': 'Cloud Storage API (Read Only, Vault/Takeout Download, Cloud Storage)',
+   'api': STORAGEREAD,
+   'subscopes': [],
+   'offByDefault': True,
+   'scope': STORAGE_READONLY_SCOPE},
+  {'name': 'Cloud Storage API (Read/Write, Vault/Takeout Copy/Download, Cloud Storage)',
+   'api': STORAGEWRITE,
+   'subscopes': [],
+   'offByDefault': True,
+   'scope': STORAGE_READWRITE_SCOPE},
+  {'name': 'Contacts API - Domain Shared Contacts and GAL',
+   'api': CONTACTS,
+   'subscopes': [],
+   'scope': 'https://www.google.com/m8/feeds'},
+  {'name': 'Contact Delegation API',
+   'api': CONTACTDELEGATION,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.contact.delegation'},
+  {'name': 'Data Transfer API',
+   'api': DATATRANSFER,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.datatransfer'},
+  {'name': 'Directory API - Chrome OS Devices',
+   'api': DIRECTORY,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.directory.device.chromeos'},
+  {'name': 'Directory API - Customers',
+   'api': DIRECTORY,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.directory.customer'},
+  {'name': 'Directory API - Domains',
+   'api': DIRECTORY,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.directory.domain'},
+  {'name': 'Directory API - Groups',
+   'api': DIRECTORY,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.directory.group'},
+  {'name': 'Directory API - Mobile Devices Directory',
+   'api': DIRECTORY,
+   'subscopes': ['readonly', 'action'],
+   'scope': 'https://www.googleapis.com/auth/admin.directory.device.mobile'},
+  {'name': 'Directory API - Organizational Units',
+   'api': DIRECTORY,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.directory.orgunit'},
+  {'name': 'Directory API - Resource Calendars',
+   'api': DIRECTORY,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.directory.resource.calendar'},
+  {'name': 'Directory API - Roles',
+   'api': DIRECTORY,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.directory.rolemanagement'},
+  {'name': 'Directory API - User Schemas',
+   'api': DIRECTORY,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.directory.userschema'},
+  {'name': 'Directory API - User Security',
+   'api': DIRECTORY,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/admin.directory.user.security'},
+  {'name': 'Directory API - Users',
+   'api': DIRECTORY,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/admin.directory.user'},
+  {'name': 'Email Audit API',
+   'api': EMAIL_AUDIT,
+   'subscopes': [],
+   'offByDefault': True,
+   'scope': 'https://apps-apis.google.com/a/feeds/compliance/audit/'},
+  {'name': 'Groups Migration API',
+   'api': GROUPSMIGRATION,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/apps.groups.migration'},
+  {'name': 'Groups Settings API',
+   'api': GROUPSSETTINGS,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/apps.groups.settings'},
+  {'name': 'License Manager API',
+   'api': LICENSING,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/apps.licensing'},
+  {'name': 'People Directory API - read only',
+   'api': PEOPLE_DIRECTORY,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/directory.readonly'},
+  {'name': 'People API',
+   'api': PEOPLE,
+   'subscopes': READONLY,
+   'scope': PEOPLE_SCOPE},
+  {'name': 'Pub / Sub API',
+   'api': PUBSUB,
+   'subscopes': [],
+   'offByDefault': True,
+   'scope': 'https://www.googleapis.com/auth/pubsub'},
+  {'name': 'Reports API - Audit Reports',
+   'api': REPORTS,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/admin.reports.audit.readonly'},
+  {'name': 'Reports API - Usage Reports',
+   'api': REPORTS,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/admin.reports.usage.readonly'},
+  {'name': 'Reseller API',
+   'api': RESELLER,
+   'subscopes': [],
+   'offByDefault': True,
+   'scope': 'https://www.googleapis.com/auth/apps.order'},
+  {'name': 'Service Account Lookup pseudo-API',
+   'api': SERVICEACCOUNTLOOKUP,
+   'subscopes': [],
+   'scope': ''},
+  {'name': 'Site Verification API',
+   'api': SITEVERIFICATION,
+   'subscopes': [],
+   'offByDefault': True,
+   'scope': 'https://www.googleapis.com/auth/siteverification'},
+  {'name': 'Sites API',
+   'api': SITES,
+   'subscopes': [],
+   'offByDefault': True,
+   'scope': 'https://sites.google.com/feeds'},
+  {'name': 'Vault API',
+   'api': VAULT,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/ediscovery'},
+  ]
+
+_TODRIVE_CLIENT_SCOPES = [
+  {'name': 'Drive API - todrive_clientaccess',
+   'api': DRIVE3,
+   'subscopes': [],
+   'scope': DRIVE_SCOPE},
+  {'name': 'Drive File API - todrive_clientaccess',
+   'api': DRIVE3,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/drive.file'},
+  {'name': 'Gmail API - todrive_clientaccess',
+   'api': GMAIL,
+   'subscopes': [],
+   'scope': GMAIL_SEND_SCOPE},
+  {'name': 'Sheets API - todrive_clientaccess',
+   'api': SHEETS,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/spreadsheets'},
+  ]
+
+OAUTH2SA_SCOPES = 'us_scopes'
+
+_SVCACCT_SCOPES = [
+  {'name': 'AlertCenter API',
+   'api': ALERTCENTER,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/apps.alerts'},
+  {'name': 'Analytics API - read only',
+   'api': ANALYTICS,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/analytics.readonly'},
+  {'name': 'Analytics Admin API - read only',
+   'api': ANALYTICS_ADMIN,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/analytics.readonly'},
+  {'name': 'Calendar API',
+   'api': CALENDAR,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/calendar'},
+  {'name': 'Chat API - Memberships',
+   'api': CHAT_MEMBERSHIPS,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/chat.memberships'},
+  {'name': 'Chat API - Memberships Admin',
+   'api': CHAT_MEMBERSHIPS_ADMIN,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/chat.admin.memberships'},
+  {'name': 'Chat API - Messages',
+   'api': CHAT_MESSAGES,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/chat.messages'},
+  {'name': 'Chat API - Spaces',
+   'api': CHAT_SPACES,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/chat.spaces'},
+  {'name': 'Chat API - Spaces Admin',
+   'api': CHAT_SPACES_ADMIN,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/chat.admin.spaces'},
+  {'name': 'Chat API - Spaces Delete',
+   'api': CHAT_SPACES_DELETE,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/chat.delete'},
+  {'name': 'Chat API - Spaces Delete Admin',
+   'api': CHAT_SPACES_DELETE_ADMIN,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/chat.admin.delete'},
+  {'name': 'Classroom API - Course Announcements',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.announcements'},
+  {'name': 'Classroom API - Course Topics',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.topics'},
+  {'name': 'Classroom API - Course Work/Materials',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.courseworkmaterials'},
+  {'name': 'Classroom API - Course Work/Submissions',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.coursework.students'},
+  {'name': 'Classroom API - Profile Emails',
+   'api': CLASSROOM,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/classroom.profile.emails'},
+  {'name': 'Classroom API - Profile Photos',
+   'api': CLASSROOM,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/classroom.profile.photos'},
+  {'name': 'Classroom API - Rosters',
+   'api': CLASSROOM,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/classroom.rosters'},
+  {'name': 'Cloud Identity Devices API',
+   'api': CLOUDIDENTITY_DEVICES,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/cloud-identity'},
+#  {'name': 'Cloud Identity User Invitations API',
+#   'api': CLOUDIDENTITY_USERINVITATIONS,
+#   'subscopes': READONLY,
+#   'scope': 'https://www.googleapis.com/auth/cloud-identity'},
+#  {'name': 'Contacts API - Users',
+#   'api': CONTACTS,
+#   'subscopes': [],
+#   'scope': 'https://www.google.com/m8/feeds'},
+  {'name': 'Drive API',
+   'api': DRIVE3,
+   'subscopes': READONLY,
+   'scope': DRIVE_SCOPE},
+  {'name': 'Drive Activity API v2 - must pair with Drive API',
+   'api': DRIVEACTIVITY,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/drive.activity'},
+  {'name': 'Drive Labels API - Admin',
+   'api': DRIVELABELS_ADMIN,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/drive.admin.labels'},
+  {'name': 'Drive Labels API - User',
+   'api': DRIVELABELS_USER,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/drive.labels'},
+  {'name': 'Docs API',
+   'api': DOCS,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/documents'},
+  {'name': 'Forms API',
+   'api': FORMS,
+   'subscopes': [],
+   'scope': DRIVE_SCOPE},
+  {'name': 'Gmail API - Full Access (Labels, Messages)',
+   'api': GMAIL,
+   'subscopes': [],
+   'scope': 'https://mail.google.com/'},
+  {'name': 'Gmail API - Full Access (Labels, Messages) except delete message',
+   'api': GMAIL,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/gmail.modify'},
+  {'name': 'Gmail API - Basic Settings (Filters,IMAP, Language, POP, Vacation) - read/write, Sharing Settings (Delegates, Forwarding, SendAs) - read',
+   'api': GMAIL,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/gmail.settings.basic'},
+  {'name': 'Gmail API - Sharing Settings (Delegates, Forwarding, SendAs) - write',
+   'api': GMAIL,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/gmail.settings.sharing'},
+  {'name': 'Identity and Access Management API',
+   'api': IAM,
+   'subscopes': [],
+   'scope': CLOUD_PLATFORM_SCOPE},
+  {'name': 'Keep API',
+   'api': KEEP,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/keep'},
+  {'name': 'Looker Studio API',
+   'api': LOOKERSTUDIO,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/datastudio'},
+  {'name': 'Meet API',
+   'api': MEET,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/meetings.space.created',
+   'roscope': 'https://www.googleapis.com/auth/meetings.space.readonly'},
+  {'name': 'OAuth2 API',
+   'api': OAUTH2,
+   'subscopes': [],
+   'scope': USERINFO_PROFILE_SCOPE},
+  {'name': 'People API',
+   'api': PEOPLE,
+   'subscopes': READONLY,
+   'scope': PEOPLE_SCOPE},
+  {'name': 'People Directory API - read only',
+   'api': PEOPLE_DIRECTORY,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/directory.readonly'},
+  {'name': 'People API - Other Contacts - read only',
+   'api': PEOPLE_OTHERCONTACTS,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/contacts.other.readonly'},
+  {'name': 'Sheets API',
+   'api': SHEETS,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/spreadsheets'},
+  {'name': 'Sites API',
+   'api': SITES,
+   'subscopes': [],
+   'scope': 'https://sites.google.com/feeds'},
+  {'name': 'Tasks API',
+   'api': TASKS,
+   'subscopes': READONLY,
+   'scope': 'https://www.googleapis.com/auth/tasks'},
+  {'name': 'Youtube API - read only',
+   'api': YOUTUBE,
+   'subscopes': [],
+   'offByDefault': True,
+   'scope': 'https://www.googleapis.com/auth/youtube.readonly'},
+  ]
+
+_SVCACCT_SPECIAL_SCOPES = [
+  {'name': 'Drive API - todrive',
+   'api': DRIVETD,
+   'subscopes': [],
+   'scope': DRIVE_SCOPE},
+  {'name': 'Gmail API - Full Access - read only',
+   'api': GMAIL,
+   'subscopes': [],
+   'offByDefault': True,
+   'scope': 'https://www.googleapis.com/auth/gmail.readonly'},
+  {'name': 'Gmail API - Send Messages - including todrive',
+   'api': GMAIL,
+   'subscopes': [],
+   'offByDefault': True,
+   'scope': GMAIL_SEND_SCOPE},
+  {'name': 'Sheets API - todrive',
+   'api': SHEETSTD,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/spreadsheets'},
+  ]
+
+_USER_SVCACCT_ONLY_SCOPES = [
+  {'name': 'Groups Migration API',
+   'api': GROUPSMIGRATION,
+   'subscopes': [],
+   'scope': 'https://www.googleapis.com/auth/apps.groups.migration'},
+  ]
+
+DRIVE3_TO_DRIVE2_ABOUT_FIELDS_MAP = {
+  'displayName': 'name',
+  'limit': 'quotaBytesTotal',
+  'usage': 'quotaBytesUsedAggregate',
+  'usageInDrive': 'quotaBytesUsed',
+  'usageInDriveTrash': 'quotaBytesUsedInTrash',
+  }
+
+DRIVE3_TO_DRIVE2_CAPABILITIES_FIELDS_MAP = {
+  'canComment': 'canComment',
+  'canReadRevisions': 'canReadRevisions',
+  'canCopy': 'copyable',
+  'canEdit': 'editable',
+  'canShare': 'shareable',
+  }
+
+DRIVE3_TO_DRIVE2_CAPABILITIES_NAMES_MAP = {
+  'canChangeViewersCanCopyContent': 'canChangeRestrictedDownload',
+  }
+
+DRIVE3_TO_DRIVE2_FILES_FIELDS_MAP = {
+  'allowFileDiscovery': 'withLink',
+  'createdTime': 'createdDate',
+  'expirationTime': 'expirationDate',
+  'modifiedByMe': 'modified',
+  'modifiedByMeTime': 'modifiedByMeDate',
+  'modifiedTime': 'modifiedDate',
+  'name': 'title',
+  'restrictionTime': 'restrictionDate',
+  'sharedWithMeTime': 'sharedWithMeDate',
+  'size': 'fileSize',
+  'trashedTime': 'trashedDate',
+  'viewedByMe': 'viewed',
+  'viewedByMeTime': 'lastViewedByMeDate',
+  'webViewLink': 'alternateLink',
+  }
+
+DRIVE3_TO_DRIVE2_LABELS_MAP = {
+  'modifiedByMe': 'modified',
+  'starred': 'starred',
+  'trashed': 'trashed',
+  'viewedByMe': 'viewed',
+  }
+
+DRIVE3_TO_DRIVE2_REVISIONS_FIELDS_MAP = {
+  'modifiedTime': 'modifiedDate',
+  'keepForever': 'pinned',
+  'size': 'fileSize',
+  }
+
+def getAPIName(api):
+  return _INFO[api]['name']
+
+def getVersion(api):
+  version = _INFO[api]['version']
+  v2discovery = _INFO[api]['v2discovery']
+  api = _INFO[api].get('mappedAPI', api)
+  return (api, version, v2discovery)
+
+def getClientScopesSet(api):
+  return {scope['scope'] for scope in _CLIENT_SCOPES if scope['api'] == api}
+
+def getClientScopesList(todriveClientAccess):
+  caScopes = _CLIENT_SCOPES[:]
+  if todriveClientAccess:
+    caScopes.extend(_TODRIVE_CLIENT_SCOPES)
+  return sorted(caScopes, key=lambda k: k['name'])
+
+def getClientScopesURLs(todriveClientAccess):
+  caScopes = _CLIENT_SCOPES[:]
+  if todriveClientAccess:
+    caScopes.extend(_TODRIVE_CLIENT_SCOPES)
+  return sorted({scope['scope'] for scope in _CLIENT_SCOPES})
+
+def getSvcAcctScopeAPI(uscope):
+  for scope in _SVCACCT_SCOPES:
+    if uscope == scope['scope'] or (uscope.endswith('.readonly') and 'readonly' in scope['subscopes']):
+      return scope['api']
+  return None
+
+def getSvcAcctScopes(userServiceAccountAccessOnly, svcAcctSpecialScopes):
+  saScopes = [scope['scope'] for scope in _SVCACCT_SCOPES]
+  if userServiceAccountAccessOnly:
+    saScopes.extend([scope['scope'] for scope in _USER_SVCACCT_ONLY_SCOPES])
+  if svcAcctSpecialScopes:
+    saScopes.extend([scope['scope'] for scope in _SVCACCT_SPECIAL_SCOPES])
+  return saScopes
+
+def getSvcAcctScopesList(userServiceAccountAccessOnly, svcAcctSpecialScopes):
+  saScopes = _SVCACCT_SCOPES[:]
+  if userServiceAccountAccessOnly:
+    saScopes.extend(_USER_SVCACCT_ONLY_SCOPES)
+  if svcAcctSpecialScopes:
+    saScopes.extend(_SVCACCT_SPECIAL_SCOPES)
+  return sorted(saScopes, key=lambda k: k['name'])
+
+def hasLocalJSON(api):
+  return _INFO[api].get('localjson', False)

src/gam/gamlib/glcfg.py

@@ -0,0 +1,616 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2024 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""GAM gam.cfg variables
+
+"""
+
+import os
+
+TRUE = 'true'
+FALSE = 'false'
+DEFAULT_CHARSET = 'utf-8'
+MY_CUSTOMER = 'my_customer'
+NEVER = 'Never'
+TLS_CHOICE_MAP = {
+  '': '',
+  'tlsv1_2': 'TLSv1_2', 'tlsv1.2': 'TLSv1_2',
+  'tlsv1_3': 'TLSv1_3', 'tlsv1.3': 'TLSv1_3',
+  }
+
+FN_CACERTS_PEM = 'cacerts.pem'
+FN_CLIENT_SECRETS_JSON = 'client_secrets.json'
+FN_EXTRA_ARGS_TXT = 'extra-args.txt'
+FN_OAUTH2_TXT = 'oauth2.txt'
+FN_OAUTH2SERVICE_JSON = 'oauth2service.json'
+
+# Global variables defined in gam.cfg
+
+# The following XXX constants are the names of the items in gam.cfg
+# When retrieving lists of Google Drive activities from API, how many should be retrieved in each chunk
+ACTIVITY_MAX_RESULTS = 'activity_max_results'
+# Admin email address, required when enable_dasa is true, overrides oauth2.txt value otherwise
+ADMIN_EMAIL = 'admin_email'
+# Check if API calls rate exceeds limit
+API_CALLS_RATE_CHECK = 'api_calls_rate_check'
+# API calls per 100 seconds limit
+API_CALLS_RATE_LIMIT = 'api_calls_rate_limit'
+# API calls tries limit
+API_CALLS_TRIES_LIMIT = 'api_calls_tries_limit'
+# Automatically generate gam batch command if number of users specified in gam users xxx command exceeds this number
+# Default: 0, do not automatically generate gam batch commands
+AUTO_BATCH_MIN = 'auto_batch_min'
+# When bailing on internal errors, how many total tries should be performed
+BAIL_ON_INTERNAL_ERROR_TRIES = 'bail_on_internal_error_tries'
+# When processing items in batches, how many should be processed in each batch
+BATCH_SIZE = 'batch_size'
+# Location of cacerts.pem for API calls
+CACERTS_PEM = 'cacerts_pem'
+# GAM cache directory
+CACHE_DIR = 'cache_dir'
+# GAM cache discovery only. If no_cache is False, only API discovery calls will be cached
+CACHE_DISCOVERY_ONLY = 'cache_discovery_only'
+# Channel custmerId from gam.cfg
+CHANNEL_CUSTOMER_ID = 'channel_customer_id'
+# Character set of batch, csv, data files
+CHARSET = 'charset'
+# When retrieving lists of Google Classroom items from API, how many should be retrieved in each chunk
+CLASSROOM_MAX_RESULTS = 'classroom_max_results'
+# Path to client_secrets.json
+CLIENT_SECRETS_JSON = 'client_secrets_json'
+# Allowed clock skew in seconds
+CLOCK_SKEW_IN_SECONDS = 'clock_skew_in_seconds'
+# Command logging filename
+CMDLOG = 'cmdlog'
+# Bogus Command logging maximum number of backup log files
+CMDLOG_MAX__BACKUPS = 'cmdlog_max__backups'
+# Command logging maximum number of backup log files
+CMDLOG_MAX_BACKUPS = 'cmdlog_max_backups'
+# Command logging max kilo bytes per log file
+CMDLOG_MAX_KILO_BYTES = 'cmdlog_max_kilo_bytes'
+# GAM config directory containing client_secrets.json, oauth2.txt, oauth2service.json, extra_args.txt
+CONFIG_DIR = 'config_dir'
+# When retrieving lists of Google Contacts from API, how many should be retrieved in each chunk
+CONTACT_MAX_RESULTS = 'contact_max_results'
+# Column delimiter in CSV input file
+CSV_INPUT_COLUMN_DELIMITER = 'csv_input_column_delimiter'
+# No escape character in CSV input file
+CSV_INPUT_NO_ESCAPE_CHAR = 'csv_input_no_escape_char'
+# Quote character in CSV input file
+CSV_INPUT_QUOTE_CHAR = 'csv_input_quote_char'
+# Filter for input column values
+CSV_INPUT_ROW_FILTER = 'csv_input_row_filter'
+# Mode (and|or) for input column values
+CSV_INPUT_ROW_FILTER_MODE = 'csv_input_row_filter_mode'
+# Filter for input column drop values
+CSV_INPUT_ROW_DROP_FILTER = 'csv_input_row_drop_filter'
+# Mode (and|or) for input column drop values
+CSV_INPUT_ROW_DROP_FILTER_MODE = 'csv_input_row_drop_filter_mode'
+# Limit number of input rows
+CSV_INPUT_ROW_LIMIT = 'csv_input_row_limit'
+# Convert newlines in text fields to "\n" in CSV output file
+CSV_OUTPUT_CONVERT_CR_NL = 'csv_output_convert_cr_nl'
+# Column delimiter in CSV output file
+CSV_OUTPUT_COLUMN_DELIMITER = 'csv_output_column_delimiter'
+# No escape character in CSV output file
+CSV_OUTPUT_NO_ESCAPE_CHAR = 'csv_output_no_escape_char'
+# Field list delimiter in CSV output file
+CSV_OUTPUT_FIELD_DELIMITER = 'csv_output_field_delimiter'
+# Filter for output column headers
+CSV_OUTPUT_HEADER_FILTER = 'csv_output_header_filter'
+# Filter for output column headers to drop
+CSV_OUTPUT_HEADER_DROP_FILTER = 'csv_output_header_drop_filter'
+# Force output column headers
+CSV_OUTPUT_HEADER_FORCE = 'csv_output_header_force'
+# Orde output column headers
+CSV_OUTPUT_HEADER_ORDER = 'csv_output_header_order'
+# Line terminator in CSV output file
+CSV_OUTPUT_LINE_TERMINATOR = 'csv_output_line_terminator'
+# Quote character in CSV output file
+CSV_OUTPUT_QUOTE_CHAR = 'csv_output_quote_char'
+# Filter for output column values
+CSV_OUTPUT_ROW_FILTER = 'csv_output_row_filter'
+# Mode (and|or) for output column values
+CSV_OUTPUT_ROW_FILTER_MODE = 'csv_output_row_filter_mode'
+# Filter for output column drop values
+CSV_OUTPUT_ROW_DROP_FILTER = 'csv_output_row_drop_filter'
+# Mode (and|or) for output column drop values
+CSV_OUTPUT_ROW_DROP_FILTER_MODE = 'csv_output_row_drop_filter_mode'
+# Limit number of output rows
+CSV_OUTPUT_ROW_LIMIT = 'csv_output_row_limit'
+# Output sort headers
+CSV_OUTPUT_SORT_HEADERS = 'csv_output_sort_headers'
+# Column header subfield name delimiter in CSV output file
+CSV_OUTPUT_SUBFIELD_DELIMITER = 'csv_output_subfield_delimiter'
+# Add timestamp column to CSV output file
+CSV_OUTPUT_TIMESTAMP_COLUMN = 'csv_output_timestamp_column'
+# Output rows for users even if they do not have the print object (delegate, filters, ...)
+CSV_OUTPUT_USERS_AUDIT = 'csv_output_users_audit'
+# custmerId from gam.cfg or retrieved from Google
+CUSTOMER_ID = 'customer_id'
+# If debug_level > 0: extra_args['prettyPrint'] = True, httplib2.debuglevel = gam_debug_level, appsObj.debug = True
+DEBUG_LEVEL = 'debug_level'
+# When retrieving lists of ChromeOS devices from API, how many should be retrieved in each chunk
+DEVICE_MAX_RESULTS = 'device_max_results'
+# Domain obtained from gam.cfg or oauth2.txt
+DOMAIN = 'domain'
+# Google Drive download directory
+DRIVE_DIR = 'drive_dir'
+# When retrieving lists of Drive files/folders from API, how many should be retrieved in each chunk
+DRIVE_MAX_RESULTS = 'drive_max_results'
+# Use Drive V3 beta
+DRIVE_V3_BETA = 'drive_v3_beta'
+# Use Drive V3 ntive names
+DRIVE_V3_NATIVE_NAMES = 'drive_v3_native_names'
+# When processing email messages in batches, how many should be processed in each batch
+EMAIL_BATCH_SIZE = 'email_batch_size'
+# Enable Delegated Admin Service Account
+ENABLE_DASA = 'enable_dasa'
+# Enable Cloud Session Reauthentication by borrowing a RAPT token from gcloud command
+ENABLE_GCLOUD_REAUTH = 'enable_gcloud_reauth'
+# When retrieving lists of calendar events from API, how many should be retrieved in each chunk
+EVENT_MAX_RESULTS = 'event_max_results'
+# Path to extra_args.txt
+EXTRA_ARGS = 'extra_args'
+# Gmail CSE certificates directory
+GMAIL_CSE_INCERT_DIR = 'gmail_cse_incert_dir'
+# Gmail CSE KACL wrapped key files
+GMAIL_CSE_INKEY_DIR = 'gmail_cse_inkey_dir'
+# When processing items in batches, how many seconds should GAM wait between batches
+INTER_BATCH_WAIT = 'inter_batch_wait'
+# When retrieving lists of licenses from API, how many should be retrieved in each chunk
+LICENSE_MAX_RESULTS = 'license_max_results'
+# License SKUs to process
+LICENSE_SKUS = 'license_skus'
+# When retrieving lists of Google Group members from API, how many should be retrieved in each chunk
+MEMBER_MAX_RESULTS = 'member_max_results'
+# When deleting or modifying Gmail messages, how many should be processed in each batch
+MESSAGE_BATCH_SIZE = 'message_batch_size'
+# When retrieving lists of Gmail messages from API, how many should be retrieved in each chunk
+MESSAGE_MAX_RESULTS = 'message_max_results'
+# When retrieving lists of Mobile devices from API, how many should be retrieved in each chunk
+MOBILE_MAX_RESULTS = 'mobile_max_results'
+# Number of parallel multiprocess pool.apply_async calls; -1: no limit, 0: NUM_THREADS, >0: specific limit
+MULTIPROCESS_POOL_LIMIT = 'multiprocess_pool_limit'
+# Value to substitute for NEVER_TIME
+NEVER_TIME = 'never_time'
+# If no_browser is False, writeCSVfile won't open a browser when todrive is set
+# and doOAuthRequest prints a link and waits for the verification code when oauth2.txt is being created
+NO_BROWSER = 'no_browser'
+# Disable GAM API caching
+NO_CACHE = 'no_cache'
+# Do noit use URL shortner for authentication URLs
+NO_SHORT_URLS = 'no_short_urls'
+# Disable GAM update check
+NO_UPDATE_CHECK = 'no_update_check'
+# Disable SSL certificate validation
+NO_VERIFY_SSL = 'no_verify_ssl'
+# Number of threads for gam tbatch
+NUM_TBATCH_THREADS = 'num_tbatch_threads'
+# Number of threads for gam batch/csv
+NUM_THREADS = 'num_threads'
+# Path to oauth2.txt
+OAUTH2_TXT = 'oauth2_txt'
+# Path to oauth2service.json
+OAUTH2SERVICE_JSON = 'oauth2service_json'
+# Output date format, empty defalts to ISOFormat
+OUTPUT_DATEFORMAT = 'output_dateformat'
+# Output time format, empty defalts to ISOFormat
+OUTPUT_TIMEFORMAT = 'output_timeformat'
+# When retrieving lists of people from API, how many should be retrieved in each chunk
+PEOPLE_MAX_RESULTS = 'people_max_results'
+# Domains for print alises|groups|users
+PRINT_AGU_DOMAINS = 'print_agu_domains'
+# OrgUnits for print cros
+PRINT_CROS_OUS = 'print_cros_ous'
+# OrgUnits and children for print cros
+PRINT_CROS_OUS_AND_CHILDREN = 'print_cros_ous_and_children'
+# Number of seconds to wait for batch/csv processes to complete
+PROCESS_WAIT_LIMIT = 'process_wait_limit'
+# Use quick method to move Chromebooks to OU
+QUICK_CROS_MOVE = 'quick_cros_move'
+# Quick info user: nogroups nolicenses noschemas
+QUICK_INFO_USER = 'quick_info_user'
+# resellerId from gam.cfg or retrieved from Google
+RESELLER_ID = 'reseller_id'
+# Retry service not available errors on API calls
+RETRY_API_SERVICE_NOT_AVAILABLE = 'retry_api_service_not_available'
+# Default section to use for processing
+SECTION = 'section'
+# Show API calls retry data
+SHOW_API_CALLS_RETRY_DATA = 'show_api_calls_retry_data'
+# Show commands when processing batch/csv/loop
+SHOW_COMMANDS = 'show_commands'
+# Convert newlines in text fields to "\n" in show commands
+SHOW_CONVERT_CR_NL = 'show_convert_cr_nl'
+# Add (n/m) to end of messages if number of items to be processed exceeds this number
+SHOW_COUNTS_MIN = 'show_counts_min'
+# Enable/disable "Getting ... " messages
+SHOW_GETTINGS = 'show_gettings'
+# Enable/disable NL at end of "Got ..." messages
+SHOW_GETTINGS_GOT_NL = 'show_gettings_got_nl'
+# Enable/disable showing multiprocess info in redirected stdout/stderr
+SHOW_MULTIPROCESS_INFO = 'show_multiprocess_info'
+# SMTP fqdn
+SMTP_FQDN = 'smtp_fqdn'
+# SMTP host
+SMTP_HOST = 'smtp_host'
+# SMTP username
+SMTP_USERNAME = 'smtp_username'
+# SMTP password
+SMTP_PASSWORD = 'smtp_password'
+## Minimum TLS Version required for HTTPS connections
+TLS_MIN_VERSION = 'tls_min_version'
+## Maximum TLS Version used for HTTPS connections
+TLS_MAX_VERSION = 'tls_max_version'
+# Time Zone
+TIMEZONE = 'timezone'
+# Clear basic filter when updating an existing sheet
+TODRIVE_CLEARFILTER = 'todrive_clearfilter'
+# Use client access for todrive
+TODRIVE_CLIENTACCESS = 'todrive_clientaccess'
+# Enable conversion to Google Sheets when uploading todrive files
+TODRIVE_CONVERSION = 'todrive_conversion'
+# Save local copy of CSV file
+TODRIVE_LOCALCOPY = 'todrive_localcopy'
+# Specify locale for Google Sheets
+TODRIVE_LOCALE = 'todrive_locale'
+# Suppress opening browser on todrive upload
+TODRIVE_NOBROWSER = 'todrive_nobrowser'
+# Suppress sending email on todrive upload
+TODRIVE_NOEMAIL = 'todrive_noemail'
+# No escape character in CSV output file
+TODRIVE_NO_ESCAPE_CHAR = 'todrive_no_escape_char'
+# ID/Name of parent folder for todrive files
+TODRIVE_PARENT = 'todrive_parent'
+# Append timestamp to todrive sheet name
+TODRIVE_SHEET_TIMESTAMP = 'todrive_sheet_timestamp'
+# Sheet timestamp format, empty defalts to ISOFormat
+TODRIVE_SHEET_TIMEFORMAT = 'todrive_sheet_timeformat'
+# Append timestamp to todrive file name
+TODRIVE_TIMESTAMP = 'todrive_timestamp'
+# Timestamp format, empty defalts to ISOFormat
+TODRIVE_TIMEFORMAT = 'todrive_timeformat'
+# Specify timezone for Google Sheets
+TODRIVE_TIMEZONE = 'todrive_timezone'
+# Upload data files with no data
+TODRIVE_UPLOAD_NODATA = 'todrive_upload_nodata'
+# User for todrive files
+TODRIVE_USER = 'todrive_user'
+# Truncate Client ID
+TRUNCATE_CLIENT_ID  = 'truncate_client_id'
+# Update CrOS org unit with orgUnitId
+UPDATE_CROS_OU_WITH_ID = 'update_cros_ou_with_id'
+# Use admin access for chat where possible
+USE_CHAT_ADMIN_ACCESS = 'use_chat_admin_access'
+# Use course owner for course access
+USE_COURSE_OWNER_ACCESS = 'use_course_owner_access'
+# Use Project ID as Project Name and App Name
+USE_PROJECTID_AS_NAME = 'use_projectid_as_name'
+# When retrieving lists of Users from API, how many should be retrieved in each chunk
+USER_MAX_RESULTS = 'user_max_results'
+# User service account access only, no client access
+USER_SERVICE_ACCOUNT_ACCESS_ONLY = 'user_service_account_access_only'
+
+CSV_INPUT_ROW_FILTER_ITEMS = {CSV_INPUT_ROW_FILTER, CSV_INPUT_ROW_FILTER_MODE,
+                              CSV_INPUT_ROW_DROP_FILTER, CSV_INPUT_ROW_DROP_FILTER_MODE,
+                              CSV_INPUT_ROW_LIMIT}
+
+CSV_OUTPUT_ROW_FILTER_ITEMS = {CSV_OUTPUT_HEADER_FILTER, CSV_OUTPUT_HEADER_DROP_FILTER,
+                               CSV_OUTPUT_HEADER_FORCE, CSV_OUTPUT_HEADER_ORDER,
+                               CSV_OUTPUT_ROW_FILTER, CSV_OUTPUT_ROW_FILTER_MODE,
+                               CSV_OUTPUT_ROW_DROP_FILTER, CSV_OUTPUT_ROW_DROP_FILTER_MODE,
+                               CSV_OUTPUT_ROW_LIMIT}
+
+Defaults = {
+  ACTIVITY_MAX_RESULTS: '100',
+  ADMIN_EMAIL: '',
+  API_CALLS_RATE_CHECK: FALSE,
+  API_CALLS_RATE_LIMIT: '100',
+  API_CALLS_TRIES_LIMIT: '10',
+  AUTO_BATCH_MIN: '0',
+  BAIL_ON_INTERNAL_ERROR_TRIES: '2',
+  BATCH_SIZE: '50',
+  CACERTS_PEM: '',
+  CACHE_DIR: '',
+  CACHE_DISCOVERY_ONLY: TRUE,
+  CHARSET: DEFAULT_CHARSET,
+  CHANNEL_CUSTOMER_ID: '',
+  CLASSROOM_MAX_RESULTS: '0',
+  CLIENT_SECRETS_JSON: FN_CLIENT_SECRETS_JSON,
+  CLOCK_SKEW_IN_SECONDS: '10',
+  CMDLOG: '',
+  CMDLOG_MAX_BACKUPS: 5,
+  CMDLOG_MAX_KILO_BYTES: 1000,
+  CONFIG_DIR: '',
+  CONTACT_MAX_RESULTS: '100',
+  CSV_INPUT_COLUMN_DELIMITER: ',',
+  CSV_INPUT_NO_ESCAPE_CHAR: TRUE,
+  CSV_INPUT_QUOTE_CHAR: '\'"\'',
+  CSV_INPUT_ROW_FILTER: '',
+  CSV_INPUT_ROW_FILTER_MODE: 'allmatch',
+  CSV_INPUT_ROW_DROP_FILTER: '',
+  CSV_INPUT_ROW_DROP_FILTER_MODE: 'anymatch',
+  CSV_INPUT_ROW_LIMIT: '0',
+  CSV_OUTPUT_COLUMN_DELIMITER: ',',
+  CSV_OUTPUT_CONVERT_CR_NL: FALSE,
+  CSV_OUTPUT_NO_ESCAPE_CHAR: FALSE,
+  CSV_OUTPUT_FIELD_DELIMITER: "' '",
+  CSV_OUTPUT_HEADER_FILTER: '',
+  CSV_OUTPUT_HEADER_DROP_FILTER: '',
+  CSV_OUTPUT_HEADER_FORCE: '',
+  CSV_OUTPUT_HEADER_ORDER: '',
+  CSV_OUTPUT_LINE_TERMINATOR: 'lf',
+  CSV_OUTPUT_QUOTE_CHAR: '\'"\'',
+  CSV_OUTPUT_ROW_FILTER: '',
+  CSV_OUTPUT_ROW_FILTER_MODE: 'allmatch',
+  CSV_OUTPUT_ROW_DROP_FILTER: '',
+  CSV_OUTPUT_ROW_DROP_FILTER_MODE: 'anymatch',
+  CSV_OUTPUT_ROW_LIMIT: '0',
+  CSV_OUTPUT_SORT_HEADERS: '',
+  CSV_OUTPUT_SUBFIELD_DELIMITER: '.',
+  CSV_OUTPUT_TIMESTAMP_COLUMN: '',
+  CSV_OUTPUT_USERS_AUDIT: FALSE,
+  CUSTOMER_ID: MY_CUSTOMER,
+  DEBUG_LEVEL: '0',
+  DEVICE_MAX_RESULTS: '200',
+  DOMAIN: '',
+  DRIVE_DIR: '',
+  DRIVE_MAX_RESULTS: '1000',
+  DRIVE_V3_BETA: FALSE,
+  DRIVE_V3_NATIVE_NAMES: TRUE,
+  EMAIL_BATCH_SIZE: '50',
+  ENABLE_DASA: FALSE,
+  ENABLE_GCLOUD_REAUTH: FALSE,
+  EVENT_MAX_RESULTS: '250',
+  EXTRA_ARGS: '',
+  GMAIL_CSE_INCERT_DIR: '',
+  GMAIL_CSE_INKEY_DIR: '',
+  INTER_BATCH_WAIT: '0',
+  LICENSE_MAX_RESULTS: '100',
+  LICENSE_SKUS: '',
+  MEMBER_MAX_RESULTS: '200',
+  MESSAGE_BATCH_SIZE: '50',
+  MESSAGE_MAX_RESULTS: '500',
+  MOBILE_MAX_RESULTS: '100',
+  MULTIPROCESS_POOL_LIMIT: '0',
+  NEVER_TIME: NEVER,
+  NO_BROWSER: FALSE,
+  NO_CACHE: FALSE,
+  NO_SHORT_URLS: TRUE,
+  NO_UPDATE_CHECK: TRUE,
+  NO_VERIFY_SSL: FALSE,
+  NUM_TBATCH_THREADS: '2',
+  NUM_THREADS: '5',
+  OAUTH2_TXT: FN_OAUTH2_TXT,
+  OAUTH2SERVICE_JSON: FN_OAUTH2SERVICE_JSON,
+  OUTPUT_DATEFORMAT: '',
+  OUTPUT_TIMEFORMAT: '',
+  PEOPLE_MAX_RESULTS: '100',
+  PRINT_AGU_DOMAINS: '',
+  PRINT_CROS_OUS: '',
+  PRINT_CROS_OUS_AND_CHILDREN: '',
+  PROCESS_WAIT_LIMIT: '0',
+  QUICK_CROS_MOVE: FALSE,
+  QUICK_INFO_USER: FALSE,
+  RESELLER_ID: '',
+  RETRY_API_SERVICE_NOT_AVAILABLE: FALSE,
+  SECTION: '',
+  SHOW_API_CALLS_RETRY_DATA: FALSE,
+  SHOW_COMMANDS: FALSE,
+  SHOW_CONVERT_CR_NL: FALSE,
+  SHOW_COUNTS_MIN: '1',
+  SHOW_GETTINGS: TRUE,
+  SHOW_GETTINGS_GOT_NL: FALSE,
+  SHOW_MULTIPROCESS_INFO: FALSE,
+  SMTP_FQDN: '',
+  SMTP_HOST: '',
+  SMTP_USERNAME: '',
+  SMTP_PASSWORD: '',
+  TLS_MIN_VERSION: 'TLSv1_3',
+  TLS_MAX_VERSION: '',
+  TIMEZONE: 'utc',
+  TODRIVE_CLEARFILTER: FALSE,
+  TODRIVE_CLIENTACCESS: FALSE,
+  TODRIVE_CONVERSION: TRUE,
+  TODRIVE_LOCALCOPY: FALSE,
+  TODRIVE_LOCALE: '',
+  TODRIVE_NOBROWSER: '',
+  TODRIVE_NOEMAIL: '',
+  TODRIVE_NO_ESCAPE_CHAR: TRUE,
+  TODRIVE_PARENT: 'root',
+  TODRIVE_SHEET_TIMESTAMP: 'copy', # copy from TODRIVE_TIMESTAMP
+  TODRIVE_SHEET_TIMEFORMAT: 'copy', # copy from TODRIVE_TIMEFORMAT
+  TODRIVE_TIMESTAMP: FALSE,
+  TODRIVE_TIMEFORMAT: '',
+  TODRIVE_TIMEZONE: '',
+  TODRIVE_UPLOAD_NODATA: TRUE,
+  TODRIVE_USER: '',
+  TRUNCATE_CLIENT_ID: FALSE,
+  UPDATE_CROS_OU_WITH_ID: FALSE,
+  USE_CHAT_ADMIN_ACCESS: FALSE,
+  USE_COURSE_OWNER_ACCESS: FALSE,
+  USE_PROJECTID_AS_NAME: FALSE,
+  USER_MAX_RESULTS: '500',
+  USER_SERVICE_ACCOUNT_ACCESS_ONLY: FALSE,
+  }
+
+Values = {DEBUG_LEVEL: 0}
+
+TYPE_BOOLEAN = 'bool'
+TYPE_CHARACTER = 'char'
+TYPE_CHOICE = 'choi'
+TYPE_CHOICE_LIST = 'chol'
+TYPE_DATETIME = 'datm'
+TYPE_DIRECTORY = 'dire'
+TYPE_EMAIL = 'emai'
+TYPE_EMAIL_OPTIONAL = 'emao'
+TYPE_FILE = 'file'
+TYPE_FLOAT = 'floa'
+TYPE_HEADERFILTER = 'heaf'
+TYPE_HEADERFORCE = 'hefo'
+TYPE_HEADERORDER = 'heor'
+TYPE_INTEGER = 'inte'
+TYPE_LANGUAGE = 'lang'
+TYPE_LOCALE = 'locl'
+TYPE_PASSWORD = 'pass'
+TYPE_ROWFILTER = 'rowf'
+TYPE_STRING = 'stri'
+TYPE_STRINGLIST = 'strl'
+TYPE_TIMEZONE = 'tmzn'
+
+VAR_TYPE = 'type'
+VAR_ENVVAR = 'enva'
+VAR_CHOICES = 'chod'
+VAR_LIMITS = 'lmit'
+VAR_SFFT = 'sfft'
+VAR_SIGFILE = 'sigf'
+VAR_ACCESS = 'aces'
+
+VAR_INFO = {
+  ACTIVITY_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 500)},
+  ADMIN_EMAIL: {VAR_TYPE: TYPE_STRING, VAR_ENVVAR: 'GA_ADMIN_EMAIL', VAR_LIMITS: (0, None)},
+  API_CALLS_RATE_CHECK: {VAR_TYPE: TYPE_BOOLEAN},
+  API_CALLS_RATE_LIMIT: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (50, None)},
+  API_CALLS_TRIES_LIMIT: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (3, 30)},
+  AUTO_BATCH_MIN: {VAR_TYPE: TYPE_INTEGER, VAR_ENVVAR: 'GAM_AUTOBATCH', VAR_LIMITS: (0, 100)},
+  BAIL_ON_INTERNAL_ERROR_TRIES: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 10)},
+  BATCH_SIZE: {VAR_TYPE: TYPE_INTEGER, VAR_ENVVAR: 'GAM_BATCH_SIZE', VAR_LIMITS: (1, 1000)},
+  CACERTS_PEM: {VAR_TYPE: TYPE_FILE, VAR_ENVVAR: 'GAM_CA_FILE', VAR_ACCESS: os.R_OK},
+  CACHE_DIR: {VAR_TYPE: TYPE_DIRECTORY, VAR_ENVVAR: 'GAMCACHEDIR'},
+  CACHE_DISCOVERY_ONLY: {VAR_TYPE: TYPE_BOOLEAN, VAR_SIGFILE: 'allcache.txt', VAR_SFFT: (TRUE, FALSE)},
+  CHARSET: {VAR_TYPE: TYPE_STRING, VAR_ENVVAR: 'GAM_CHARSET', VAR_LIMITS: (1, None)},
+  CHANNEL_CUSTOMER_ID: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  CLASSROOM_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (0, 1000)},
+  CLIENT_SECRETS_JSON: {VAR_TYPE: TYPE_FILE, VAR_ENVVAR: 'CLIENTSECRETS', VAR_ACCESS: os.R_OK},
+  CLOCK_SKEW_IN_SECONDS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (10, 3600)},
+  CMDLOG: {VAR_TYPE: TYPE_FILE, VAR_ACCESS: os.W_OK},
+  CMDLOG_MAX_BACKUPS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 10)},
+  CMDLOG_MAX_KILO_BYTES: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (100, 10000)},
+  CONFIG_DIR: {VAR_TYPE: TYPE_DIRECTORY, VAR_ENVVAR: 'GAMUSERCONFIGDIR'},
+  CONTACT_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 10000)},
+  CSV_INPUT_COLUMN_DELIMITER: {VAR_TYPE: TYPE_CHARACTER},
+  CSV_INPUT_NO_ESCAPE_CHAR: {VAR_TYPE: TYPE_BOOLEAN},
+  CSV_INPUT_QUOTE_CHAR: {VAR_TYPE: TYPE_CHARACTER},
+  CSV_INPUT_ROW_FILTER: {VAR_TYPE: TYPE_ROWFILTER},
+  CSV_INPUT_ROW_FILTER_MODE: {VAR_TYPE: TYPE_CHOICE, VAR_CHOICES: {'allmatch': True, 'anymatch': False}},
+  CSV_INPUT_ROW_DROP_FILTER: {VAR_TYPE: TYPE_ROWFILTER},
+  CSV_INPUT_ROW_DROP_FILTER_MODE: {VAR_TYPE: TYPE_CHOICE, VAR_CHOICES: {'allmatch': True, 'anymatch': False}},
+  CSV_INPUT_ROW_LIMIT: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (0, None)},
+  CSV_OUTPUT_COLUMN_DELIMITER: {VAR_TYPE: TYPE_CHARACTER},
+  CSV_OUTPUT_CONVERT_CR_NL: {VAR_TYPE: TYPE_BOOLEAN},
+  CSV_OUTPUT_NO_ESCAPE_CHAR: {VAR_TYPE: TYPE_BOOLEAN},
+  CSV_OUTPUT_FIELD_DELIMITER: {VAR_TYPE: TYPE_CHARACTER},
+  CSV_OUTPUT_HEADER_FILTER: {VAR_TYPE: TYPE_HEADERFILTER},
+  CSV_OUTPUT_HEADER_DROP_FILTER: {VAR_TYPE: TYPE_HEADERFILTER},
+  CSV_OUTPUT_HEADER_FORCE: {VAR_TYPE: TYPE_HEADERFORCE},
+  CSV_OUTPUT_HEADER_ORDER: {VAR_TYPE: TYPE_HEADERORDER},
+  CSV_OUTPUT_LINE_TERMINATOR: {VAR_TYPE: TYPE_CHOICE, VAR_CHOICES: {'cr': '\r', 'lf': '\n', 'crlf': '\r\n'}},
+  CSV_OUTPUT_QUOTE_CHAR: {VAR_TYPE: TYPE_CHARACTER},
+  CSV_OUTPUT_ROW_FILTER: {VAR_TYPE: TYPE_ROWFILTER},
+  CSV_OUTPUT_ROW_FILTER_MODE: {VAR_TYPE: TYPE_CHOICE, VAR_CHOICES: {'allmatch': True, 'anymatch': False}},
+  CSV_OUTPUT_ROW_DROP_FILTER: {VAR_TYPE: TYPE_ROWFILTER},
+  CSV_OUTPUT_ROW_DROP_FILTER_MODE: {VAR_TYPE: TYPE_CHOICE, VAR_CHOICES: {'allmatch': True, 'anymatch': False}},
+  CSV_OUTPUT_ROW_LIMIT: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (0, None)},
+  CSV_OUTPUT_SORT_HEADERS: {VAR_TYPE: TYPE_STRINGLIST},
+  CSV_OUTPUT_SUBFIELD_DELIMITER: {VAR_TYPE: TYPE_CHARACTER},
+  CSV_OUTPUT_TIMESTAMP_COLUMN: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  CSV_OUTPUT_USERS_AUDIT: {VAR_TYPE: TYPE_BOOLEAN},
+  CUSTOMER_ID: {VAR_TYPE: TYPE_STRING, VAR_ENVVAR: 'CUSTOMER_ID', VAR_LIMITS: (0, None)},
+  DEBUG_LEVEL: {VAR_TYPE: TYPE_INTEGER, VAR_SIGFILE: 'debug.gam', VAR_LIMITS: (0, None), VAR_SFFT: ('0', '4')},
+  DEVICE_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 200)},
+  DOMAIN: {VAR_TYPE: TYPE_STRING, VAR_ENVVAR: 'GA_DOMAIN', VAR_LIMITS: (0, None)},
+  DRIVE_DIR: {VAR_TYPE: TYPE_DIRECTORY, VAR_ENVVAR: 'GAMDRIVEDIR'},
+  DRIVE_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 1000)},
+  DRIVE_V3_BETA: {VAR_TYPE: TYPE_BOOLEAN},
+  DRIVE_V3_NATIVE_NAMES: {VAR_TYPE: TYPE_BOOLEAN},
+  EMAIL_BATCH_SIZE: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 100)},
+  ENABLE_DASA: {VAR_TYPE: TYPE_BOOLEAN, VAR_SIGFILE: 'enabledasa.txt', VAR_SFFT: (FALSE, TRUE)},
+  ENABLE_GCLOUD_REAUTH: {VAR_TYPE: TYPE_BOOLEAN},
+  EVENT_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 2500)},
+  EXTRA_ARGS: {VAR_TYPE: TYPE_FILE, VAR_SIGFILE: FN_EXTRA_ARGS_TXT, VAR_SFFT: ('', FN_EXTRA_ARGS_TXT), VAR_ACCESS: os.R_OK},
+  GMAIL_CSE_INCERT_DIR: {VAR_TYPE: TYPE_DIRECTORY},
+  GMAIL_CSE_INKEY_DIR: {VAR_TYPE: TYPE_DIRECTORY},
+  INTER_BATCH_WAIT: {VAR_TYPE: TYPE_FLOAT, VAR_LIMITS: (0.0, 60.0)},
+  LICENSE_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (10, 1000)},
+  LICENSE_SKUS: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  MEMBER_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 200)},
+  MESSAGE_BATCH_SIZE: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 1000)},
+  MESSAGE_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 10000)},
+  MOBILE_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 100)},
+  MULTIPROCESS_POOL_LIMIT: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (-1, None)},
+  NEVER_TIME: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  NO_BROWSER: {VAR_TYPE: TYPE_BOOLEAN, VAR_SIGFILE: 'nobrowser.txt', VAR_SFFT: (FALSE, TRUE)},
+  NO_CACHE: {VAR_TYPE: TYPE_BOOLEAN, VAR_SIGFILE: 'nocache.txt', VAR_SFFT: (FALSE, TRUE)},
+  NO_SHORT_URLS: {VAR_TYPE: TYPE_BOOLEAN, VAR_SIGFILE: 'noshorturls.txt', VAR_SFFT: (FALSE, TRUE)},
+  NO_UPDATE_CHECK: {VAR_TYPE: TYPE_BOOLEAN},
+  NO_VERIFY_SSL: {VAR_TYPE: TYPE_BOOLEAN},
+  NUM_TBATCH_THREADS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 1000)},
+  NUM_THREADS: {VAR_TYPE: TYPE_INTEGER, VAR_ENVVAR: 'GAM_THREADS', VAR_LIMITS: (1, 1000)},
+  OAUTH2_TXT: {VAR_TYPE: TYPE_FILE, VAR_ENVVAR: 'OAUTHFILE', VAR_ACCESS: os.R_OK | os.W_OK},
+  OAUTH2SERVICE_JSON: {VAR_TYPE: TYPE_FILE, VAR_ENVVAR: 'OAUTHSERVICEFILE', VAR_ACCESS: os.R_OK | os.W_OK},
+  OUTPUT_DATEFORMAT: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  OUTPUT_TIMEFORMAT: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  PEOPLE_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (0, 1000)},
+  PRINT_AGU_DOMAINS: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  PRINT_CROS_OUS: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  PRINT_CROS_OUS_AND_CHILDREN: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  PROCESS_WAIT_LIMIT: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (0, None)},
+  QUICK_CROS_MOVE: {VAR_TYPE: TYPE_BOOLEAN},
+  QUICK_INFO_USER: {VAR_TYPE: TYPE_BOOLEAN},
+  RESELLER_ID: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  RETRY_API_SERVICE_NOT_AVAILABLE: {VAR_TYPE: TYPE_BOOLEAN},
+  SECTION: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  SHOW_API_CALLS_RETRY_DATA: {VAR_TYPE: TYPE_BOOLEAN},
+  SHOW_COMMANDS: {VAR_TYPE: TYPE_BOOLEAN},
+  SHOW_CONVERT_CR_NL: {VAR_TYPE: TYPE_BOOLEAN},
+  SHOW_COUNTS_MIN: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (0, 100)},
+  SHOW_GETTINGS: {VAR_TYPE: TYPE_BOOLEAN},
+  SHOW_GETTINGS_GOT_NL: {VAR_TYPE: TYPE_BOOLEAN},
+  SHOW_MULTIPROCESS_INFO: {VAR_TYPE: TYPE_BOOLEAN},
+  SMTP_FQDN: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  SMTP_HOST: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  SMTP_USERNAME: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  SMTP_PASSWORD: {VAR_TYPE: TYPE_PASSWORD, VAR_LIMITS: (0, None)},
+  TLS_MIN_VERSION: {VAR_TYPE: TYPE_CHOICE, VAR_ENVVAR: 'GAM_TLS_MIN_VERSION', VAR_CHOICES: TLS_CHOICE_MAP},
+  TLS_MAX_VERSION: {VAR_TYPE: TYPE_CHOICE, VAR_ENVVAR: 'GAM_TLS_MAX_VERSION', VAR_CHOICES: TLS_CHOICE_MAP},
+  TIMEZONE: {VAR_TYPE: TYPE_TIMEZONE},
+  TODRIVE_CLEARFILTER: {VAR_TYPE: TYPE_BOOLEAN},
+  TODRIVE_CLIENTACCESS: {VAR_TYPE: TYPE_BOOLEAN},
+  TODRIVE_CONVERSION: {VAR_TYPE: TYPE_BOOLEAN},
+  TODRIVE_LOCALCOPY: {VAR_TYPE: TYPE_BOOLEAN},
+  TODRIVE_LOCALE: {VAR_TYPE: TYPE_LOCALE},
+  TODRIVE_NOBROWSER: {VAR_TYPE: TYPE_BOOLEAN, VAR_SIGFILE: 'nobrowser.txt', VAR_SFFT: (FALSE, TRUE)},
+  TODRIVE_NOEMAIL: {VAR_TYPE: TYPE_BOOLEAN, VAR_SIGFILE: 'notdemail.txt', VAR_SFFT: (FALSE, TRUE)},
+  TODRIVE_NO_ESCAPE_CHAR: {VAR_TYPE: TYPE_BOOLEAN},
+  TODRIVE_PARENT: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  TODRIVE_SHEET_TIMESTAMP: {VAR_TYPE: TYPE_BOOLEAN},
+  TODRIVE_SHEET_TIMEFORMAT: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  TODRIVE_TIMESTAMP: {VAR_TYPE: TYPE_BOOLEAN},
+  TODRIVE_TIMEFORMAT: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  TODRIVE_TIMEZONE: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  TODRIVE_UPLOAD_NODATA: {VAR_TYPE: TYPE_BOOLEAN},
+  TODRIVE_USER: {VAR_TYPE: TYPE_STRING, VAR_LIMITS: (0, None)},
+  TRUNCATE_CLIENT_ID: {VAR_TYPE: TYPE_BOOLEAN},
+  UPDATE_CROS_OU_WITH_ID: {VAR_TYPE: TYPE_BOOLEAN},
+  USE_CHAT_ADMIN_ACCESS: {VAR_TYPE: TYPE_BOOLEAN},
+  USE_COURSE_OWNER_ACCESS: {VAR_TYPE: TYPE_BOOLEAN},
+  USE_PROJECTID_AS_NAME: {VAR_TYPE: TYPE_BOOLEAN},
+  USER_MAX_RESULTS: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 500)},
+  USER_SERVICE_ACCOUNT_ACCESS_ONLY: {VAR_TYPE: TYPE_BOOLEAN},
+  }

src/gam/gamlib/glentity.py

@@ -0,0 +1,831 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2024 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""GAM entity processing
+
+"""
+
+class GamEntity():
+
+  ROLE_MANAGER = 'MANAGER'
+  ROLE_MEMBER = 'MEMBER'
+  ROLE_OWNER = 'OWNER'
+  ROLE_LIST = [ROLE_MANAGER, ROLE_MEMBER, ROLE_OWNER]
+  ROLE_USER = 'USER'
+  ROLE_MANAGER_MEMBER = ','.join([ROLE_MANAGER, ROLE_MEMBER])
+  ROLE_MANAGER_OWNER = ','.join([ROLE_MANAGER, ROLE_OWNER])
+  ROLE_MEMBER_OWNER = ','.join([ROLE_MEMBER, ROLE_OWNER])
+  ROLE_MANAGER_MEMBER_OWNER = ','.join(ROLE_LIST)
+  ROLE_PUBLIC = 'PUBLIC'
+  ROLE_ALL = ROLE_MANAGER_MEMBER_OWNER
+
+  TYPE_CBCM_BROWSER = 'CBCM_BROWSER'
+  TYPE_CUSTOMER = 'CUSTOMER'
+  TYPE_EXTERNAL = 'EXTERNAL'
+  TYPE_OTHER = 'OTHER'
+  TYPE_GROUP = 'GROUP'
+  TYPE_SERVICE_ACCOUNT = 'SERVICE_ACCOUNT'
+  TYPE_USER = 'USER'
+
+# Keys into NAMES; arbitrary values but must be unique
+  ACCESS_TOKEN = 'atok'
+  ACCOUNT = 'acct'
+  ACTION = 'actn'
+  ACTIVITY = 'actv'
+  ADMINISTRATOR = 'admi'
+  ADMIN_ROLE = 'adro'
+  ADMIN_ROLE_ASSIGNMENT = 'adra'
+  ALERT = 'alrt'
+  ALERT_ID = 'alri'
+  ALERT_FEEDBACK = 'alfb'
+  ALERT_FEEDBACK_ID = 'alfi'
+  ALIAS = 'alia'
+  ALIAS_EMAIL = 'alie'
+  ALIAS_TARGET = 'alit'
+  ANALYTIC_ACCOUNT = 'anac'
+  ANALYTIC_ACCOUNT_SUMMARY = 'anas'
+  ANALYTIC_DATASTREAM = 'anad'
+  ANALYTIC_PROPERTY = 'anap'
+  ANALYTIC_UA_PROPERTY = 'anau'
+  API = 'api '
+  APP_ACCESS_SETTINGS = 'apps'
+  APP_ID = 'appi'
+  APP_NAME = 'appn'
+  APPLICATION_SPECIFIC_PASSWORD = 'aspa'
+  ARROWS_ENABLED = 'arro'
+  ATTACHMENT = 'atta'
+  ATTENDEE = 'atnd'
+  AUDIT_ACTIVITY_REQUEST = 'auda'
+  AUDIT_EXPORT_REQUEST = 'audx'
+  AUDIT_MONITOR_REQUEST = 'audm'
+  BACKUP_VERIFICATION_CODES = 'buvc'
+  BUILDING = 'bldg'
+  BUILDING_ID = 'bldi'
+  CAA_LEVEL = 'calv'
+  CALENDAR = 'cale'
+  CALENDAR_ACL = 'cacl'
+  CALENDAR_SETTINGS = 'cset'
+  CHANNEL_CUSTOMER = 'chcu'
+  CHANNEL_CUSTOMER_ENTITLEMENT = 'chce'
+  CHANNEL_OFFER = 'chof'
+  CHANNEL_PRODUCT = 'chpr'
+  CHANNEL_SKU = 'chsk'
+  CHAT_BOT = 'chbo'
+  CHAT_ADMIN = 'chad'
+  CHAT_EVENT = 'chev'
+  CHAT_MANAGER_USER = 'chgu'
+  CHAT_MEMBER = 'chme'
+  CHAT_MEMBER_GROUP = 'chmg'
+  CHAT_MEMBER_USER = 'chmu'
+  CHAT_MESSAGE = 'chms'
+  CHAT_MESSAGE_ID = 'chmi'
+  CHAT_SPACE = 'chsp'
+  CHAT_THREAD = 'chth'
+  CHILD_ORGANIZATIONAL_UNIT = 'corg'
+  CHROME_APP = 'capp'
+  CHROME_APP_DEVICE = 'capd'
+  CHROME_BROWSER = 'chbr'
+  CHROME_BROWSER_ENROLLMENT_TOKEN = 'cbet'
+  CHROME_CHANNEL = 'chan'
+  CHROME_DEVICE = 'chdv'
+  CHROME_MODEL = 'chmo'
+  CHROME_NETWORK_ID = 'chni'
+  CHROME_NETWORK_NAME = 'chnn'
+  CHROME_PLATFORM = 'cpla'
+  CHROME_POLICY = 'cpol'
+  CHROME_POLICY_IMAGE = 'cpim'
+  CHROME_POLICY_SCHEMA = 'cpsc'
+  CHROME_PROFILE = 'cpro'
+  CHROME_RELEASE = 'crel'
+  CHROME_VERSION = 'cver'
+  CLASSIFICATION_LABEL = 'dlab'
+  CLASSIFICATION_LABEL_FIELD_ID = 'dlfi'
+  CLASSIFICATION_LABEL_ID = 'dlid'
+  CLASSIFICATION_LABEL_NAME = 'dlna'
+  CLASSIFICATION_LABEL_PERMISSION = 'dlpe'
+  CLASSIFICATION_LABEL_PERMISSION_NAME = 'dlpn'
+  CLASSROOM_INVITATION = 'clai'
+  CLASSROOM_INVITATION_OWNER = 'clio'
+  CLASSROOM_INVITATION_STUDENT = 'clis'
+  CLASSROOM_INVITATION_TEACHER = 'clit'
+  CLASSROOM_OAUTH2_TXT_FILE = 'coa'
+  CLASSROOM_USER_PROFILE = 'clup'
+  CLIENT_ID = 'clid'
+  CLIENT_SECRETS_JSON_FILE = 'csjf'
+  CLOUD_IDENTITY_GROUP = 'cidg'
+  CLOUD_STORAGE_BUCKET = 'clsb'
+  CLOUD_STORAGE_FILE = 'clsf'
+  COLLABORATOR = 'cola'
+  COMMAND_ID = 'cmdi'
+  COMPANY_DEVICE = 'codv'
+  CONFIG_FILE = 'conf'
+  CONTACT = 'cont'
+  CONTACT_DELEGATE = 'cond'
+  CONTACT_GROUP = 'cogr'
+  CONTACT_GROUP_NAME = 'cogn'
+  COPYFROM_COURSE = 'cfco'
+  COPYFROM_GROUP = 'cfgr'
+  COURSE = 'cour'
+  COURSE_ALIAS = 'coal'
+  COURSE_ANNOUNCEMENT = 'cann'
+  COURSE_ANNOUNCEMENT_ID = 'caid'
+  COURSE_ANNOUNCEMENT_STATE = 'cast'
+  COURSE_MATERIAL_DRIVEFILE = 'comd'
+  COURSE_MATERIAL_FORM = 'comf'
+  COURSE_MATERIAL = 'cmtl'
+  COURSE_MATERIAL_ID = 'cmid'
+  COURSE_MATERIAL_STATE = 'cmst'
+  COURSE_NAME = 'cona'
+  COURSE_STATE = 'cost'
+  COURSE_SUBMISSION_ID = 'csid'
+  COURSE_SUBMISSION_STATE = 'csst'
+  COURSE_TOPIC = 'ctop'
+  COURSE_TOPIC_ID = 'ctoi'
+  COURSE_WORK = 'cwrk'
+  COURSE_WORK_ID = 'cwid'
+  COURSE_WORK_STATE = 'cwst'
+  CREATOR_ID = 'crid'
+  CREDENTIALS = 'cred'
+  CRITERIA = 'crit'
+  CROS_DEVICE = 'cros'
+  CROS_SERIAL_NUMBER = 'crsn'
+  CSE_IDENTITY = 'csei'
+  CSE_KEYPAIR = 'csek'
+  CUSTOMER_DOMAIN = 'cudo'
+  CUSTOMER_ID = 'cuid'
+  DATE = 'date'
+  DEFAULT_LANGUAGE = 'dfla'
+  DELEGATE = 'dele'
+  DELETED_USER = 'delu'
+  DELIVERY = 'deli'
+  DEVICE = 'devi'
+  DEVICE_FILE = 'devf'
+  DIRECTORY = 'drct'
+  DEVICE_USER = 'devu'
+  DEVICE_USER_CLIENT_STATE = 'ducs'
+  DISCOVERY_JSON_FILE = 'disc'
+  DOCUMENT = 'docu'
+  DOMAIN = 'doma'
+  DOMAIN_ALIAS = 'doal'
+  DOMAIN_CONTACT = 'doco'
+  DOMAIN_PEOPLE_CONTACT = 'dopc'
+  DOMAIN_PROFILE = 'dopr'
+  DRIVE_DISK_USAGE = 'drdu'
+  DRIVE_FILE = 'dfil'
+  DRIVE_FILE_COMMENT = 'filc'
+  DRIVE_FILE_ID = 'fili'
+  DRIVE_FILE_NAME = 'filn'
+  DRIVE_FILE_RENAMED = 'firn'
+  DRIVE_FILE_REVISION = 'filr'
+  DRIVE_FILE_SHORTCUT = 'fils'
+  DRIVE_FILE_OR_FOLDER = 'fifo'
+  DRIVE_FILE_OR_FOLDER_ACL = 'fiac'
+  DRIVE_FILE_OR_FOLDER_ID = 'fifi'
+  DRIVE_FOLDER = 'fold'
+  DRIVE_FOLDER_ID = 'foli'
+  DRIVE_FOLDER_NAME = 'foln'
+  DRIVE_FOLDER_PATH = 'folp'
+  DRIVE_FOLDER_RENAMED = 'forn'
+  DRIVE_FOLDER_SHORTCUT = 'fols'
+  DRIVE_ORPHAN_FILE_OR_FOLDER = 'orph'
+  DRIVE_PARENT_FOLDER = 'fipf'
+  DRIVE_PARENT_FOLDER_ID = 'fipi'
+  DRIVE_PARENT_FOLDER_REFERENCE = 'pfrf'
+  DRIVE_PATH = 'drvp'
+  DRIVE_SETTINGS = 'drvs'
+  DRIVE_SHORTCUT = 'drsc'
+  DRIVE_SHORTCUT_ID = 'dsci'
+  DRIVE_3PSHORTCUT = 'dr3s'
+  DRIVE_TRASH = 'drvt'
+  EMAIL = 'emai'
+  EMAIL_ALIAS = 'emal'
+  EMAIL_SETTINGS = 'emse'
+  END_TIME = 'endt'
+  ENTITY = 'enti'
+  EVENT = 'evnt'
+  EVENT_BIRTHDAY = 'evbd'
+  EVENT_FOCUSTIME = 'evft'
+  EVENT_OUTOFOFFICE = 'evoo'
+  EVENT_WORKINGLOCATION = 'evwl'
+  FEATURE = 'feat'
+  FIELD = 'fiel'
+  FILE = 'file'
+  FILE_PARENT_TREE = 'fptr'
+  FILTER = 'filt'
+  FORM = 'form'
+  FORM_RESPONSE = 'frmr'
+  FORWARD_ENABLED = 'fwde'
+  FORWARDING_ADDRESS = 'fwda'
+  GCP_FOLDER = 'gcpf'
+  GCP_FOLDER_NAME = 'gcpn'
+  GMAIL_PROFILE = 'gmpr'
+  GROUP = 'grou'
+  GROUP_ALIAS = 'gali'
+  GROUP_EMAIL = 'gale'
+  GROUP_MEMBERSHIP = 'gmem'
+  GROUP_MEMBERSHIP_TREE = 'gmtr'
+  GROUP_SETTINGS = 'gset'
+  GROUP_TREE = 'gtre'
+  GUARDIAN = 'guar'
+  GUARDIAN_INVITATION = 'gari'
+  GUARDIAN_AND_INVITATION = 'gani'
+  IAM_POLICY = 'iamp'
+  IMAP_ENABLED = 'imap'
+  INBOUND_SSO_ASSIGNMENT = 'insa'
+  INBOUND_SSO_CREDENTIALS = 'insc'
+  INBOUND_SSO_PROFILE = 'insp'
+  INSTANCE = 'inst'
+  ITEM = 'item'
+  ISSUER_CN = 'iscn'
+  KEYBOARD_SHORTCUTS_ENABLED = 'kbsc'
+  LABEL = 'labe'
+  LABEL_ID = 'labi'
+  LANGUAGE = 'lang'
+  LICENSE = 'lice'
+  LOCATION = 'loca'
+  LOOKERSTUDIO_ASSET = 'lsas'
+  LOOKERSTUDIO_ASSET_DATASOURCE = 'lsad'
+  LOOKERSTUDIO_ASSETID = 'lsai'
+  LOOKERSTUDIO_ASSET_REPORT = 'lsar'
+  LOOKERSTUDIO_PERMISSION = 'lspe'
+  MD5HASH = 'md5h'
+  MEET_SPACE = 'mesp'
+  MEET_CONFERENCE = 'msco'
+  MEET_PARTICIPANT = 'msps'
+  MEET_RECORDING = 'msre'
+  MEET_TRANSCRIPT = 'mstr'
+  MEMBER = 'memb'
+  MEMBER_NOT_ARCHIVED = 'mena'
+  MEMBER_ARCHIVED = 'mear'
+  MEMBER_NOT_SUSPENDED = 'mens'
+  MEMBER_SUSPENDED = 'mesu'
+  MEMBER_NOT_SUSPENDED_NOT_ARCHIVED = 'nsna'
+  MEMBER_SUSPENDED_ARCHIVED = 'suar'
+  MEMBER_RESTRICTION = 'memr'
+  MEMBER_URI = 'memu'
+  MEMBERSHIP_TREE = 'metr'
+  MESSAGE = 'mesg'
+  MIMETYPE = 'mime'
+  MOBILE_DEVICE = 'mobi'
+  NAME = 'name'
+  NOTE = 'note'
+  NOTE_ACL = 'nota'
+  NOTES_ACLS = 'naac'
+  NONEDITABLE_ALIAS = 'neal'
+  OAUTH2_TXT_FILE = 'oaut'
+  OAUTH2SERVICE_JSON_FILE = 'oau2'
+  ORGANIZATIONAL_UNIT = 'orgu'
+  OTHER_CONTACT = 'otco'
+  OWNER = 'ownr'
+  OWNER_ID = 'owid'
+  PAGE_SIZE = 'page'
+  PARENT_ORGANIZATIONAL_UNIT = 'porg'
+  PARTICIPANT = 'part'
+  PEOPLE_CONTACT = 'peco'
+  PEOPLE_CONTACT_GROUP = 'pecg'
+  PEOPLE_PHOTO = 'peph'
+  PEOPLE_PROFILE = 'pepr'
+  PERMISSION = 'perm'
+  PERMISSION_ID = 'peid'
+  PERMITTEE = 'prmt'
+  PERSONAL_DEVICE = 'pedv'
+  PHOTO = 'phot'
+  POLICY = 'poli'
+  POP_ENABLED = 'popa'
+  PRESENTATION = 'pres'
+  PRINTER = 'prin'
+  PRINTER_ID = 'prid'
+  PRINTER_MODEL = 'prmd'
+  PRIVILEGE = 'priv'
+  PRODUCT = 'prod'
+  PROFILE_SHARING_ENABLED = 'prof'
+  PROJECT = 'proj'
+  PROJECT_FOLDER = 'prjf'
+  PROJECT_ID = 'prji'
+  PUBLIC_KEY = 'pubk'
+  QUERY = 'quer'
+  RECIPIENT = 'recp'
+  RECIPIENT_BCC = 'rebc'
+  RECIPIENT_CC = 'recc'
+  REPORT = 'rept'
+  REQUEST_ID = 'reqi'
+  RESOURCE_CALENDAR = 'resc'
+  RESOURCE_ID = 'resi'
+  ROLE = 'role'
+  ROW = 'row '
+  SCOPE = 'scop'
+  SECTION = 'sect'
+  SENDAS_ADDRESS = 'sasa'
+  SENDER = 'send'
+  SERVICE = 'serv'
+  SHAREDDRIVE = 'tdrv'
+  SHAREDDRIVE_ACL = 'tdac'
+  SHAREDDRIVE_FOLDER = 'tdfo'
+  SHAREDDRIVE_ID = 'tdid'
+  SHAREDDRIVE_NAME = 'tdna'
+  SHAREDDRIVE_THEME = 'tdth'
+  SHEET = 'shet'
+  SHEET_ID = 'shti'
+  SIGNATURE = 'sign'
+  SITE = 'site'
+  SITE_ACL = 'sacl'
+  SIZE = 'size'
+  SKU = 'sku '
+  SMIME_ID = 'smid'
+  SNIPPETS_ENABLED = 'snip'
+  SSO_KEY = 'ssok'
+  SSO_SETTINGS = 'ssos'
+  SOURCE_USER = 'src'
+  SPREADSHEET = 'sprd'
+  SPREADSHEET_RANGE = 'ssrn'
+  START_TIME = 'strt'
+  STATUS = 'stat'
+  STUDENT = 'stud'
+  SUBSCRIPTION = 'subs'
+  SVCACCT = 'svac'
+  SVCACCT_KEY = 'svky'
+  TARGET_USER = 'tgt'
+  TASK = 'task'
+  TASKLIST = 'tali'
+  TEACHER = 'teac'
+  THREAD = 'thre'
+  TRANSFER_APPLICATION = 'trap'
+  TRANSFER_ID = 'trid'
+  TRANSFER_REQUEST = 'trnr'
+  TRASHED_EVENT = 'trev'
+  TRUSTED_APPLICATION = 'trus'
+  TYPE = 'type'
+  UNICODE_ENCODING_ENABLED = 'unic'
+  UNIQUE_ID = 'uniq'
+  URL = 'url '
+  USER = 'user'
+  USER_ALIAS = 'uali'
+  USER_EMAIL = 'uema'
+  USER_INVITATION = 'uinv'
+  USER_NOT_SUSPENDED = 'uns'
+  USER_SCHEMA = 'usch'
+  USER_SUSPENDED = 'usup'
+  VACATION = 'vaca'
+  VACATION_ENABLED = 'vace'
+  VALUE = 'val'
+  VAULT_EXPORT = 'vlte'
+  VAULT_HOLD = 'vlth'
+  VAULT_MATTER = 'vltm'
+  VAULT_MATTER_ARTIFACT = 'vlma'
+  VAULT_MATTER_ID = 'vlmi'
+  VAULT_OPERATION = 'vlto'
+  VAULT_QUERY = 'vltq'
+  WEBCLIPS_ENABLED = 'webc'
+  YOUTUBE_CHANNEL = 'ytch'
+  # _NAMES[0] is plural, _NAMES[1] is singular unless the item name is explicitly plural (Calendar Settings)
+  # For items with Boolean values, both entries are singular (Forward, POP)
+  # These values can be translated into other languages
+  _NAMES = {
+    ACCESS_TOKEN: ['Access Tokens', 'Access Token'],
+    ACCOUNT: ['Google Workspace Accounts', 'Google Workspace Account'],
+    ACTION: ['Actions', 'Action'],
+    ACTIVITY: ['Activities', 'Activity'],
+    ADMINISTRATOR: ['Administrators', 'Administrator'],
+    ADMIN_ROLE: ['Admin Roles', 'Admin Role'],
+    ADMIN_ROLE_ASSIGNMENT: ['Admin Role Assignments', 'Admin Role Assignment'],
+    ALERT: ['Alerts', 'Alert'],
+    ALERT_ID: ['Alert IDs', 'Alert ID'],
+    ALERT_FEEDBACK: ['Alert Feedbacks', 'Alert Feedback'],
+    ALERT_FEEDBACK_ID: ['Alert Feedback IDs', 'Alert Feedback ID'],
+    ALIAS: ['Aliases', 'Alias'],
+    ALIAS_EMAIL: ['Alias Emails', 'Alias Email'],
+    ALIAS_TARGET: ['Alias Targets', 'Alias Target'],
+    ANALYTIC_ACCOUNT: ['Analytic Accounts', 'Analytic Account'],
+    ANALYTIC_ACCOUNT_SUMMARY: ['Analytic Account Summaries', 'Analytic Account Summary'],
+    ANALYTIC_DATASTREAM: ['Analytic Datastreams', 'Analytic Datastream'],
+    ANALYTIC_PROPERTY: ['Analytic GA4 Properties', 'Analytic GA4 Property'],
+    ANALYTIC_UA_PROPERTY: ['Analytic UA Properties', 'Analytic UA Property'],
+    API: ['APIs', 'API'],
+    APP_ACCESS_SETTINGS: ['Application Access Settings', 'Application Access Settings'],
+    APP_ID: ['Application IDs', 'Application ID'],
+    APP_NAME: ['Application Names', 'Application Name'],
+    APPLICATION_SPECIFIC_PASSWORD: ['Application Specific Password IDs', 'Application Specific Password ID'],
+    ARROWS_ENABLED: ['Personal Indicator Arrows Enabled', 'Personal Indicator Arrows Enabled'],
+    ATTACHMENT: ['Attachments', 'Attachment'],
+    ATTENDEE: ['Attendees', 'Attendee'],
+    AUDIT_ACTIVITY_REQUEST: ['Audit Activity Requests', 'Audit Activity Request'],
+    AUDIT_EXPORT_REQUEST: ['Audit Export Requests', 'Audit Export Request'],
+    AUDIT_MONITOR_REQUEST: ['Audit Monitor Requests', 'Audit Monitor Request'],
+    BACKUP_VERIFICATION_CODES: ['Backup Verification Codes', 'Backup Verification Codes'],
+    BUILDING: ['Buildings', 'Building'],
+    BUILDING_ID: ['Building IDs', 'Building ID'],
+    CAA_LEVEL: ['CAA Levels', 'CAA Level'],
+    CALENDAR: ['Calendars', 'Calendar'],
+    CALENDAR_ACL: ['Calendar ACLs', 'Calendar ACL'],
+    CALENDAR_SETTINGS: ['Calendar Settings', 'Calendar Settings'],
+    CHANNEL_CUSTOMER: ['Channel Customers', 'Channel Customer'],
+    CHANNEL_CUSTOMER_ENTITLEMENT: ['Channel Customer Entitlements', 'Channel Customer Entitlement'],
+    CHANNEL_OFFER: ['Channel Offers', 'Channel Offer'],
+    CHANNEL_PRODUCT: ['Channel Products', 'Channel Product'],
+    CHANNEL_SKU: ['Channel SKUs', 'Channel SKU'],
+    CHAT_BOT: ['Chat BOTs', 'Chat BOT'],
+    CHAT_ADMIN: ['Chat Admins', 'Chat Admin'],
+    CHAT_EVENT: ['Chat Events', 'Chat Event'],
+    CHAT_MANAGER_USER: ['Chat User Managers', 'Chat User Manager'],
+    CHAT_MESSAGE: ['Chat Messages', 'Chat Message'],
+    CHAT_MESSAGE_ID: ['Chat Message IDs', 'Chat Message ID'],
+    CHAT_MEMBER: ['Chat Members', 'Chat Member'],
+    CHAT_MEMBER_GROUP: ['Chat Group Members', 'Chat Group Member'],
+    CHAT_MEMBER_USER: ['Chat User Members', 'Chat User Member'],
+    CHAT_SPACE: ['Chat Spaces', 'Chat Space'],
+    CHAT_THREAD: ['Chat Threads', 'Chat Thread'],
+    CHILD_ORGANIZATIONAL_UNIT: ['Child Organizational Units', 'Child Organizational Unit'],
+    CHROME_APP: ['Chrome Applications', 'Chrome Application'],
+    CHROME_APP_DEVICE: ['Chrome Application Devices', 'Chrome Application Device'],
+    CHROME_BROWSER: ['Chrome Browsers', 'Chrome Browser'],
+    CHROME_BROWSER_ENROLLMENT_TOKEN: ['Chrome Browser Enrollment Tokens', 'Chrome Browser Enrollment Token'],
+    CHROME_CHANNEL: ['Chrome Channels', 'Chrome Channel'],
+    CHROME_DEVICE: ['Chrome Devices', 'Chrome Device'],
+    CHROME_MODEL: ['Chrome Models', 'Chrome Model'],
+    CHROME_NETWORK_ID: ['Chrome Network IDs', 'Chrome Network ID'],
+    CHROME_NETWORK_NAME: ['Chrome Network Names', 'Chrome Network Name'],
+    CHROME_PLATFORM: ['Chrome Platforms', 'Chrome Platform'],
+    CHROME_POLICY: ['Chrome Policies', 'Chrome Policy'],
+    CHROME_POLICY_IMAGE: ['Chrome Policy Images', 'Chrome Policy Image'],
+    CHROME_POLICY_SCHEMA: ['Chrome Policy Schemas', 'Chrome Policy Schema'],
+    CHROME_PROFILE: ['Chrome Profiles', 'Chrome Profile'],
+    CHROME_RELEASE: ['Chrome Releases', 'Chrome Release'],
+    CHROME_VERSION: ['Chrome Versions', 'Chrome Version'],
+    CLASSIFICATION_LABEL: ['Classification Labels', 'Classification Label'],
+    CLASSIFICATION_LABEL_FIELD_ID: ['Classification Label Field IDs', 'Classification Label Field ID'],
+    CLASSIFICATION_LABEL_ID: ['Classification Label IDs', 'Classification Label ID'],
+    CLASSIFICATION_LABEL_NAME: ['Classification Label Names', 'Classification Label Name'],
+    CLASSIFICATION_LABEL_PERMISSION: ['Classification Label Permissions', 'Classification Label Permission'],
+    CLASSIFICATION_LABEL_PERMISSION_NAME: ['Classification Label Permission Names', 'Classification Label Permission Name'],
+    CLASSROOM_INVITATION: ['Classroom Invitations', 'Classroom Invitation'],
+    CLASSROOM_INVITATION_OWNER: ['Classroom Owner Invitations', 'Classroom Owner Invitation'],
+    CLASSROOM_INVITATION_STUDENT: ['Classroom Student Invitations', 'Classroom Student Invitation'],
+    CLASSROOM_INVITATION_TEACHER: ['Classroom Teacher Invitations', 'Classroom Teacher Invitation'],
+    CLASSROOM_OAUTH2_TXT_FILE: ['Classroom OAuth2 File', 'Classroom OAuth2 File'],
+    CLASSROOM_USER_PROFILE: ['Classroom User Profile', 'Classroom User Profile'],
+    CLIENT_ID: ['Client IDs', 'Client ID'],
+    CLIENT_SECRETS_JSON_FILE: ['Client Secrets File', 'Client Secrets File'],
+    CLOUD_IDENTITY_GROUP: ['Cloud Identity Groups', 'Cloud Identity Group'],
+    CLOUD_STORAGE_BUCKET: ['Cloud Storage Buckets', 'Cloud Storage Bucket'],
+    CLOUD_STORAGE_FILE: ['Cloud Storage Files', 'Cloud Storage File'],
+    COLLABORATOR: ['Collaborators', 'Collaborator'],
+    COMMAND_ID: ['Command IDs', 'Command ID'],
+    COMPANY_DEVICE: ['Company Devices', 'Company Device'],
+    CONFIG_FILE: ['Config File', 'Config File'],
+    CONTACT: ['Contacts', 'Contact'],
+    CONTACT_DELEGATE: ['Contact Delegates', 'Contact Delegate'],
+    CONTACT_GROUP: ['Contact Groups', 'Contact Group'],
+    CONTACT_GROUP_NAME: ['Contact Group Names', 'Contact Group Name'],
+    COPYFROM_COURSE: ['Copy From Courses', 'CopyFrom Course'],
+    COPYFROM_GROUP: ['Copy From Groups', 'CopyFrom Group'],
+    COURSE: ['Courses', 'Course'],
+    COURSE_ALIAS: ['Course Aliases', 'Course Alias'],
+    COURSE_ANNOUNCEMENT: ['Course Announcements', 'Course Announcement'],
+    COURSE_ANNOUNCEMENT_ID: ['Course Announcement IDs', 'Course Announcement ID'],
+    COURSE_ANNOUNCEMENT_STATE: ['Course Announcement States', 'Course Announcement State'],
+    COURSE_MATERIAL_DRIVEFILE: ['Course Material Drive Files', 'Course Material Drive File'],
+    COURSE_MATERIAL_FORM: ['Course Material Forms', 'Course Material Form'],
+    COURSE_MATERIAL: ['Course Materials', 'Course Material'],
+    COURSE_MATERIAL_ID: ['Course Material IDs', 'Course Material ID'],
+    COURSE_MATERIAL_STATE: ['Course Material States', 'Course Material State'],
+    COURSE_NAME: ['Course Names', 'Course Name'],
+    COURSE_STATE: ['Course States', 'Course State'],
+    COURSE_SUBMISSION_ID: ['Course Submission IDs', 'Course Submission ID'],
+    COURSE_SUBMISSION_STATE: ['Course Submission States', 'Course Submission State'],
+    COURSE_TOPIC: ['Course Topics', 'Course Topic'],
+    COURSE_TOPIC_ID: ['Course Topic IDs', 'Course Topic ID'],
+    COURSE_WORK: ['Course Works', 'Course Work'],
+    COURSE_WORK_ID: ['Course Work IDs', 'Course Work ID'],
+    COURSE_WORK_STATE: ['Course Work States', 'Course Work State'],
+    CREATOR_ID: ['Creator IDs', 'Creator ID'],
+    CREDENTIALS: ['Credentials', 'Credentials'],
+    CRITERIA: ['Criteria', 'Criteria'],
+    CROS_DEVICE: ['CrOS Devices', 'CrOS Device'],
+    CROS_SERIAL_NUMBER: ['CrOS Serial Numbers', 'CrOS Serial Numbers'],
+    CSE_IDENTITY: ['CSE Identities', 'CSE Identity'],
+    CSE_KEYPAIR: ['CSE KeyPairs', 'CSE KeyPair'],
+    CUSTOMER_DOMAIN: ['Customer Domains', 'Customer Domain'],
+    CUSTOMER_ID: ['Customer IDs', 'Customer ID'],
+    DATE: ['Dates', 'Date'],
+    DEFAULT_LANGUAGE: ['Default Language', 'Default Language'],
+    DELEGATE: ['Delegates', 'Delegate'],
+    DELETED_USER: ['Deleted Users', 'Deleted User'],
+    DELIVERY: ['Delivery', 'Delivery'],
+    DEVICE: ['Devices', 'Device'],
+    DEVICE_FILE: ['Device Files', 'Device File'],
+    DEVICE_USER: ['Device Users', 'Device User'],
+    DEVICE_USER_CLIENT_STATE: ['Device Users Client States', 'Device User Client State'],
+    DIRECTORY: ['Directories', 'Directory'],
+    DISCOVERY_JSON_FILE: ['Discovery File', 'Discovery File'],
+    DOCUMENT: ['Documents', 'Document'],
+    DOMAIN: ['Domains', 'Domain'],
+    DOMAIN_ALIAS: ['Domain Aliases', 'Domain Alias'],
+    DOMAIN_CONTACT: ['Domain Contacts', 'Domain Contact'],
+    DOMAIN_PEOPLE_CONTACT: ['Domain People Contacts', 'Domain People Contact'],
+    DOMAIN_PROFILE: ['Domain Profiles', 'Domain Profile'],
+    DRIVE_DISK_USAGE: ['Drive Disk Usages', 'Drive Disk Usage'],
+    DRIVE_FILE: ['Drive Files', 'Drive File'],
+    DRIVE_FILE_COMMENT: ['Drive File Comments', 'Drive File Comment'],
+    DRIVE_FILE_ID: ['Drive File IDs', 'Drive File ID'],
+    DRIVE_FILE_NAME: ['Drive File Names', 'Drive File Name'],
+    DRIVE_FILE_REVISION: ['Drive File Revisions', 'Drive File Revision'],
+    DRIVE_FILE_RENAMED: ['Drive Files Renamed', 'Drive File Renamed'],
+    DRIVE_FILE_SHORTCUT: ['Drive File Shortcuts', 'Drive File Shortcut'],
+    DRIVE_FILE_OR_FOLDER: ['Drive Files/Folders', 'Drive File/Folder'],
+    DRIVE_FILE_OR_FOLDER_ACL: ['Drive File/Folder ACLs', 'Drive File/Folder ACL'],
+    DRIVE_FILE_OR_FOLDER_ID: ['Drive File/Folder IDs', 'Drive File/Folder ID'],
+    DRIVE_FOLDER: ['Drive Folders', 'Drive Folder'],
+    DRIVE_FOLDER_ID: ['Drive Folder IDs', 'Drive Folder ID'],
+    DRIVE_FOLDER_NAME: ['Drive Folder Names', 'Drive Folder Name'],
+    DRIVE_FOLDER_PATH: ['Drive Folder Paths', 'Drive Folder Path'],
+    DRIVE_FOLDER_RENAMED: ['Drive Folders Renamed', 'Drive Folder Renamed'],
+    DRIVE_FOLDER_SHORTCUT: ['Drive Folder Shortcuts', 'Drive Folder Shortcut'],
+    DRIVE_ORPHAN_FILE_OR_FOLDER: ['Drive Orphan Files/Folders', 'Drive Orphan File/Folder'],
+    DRIVE_PARENT_FOLDER: ['Drive Parent Folders', 'Drive Parent Folder'],
+    DRIVE_PARENT_FOLDER_ID: ['Drive Parent Folder IDs', 'Drive Parent Folder ID'],
+    DRIVE_PARENT_FOLDER_REFERENCE: ['Drive Parent Folder References', 'Drive Parent Folder Reference'],
+    DRIVE_PATH: ['Drive Paths', 'Drive Path'],
+    DRIVE_SETTINGS: ['Drive Settings', 'Drive Settings'],
+    DRIVE_SHORTCUT: ['Drive Shortcuts', 'Drive Shortcut'],
+    DRIVE_SHORTCUT_ID: ['Drive Shortcut IDs', 'Drive Shortcut ID'],
+    DRIVE_3PSHORTCUT: ['Drive 3rd Party Shortcuts', 'Drive 3rd Party Shortcut'],
+    DRIVE_TRASH: ['Drive Trash', 'Drive Trash'],
+    EMAIL: ['Email Addresses', 'Email Address'],
+    EMAIL_ALIAS: ['Email Aliases', 'Email Alias'],
+    EMAIL_SETTINGS: ['Email Settings', 'Email Settings'],
+    END_TIME: ['End Times', 'End Time'],
+    ENTITY: ['Entities', 'Entity'],
+    EVENT: ['Events', 'Event'],
+    EVENT_BIRTHDAY: ['Borthday Events', 'Birthday Event'],
+    EVENT_FOCUSTIME: ['Focus Time Events', 'Focus Time Event'],
+    EVENT_OUTOFOFFICE: ['Out of Office Events', 'Out of Office Event'],
+    EVENT_WORKINGLOCATION: ['Working Location Events', 'Working Location Event'],
+    FEATURE: ['Features', 'Feature'],
+    FIELD: ['Fields', 'Field'],
+    FILE: ['Files', 'File'],
+    FILE_PARENT_TREE: ['File Parent Trees', 'File Parent Tree'],
+    FILTER: ['Filters', 'Filter'],
+    FORM: ['Forms', 'Form'],
+    FORM_RESPONSE: ['Form Responses', 'Form Response'],
+    FORWARD_ENABLED: ['Forward Enabled', 'Forward Enabled'],
+    FORWARDING_ADDRESS: ['Forwarding Addresses', 'Forwarding Address'],
+    GCP_FOLDER: ['GCP Folders', 'GCP Folder'],
+    GCP_FOLDER_NAME: ['GCP Folder Names', 'GCP Folder Name'],
+    GMAIL_PROFILE: ['Gmail Profile', 'Gmail Profile'],
+    GROUP: ['Groups', 'Group'],
+    GROUP_ALIAS: ['Group Aliases', 'Group Alias'],
+    GROUP_EMAIL: ['Group Emails', 'Group Email'],
+    GROUP_MEMBERSHIP: ['Group Memberships', 'Group Membership'],
+    GROUP_MEMBERSHIP_TREE: ['Group Membership Trees', 'Group Membership Tree'],
+    GROUP_SETTINGS: ['Group Settings', 'Group Settings'],
+    GROUP_TREE: ['Group Trees', 'Group Tree'],
+    GUARDIAN: ['Guardians', 'Guardian'],
+    GUARDIAN_INVITATION: ['Guardian Invitations', 'Guardian Invitation'],
+    GUARDIAN_AND_INVITATION: ['Guardians and Invitations', 'Guardian and Invitation'],
+    IAM_POLICY: ['IAM Policies', 'IAM Policy'],
+    IMAP_ENABLED: ['IMAP Enabled', 'IMAP Enabled'],
+    INBOUND_SSO_ASSIGNMENT: ['Inbound SSO Assignments', 'Inbound SSO Assignment'],
+    INBOUND_SSO_CREDENTIALS: ['Inbound SSO Credentials', 'Inbound SSO Credential'],
+    INBOUND_SSO_PROFILE: ['Inbound SSO Profiles', 'Inbound SSO Profile'],
+    INSTANCE: ['Instances', 'Instance'],
+    ISSUER_CN: ['Issuer CNs', 'Issuer CN'],
+    ITEM: ['Items', 'Item'],
+    KEYBOARD_SHORTCUTS_ENABLED: ['Keyboard Shortcuts Enabled', 'Keyboard Shortcuts Enabled'],
+    LABEL: ['Labels', 'Label'],
+    LABEL_ID: ['Label IDs', 'Label ID'],
+    LANGUAGE: ['Languages', 'Language'],
+    LICENSE: ['Licenses', 'License'],
+    LOCATION: ['Locations', 'Location'],
+    LOOKERSTUDIO_ASSET: ['Looker Studio Assets', 'Looker Studio Asset'],
+    LOOKERSTUDIO_ASSET_DATASOURCE: ['Looker Studio DATA_SOURCE Assets', 'Looker Studio DATA_SOURCE Asset'],
+    LOOKERSTUDIO_ASSETID: ['Looker Studio Asset IDs', 'Looker Studio Asset ID'],
+    LOOKERSTUDIO_ASSET_REPORT: ['Looker Studio REPORT Assets', 'Looker Studio REPORT Asset'],
+    LOOKERSTUDIO_PERMISSION: ['Looker Studio Permissions', 'Looker Studio Permission'],
+    MD5HASH: ['MD5 hash', 'MD5 Hash'],
+    MEET_SPACE: ['Meet Spaces', 'Meet Space'],
+    MEET_CONFERENCE: ['Meet Conferences', 'Meet Conference'],
+    MEET_PARTICIPANT: ['Meet Participants', 'Meet Participant'],
+    MEET_RECORDING: ['Meet Recordings', 'Meet Recording'],
+    MEET_TRANSCRIPT: ['Meet Transcripts', 'Meet Transcript'],
+    MEMBER: ['Members', 'Member'],
+    MEMBER_NOT_ARCHIVED: ['Members (Not Archived)', 'Member (Not Archived)'],
+    MEMBER_ARCHIVED: ['Members (Archived)', 'Member (Archived)'],
+    MEMBER_NOT_SUSPENDED: ['Members (Not Suspended)', 'Member (Not Suspended)'],
+    MEMBER_SUSPENDED: ['Members (Suspended)', 'Member (Suspended)'],
+    MEMBER_NOT_SUSPENDED_NOT_ARCHIVED: ['Members (Not Suspended & Not Archived)', 'Member (Not Suspended & Not Archived)'],
+    MEMBER_SUSPENDED_ARCHIVED: ['Members (Suspended & Archived)', 'Member (Suspended & Archived)'],
+    MEMBER_RESTRICTION: ['Member Restrictions', 'Member Restriction'],
+    MEMBER_URI: ['Member URIs', 'Member URI'],
+    MEMBERSHIP_TREE: ['Membership Trees', 'Membership Tree'],
+    MESSAGE: ['Messages', 'Message'],
+    MIMETYPE: ['MIME Types', 'MIME Type'],
+    MOBILE_DEVICE: ['Mobile Devices', 'Mobile Device'],
+    NAME: ['Names', 'Name'],
+    NOTE: ['Notes', 'Note'],
+    NOTE_ACL: ['Note ACLs', 'Note ACL'],
+    NOTES_ACLS: ["'Note's ACLs", "Note's ACLs"],
+    NONEDITABLE_ALIAS: ['Non-Editable Aliases', 'Non-Editable Alias'],
+    OAUTH2_TXT_FILE: ['Client OAuth2 File', 'Client OAuth2 File'],
+    OAUTH2SERVICE_JSON_FILE: ['Service Account OAuth2 File', 'Service Account OAuth2 File'],
+    ORGANIZATIONAL_UNIT: ['Organizational Units', 'Organizational Unit'],
+    OTHER_CONTACT: ['Other Contacts', 'Other Contact'],
+    OWNER: ['Owners', 'Owner'],
+    OWNER_ID: ['Owner IDs', 'Owner ID'],
+    PAGE_SIZE: ['Page Size', 'Page Size'],
+    PARENT_ORGANIZATIONAL_UNIT: ['Parent Organizational Units', 'Parent Organizational Unit'],
+    PARTICIPANT: ['Participants', 'Participant'],
+    PEOPLE_CONTACT: ['People Contacts', 'Person Contact'],
+    PEOPLE_CONTACT_GROUP: ['People Contact Groups', 'People Contact Group'],
+    PEOPLE_PHOTO: ['People Photos', 'Person Photo'],
+    PEOPLE_PROFILE: ['People Profiles', 'People Profile'],
+    PERMISSION: ['Permissions', 'Permission'],
+    PERMISSION_ID: ['Permission IDs', 'Permission ID'],
+    PERMITTEE: ['Permittees', 'Permittee'],
+    PERSONAL_DEVICE: ['Personal Devices', 'Personal Device'],
+    PHOTO: ['Photos', 'Photo'],
+    POLICY: ['Policies', 'Policy'],
+    POP_ENABLED: ['POP Enabled', 'POP Enabled'],
+    PRESENTATION: ['Presentations', 'Presentation'],
+    PRINTER: ['Printers', 'Printer'],
+    PRINTER_ID: ['Printer IDs', 'Printer ID'],
+    PRINTER_MODEL: ['Printer Models', 'Printer Model'],
+    PRIVILEGE: ['Privileges', 'Privilege'],
+    PRODUCT: ['Products', 'Product'],
+    PROFILE_SHARING_ENABLED: ['Profile Sharing Enabled', 'Profile Sharing Enabled'],
+    PROJECT: ['Projects', 'Project'],
+    PROJECT_FOLDER: ['Project Folders', 'Project Folder'],
+    PROJECT_ID: ['Project IDs', 'Project ID'],
+    PUBLIC_KEY: ['Public Key', 'Public Key'],
+    QUERY: ['Queries', 'Query'],
+    RECIPIENT: ['Recipients', 'Recipient'],
+    RECIPIENT_BCC: ['Recipients (BCC)', 'Recipient (BCC)'],
+    RECIPIENT_CC: ['Recipients (CC)', 'Recipient (CC)'],
+    REPORT: ['Reports', 'Report'],
+    REQUEST_ID: ['Request IDs', 'Request ID'],
+    RESOURCE_CALENDAR: ['Resource Calendars', 'Resource Calendar'],
+    RESOURCE_ID: ['Resource IDs', 'Resource ID'],
+    ROLE: ['Roles', 'Role'],
+    ROW: ['Rows', 'Row'],
+    SCOPE: ['Scopes', 'Scope'],
+    SECTION: ['Sections', 'Section'],
+    SENDAS_ADDRESS: ['SendAs Addresses', 'SendAs Address'],
+    SENDER: ['Senders', 'Sender'],
+    SERVICE: ['Services', 'Service'],
+    SHAREDDRIVE: ['Shared Drives', 'Shared Drive'],
+    SHAREDDRIVE_ACL: ['Shared Drive ACLs', 'Shared Drive ACL'],
+    SHAREDDRIVE_FOLDER: ['Shared Drive Folders', 'Shared Drive Folder'],
+    SHAREDDRIVE_ID: ['Shared Drive IDs', 'Shared Drive ID'],
+    SHAREDDRIVE_NAME: ['Shared Drive Names', 'Shared Drive Name'],
+    SHAREDDRIVE_THEME: ['Shared Drive Themes', 'Shared Drive Theme'],
+    SHEET: ['Sheets', 'Sheet'],
+    SHEET_ID: ['Sheet IDs', 'Sheet ID'],
+    SIGNATURE: ['Signatures', 'Signature'],
+    SITE: ['Sites', 'Site'],
+    SITE_ACL: ['Site ACLs', 'Site ACL'],
+    SIZE: ['Sizes', 'Size'],
+    SKU: ['SKUs', 'SKU'],
+    SMIME_ID: ['S/MIME Certificate IDs', 'S/MIME Certificate ID'],
+    SNIPPETS_ENABLED: ['Preview Snippets Enabled', 'Preview Snippets Enabled'],
+    SSO_KEY: ['SSO Key', 'SSO Key'],
+    SSO_SETTINGS: ['SSO Settings', 'SSO Settings'],
+    SOURCE_USER: ['Source Users', 'Source User'],
+    SPREADSHEET: ['Spreadsheets', 'Spreadsheet'],
+    SPREADSHEET_RANGE: ['Spreadsheet Ranges', 'Spreadsheet Range'],
+    START_TIME: ['Start Times', 'Start Time'],
+    STATUS: ['Status', 'Status'],
+    STUDENT: ['Students', 'Student'],
+    SUBSCRIPTION: ['Subscriptions', 'Subscription'],
+    SVCACCT: ['Service Accounts', 'Service Account'],
+    SVCACCT_KEY: ['Service Account Keys', 'Service Account Key'],
+    TARGET_USER: ['Target Users', 'Target User'],
+    TASK: ['Tasks', 'Task'],
+    TASKLIST: ['Tasklists', 'Tasklist'],
+    TEACHER: ['Teachers', 'Teacher'],
+    THREAD: ['Threads', 'Thread'],
+    TRANSFER_APPLICATION: ['Transfer Applications', 'Transfer Application'],
+    TRANSFER_ID: ['Transfer IDs', 'Transfer ID'],
+    TRANSFER_REQUEST: ['Transfer Requests', 'Transfer Request'],
+    TRASHED_EVENT: ['Trashed Events', 'Trashed Event'],
+    TRUSTED_APPLICATION: ['Trusted Applications', 'Trusted Application'],
+    TYPE: ['Types', 'Type'],
+    UNICODE_ENCODING_ENABLED: ['UTF-8 Encoding Enabled', 'UTF-8 Encoding Enabled'],
+    UNIQUE_ID: ['Unique IDs', 'Unique ID'],
+    URL: ['URLs', 'URL'],
+    USER: ['Users', 'User'],
+    USER_ALIAS: ['User Aliases', 'User Alias'],
+    USER_EMAIL: ['User Emails', 'User Email'],
+    USER_INVITATION: ['User Invitations', 'User Invitation'],
+    USER_NOT_SUSPENDED: ['Users (Not suspended)', 'User (Not suspended)'],
+    USER_SCHEMA: ['Schemas', 'Schema'],
+    USER_SUSPENDED: ['Users (Suspended)', 'User (Suspended)'],
+    VACATION: ['Vacation', 'Vacation'],
+    VACATION_ENABLED: ['Vacation Enabled', 'Vacation Enabled'],
+    VALUE: ['Values', 'Value'],
+    VAULT_EXPORT: ['Vault Exports', 'Vault Export'],
+    VAULT_HOLD: ['Vault Holds', 'Vault Hold'],
+    VAULT_MATTER: ['Vault Matters', 'Vault Matter'],
+    VAULT_MATTER_ARTIFACT: ['Vault Matter Artifacts', 'Vault Matter Artifact'],
+    VAULT_MATTER_ID: ['Vault Matter IDs', 'Vault Matter ID'],
+    VAULT_OPERATION: ['Vault Operations', 'Vault Operation'],
+    VAULT_QUERY: ['Vault Queries', 'Vault Query'],
+    WEBCLIPS_ENABLED: ['Web Clips Enabled', 'Web Clips Enabled'],
+    YOUTUBE_CHANNEL: ['YouTube Channels', 'YouTube Channel'],
+    ROLE_MANAGER: ['Managers', 'Manager'],
+    ROLE_MEMBER: ['Members', 'Member'],
+    ROLE_OWNER: ['Owners', 'Owner'],
+    ROLE_ALL: ['Members, Managers, Owners', 'Member, Manager, Owner'],
+    ROLE_USER: ['Users', 'User'],
+    ROLE_MANAGER_MEMBER: ['Members, Managers', 'Member, Manager'],
+    ROLE_MANAGER_OWNER: ['Managers, Owners', 'Manager, Owner'],
+    ROLE_MEMBER_OWNER: ['Members, Owners', 'Member, Owner'],
+    ROLE_MANAGER_MEMBER_OWNER: ['Members, Managers, Owners', 'Member, Manager, Owner'],
+    ROLE_PUBLIC: ['Public', 'Public'],
+    }
+
+  def __init__(self):
+    self.entityType = None
+    self.forWhom = None
+    self.preQualifier = ''
+    self.postQualifier = ''
+
+  def SetGetting(self, entityType):
+    self.entityType = entityType
+    self.preQualifier = self.postQualifier = ''
+
+  def SetGettingQuery(self, entityType, query):
+    self.entityType = entityType
+    self.preQualifier = f' that match query ({query})'
+    self.postQualifier = f' that matched query ({query})'
+
+  def SetGettingQualifier(self, entityType, qualifier):
+    self.entityType = entityType
+    self.preQualifier = self.postQualifier = qualifier
+
+  def Getting(self):
+    return self.entityType
+
+  def GettingPreQualifier(self):
+    return self.preQualifier
+
+  def GettingPostQualifier(self):
+    return self.postQualifier
+
+  def SetGettingForWhom(self, forWhom):
+    self.forWhom = forWhom
+
+  def GettingForWhom(self):
+    return self.forWhom
+
+  def Choose(self, entityType, count):
+    return self._NAMES[entityType][[0, 1][count == 1]]
+
+  def ChooseGetting(self, count):
+    return self._NAMES[self.entityType][[0, 1][count == 1]]
+
+  def Plural(self, entityType):
+    return self._NAMES[entityType][0]
+
+  def PluralGetting(self):
+    return self._NAMES[self.entityType][0]
+
+  def Singular(self, entityType):
+    return self._NAMES[entityType][1]
+
+  def SingularGetting(self):
+    return self._NAMES[self.entityType][1]
+
+  def MayTakeTime(self, entityType):
+    if entityType:
+      return f', may take some time on a large {self.Singular(entityType)}...'
+    return ''
+
+  def FormatEntityValueList(self, entityValueList):
+    evList = []
+    for j in range(0, len(entityValueList), 2):
+      evList.append(self.Singular(entityValueList[j]))
+      evList.append(entityValueList[j+1])
+    return evList
+
+  def TypeMessage(self, entityType, message):
+    return f'{self.Singular(entityType)}: {message}'
+
+  def TypeName(self, entityType, entityName):
+    return f'{self.Singular(entityType)}: {entityName}'
+
+  def TypeNameMessage(self, entityType, entityName, message):
+    return f'{self.Singular(entityType)}: {entityName} {message}'

src/gam/gamlib/glgapi.py

@@ -0,0 +1,817 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2024 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""GAM GAPI resources
+
+"""
+# callGAPI throw reasons
+ABORTED = 'aborted'
+ABUSIVE_CONTENT_RESTRICTION = 'abusiveContentRestriction'
+ACCESS_NOT_CONFIGURED = 'accessNotConfigured'
+ALREADY_EXISTS = 'alreadyExists'
+APPLY_LABEL_FORBIDDEN = 'applyLabelForbidden'
+AUTH_ERROR = 'authError'
+BACKEND_ERROR = 'backendError'
+BAD_GATEWAY = 'badGateway'
+BAD_REQUEST = 'badRequest'
+CANNOT_ADD_PARENT = 'cannotAddParent'
+CANNOT_CHANGE_ORGANIZER = 'cannotChangeOrganizer'
+CANNOT_CHANGE_ORGANIZER_OF_INSTANCE = 'cannotChangeOrganizerOfInstance'
+CANNOT_CHANGE_OWN_ACL = 'cannotChangeOwnAcl'
+CANNOT_CHANGE_OWNER_ACL = 'cannotChangeOwnerAcl'
+CANNOT_CHANGE_OWN_PRIMARY_SUBSCRIPTION = 'cannotChangeOwnPrimarySubscription'
+CANNOT_COPY_FILE = 'cannotCopyFile'
+CANNOT_DELETE_ONLY_REVISION = 'cannotDeleteOnlyRevision'
+CANNOT_DELETE_PRIMARY_CALENDAR = 'cannotDeletePrimaryCalendar'
+CANNOT_DELETE_PRIMARY_SENDAS = 'cannotDeletePrimarySendAs'
+CANNOT_DELETE_RESOURCE_WITH_CHILDREN = 'cannotDeleteResourceWithChildren'
+CANNOT_MODIFY_INHERITED_TEAMDRIVE_PERMISSION = 'cannotModifyInheritedTeamDrivePermission'
+CANNOT_MODIFY_RESTRICTED_LABEL = 'cannotModifyRestrictedLabel'
+CANNOT_MODIFY_VIEWERS_CAN_COPY_CONTENT = 'cannotModifyViewersCanCopyContent'
+CANNOT_MOVE_TRASHED_ITEM_INTO_TEAMDRIVE = 'cannotMoveTrashedItemIntoTeamDrive'
+CANNOT_MOVE_TRASHED_ITEM_OUT_OF_TEAMDRIVE = 'cannotMoveTrashedItemOutOfTeamDrive'
+CANNOT_REMOVE_OWNER = 'cannotRemoveOwner'
+CANNOT_SET_EXPIRATION = 'cannotSetExpiration'
+CANNOT_SHARE_GROUPS_WITHLINK = 'cannotShareGroupsWithLink'
+CANNOT_SHARE_USERS_WITHLINK = 'cannotShareUsersWithLink'
+CANNOT_SHARE_TEAMDRIVE_TOPFOLDER_WITH_ANYONEORDOMAINS = 'cannotShareTeamDriveTopFolderWithAnyoneOrDomains'
+CANNOT_SHARE_TEAMDRIVE_WITH_NONGOOGLE_ACCOUNTS = 'cannotShareTeamDriveWithNonGoogleAccounts'
+CANNOT_UPDATE_PERMISSION = 'cannotUpdatePermission'
+CONDITION_NOT_MET = 'conditionNotMet'
+CONFLICT = 'conflict'
+CONTENT_OWNER_ACCOUNT_NOT_FOUND = 'contentOwnerAccountNotFound'
+CROSS_DOMAIN_MOVE_RESTRICTION = 'crossDomainMoveRestriction'
+CUSTOMER_EXCEEDED_ROLE_ASSIGNMENTS_LIMIT = 'CUSTOMER_EXCEEDED_ROLE_ASSIGNMENTS_LIMIT'
+CUSTOMER_NOT_FOUND = 'customerNotFound'
+CYCLIC_MEMBERSHIPS_NOT_ALLOWED = 'cyclicMembershipsNotAllowed'
+DAILY_LIMIT_EXCEEDED = 'dailyLimitExceeded'
+DELETED = 'deleted'
+DELETED_USER_NOT_FOUND = 'deletedUserNotFound'
+DOMAIN_ALIAS_NOT_FOUND = 'domainAliasNotFound'
+DOMAIN_CANNOT_USE_APIS = 'domainCannotUseApis'
+DOMAIN_NOT_FOUND = 'domainNotFound'
+DOMAIN_NOT_VERIFIED_SECONDARY = 'domainNotVerifiedSecondary'
+DOMAIN_POLICY = 'domainPolicy'
+DOWNLOAD_QUOTA_EXCEEDED = 'downloadQuotaExceeded'
+DUPLICATE = 'duplicate'
+EVENT_DURATION_EXCEEDS_LIMIT = 'eventDurationExceedsLimit'
+EXPIRATION_DATE_NOT_ALLOWED_FOR_SHARED_DRIVE_MEMBERS = 'expirationDateNotAllowedForSharedDriveMembers'
+FAILED_PRECONDITION = 'failedPrecondition'
+FIELD_IN_USE = 'fieldInUse'
+FIELD_NOT_WRITABLE = 'fieldNotWritable'
+FILE_NEVER_WRITABLE = 'fileNeverWritable'
+FILE_NOT_FOUND = 'fileNotFound'
+FILE_ORGANIZER_NOT_YET_ENABLED_FOR_THIS_TEAMDRIVE = 'fileOrganizerNotYetEnabledForThisTeamDrive'
+FILE_ORGANIZER_ON_FOLDERS_IN_SHARED_DRIVE_ONLY = 'fileOrganizerOnFoldersInSharedDriveOnly'
+FILE_ORGANIZER_ON_NON_TEAMDRIVE_NOT_SUPPORTED = 'fileOrganizerOnNonTeamDriveNotSupported'
+FILE_OWNER_NOT_MEMBER_OF_TEAMDRIVE = 'fileOwnerNotMemberOfTeamDrive'
+FILE_OWNER_NOT_MEMBER_OF_WRITER_DOMAIN = 'fileOwnerNotMemberOfWriterDomain'
+FILE_WRITER_TEAMDRIVE_MOVE_IN_DISABLED = 'fileWriterTeamDriveMoveInDisabled'
+FORBIDDEN = 'forbidden'
+GATEWAY_TIMEOUT = 'gatewayTimeout'
+GROUP_NOT_FOUND = 'groupNotFound'
+ILLEGAL_ACCESS_ROLE_FOR_DEFAULT = 'illegalAccessRoleForDefault'
+INSUFFICIENT_ADMINISTRATOR_PRIVILEGES = 'insufficientAdministratorPrivileges'
+INSUFFICIENT_ARCHIVED_USER_LICENSES = 'insufficientArchivedUserLicenses'
+INSUFFICIENT_FILE_PERMISSIONS = 'insufficientFilePermissions'
+INSUFFICIENT_PARENT_PERMISSIONS = 'insufficientParentPermissions'
+INSUFFICIENT_PERMISSIONS = 'insufficientPermissions'
+INTERNAL_ERROR = 'internalError'
+INVALID = 'invalid'
+INVALID_ARGUMENT = 'invalidArgument'
+INVALID_ATTRIBUTE_VALUE = 'invalidAttributeValue'
+INVALID_CUSTOMER_ID = 'invalidCustomerId'
+INVALID_INPUT = 'invalidInput'
+INVALID_LINK_VISIBILITY = 'invalidLinkVisibility'
+INVALID_MEMBER = 'invalidMember'
+INVALID_MESSAGE_ID = 'invalidMessageId'
+INVALID_ORGUNIT = 'invalidOrgunit'
+INVALID_ORGUNIT_NAME = 'invalidOrgunitName'
+INVALID_OWNERSHIP_TRANSFER = 'invalidOwnershipTransfer'
+INVALID_PARAMETER = 'invalidParameter'
+INVALID_PARENT_ORGUNIT = 'invalidParentOrgunit'
+INVALID_QUERY = 'invalidQuery'
+INVALID_RESOURCE = 'invalidResource'
+INVALID_SCHEMA_VALUE = 'invalidSchemaValue'
+INVALID_SCOPE_VALUE = 'invalidScopeValue'
+INVALID_SHARING_REQUEST = 'invalidSharingRequest'
+LABEL_MULTIPLE_VALUES_FOR_SINGULAR_FIELD = 'labelMultipleValuesForSingularField'
+LABEL_MUTATION_FORBIDDEN = 'labelMutationForbidden'
+LABEL_MUTATION_ILLEGAL_SELECTION = 'labelMutationIllegalSelection'
+LABEL_MUTATION_UNKNOWN_FIELD = 'labelMutationUnknownField'
+LIMIT_EXCEEDED = 'limitExceeded'
+LOGIN_REQUIRED = 'loginRequired'
+MALFORMED_WORKING_LOCATION_EVENT = 'malformedWorkingLocationEvent'
+MEMBER_NOT_FOUND = 'memberNotFound'
+MYDRIVE_HIERARCHY_DEPTH_LIMIT_EXCEEDED = 'myDriveHierarchyDepthLimitExceeded'
+NO_LIST_TEAMDRIVES_ADMINISTRATOR_PRIVILEGE = 'noListTeamDrivesAdministratorPrivilege'
+NO_MANAGE_TEAMDRIVE_ADMINISTRATOR_PRIVILEGE = 'noManageTeamDriveAdministratorPrivilege'
+NOT_A_CALENDAR_USER = 'notACalendarUser'
+NOT_FOUND = 'notFound'
+NOT_IMPLEMENTED = 'notImplemented'
+NUM_CHILDREN_IN_NON_ROOT_LIMIT_EXCEEDED = 'numChildrenInNonRootLimitExceeded'
+OPERATION_NOT_SUPPORTED = 'operationNotSupported'
+ORGANIZER_ON_NON_TEAMDRIVE_NOT_SUPPORTED = 'organizerOnNonTeamDriveNotSupported'
+ORGANIZER_ON_NON_TEAMDRIVE_ITEM_NOT_SUPPORTED = 'organizerOnNonTeamDriveItemNotSupported'
+ORGUNIT_NOT_FOUND = 'orgunitNotFound'
+OWNER_ON_TEAMDRIVE_ITEM_NOT_SUPPORTED = 'ownerOnTeamDriveItemNotSupported'
+OWNERSHIP_CHANGE_ACROSS_DOMAIN_NOT_PERMITTED = 'ownershipChangeAcrossDomainNotPermitted'
+PARTICIPANT_IS_NEITHER_ORGANIZER_NOR_ATTENDEE = 'participantIsNeitherOrganizerNorAttendee'
+PERMISSION_DENIED = 'permissionDenied'
+PERMISSION_NOT_FOUND = 'permissionNotFound'
+PHOTO_NOT_FOUND = 'photoNotFound'
+PUBLISH_OUT_NOT_PERMITTED = 'publishOutNotPermitted'
+QUERY_REQUIRES_ADMIN_CREDENTIALS = 'queryRequiresAdminCredentials'
+QUOTA_EXCEEDED = 'quotaExceeded'
+RATE_LIMIT_EXCEEDED = 'rateLimitExceeded'
+REQUIRED = 'required'
+REQUIRED_ACCESS_LEVEL = 'requiredAccessLevel'
+RESOURCE_EXHAUSTED = 'resourceExhausted'
+RESOURCE_ID_NOT_FOUND = 'resourceIdNotFound'
+RESOURCE_NOT_FOUND = 'resourceNotFound'
+RESPONSE_PREPARATION_FAILURE = 'responsePreparationFailure'
+REVISION_DELETION_NOT_SUPPORTED = 'revisionDeletionNotSupported'
+REVISION_NOT_FOUND = 'revisionNotFound'
+REVISIONS_NOT_SUPPORTED = 'revisionsNotSupported'
+SERVICE_LIMIT = 'serviceLimit'
+SERVICE_NOT_AVAILABLE = 'serviceNotAvailable'
+SHARE_IN_NOT_PERMITTED = 'shareInNotPermitted'
+SHARE_OUT_NOT_PERMITTED = 'shareOutNotPermitted'
+SHARE_OUT_NOT_PERMITTED_TO_USER = 'shareOutNotPermittedToUser'
+SHARING_RATE_LIMIT_EXCEEDED = 'sharingRateLimitExceeded'
+SHORTCUT_TARGET_INVALID = 'shortcutTargetInvalid'
+STORAGE_QUOTA_EXCEEDED = 'storageQuotaExceeded'
+SYSTEM_ERROR = 'systemError'
+TARGET_USER_ROLE_LIMITED_BY_LICENSE_RESTRICTION = 'targetUserRoleLimitedByLicenseRestriction'
+TEAMDRIVE_ALREADY_EXISTS = 'teamDriveAlreadyExists'
+TEAMDRIVE_DOMAIN_USERS_ONLY_RESTRICTION = 'teamDriveDomainUsersOnlyRestriction'
+TEAMDRIVE_TEAM_MEMBERS_ONLY_RESTRICTION = 'teamDriveTeamMembersOnlyRestriction'
+TEAMDRIVE_FILE_LIMIT_EXCEEDED = 'teamDriveFileLimitExceeded'
+TEAMDRIVE_HIERARCHY_TOO_DEEP = 'teamDriveHierarchyTooDeep'
+TEAMDRIVE_MEMBERSHIP_REQUIRED = 'teamDriveMembershipRequired'
+TEAMDRIVES_FOLDER_MOVE_IN_NOT_SUPPORTED = 'teamDrivesFolderMoveInNotSupported'
+TEAMDRIVES_FOLDER_SHARING_NOT_SUPPORTED = 'teamDrivesFolderSharingNotSupported'
+TEAMDRIVES_PARENT_LIMIT = 'teamDrivesParentLimit'
+TEAMDRIVES_SHARING_RESTRICTION_NOT_ALLOWED = 'teamDrivesSharingRestrictionNotAllowed'
+TEAMDRIVES_SHORTCUT_FILE_NOT_SUPPORTED = 'teamDrivesShortcutFileNotSupported'
+TIME_RANGE_EMPTY = 'timeRangeEmpty'
+TRANSIENT_ERROR = 'transientError'
+UNKNOWN_ERROR = 'unknownError'
+UNSUPPORTED_LANGUAGE_CODE = 'unsupportedLanguageCode'
+UNSUPPORTED_SUPERVISED_ACCOUNT = 'unsupportedSupervisedAccount'
+UPLOAD_TOO_LARGE = 'uploadTooLarge'
+USER_CANNOT_CREATE_TEAMDRIVES = 'userCannotCreateTeamDrives'
+USER_ACCESS = 'userAccess'
+USER_NOT_FOUND = 'userNotFound'
+USER_RATE_LIMIT_EXCEEDED = 'userRateLimitExceeded'
+#
+DEFAULT_RETRY_REASONS = [QUOTA_EXCEEDED, RATE_LIMIT_EXCEEDED, SHARING_RATE_LIMIT_EXCEEDED, USER_RATE_LIMIT_EXCEEDED,
+                         BACKEND_ERROR, BAD_GATEWAY, GATEWAY_TIMEOUT, INTERNAL_ERROR, TRANSIENT_ERROR]
+SERVICE_NOT_AVAILABLE_RETRY_REASONS = [SERVICE_NOT_AVAILABLE]
+ACTIVITY_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST]
+ALERT_THROW_REASONS = [SERVICE_NOT_AVAILABLE, AUTH_ERROR, PERMISSION_DENIED]
+CALENDAR_THROW_REASONS = [SERVICE_NOT_AVAILABLE, AUTH_ERROR, NOT_A_CALENDAR_USER]
+CIGROUP_CREATE_THROW_REASONS = [SERVICE_NOT_AVAILABLE, ALREADY_EXISTS, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, INVALID, INVALID_ARGUMENT, PERMISSION_DENIED, FAILED_PRECONDITION]
+CIGROUP_GET_THROW_REASONS = [SERVICE_NOT_AVAILABLE, NOT_FOUND, GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR, PERMISSION_DENIED]
+CIGROUP_LIST_THROW_REASONS = [SERVICE_NOT_AVAILABLE, RESOURCE_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, INVALID_ARGUMENT, SYSTEM_ERROR, PERMISSION_DENIED]
+CIGROUP_LIST_USERKEY_THROW_REASONS = CIGROUP_LIST_THROW_REASONS+[INVALID_ARGUMENT]
+CIGROUP_UPDATE_THROW_REASONS = [SERVICE_NOT_AVAILABLE, NOT_FOUND, GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS,
+                                FORBIDDEN, BAD_REQUEST, INVALID, INVALID_INPUT, INVALID_ARGUMENT,
+                                SYSTEM_ERROR, PERMISSION_DENIED, FAILED_PRECONDITION]
+CIGROUP_RETRY_REASONS = [INVALID, SYSTEM_ERROR, SERVICE_NOT_AVAILABLE]
+CIMEMBERS_THROW_REASONS = [SERVICE_NOT_AVAILABLE, MEMBER_NOT_FOUND, INVALID_MEMBER]
+CISSO_CREATE_THROW_REASONS = [SERVICE_NOT_AVAILABLE, FAILED_PRECONDITION, NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, INVALID, INVALID_ARGUMENT, PERMISSION_DENIED, INTERNAL_ERROR]
+CISSO_GET_THROW_REASONS = [SERVICE_NOT_AVAILABLE, NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR, PERMISSION_DENIED, INTERNAL_ERROR]
+CISSO_LIST_THROW_REASONS = [SERVICE_NOT_AVAILABLE, NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR, PERMISSION_DENIED, INTERNAL_ERROR]
+CISSO_UPDATE_THROW_REASONS = [SERVICE_NOT_AVAILABLE, NOT_FOUND, FAILED_PRECONDITION, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS,
+                              FORBIDDEN, BAD_REQUEST, INVALID, INVALID_INPUT, INVALID_ARGUMENT,
+                              SYSTEM_ERROR, PERMISSION_DENIED, INTERNAL_ERROR]
+CONTACT_DELEGATE_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST, FAILED_PRECONDITION, PERMISSION_DENIED, FORBIDDEN, INVALID_ARGUMENT]
+COURSE_ACCESS_THROW_REASONS = [NOT_FOUND, INSUFFICIENT_PERMISSIONS, PERMISSION_DENIED, FORBIDDEN, INVALID_ARGUMENT]
+DRIVE_USER_THROW_REASONS = [SERVICE_NOT_AVAILABLE, AUTH_ERROR, DOMAIN_POLICY]
+DRIVE_ACCESS_THROW_REASONS = DRIVE_USER_THROW_REASONS+[FILE_NOT_FOUND, FORBIDDEN, INTERNAL_ERROR, INSUFFICIENT_FILE_PERMISSIONS, UNKNOWN_ERROR, INVALID]
+DRIVE_COPY_THROW_REASONS = DRIVE_ACCESS_THROW_REASONS+[CANNOT_COPY_FILE, BAD_REQUEST, RESPONSE_PREPARATION_FAILURE, TEAMDRIVES_SHARING_RESTRICTION_NOT_ALLOWED,
+                                                       FIELD_NOT_WRITABLE, RATE_LIMIT_EXCEEDED, USER_RATE_LIMIT_EXCEEDED,
+                                                       STORAGE_QUOTA_EXCEEDED, TEAMDRIVE_FILE_LIMIT_EXCEEDED, TEAMDRIVE_HIERARCHY_TOO_DEEP]
+DRIVE_GET_THROW_REASONS = DRIVE_USER_THROW_REASONS+[FILE_NOT_FOUND, DOWNLOAD_QUOTA_EXCEEDED]
+DRIVE3_CREATE_ACL_THROW_REASONS = [BAD_REQUEST, INVALID, INVALID_SHARING_REQUEST, OWNERSHIP_CHANGE_ACROSS_DOMAIN_NOT_PERMITTED,
+                                   CANNOT_SET_EXPIRATION, EXPIRATION_DATE_NOT_ALLOWED_FOR_SHARED_DRIVE_MEMBERS,
+                                   NOT_FOUND, TEAMDRIVE_DOMAIN_USERS_ONLY_RESTRICTION, TEAMDRIVE_TEAM_MEMBERS_ONLY_RESTRICTION,
+                                   TARGET_USER_ROLE_LIMITED_BY_LICENSE_RESTRICTION, INSUFFICIENT_ADMINISTRATOR_PRIVILEGES, SHARING_RATE_LIMIT_EXCEEDED,
+                                   PUBLISH_OUT_NOT_PERMITTED, SHARE_IN_NOT_PERMITTED, SHARE_OUT_NOT_PERMITTED, SHARE_OUT_NOT_PERMITTED_TO_USER,
+                                   CANNOT_SHARE_TEAMDRIVE_TOPFOLDER_WITH_ANYONEORDOMAINS,
+                                   CANNOT_SHARE_TEAMDRIVE_WITH_NONGOOGLE_ACCOUNTS,
+                                   OWNER_ON_TEAMDRIVE_ITEM_NOT_SUPPORTED,
+                                   ORGANIZER_ON_NON_TEAMDRIVE_NOT_SUPPORTED,
+                                   ORGANIZER_ON_NON_TEAMDRIVE_ITEM_NOT_SUPPORTED,
+                                   FILE_ORGANIZER_NOT_YET_ENABLED_FOR_THIS_TEAMDRIVE,
+                                   FILE_ORGANIZER_ON_FOLDERS_IN_SHARED_DRIVE_ONLY,
+                                   FILE_ORGANIZER_ON_NON_TEAMDRIVE_NOT_SUPPORTED,
+                                   TEAMDRIVES_FOLDER_SHARING_NOT_SUPPORTED, INVALID_LINK_VISIBILITY, ABUSIVE_CONTENT_RESTRICTION]
+DRIVE3_GET_ACL_REASONS = DRIVE_USER_THROW_REASONS+[FILE_NOT_FOUND, FORBIDDEN, INTERNAL_ERROR,
+                                                   INSUFFICIENT_ADMINISTRATOR_PRIVILEGES, INSUFFICIENT_FILE_PERMISSIONS,
+                                                   UNKNOWN_ERROR, INVALID]
+DRIVE3_UPDATE_ACL_THROW_REASONS = [BAD_REQUEST, INVALID_OWNERSHIP_TRANSFER, CANNOT_REMOVE_OWNER,
+                                   CANNOT_SET_EXPIRATION, EXPIRATION_DATE_NOT_ALLOWED_FOR_SHARED_DRIVE_MEMBERS,
+                                   OWNERSHIP_CHANGE_ACROSS_DOMAIN_NOT_PERMITTED,
+                                   NOT_FOUND, TEAMDRIVE_DOMAIN_USERS_ONLY_RESTRICTION, TEAMDRIVE_TEAM_MEMBERS_ONLY_RESTRICTION,
+                                   TARGET_USER_ROLE_LIMITED_BY_LICENSE_RESTRICTION, INSUFFICIENT_ADMINISTRATOR_PRIVILEGES, SHARING_RATE_LIMIT_EXCEEDED,
+                                   PUBLISH_OUT_NOT_PERMITTED, SHARE_IN_NOT_PERMITTED, SHARE_OUT_NOT_PERMITTED, SHARE_OUT_NOT_PERMITTED_TO_USER,
+                                   CANNOT_SHARE_TEAMDRIVE_TOPFOLDER_WITH_ANYONEORDOMAINS,
+                                   CANNOT_SHARE_TEAMDRIVE_WITH_NONGOOGLE_ACCOUNTS,
+                                   OWNER_ON_TEAMDRIVE_ITEM_NOT_SUPPORTED,
+                                   ORGANIZER_ON_NON_TEAMDRIVE_NOT_SUPPORTED,
+                                   ORGANIZER_ON_NON_TEAMDRIVE_ITEM_NOT_SUPPORTED,
+                                   FILE_ORGANIZER_NOT_YET_ENABLED_FOR_THIS_TEAMDRIVE,
+                                   FILE_ORGANIZER_ON_FOLDERS_IN_SHARED_DRIVE_ONLY,
+                                   FILE_ORGANIZER_ON_NON_TEAMDRIVE_NOT_SUPPORTED,
+                                   CANNOT_UPDATE_PERMISSION,
+                                   CANNOT_MODIFY_INHERITED_TEAMDRIVE_PERMISSION,
+                                   FIELD_NOT_WRITABLE, PERMISSION_NOT_FOUND]
+DRIVE3_DELETE_ACL_THROW_REASONS = [BAD_REQUEST, CANNOT_REMOVE_OWNER,
+                                   CANNOT_MODIFY_INHERITED_TEAMDRIVE_PERMISSION,
+                                   INSUFFICIENT_ADMINISTRATOR_PRIVILEGES, SHARING_RATE_LIMIT_EXCEEDED,
+                                   NOT_FOUND, PERMISSION_NOT_FOUND]
+DRIVE3_MODIFY_LABEL_THROW_REASONS = DRIVE_USER_THROW_REASONS+[FILE_NOT_FOUND, NOT_FOUND, FORBIDDEN, INTERNAL_ERROR,
+                                                              FILE_NEVER_WRITABLE, APPLY_LABEL_FORBIDDEN,
+                                                              INSUFFICIENT_ADMINISTRATOR_PRIVILEGES, INSUFFICIENT_FILE_PERMISSIONS,
+                                                              UNKNOWN_ERROR, INVALID_INPUT, BAD_REQUEST,
+                                                              LABEL_MULTIPLE_VALUES_FOR_SINGULAR_FIELD, LABEL_MUTATION_FORBIDDEN,
+                                                              LABEL_MUTATION_ILLEGAL_SELECTION, LABEL_MUTATION_UNKNOWN_FIELD]
+DOCS_ACCESS_THROW_REASONS = DRIVE_USER_THROW_REASONS+[NOT_FOUND, PERMISSION_DENIED, FORBIDDEN, INTERNAL_ERROR, INSUFFICIENT_FILE_PERMISSIONS,
+                                                      BAD_REQUEST, INVALID, INVALID_ARGUMENT, FAILED_PRECONDITION]
+GMAIL_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST]
+GMAIL_LIST_THROW_REASONS = [FAILED_PRECONDITION, PERMISSION_DENIED, INVALID, INVALID_ARGUMENT]
+GMAIL_SMIME_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST, INVALID_ARGUMENT, FORBIDDEN, NOT_FOUND, PERMISSION_DENIED]
+GROUP_GET_THROW_REASONS = [GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR]
+GROUP_GET_RETRY_REASONS = [INVALID, SYSTEM_ERROR, SERVICE_NOT_AVAILABLE]
+GROUP_CREATE_THROW_REASONS = [DUPLICATE, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, INVALID, INVALID_INPUT]
+GROUP_UPDATE_THROW_REASONS = [GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, INVALID, INVALID_INPUT]
+GROUP_SETTINGS_THROW_REASONS = [NOT_FOUND, GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, SYSTEM_ERROR, PERMISSION_DENIED,
+                                INVALID, INVALID_ARGUMENT, INVALID_PARAMETER, INVALID_ATTRIBUTE_VALUE, INVALID_INPUT,
+                                SERVICE_LIMIT, SERVICE_NOT_AVAILABLE, AUTH_ERROR, REQUIRED]
+GROUP_SETTINGS_RETRY_REASONS = [INVALID, SERVICE_LIMIT, SERVICE_NOT_AVAILABLE]
+GROUP_LIST_THROW_REASONS = [RESOURCE_NOT_FOUND, DOMAIN_NOT_FOUND, FORBIDDEN, BAD_REQUEST]
+GROUP_LIST_USERKEY_THROW_REASONS = GROUP_LIST_THROW_REASONS+[INVALID_MEMBER, INVALID_INPUT]
+KEEP_THROW_REASONS = [AUTH_ERROR, BAD_REQUEST, PERMISSION_DENIED, INVALID_ARGUMENT, NOT_FOUND]
+LOOKERSTUDIO_THROW_REASONS = [INVALID_ARGUMENT, SERVICE_NOT_AVAILABLE, BAD_REQUEST, NOT_FOUND, PERMISSION_DENIED, INTERNAL_ERROR]
+MEMBERS_THROW_REASONS = [GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, INVALID, FORBIDDEN, SERVICE_NOT_AVAILABLE]
+MEMBERS_RETRY_REASONS = [SYSTEM_ERROR, SERVICE_NOT_AVAILABLE]
+ORGUNIT_GET_THROW_REASONS = [INVALID_ORGUNIT, ORGUNIT_NOT_FOUND, BACKEND_ERROR, BAD_REQUEST, INVALID_CUSTOMER_ID, LOGIN_REQUIRED]
+PEOPLE_ACCESS_THROW_REASONS = [SERVICE_NOT_AVAILABLE, FORBIDDEN, PERMISSION_DENIED]
+RESELLER_THROW_REASONS = [BAD_REQUEST, RESOURCE_NOT_FOUND, FORBIDDEN, INVALID]
+SHEETS_ACCESS_THROW_REASONS = DRIVE_USER_THROW_REASONS+[NOT_FOUND, PERMISSION_DENIED, FORBIDDEN, INTERNAL_ERROR, INSUFFICIENT_FILE_PERMISSIONS,
+                                                        BAD_REQUEST, INVALID, INVALID_ARGUMENT, FAILED_PRECONDITION]
+TASK_THROW_REASONS = [BAD_REQUEST, PERMISSION_DENIED, INVALID, NOT_FOUND, ACCESS_NOT_CONFIGURED]
+TASKLIST_THROW_REASONS = [BAD_REQUEST, PERMISSION_DENIED, INVALID, NOT_FOUND, ACCESS_NOT_CONFIGURED]
+USER_GET_THROW_REASONS = [USER_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, SYSTEM_ERROR]
+YOUTUBE_THROW_REASONS = [SERVICE_NOT_AVAILABLE, AUTH_ERROR, UNSUPPORTED_SUPERVISED_ACCOUNT, UNSUPPORTED_LANGUAGE_CODE, CONTENT_OWNER_ACCOUNT_NOT_FOUND]
+
+REASON_MESSAGE_MAP = {
+  ABORTED: [
+    ('Label name exists or conflicts', DUPLICATE),
+    ('The operation was aborted', ABORTED),
+    ],
+  CONDITION_NOT_MET: [
+    ('Cyclic memberships not allowed', CYCLIC_MEMBERSHIPS_NOT_ALLOWED),
+    ('undelete', DELETED_USER_NOT_FOUND),
+    ],
+  FAILED_PRECONDITION: [
+    ('Bad Request', BAD_REQUEST),
+    ('Mail service not enabled', SERVICE_NOT_AVAILABLE),
+    ],
+  INVALID: [
+    ('userId', USER_NOT_FOUND),
+    ('memberKey', INVALID_MEMBER),
+    ('A system error has occurred', SYSTEM_ERROR),
+    ('Invalid attribute value', INVALID_ATTRIBUTE_VALUE),
+    ('Invalid Customer Id', INVALID_CUSTOMER_ID),
+    ('Invalid Input: INVALID_OU_ID', INVALID_ORGUNIT),
+    ('Invalid Input: custom_schema', INVALID_SCHEMA_VALUE),
+    ('Invalid Input: groupKey', INVALID_INPUT),
+    ('Invalid Input: resource', INVALID_RESOURCE),
+    ('Invalid Input:', INVALID_INPUT),
+    ('Invalid Input', INVALID_INPUT),
+    ('Invalid Org Unit', INVALID_ORGUNIT),
+    ('Invalid Ou Id', INVALID_ORGUNIT),
+    ('Invalid Ou Name', INVALID_ORGUNIT_NAME),
+    ('Invalid Parent Orgunit Id', INVALID_PARENT_ORGUNIT),
+    ('Invalid query', INVALID_QUERY),
+    ('Invalid scope value', INVALID_SCOPE_VALUE),
+    ('Invalid value', INVALID_INPUT),
+    ('New domain name is not a verified secondary domain', DOMAIN_NOT_VERIFIED_SECONDARY),
+    ('PermissionDenied', PERMISSION_DENIED),
+    ],
+  INVALID_ARGUMENT: [
+    ('Cannot delete primary send-as', CANNOT_DELETE_PRIMARY_SENDAS),
+    ('Invalid id value', INVALID_MESSAGE_ID),
+    ('Invalid ids value', INVALID_MESSAGE_ID),
+    ],
+  NOT_FOUND: [
+    ('userKey', USER_NOT_FOUND),
+    ('groupKey', GROUP_NOT_FOUND),
+    ('memberKey', MEMBER_NOT_FOUND),
+    ('photo', PHOTO_NOT_FOUND),
+    ('resource_id', RESOURCE_ID_NOT_FOUND),
+    ('resourceId', RESOURCE_ID_NOT_FOUND),
+    ('Customer doesn\'t exist', CUSTOMER_NOT_FOUND),
+    ('Domain alias does not exist', DOMAIN_ALIAS_NOT_FOUND),
+    ('Domain not found', DOMAIN_NOT_FOUND),
+    ('domain', DOMAIN_NOT_FOUND),
+    ('File not found', FILE_NOT_FOUND),
+    ('Org unit not found', ORGUNIT_NOT_FOUND),
+    ('Permission not found', PERMISSION_NOT_FOUND),
+    ('Resource Not Found', RESOURCE_NOT_FOUND),
+    ('Revision not found', REVISION_NOT_FOUND),
+    ('Shared Drive not found', NOT_FOUND),
+    ('Not Found', NOT_FOUND),
+    ],
+  REQUIRED: [
+    ('Login Required', LOGIN_REQUIRED),
+    ('memberKey', MEMBER_NOT_FOUND),
+    ],
+  RESOURCE_NOT_FOUND: [
+    ('resourceId', RESOURCE_ID_NOT_FOUND),
+    ],
+  }
+
+class aborted(Exception):
+  pass
+class abusiveContentRestriction(Exception):
+  pass
+class accessNotConfigured(Exception):
+  pass
+class alreadyExists(Exception):
+  pass
+class applyLabelForbidden(Exception):
+  pass
+class authError(Exception):
+  pass
+class backendError(Exception):
+  pass
+class badRequest(Exception):
+  pass
+class cannotAddParent(Exception):
+  pass
+class cannotChangeOrganizer(Exception):
+  pass
+class cannotChangeOrganizerOfInstance(Exception):
+  pass
+class cannotChangeOwnAcl(Exception):
+  pass
+class cannotChangeOwnerAcl(Exception):
+  pass
+class cannotChangeOwnPrimarySubscription(Exception):
+  pass
+class cannotCopyFile(Exception):
+  pass
+class cannotDeleteOnlyRevision(Exception):
+  pass
+class cannotDeletePrimaryCalendar(Exception):
+  pass
+class cannotDeletePrimarySendAs(Exception):
+  pass
+class cannotDeleteResourceWithChildren(Exception):
+  pass
+class cannotModifyInheritedTeamDrivePermission(Exception):
+  pass
+class cannotModifyRestrictedLabel(Exception):
+  pass
+class cannotModifyViewersCanCopyContent(Exception):
+  pass
+class cannotMoveTrashedItemIntoTeamDrive(Exception):
+  pass
+class cannotMoveTrashedItemOutOfTeamDrive(Exception):
+  pass
+class cannotRemoveOwner(Exception):
+  pass
+class cannotSetExpiration(Exception):
+  pass
+class cannotShareGroupsWithLink(Exception):
+  pass
+class cannotShareUsersWithLink(Exception):
+  pass
+class cannotShareTeamDriveTopFolderWithAnyoneOrDomains(Exception):
+  pass
+class cannotShareTeamDriveWithNonGoogleAccounts(Exception):
+  pass
+class cannotUpdatePermission(Exception):
+  pass
+class conditionNotMet(Exception):
+  pass
+class conflict(Exception):
+  pass
+class contentOwnerAccountNotFound(Exception):
+  pass
+class crossDomainMoveRestriction(Exception):
+  pass
+class customerExceededRoleAssignmentsLimit(Exception):
+  pass
+class customerNotFound(Exception):
+  pass
+class cyclicMembershipsNotAllowed(Exception):
+  pass
+class deleted(Exception):
+  pass
+class deletedUserNotFound(Exception):
+  pass
+class domainAliasNotFound(Exception):
+  pass
+class domainCannotUseApis(Exception):
+  pass
+class domainNotFound(Exception):
+  pass
+class domainNotVerifiedSecondary(Exception):
+  pass
+class domainPolicy(Exception):
+  pass
+class downloadQuotaExceeded(Exception):
+  pass
+class duplicate(Exception):
+  pass
+class eventDurationExceedsLimit(Exception):
+  pass
+class expirationDateNotAllowedForSharedDriveMembers(Exception):
+  pass
+class failedPrecondition(Exception):
+  pass
+class fieldInUse(Exception):
+  pass
+class fieldNotWritable(Exception):
+  pass
+class fileNeverWritable(Exception):
+  pass
+class fileNotFound(Exception):
+  pass
+class fileOrganizerNotYetEnabledForThisTeamDrive(Exception):
+  pass
+class fileOrganizerOnFoldersInSharedDriveOnly(Exception):
+  pass
+class fileOrganizerOnNonTeamDriveNotSupported(Exception):
+  pass
+class fileOwnerNotMemberOfTeamDrive(Exception):
+  pass
+class fileOwnerNotMemberOfWriterDomain(Exception):
+  pass
+class fileWriterTeamDriveMoveInDisabled(Exception):
+  pass
+class forbidden(Exception):
+  pass
+class groupNotFound(Exception):
+  pass
+class illegalAccessRoleForDefault(Exception):
+  pass
+class insufficientAdministratorPrivileges(Exception):
+  pass
+class insufficientArchivedUserLicenses(Exception):
+  pass
+class insufficientFilePermissions(Exception):
+  pass
+class insufficientParentPermissions(Exception):
+  pass
+class insufficientPermissions(Exception):
+  pass
+class internalError(Exception):
+  pass
+class invalid(Exception):
+  pass
+class invalidArgument(Exception):
+  pass
+class invalidAttributeValue(Exception):
+  pass
+class invalidCustomerId(Exception):
+  pass
+class invalidInput(Exception):
+  pass
+class invalidLinkVisibility(Exception):
+  pass
+class invalidMember(Exception):
+  pass
+class invalidMessageId(Exception):
+  pass
+class invalidOrgunit(Exception):
+  pass
+class invalidOrgunitName(Exception):
+  pass
+class invalidOwnershipTransfer(Exception):
+  pass
+class invalidParameter(Exception):
+  pass
+class invalidParentOrgunit(Exception):
+  pass
+class invalidQuery(Exception):
+  pass
+class invalidResource(Exception):
+  pass
+class invalidSchemaValue(Exception):
+  pass
+class invalidScopeValue(Exception):
+  pass
+class invalidSharingRequest(Exception):
+  pass
+class labelMultipleValuesForSingularField(Exception):
+  pass
+class labelMutationForbidden(Exception):
+  pass
+class labelMutationIllegalSelection(Exception):
+  pass
+class labelMutationUnknownField(Exception):
+  pass
+class limitExceeded(Exception):
+  pass
+class loginRequired(Exception):
+  pass
+class malformedWorkingLocationEvent(Exception):
+  pass
+class memberNotFound(Exception):
+  pass
+class noListTeamDrivesAdministratorPrivilege(Exception):
+  pass
+class noManageTeamDriveAdministratorPrivilege(Exception):
+  pass
+class notACalendarUser(Exception):
+  pass
+class notFound(Exception):
+  pass
+class notImplemented(Exception):
+  pass
+class operationNotSupported(Exception):
+  pass
+class organizerOnNonTeamDriveNotSupported(Exception):
+  pass
+class organizerOnNonTeamDriveItemNotSupported(Exception):
+  pass
+class orgunitNotFound(Exception):
+  pass
+class ownerOnTeamDriveItemNotSupported(Exception):
+  pass
+class ownershipChangeAcrossDomainNotPermitted(Exception):
+  pass
+class participantIsNeitherOrganizerNorAttendee(Exception):
+  pass
+class permissionDenied(Exception):
+  pass
+class permissionNotFound(Exception):
+  pass
+class photoNotFound(Exception):
+  pass
+class publishOutNotPermitted(Exception):
+  pass
+class queryRequiresAdminCredentials(Exception):
+  pass
+class quotaExceeded(Exception):
+  pass
+class rateLimitExceeded(Exception):
+  pass
+class required(Exception):
+  pass
+class requiredAccessLevel(Exception):
+  pass
+class resourceExhausted(Exception):
+  pass
+class resourceIdNotFound(Exception):
+  pass
+class resourceNotFound(Exception):
+  pass
+class responsePreparationFailure(Exception):
+  pass
+class revisionDeletionNotSupported(Exception):
+  pass
+class revisionNotFound(Exception):
+  pass
+class revisionsNotSupported(Exception):
+  pass
+class serviceLimit(Exception):
+  pass
+class serviceNotAvailable(Exception):
+  pass
+class shareInNotPermitted(Exception):
+  pass
+class shareOutNotPermitted(Exception):
+  pass
+class shareOutNotPermittedToUser(Exception):
+  pass
+class sharingRateLimitExceeded(Exception):
+  pass
+class shortcutTargetInvalid(Exception):
+  pass
+class storageQuotaExceeded(Exception):
+  pass
+class systemError(Exception):
+  pass
+class targetUserRoleLimitedByLicenseRestriction(Exception):
+  pass
+class teamDriveAlreadyExists(Exception):
+  pass
+class teamDriveDomainUsersOnlyRestriction(Exception):
+  pass
+class teamDriveTeamMembersOnlyRestriction(Exception):
+  pass
+class teamDriveFileLimitExceeded(Exception):
+  pass
+class teamDriveHierarchyTooDeep(Exception):
+  pass
+class teamDriveMembershipRequired(Exception):
+  pass
+class teamDrivesFolderMoveInNotSupported(Exception):
+  pass
+class teamDrivesFolderSharingNotSupported(Exception):
+  pass
+class teamDrivesParentLimit(Exception):
+  pass
+class teamDrivesSharingRestrictionNotAllowed(Exception):
+  pass
+class teamDrivesShortcutFileNotSupported(Exception):
+  pass
+class timeRangeEmpty(Exception):
+  pass
+class transientError(Exception):
+  pass
+class unknownError(Exception):
+  pass
+class unsupportedLanguageCode(Exception):
+  pass
+class unsupportedSupervisedAccount(Exception):
+  pass
+class uploadTooLarge(Exception):
+  pass
+class userCannotCreateTeamDrives(Exception):
+  pass
+class userAccess(Exception):
+  pass
+class userNotFound(Exception):
+  pass
+class userRateLimitExceeded(Exception):
+  pass
+
+REASON_EXCEPTION_MAP = {
+  ABORTED: aborted,
+  ABUSIVE_CONTENT_RESTRICTION: abusiveContentRestriction,
+  ACCESS_NOT_CONFIGURED: accessNotConfigured,
+  ALREADY_EXISTS: alreadyExists,
+  APPLY_LABEL_FORBIDDEN: applyLabelForbidden,
+  AUTH_ERROR: authError,
+  BACKEND_ERROR: backendError,
+  BAD_REQUEST: badRequest,
+  CANNOT_ADD_PARENT: cannotAddParent,
+  CANNOT_CHANGE_ORGANIZER: cannotChangeOrganizer,
+  CANNOT_CHANGE_ORGANIZER_OF_INSTANCE: cannotChangeOrganizerOfInstance,
+  CANNOT_CHANGE_OWN_ACL: cannotChangeOwnAcl,
+  CANNOT_CHANGE_OWNER_ACL: cannotChangeOwnerAcl,
+  CANNOT_CHANGE_OWN_PRIMARY_SUBSCRIPTION: cannotChangeOwnPrimarySubscription,
+  CANNOT_COPY_FILE: cannotCopyFile,
+  CANNOT_DELETE_ONLY_REVISION: cannotDeleteOnlyRevision,
+  CANNOT_DELETE_PRIMARY_CALENDAR: cannotDeletePrimaryCalendar,
+  CANNOT_DELETE_PRIMARY_SENDAS: cannotDeletePrimarySendAs,
+  CANNOT_DELETE_RESOURCE_WITH_CHILDREN: cannotDeleteResourceWithChildren,
+  CANNOT_MODIFY_INHERITED_TEAMDRIVE_PERMISSION: cannotModifyInheritedTeamDrivePermission,
+  CANNOT_MODIFY_RESTRICTED_LABEL: cannotModifyRestrictedLabel,
+  CANNOT_MODIFY_VIEWERS_CAN_COPY_CONTENT: cannotModifyViewersCanCopyContent,
+  CANNOT_MOVE_TRASHED_ITEM_INTO_TEAMDRIVE: cannotMoveTrashedItemIntoTeamDrive,
+  CANNOT_MOVE_TRASHED_ITEM_OUT_OF_TEAMDRIVE: cannotMoveTrashedItemOutOfTeamDrive,
+  CANNOT_REMOVE_OWNER: cannotRemoveOwner,
+  CANNOT_SET_EXPIRATION: cannotSetExpiration,
+  CANNOT_SHARE_GROUPS_WITHLINK: cannotShareGroupsWithLink,
+  CANNOT_SHARE_USERS_WITHLINK: cannotShareUsersWithLink,
+  CANNOT_SHARE_TEAMDRIVE_TOPFOLDER_WITH_ANYONEORDOMAINS: cannotShareTeamDriveTopFolderWithAnyoneOrDomains,
+  CANNOT_SHARE_TEAMDRIVE_WITH_NONGOOGLE_ACCOUNTS: cannotShareTeamDriveWithNonGoogleAccounts,
+  CANNOT_UPDATE_PERMISSION: cannotUpdatePermission,
+  CONDITION_NOT_MET: conditionNotMet,
+  CONFLICT: conflict,
+  CONTENT_OWNER_ACCOUNT_NOT_FOUND: contentOwnerAccountNotFound,
+  CROSS_DOMAIN_MOVE_RESTRICTION: crossDomainMoveRestriction,
+  CUSTOMER_EXCEEDED_ROLE_ASSIGNMENTS_LIMIT: customerExceededRoleAssignmentsLimit,
+  CUSTOMER_NOT_FOUND: customerNotFound,
+  CYCLIC_MEMBERSHIPS_NOT_ALLOWED: cyclicMembershipsNotAllowed,
+  DELETED: deleted,
+  DELETED_USER_NOT_FOUND: deletedUserNotFound,
+  DOMAIN_ALIAS_NOT_FOUND: domainAliasNotFound,
+  DOMAIN_CANNOT_USE_APIS: domainCannotUseApis,
+  DOMAIN_NOT_FOUND: domainNotFound,
+  DOMAIN_NOT_VERIFIED_SECONDARY: domainNotVerifiedSecondary,
+  DOMAIN_POLICY: domainPolicy,
+  DOWNLOAD_QUOTA_EXCEEDED: downloadQuotaExceeded,
+  DUPLICATE: duplicate,
+  EVENT_DURATION_EXCEEDS_LIMIT: eventDurationExceedsLimit,
+  EXPIRATION_DATE_NOT_ALLOWED_FOR_SHARED_DRIVE_MEMBERS: expirationDateNotAllowedForSharedDriveMembers,
+  FAILED_PRECONDITION: failedPrecondition,
+  FIELD_IN_USE: fieldInUse,
+  FIELD_NOT_WRITABLE: fieldNotWritable,
+  FILE_NEVER_WRITABLE: fileNeverWritable,
+  FILE_NOT_FOUND: fileNotFound,
+  FILE_ORGANIZER_NOT_YET_ENABLED_FOR_THIS_TEAMDRIVE: fileOrganizerNotYetEnabledForThisTeamDrive,
+  FILE_ORGANIZER_ON_FOLDERS_IN_SHARED_DRIVE_ONLY: fileOrganizerOnFoldersInSharedDriveOnly,
+  FILE_ORGANIZER_ON_NON_TEAMDRIVE_NOT_SUPPORTED: fileOrganizerOnNonTeamDriveNotSupported,
+  FILE_OWNER_NOT_MEMBER_OF_TEAMDRIVE: fileOwnerNotMemberOfTeamDrive,
+  FILE_OWNER_NOT_MEMBER_OF_WRITER_DOMAIN: fileOwnerNotMemberOfWriterDomain,
+  FILE_WRITER_TEAMDRIVE_MOVE_IN_DISABLED: fileWriterTeamDriveMoveInDisabled,
+  FORBIDDEN: forbidden,
+  GROUP_NOT_FOUND: groupNotFound,
+  ILLEGAL_ACCESS_ROLE_FOR_DEFAULT: illegalAccessRoleForDefault,
+  INSUFFICIENT_ADMINISTRATOR_PRIVILEGES: insufficientAdministratorPrivileges,
+  INSUFFICIENT_ARCHIVED_USER_LICENSES: insufficientArchivedUserLicenses,
+  INSUFFICIENT_FILE_PERMISSIONS: insufficientFilePermissions,
+  INSUFFICIENT_PARENT_PERMISSIONS: insufficientParentPermissions,
+  INSUFFICIENT_PERMISSIONS: insufficientPermissions,
+  INTERNAL_ERROR: internalError,
+  INVALID: invalid,
+  INVALID_ARGUMENT: invalidArgument,
+  INVALID_ATTRIBUTE_VALUE: invalidAttributeValue,
+  INVALID_CUSTOMER_ID: invalidCustomerId,
+  INVALID_INPUT: invalidInput,
+  INVALID_LINK_VISIBILITY: invalidLinkVisibility,
+  INVALID_MEMBER: invalidMember,
+  INVALID_MESSAGE_ID: invalidMessageId,
+  INVALID_ORGUNIT: invalidOrgunit,
+  INVALID_ORGUNIT_NAME: invalidOrgunitName,
+  INVALID_OWNERSHIP_TRANSFER: invalidOwnershipTransfer,
+  INVALID_PARAMETER: invalidParameter,
+  INVALID_PARENT_ORGUNIT: invalidParentOrgunit,
+  INVALID_QUERY: invalidQuery,
+  INVALID_RESOURCE: invalidResource,
+  INVALID_SCHEMA_VALUE: invalidSchemaValue,
+  INVALID_SCOPE_VALUE: invalidScopeValue,
+  INVALID_SHARING_REQUEST: invalidSharingRequest,
+  LABEL_MULTIPLE_VALUES_FOR_SINGULAR_FIELD: labelMultipleValuesForSingularField,
+  LABEL_MUTATION_FORBIDDEN: labelMutationForbidden,
+  LABEL_MUTATION_ILLEGAL_SELECTION: labelMutationIllegalSelection,
+  LABEL_MUTATION_UNKNOWN_FIELD: labelMutationUnknownField,
+  LIMIT_EXCEEDED: limitExceeded,
+  LOGIN_REQUIRED: loginRequired,
+  MALFORMED_WORKING_LOCATION_EVENT: malformedWorkingLocationEvent,
+  MEMBER_NOT_FOUND: memberNotFound,
+  NO_LIST_TEAMDRIVES_ADMINISTRATOR_PRIVILEGE: noListTeamDrivesAdministratorPrivilege,
+  NO_MANAGE_TEAMDRIVE_ADMINISTRATOR_PRIVILEGE: noManageTeamDriveAdministratorPrivilege,
+  NOT_A_CALENDAR_USER: notACalendarUser,
+  NOT_FOUND: notFound,
+  NOT_IMPLEMENTED: notImplemented,
+  OPERATION_NOT_SUPPORTED: operationNotSupported,
+  ORGANIZER_ON_NON_TEAMDRIVE_NOT_SUPPORTED: organizerOnNonTeamDriveNotSupported,
+  ORGANIZER_ON_NON_TEAMDRIVE_ITEM_NOT_SUPPORTED: organizerOnNonTeamDriveItemNotSupported,
+  ORGUNIT_NOT_FOUND: orgunitNotFound,
+  OWNER_ON_TEAMDRIVE_ITEM_NOT_SUPPORTED: ownerOnTeamDriveItemNotSupported,
+  OWNERSHIP_CHANGE_ACROSS_DOMAIN_NOT_PERMITTED: ownershipChangeAcrossDomainNotPermitted,
+  PARTICIPANT_IS_NEITHER_ORGANIZER_NOR_ATTENDEE: participantIsNeitherOrganizerNorAttendee,
+  PERMISSION_DENIED: permissionDenied,
+  PERMISSION_NOT_FOUND: permissionNotFound,
+  PHOTO_NOT_FOUND: photoNotFound,
+  PUBLISH_OUT_NOT_PERMITTED: publishOutNotPermitted,
+  QUERY_REQUIRES_ADMIN_CREDENTIALS: queryRequiresAdminCredentials,
+  QUOTA_EXCEEDED: quotaExceeded,
+  RATE_LIMIT_EXCEEDED: rateLimitExceeded,
+  REQUIRED: required,
+  REQUIRED_ACCESS_LEVEL: requiredAccessLevel,
+  RESOURCE_EXHAUSTED: resourceExhausted,
+  RESOURCE_ID_NOT_FOUND: resourceIdNotFound,
+  RESOURCE_NOT_FOUND: resourceNotFound,
+  RESPONSE_PREPARATION_FAILURE: responsePreparationFailure,
+  REVISION_DELETION_NOT_SUPPORTED: revisionDeletionNotSupported,
+  REVISION_NOT_FOUND: revisionNotFound,
+  REVISIONS_NOT_SUPPORTED: revisionsNotSupported,
+  SERVICE_LIMIT: serviceLimit,
+  SERVICE_NOT_AVAILABLE: serviceNotAvailable,
+  SHARE_IN_NOT_PERMITTED: shareInNotPermitted,
+  SHARE_OUT_NOT_PERMITTED: shareOutNotPermitted,
+  SHARE_OUT_NOT_PERMITTED_TO_USER: shareOutNotPermittedToUser,
+  SHARING_RATE_LIMIT_EXCEEDED: sharingRateLimitExceeded,
+  SHORTCUT_TARGET_INVALID: shortcutTargetInvalid,
+  STORAGE_QUOTA_EXCEEDED: storageQuotaExceeded,
+  SYSTEM_ERROR: systemError,
+  TARGET_USER_ROLE_LIMITED_BY_LICENSE_RESTRICTION: targetUserRoleLimitedByLicenseRestriction,
+  TEAMDRIVE_ALREADY_EXISTS: teamDriveAlreadyExists,
+  TEAMDRIVE_DOMAIN_USERS_ONLY_RESTRICTION: teamDriveDomainUsersOnlyRestriction,
+  TEAMDRIVE_TEAM_MEMBERS_ONLY_RESTRICTION: teamDriveTeamMembersOnlyRestriction,
+  TEAMDRIVE_FILE_LIMIT_EXCEEDED: teamDriveFileLimitExceeded,
+  TEAMDRIVE_HIERARCHY_TOO_DEEP: teamDriveHierarchyTooDeep,
+  TEAMDRIVE_MEMBERSHIP_REQUIRED: teamDriveMembershipRequired,
+  TEAMDRIVES_FOLDER_MOVE_IN_NOT_SUPPORTED: teamDrivesFolderMoveInNotSupported,
+  TEAMDRIVES_FOLDER_SHARING_NOT_SUPPORTED: teamDrivesFolderSharingNotSupported,
+  TEAMDRIVES_PARENT_LIMIT: teamDrivesParentLimit,
+  TEAMDRIVES_SHARING_RESTRICTION_NOT_ALLOWED: teamDrivesSharingRestrictionNotAllowed,
+  TEAMDRIVES_SHORTCUT_FILE_NOT_SUPPORTED: teamDrivesShortcutFileNotSupported,
+  TIME_RANGE_EMPTY: timeRangeEmpty,
+  TRANSIENT_ERROR: transientError,
+  UNKNOWN_ERROR: unknownError,
+  UNSUPPORTED_LANGUAGE_CODE: unsupportedLanguageCode,
+  UNSUPPORTED_SUPERVISED_ACCOUNT: unsupportedSupervisedAccount,
+  UPLOAD_TOO_LARGE: uploadTooLarge,
+  USER_CANNOT_CREATE_TEAMDRIVES: userCannotCreateTeamDrives,
+  USER_ACCESS: userAccess,
+  USER_NOT_FOUND: userNotFound,
+  USER_RATE_LIMIT_EXCEEDED: userRateLimitExceeded,
+  }

src/gam/gamlib/glgdata.py

@@ -0,0 +1,98 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2023 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""GAM GData resources
+
+"""
+API_DEPRECATED_MSG = 'Contacts API is being deprecated.'
+
+# callGData throw errors
+API_DEPRECATED = 612
+BAD_GATEWAY = 601
+BAD_REQUEST = 602
+DOES_NOT_EXIST = 1301
+ENTITY_EXISTS = 1300
+FORBIDDEN = 603
+GATEWAY_TIMEOUT = 612
+INSUFFICIENT_PERMISSIONS = 604
+INTERNAL_SERVER_ERROR = 1000
+INVALID_DOMAIN = 605
+INVALID_INPUT = 1317
+INVALID_VALUE = 1801
+NAME_NOT_VALID = 1303
+NOT_FOUND = 606
+NOT_IMPLEMENTED = 607
+PRECONDITION_FAILED = 608
+QUOTA_EXCEEDED = 609
+SERVICE_NOT_APPLICABLE = 1410
+SERVICE_UNAVAILABLE = 610
+TOKEN_EXPIRED = 611
+TOKEN_INVALID = 403
+UNKNOWN_ERROR = 600
+#
+NON_TERMINATING_ERRORS = [API_DEPRECATED, BAD_GATEWAY, GATEWAY_TIMEOUT, QUOTA_EXCEEDED, SERVICE_UNAVAILABLE, TOKEN_EXPIRED]
+EMAILSETTINGS_THROW_LIST = [INVALID_DOMAIN, DOES_NOT_EXIST, SERVICE_NOT_APPLICABLE, BAD_REQUEST, NAME_NOT_VALID, INTERNAL_SERVER_ERROR, INVALID_VALUE]
+#
+class apiDeprecated(Exception):
+  pass
+class badRequest(Exception):
+  pass
+class doesNotExist(Exception):
+  pass
+class entityExists(Exception):
+  pass
+class forbidden(Exception):
+  pass
+class insufficientPermissions(Exception):
+  pass
+class internalServerError(Exception):
+  pass
+class invalidDomain(Exception):
+  pass
+class invalidInput(Exception):
+  pass
+class invalidValue(Exception):
+  pass
+class nameNotValid(Exception):
+  pass
+class notFound(Exception):
+  pass
+class notImplemented(Exception):
+  pass
+class preconditionFailed(Exception):
+  pass
+class serviceNotApplicable(Exception):
+  pass
+
+ERROR_CODE_EXCEPTION_MAP = {
+  API_DEPRECATED: apiDeprecated,
+  BAD_REQUEST: badRequest,
+  DOES_NOT_EXIST: doesNotExist,
+  ENTITY_EXISTS: entityExists,
+  FORBIDDEN: forbidden,
+  INSUFFICIENT_PERMISSIONS: insufficientPermissions,
+  INTERNAL_SERVER_ERROR: internalServerError,
+  INVALID_DOMAIN: invalidDomain,
+  INVALID_INPUT: invalidInput,
+  INVALID_VALUE: invalidValue,
+  NAME_NOT_VALID: nameNotValid,
+  NOT_FOUND: notFound,
+  NOT_IMPLEMENTED: notImplemented,
+  PRECONDITION_FAILED: preconditionFailed,
+  SERVICE_NOT_APPLICABLE: serviceNotApplicable,
+  }

src/gam/gamlib/glglobals.py

@@ -0,0 +1,307 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2023 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""GAM global variables
+
+"""
+
+# The following GM_XXX constants are arbitrary but must be unique
+# Most errors print a message and bail out with a return code
+# Some commands want to set a non-zero return code but not bail
+# GAM admin user
+ADMIN = 'admin'
+# Number/length of API call retries
+API_CALLS_RETRY_DATA = 'rtry'
+# GAM cache directory. If no_cache is True, this variable will be set to None
+CACHE_DIR = 'gacd'
+# Reset GAM cache directory after discovery
+CACHE_DISCOVERY_ONLY = 'gcdo'
+# Classroom service not available
+CLASSROOM_SERVICE_NOT_AVAILABLE = 'csna'
+# Command logging
+CMDLOG_HANDLER = 'clha'
+CMDLOG_LOGGER = 'cllo'
+# Convert to local time
+CONVERT_TO_LOCAL_TIME = 'ctlt'
+# Credentials scopes
+CREDENTIALS_SCOPES = 'crsc'
+# csvfile keyfield <FieldName> [delimiter <Character>] (matchfield <FieldName> <MatchPattern>)* [datafield <FieldName>(:<FieldName>*) [delimiter <String>]]
+CSVFILE = 'csvf'
+# { key: [datafieldvalues]}
+CSV_DATA_DICT = 'csdd'
+CSV_KEY_FIELD = 'cskf'
+CSV_SUBKEY_FIELD = 'cssk'
+CSV_DATA_FIELD = 'csdf'
+# Filter for input column drop values
+CSV_INPUT_ROW_DROP_FILTER = 'cird'
+# Mode (and|or) for input column drop values
+CSV_INPUT_ROW_DROP_FILTER_MODE = 'cidm'
+# Filter for input column values
+CSV_INPUT_ROW_FILTER = 'cirf'
+# Mode (and|or) for input column values
+CSV_INPUT_ROW_FILTER_MODE = 'cirm'
+# Limit number of input rows
+CSV_INPUT_ROW_LIMIT = 'cirl'
+# Column delimiter in CSV output file
+CSV_OUTPUT_COLUMN_DELIMITER = 'codl'
+# Filter for output column headers to drop
+CSV_OUTPUT_HEADER_DROP_FILTER = 'cohd'
+# Filter for output column headers
+CSV_OUTPUT_HEADER_FILTER = 'cohf'
+# Force output column headers
+CSV_OUTPUT_HEADER_FORCE = 'cofh'
+# Order output column headers
+CSV_OUTPUT_HEADER_ORDER = 'coho'
+# No escape character in CSV output file
+CSV_OUTPUT_NO_ESCAPE_CHAR = 'cone'
+# Quote character in CSV output file
+CSV_OUTPUT_QUOTE_CHAR = 'coqc'
+# Filter for output column drop values
+CSV_OUTPUT_ROW_DROP_FILTER = 'cord'
+# Mode (and|or) for output column drop values
+CSV_OUTPUT_ROW_DROP_FILTER_MODE = 'codm'
+# Filter for output column values
+CSV_OUTPUT_ROW_FILTER = 'corf'
+# Mode (and|or) for output column values
+CSV_OUTPUT_ROW_FILTER_MODE = 'corm'
+# Limit number of output rows
+CSV_OUTPUT_ROW_LIMIT = 'corl'
+# Add timestamp column to CSV output file
+CSV_OUTPUT_TIMESTAMP_COLUMN = 'cotc'
+# Output sort headers
+CSV_OUTPUT_SORT_HEADERS = 'cosh'
+# CSV todrive options
+CSV_TODRIVE = 'todr'
+# Current API services
+CURRENT_API_SERVICES = 'caps'
+# Current Client API
+CURRENT_CLIENT_API = 'ccap'
+# Current Client API scopes
+CURRENT_CLIENT_API_SCOPES = 'ccas'
+# Current Service Account API
+CURRENT_SVCACCT_API = 'csap'
+# Current Service Account API scopes
+CURRENT_SVCACCT_API_SCOPES = 'csas'
+# Current Service Account user
+CURRENT_SVCACCT_USER = 'csa'
+# datetime.datetime.now
+DATETIME_NOW = 'dtno'
+# If debug_level > 0: extra_args['prettyPrint'] = True, httplib2.debuglevel = gam_debug_level, appsObj.debug = True
+DEBUG_LEVEL = 'dbgl'
+# Decoded ID token
+DECODED_ID_TOKEN = 'didt'
+# Index of start of <UserTypeEntity> in command line
+ENTITY_CL_DELAY_START = 'ecld'
+ENTITY_CL_START = 'ecls'
+# Extra arguments to pass to GAPI functions
+EXTRA_ARGS_LIST = 'exad'
+# gam.cfg file
+GAM_CFG_FILE = 'gcfi'
+GAM_CFG_PATH = 'gcpa'
+GAM_CFG_SECTION = 'gcse'
+GAM_CFG_SECTION_NAME = 'gcsn'
+# Path to gam
+GAM_PATH = 'gpth'
+# Python source, PyInstaller or StaticX?
+GAM_TYPE = 'gtyp'
+# Length of last Got message
+LAST_GOT_MSG_LEN = 'lgml'
+# License SKUs
+LICENSE_SKUS = 'lsku'
+# Make Building ID/Name map
+MAKE_BUILDING_ID_NAME_MAP = 'mkbm'
+# Dictionary mapping Building ID to Name
+MAP_BUILDING_ID_TO_NAME = 'bi2n'
+# Dictionary mapping Building Name to ID
+MAP_BUILDING_NAME_TO_ID = 'bn2i'
+# Dictionary mapping OrgUnit ID to Name
+MAP_ORGUNIT_ID_TO_NAME = 'oi2n'
+# Dictionary mapping Shared Drive ID to Name
+MAP_SHAREDDRIVE_ID_TO_NAME = 'si2n'
+# Make Role ID/Name map
+MAKE_ROLE_ID_NAME_MAP = 'mkrm'
+# Dictionary mapping Role ID to Name
+MAP_ROLE_ID_TO_NAME = 'ri2n'
+# Dictionary mapping Role Name to ID
+MAP_ROLE_NAME_TO_ID = 'rn2i'
+# Dictionary mapping User ID to Name
+MAP_USER_ID_TO_NAME = 'ui2n'
+# Multiprocess exit condition
+MULTIPROCESS_EXIT_CONDITION = 'mpec'
+# Multiprocess exit processing
+MULTIPROCESS_EXIT_PROCESSING = 'mpep'
+# Number of batch items
+NUM_BATCH_ITEMS = 'nbat'
+# Values retrieved from oauth2service.json
+OAUTH2SERVICE_CLIENT_ID = 'osci'
+OAUTH2SERVICE_JSON_DATA = 'osjd'
+# Values retrieved from oauth2.txt
+OAUTH2_CLIENT_ID = 'oaci'
+# oauth2.txt lock file
+OAUTH2_TXT_LOCK = 'oatl'
+# Output date format, empty defalts to ISOFormat
+OUTPUT_DATEFORMAT = 'oudf'
+# Output time format, empty defalts to ISOFormat
+OUTPUT_TIMEFORMAT = 'outf'
+# gam.cfg parser
+PARSER = 'pars'
+# Process ID
+PID = 'pid '
+# Domains for print alises|groups|users
+PRINT_AGU_DOMAINS = 'pagu'
+# OrgUnits for print cros
+PRINT_CROS_OUS = 'pcou'
+# OrgUnits and children for print cros
+PRINT_CROS_OUS_AND_CHILDREN = 'pcoc'
+# Check API calls rate
+RATE_CHECK_COUNT = 'rccn'
+RATE_CHECK_START = 'rcst'
+# Section name from outer gam, passed to inner gams
+SECTION = 'sect'
+# Enable/disable "Getting ... " messages
+SHOW_GETTINGS = 'shog'
+# Enable/disable NL at end of "Got ..." messages
+SHOW_GETTINGS_GOT_NL = 'shgn'
+# redirected files
+SAVED_STDOUT = 'svso'
+STDERR = 'stde'
+STDOUT = 'stdo'
+# Scopes values retrieved from oauth2service.json
+SVCACCT_SCOPES = 'sasc'
+# Were scopes values retrieved from oauth2service.json
+SVCACCT_SCOPES_DEFINED = 'sasd'
+# Most errors print a message and bail out with a return code
+# Some commands want to set a non-zero return code but not bail
+SYSEXITRC = 'sxrc'
+# Encodings
+SYS_ENCODING = 'syen'
+# Shared by threadBatchWorker and threadBatchGAMCommands
+TBATCH_QUEUE = 'batq'
+# redirected file fields: name, mode, encoding, write header, multiproces, queue
+REDIRECT_NAME = 'rdfn'
+REDIRECT_MODE = 'rdmo'
+REDIRECT_FD = 'rdfd'
+REDIRECT_MULTI_FD = 'rdmf'
+REDIRECT_STD = 'rdst'
+REDIRECT_ENCODING = 'rden'
+REDIRECT_WRITE_HEADER = 'rdwh'
+REDIRECT_MULTIPROCESS = 'rdmp'
+REDIRECT_QUEUE = 'rdq'
+REDIRECT_QUEUE_NAME = 'name'
+REDIRECT_QUEUE_TODRIVE = 'todrive'
+REDIRECT_QUEUE_CSVPF = 'csvpf'
+REDIRECT_QUEUE_DATA = 'rows'
+REDIRECT_QUEUE_ARGS = 'args'
+REDIRECT_QUEUE_GLOBALS = 'globals'
+REDIRECT_QUEUE_VALUES = 'values'
+REDIRECT_QUEUE_START = 'start'
+REDIRECT_QUEUE_END = 'end'
+REDIRECT_QUEUE_EOF = 'eof'
+#
+Globals = {
+  ADMIN: None,
+  API_CALLS_RETRY_DATA: {},
+  CACHE_DIR: None,
+  CACHE_DISCOVERY_ONLY: True,
+  CLASSROOM_SERVICE_NOT_AVAILABLE: False,
+  CMDLOG_HANDLER: None,
+  CMDLOG_LOGGER: None,
+  CONVERT_TO_LOCAL_TIME: False,
+  CREDENTIALS_SCOPES: set(),
+  CSVFILE: {},
+  CSV_DATA_DICT: {},
+  CSV_KEY_FIELD: None,
+  CSV_SUBKEY_FIELD: None,
+  CSV_DATA_FIELD: None,
+  CSV_INPUT_ROW_DROP_FILTER: [],
+  CSV_INPUT_ROW_DROP_FILTER_MODE: False,
+  CSV_INPUT_ROW_FILTER: [],
+  CSV_INPUT_ROW_FILTER_MODE: True,
+  CSV_INPUT_ROW_LIMIT: 0,
+  CSV_OUTPUT_COLUMN_DELIMITER: None,
+  CSV_OUTPUT_HEADER_DROP_FILTER: [],
+  CSV_OUTPUT_HEADER_FILTER: [],
+  CSV_OUTPUT_HEADER_FORCE: [],
+  CSV_OUTPUT_HEADER_ORDER: [],
+  CSV_OUTPUT_NO_ESCAPE_CHAR: None,
+  CSV_OUTPUT_QUOTE_CHAR: None,
+  CSV_OUTPUT_ROW_DROP_FILTER: [],
+  CSV_OUTPUT_ROW_DROP_FILTER_MODE: False,
+  CSV_OUTPUT_ROW_FILTER: [],
+  CSV_OUTPUT_ROW_FILTER_MODE: True,
+  CSV_OUTPUT_ROW_LIMIT: 0,
+  CSV_OUTPUT_SORT_HEADERS: [],
+  CSV_OUTPUT_TIMESTAMP_COLUMN: None,
+  CSV_TODRIVE: {},
+  CURRENT_API_SERVICES: {},
+  CURRENT_CLIENT_API: None,
+  CURRENT_CLIENT_API_SCOPES: set(),
+  CURRENT_SVCACCT_API: None,
+  CURRENT_SVCACCT_API_SCOPES: set(),
+  CURRENT_SVCACCT_USER: None,
+  DATETIME_NOW: None,
+  DEBUG_LEVEL: 0,
+  DECODED_ID_TOKEN: None,
+  ENTITY_CL_DELAY_START: 1,
+  ENTITY_CL_START: 1,
+  EXTRA_ARGS_LIST: [],
+  GAM_CFG_FILE: '',
+  GAM_CFG_PATH: '',
+  GAM_CFG_SECTION: '',
+  GAM_CFG_SECTION_NAME: '',
+  GAM_PATH: '.',
+  GAM_TYPE: '',
+  LAST_GOT_MSG_LEN: 0,
+  LICENSE_SKUS: [],
+  MAKE_BUILDING_ID_NAME_MAP: True,
+  MAKE_ROLE_ID_NAME_MAP: True,
+  MAP_BUILDING_ID_TO_NAME: {},
+  MAP_BUILDING_NAME_TO_ID: {},
+  MAP_ORGUNIT_ID_TO_NAME: {},
+  MAP_SHAREDDRIVE_ID_TO_NAME: {},
+  MAP_ROLE_ID_TO_NAME: {},
+  MAP_ROLE_NAME_TO_ID: {},
+  MAP_USER_ID_TO_NAME: {},
+  MULTIPROCESS_EXIT_CONDITION: None,
+  MULTIPROCESS_EXIT_PROCESSING: False,
+  NUM_BATCH_ITEMS: 0,
+  OAUTH2SERVICE_CLIENT_ID: None,
+  OAUTH2SERVICE_JSON_DATA: {},
+  OAUTH2_CLIENT_ID: None,
+  OAUTH2_TXT_LOCK: None,
+  OUTPUT_DATEFORMAT: '',
+  OUTPUT_TIMEFORMAT: '',
+  PARSER: None,
+  PID: 0,
+  PRINT_AGU_DOMAINS: '',
+  PRINT_CROS_OUS: '',
+  PRINT_CROS_OUS_AND_CHILDREN: '',
+  RATE_CHECK_COUNT: 0,
+  RATE_CHECK_START: 0,
+  SECTION: None,
+  SHOW_GETTINGS: True,
+  SHOW_GETTINGS_GOT_NL: False,
+  SAVED_STDOUT: None,
+  STDERR: {},
+  STDOUT: {},
+  SVCACCT_SCOPES: {},
+  SVCACCT_SCOPES_DEFINED: False,
+  SYSEXITRC: 0,
+  SYS_ENCODING: 'utf-8',
+  TBATCH_QUEUE: None
+  }

src/gam/gamlib/glindent.py

@@ -0,0 +1,46 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2023 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""GAM indent processing
+
+"""
+
+class GamIndent():
+
+  INDENT_SPACES_PER_LEVEL = '  '
+
+  def __init__(self):
+    self.indent = 0
+
+  def Reset(self):
+    self.indent = 0
+
+  def Increment(self):
+    self.indent += 1
+
+  def Decrement(self):
+    self.indent -= 1
+
+  def Spaces(self):
+    return self.INDENT_SPACES_PER_LEVEL*self.indent
+
+  def SpacesSub1(self):
+    return self.INDENT_SPACES_PER_LEVEL*(self.indent-1)
+
+  def MultiLineText(self, message, n=0):
+    return message.replace('\n', f'\n{self.INDENT_SPACES_PER_LEVEL*(self.indent+n)}').rstrip()

src/gam/gamlib/glmsgs.py

@@ -0,0 +1,525 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2024 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""GAM messages
+
+"""
+
+# These values can be translated into other languages
+# Project creation messages in order of appearance
+CREATING_PROJECT = 'Creating project "{0}"...\n'
+CHECK_INTERRUPTED = 'Check interrupted'
+CHECKING_PROJECT_CREATION_STATUS = 'Checking project creation status...\n'
+NO_RIGHTS_GOOGLE_CLOUD_ORGANIZATION = 'Looks like you have no rights to your Google Cloud Organization.\nAttempting to fix that...\n'
+YOUR_ORGANIZATION_NAME_IS = 'Your organization name is {0}\n'
+YOU_HAVE_NO_RIGHTS_TO_CREATE_PROJECTS_AND_YOU_ARE_NOT_A_SUPER_ADMIN = 'You have no rights to create projects for your organization and you don\'t seem to be a super admin! Sorry, there\'s nothing more I can do.'
+LOOKS_LIKE_NO_ONE_HAS_RIGHTS_TO_YOUR_GOOGLE_CLOUD_ORGANIZATION_ATTEMPTING_TO_GIVE_YOU_CREATE_RIGHTS = 'Looks like no one has rights to your Google Cloud Organization. Attempting to give you create rights...\n'
+THE_FOLLOWING_RIGHTS_SEEM_TO_EXIST = 'The following rights seem to exist:\n'
+GIVING_LOGIN_HINT_THE_CREATOR_ROLE = 'Giving {0} the role of {1}...\n'
+ACCEPT_CLOUD_TOS = '''
+Please go to:
+
+https://console.cloud.google.com/projectselector2/home/dashboard?supportedpurview=project
+
+sign in as {0} and accept the Terms of Service (ToS). As soon as you've accepted the ToS popup, you can return here and press enter.\n'''
+
+PROJECT_STILL_BEING_CREATED_SLEEPING = 'Project still being created. Sleeping {0} seconds\n'
+FAILED_TO_CREATE_PROJECT = 'Failed to create project: {0}\n'
+SETTING_GAM_PROJECT_CONSENT_SCREEN = 'Setting GAM project consent screen...\n'
+CREATE_PROJECT_INSTRUCTIONS = '''
+Please go to:
+
+  {0}
+
+1. Choose "Desktop App" or "Other" for "Application type".
+2. Enter "GAM" or another desired value for "Name".
+3. Click the blue "Create" button.
+4. Copy your "Client ID" value that shows on the next page.
+'''
+ENTER_YOUR_CLIENT_ID = '\nEnter your Client ID: '
+GO_BACK_TO_YOUR_BROWSER_AND_COPY_YOUR_CLIENT_SECRET_VALUE = '\n5. Go back to your browser and copy your "Client Secret" value.\n'
+ENTER_YOUR_CLIENT_SECRET = '\nEnter your Client Secret: '
+GO_BACK_TO_YOUR_BROWSER_AND_CLICK_OK_TO_CLOSE_THE_OAUTH_CLIENT_POPUP = '\n6. Go back to your browser and click OK to close the "OAuth client" popup if it\'s still open.\n'
+IS_NOT_A_VALID_CLIENT_ID = '''
+
+{0}
+
+Is not a valid Client ID.
+
+Please make sure you are following the directions exactly and that there are no extra spaces in your Client ID.
+'''
+IS_NOT_A_VALID_CLIENT_SECRET = '''
+
+{0}
+
+Is not a valid Client Secret.
+
+Please make sure you are following the directions exactly and that there are no extra spaces in your Client Secret.
+'''
+TRUST_GAM_CLIENT_ID = '''
+It's important to mark the {0} Client ID as trusted by your Workspace instance.
+
+Please go to:
+
+  https://admin.google.com/ac/owl/list?tab=configuredApps
+
+1. Click on: Configure new app > OAuth App Name Or Client ID.
+2. Enter the following Client ID value:
+
+  {1}
+
+3. Press Search, select the {0} app, press Select, check the box and press Select.
+4. Keep the default scope or select a preferred scope that includes your GAM admin.
+5. Press Continue
+6. Select Trusted radio button, press Continue and Finish.
+7. Press Confirm if Confirm parental consent pops up
+8. Press enter here on the terminal once trust is complete.
+'''
+
+ENABLE_SERVICE_ACCOUNT_PRIVATE_KEY_UPLOAD = '''
+Your workspace is configured to disable service account private key uploads.
+
+Please go to:
+
+  https://github.com/taers232c/GAMADV-XTD3/wiki/Authorization#authorize-service-account-key-uploads
+
+Follow the steps to allow a service account private key upload for the project ({0}) just created.
+Once those steps are completed, you can continue with your project authentication.
+'''
+
+YOUR_GAM_PROJECT_IS_CREATED_AND_READY_TO_USE = '''
+That\'s it! Your GAM Project is created and ready to use.
+Proceed to the authentication steps.
+'''
+
+# check|update service messages in order of appearance
+SYSTEM_TIME_STATUS = 'System time status'
+YOUR_SYSTEM_TIME_DIFFERS_FROM_GOOGLE = 'Your system time differs from {0} by {1}'
+PRESS_ENTER_ONCE_AUTHORIZATION_IS_COMPLETE = 'Press enter once authorization is complete.'
+SERVICE_ACCOUNT_API_DISABLED = '{0} not enabled. Please run "gam update project" and "gam user user@domain.com update serviceaccount"'
+SERVICE_ACCOUNT_PRIVATE_KEY_AUTHENTICATION = 'Service Account Private Key Authentication'
+SERVICE_ACCOUNT_CHECK_PRIVATE_KEY_AGE = 'Service Account Private Key age; Google recommends rotating keys on a routine basis'
+SERVICE_ACCOUNT_PRIVATE_KEY_AGE = 'Service Account Private Key age: {0} days'
+SERVICE_ACCOUNT_SKIPPING_KEY_AGE_CHECK = 'Skipping Private Key age check: {0} rotation not necessary'
+UPDATE_PROJECT_TO_VIEW_MANAGE_SAKEYS = 'Please run "gam update project" to view/manage service account keys'
+DOMAIN_WIDE_DELEGATION_AUTHENTICATION = 'Domain-wide Delegation authentication'
+SCOPE_AUTHORIZATION_PASSED = '''All scopes PASSED!
+
+Service Account Client name: {0} is fully authorized.
+'''
+SCOPE_AUTHORIZATION_UPDATE_PASSED = '''All scopes PASSED!
+To authorize them (in case some scopes were unselected), please go to the following link in your browser:
+{0}
+    {1}
+
+You will be directed to the Google Workspace admin console Security > API Controls > Domain-wide Delegation page
+The "Add a new Client ID" box will open
+Make sure that "Overwrite existing client ID" is checked
+Click AUTHORIZE
+When the box closes you're done
+After authorizing it may take some time for this test to pass so wait a few moments and then try this command again.
+'''
+SCOPE_AUTHORIZATION_FAILED = '''Some scopes FAILED!
+To authorize them, please go to the following link in your browser:
+{0}
+    {1}
+
+You will be directed to the Google Workspace admin console Security > API Controls > Domain-wide Delegation page
+The "Add a new Client ID" box will open
+Make sure that "Overwrite existing client ID" is checked
+Click AUTHORIZE
+When the box closes you're done
+After authorizing it may take some time for this test to pass so wait a few moments and then try this command again.
+'''
+# General messages
+ACCESS_FORBIDDEN = 'Access Forbidden'
+ACTION_APPLIED = 'Action Applied'
+ACTION_IN_PROGRESS = 'Action {0} in progress'
+ACTION_MAY_BE_DELAYED = 'Action may be delayed'
+ADMIN_STATUS_CHANGED_TO = 'Admin Status Changed to'
+ALL = 'All'
+ALL_FOLDER_NAMES_MUST_BE_NON_BLANK = 'All folder names must be non-blank: {0}'
+ALL_SKU_PRODUCTIDS_MUST_MATCH = 'All SKU productIds must match, {0} != {1}'
+ALREADY_WAS_OWNER = 'Already was owner'
+ALREADY_EXISTS = 'Already exists'
+ALREADY_EXISTS_IN_TARGET_FOLDER = 'Already exists in {0}: {1}'
+ALREADY_EXISTS_USE_MERGE_ARGUMENT = 'Already exists; use the "merge" argument to merge the labels'
+API_ACCESS_DENIED = 'API access Denied'
+API_CALLS_RETRY_DATA = 'API calls retry data\n'
+API_CHECK_CLIENT_AUTHORIZATION = 'Please make sure the Client ID: {0} is authorized for the appropriate API or scopes:\n{1}\n\nRun: gam oauth create\n'
+API_CHECK_SVCACCT_AUTHORIZATION = 'Please make sure the Service Account Client ID: {0} is authorized for the appropriate API or scopes:\n{1}\n\nRun: gam user {2} update serviceaccount\n'
+API_ERROR_SETTINGS = 'API error, some settings not set'
+ARE_BOTH_REQUIRED = 'Arguments {0} and {1} are both required'
+ARE_MUTUALLY_EXCLUSIVE = 'Arguments {0} and {1} are mutually exclusive'
+AS = 'as'
+ATTENDEES_ADD = 'Add Attendees'
+ATTENDEES_ADD_REMOVE = 'Add/Remove Attendees'
+ATTENDEES_REMOVE = 'Remove Attendees'
+AUTHORIZATION_FILE_ALREADY_EXISTS = '{0} already exists. Please delete or rename it before attempting to {1} project.'
+AUTHENTICATION_FLOW_COMPLETE = '\nThe authentication flow has completed.'
+AUTHENTICATION_FLOW_COMPLETE_CLOSE_BROWSER = 'The authentication flow has completed. You may close this browser window and return to {0}.'
+AUTHENTICATION_FLOW_FAILED = 'The authentication flow failed, reissue command'
+BAD_ENTITIES_IN_SOURCE = '{0} {1} {2} in source marked >>> <<< above'
+BAD_REQUEST = 'Bad Request'
+BATCH = 'Batch'
+BATCH_CSV_LOOP_DASH_DEBUG_INCOMPATIBLE = '"gam {0} - ..." is not compatible with debugging. Disable debugging by setting debug_level = 0 in gam.cfg'
+BATCH_CSV_PROCESSING_COMPLETE = '{0},0/{1},Processing complete\n'
+BATCH_CSV_TERMINATE_N_PROCESSES = '{0},0/{1},Terminating {2} running {3}\n'
+BATCH_CSV_WAIT_LIMIT = ', wait limit {0} seconds'
+BATCH_CSV_WAIT_N_PROCESSES = '{0},0/{1},Waiting for {2} running {3} to finish before terminating{4}\n'
+BATCH_NOT_PROCESSED_ERRORS = '{0}batch file: {1}, not processed, {2} {3}\n'
+CALLING_GCLOUD_FOR_REAUTH = 'Calling gcloud for reauth credentials..."\n'
+CAN_NOT_DELETE_USER_WITH_VAULT_HOLD = '{0}: The user may be (or have recently been) on Google Vault Hold and thus not eligible for deletion. You can check holds with "gam user {1} show vaultholds".'
+CAN_NOT_BE_SPECIFIED_MORE_THAN_ONCE = 'Argument {0} can not be specified more than once'
+CHAT_ADMIN_ACCESS_LIMITED_TO_ONE_USER = 'Chat adminaccess|asadmin limited to one user, {0} specified'
+CHROME_TARGET_VERSION_FORMAT = r'^([a-z]+)-(\d+)$ or ^(\d{1,4}\.){1,4}$'
+COLUMN_DOES_NOT_MATCH_ANY_INPUT_COLUMNS = '{0} column "{1}" does not match any input columns'
+COLUMN_DOES_NOT_MATCH_ANY_OUTPUT_COLUMNS = '{0} column "{1}" does not match any output columns'
+COMMAND_NOT_COMPATIBLE_WITH_ENABLE_DASA = 'gam {0} {1} is not compatible with enable_dasa = true in gam.cfg'
+COMMIT_BATCH_COMPLETE = '{0},0/{1},commit-batch - running {2} finished, proceeding\n'
+COMMIT_BATCH_WAIT_N_PROCESSES = '{0},0/{1},commit-batch - waiting for {2} running {3} to finish before proceeding\n'
+CONFIRM_WIPE_YUBIKEY_PIV = 'This will wipe all YubiKey PIV keys and configuration from your YubiKey. Are you sure? (y/N) '
+CONTACT_ADMINISTRATOR_FOR_PASSWORD = 'Contact administrator for password'
+CONTACT_PHOTO_NOT_FOUND = 'Contact photo not found'
+CONTAINS_AT_LEAST_1_ITEM = 'Contains at least 1 item'
+COUNT_N_EXCEEDS_MAX_TO_PROCESS_M = 'Count {0} exceeds maximum to {1} {2}'
+CORRUPT_FILE = 'Corrupt file'
+COULD_NOT_FIND_ANY_YUBIKEY = 'Could not find any YubiKey\n'
+COULD_NOT_FIND_YUBIKEY_WITH_SERIAL = 'Could not find YubiKey with serial number {0}\n'
+CREATE_USER_NOTIFY_MESSAGE = 'Hello #givenname# #familyname#,\n\nYou have a new account at #domain#\nAccount details:\nUsername: #user#\nPassword: #password#\nStart using your new account by signing in at\nhttps://www.google.com/accounts/AccountChooser?Email=#user#&continue=https://workspace.google.com/dashboard\n'
+CREATE_USER_NOTIFY_SUBJECT = 'Welcome to #domain#'
+CSV_DATA_ALREADY_SAVED = 'CSV data already saved'
+CSV_FILE_HEADERS = 'The CSV file ({0}) has the following headers:\n'
+CSV_SAMPLE_COMMANDS = 'Here are the first {0} commands {1} will run\n'
+DATA_FIELD_MISMATCH = 'datafield {0} does not match saved datafield {1}'
+DATA_TRANSFER_COMPLETED = 'Data Transfer completed: {0}\n'
+DATA_UPLOADED_TO_DRIVE_FILE = 'Data uploaded to Drive File'
+DEFAULT_SMIME = 'Default S/MIME'
+DELETED = 'Deleted'
+DEVICE_LIST_BUG = 'GAM hit Google internal bug 237397223. Please file a Google Support ticket stating that you are encountering this bug.'
+DEVICE_LIST_BUG_WORKAROUND_NOT_POSSIBLE = 'GAM workaround for this issue only works if orderby argument is not used and query does not contain \'register\'.'
+DEVICE_LIST_BUG_ATTEMPTING_WORKAROUND = 'GAM is attempting to work around the bug by filtering for devices created on or after the newest we\'ve seen ({0})...\n'
+DIRECTLY_IN_THE = ' directly in the {0}'
+DISABLE_TLS_MIN_MAX = 'Execute: gam select default config tls_max_version "" tls_min_version "" save\n'
+DISPLAYNAME_NOT_ALLOWED_WHEN_UPDATING_MULTIPLE_SCHEMAS = 'displayname not allowed when updating multiple schemas'
+DOES_NOT_EXIST = 'Does not exist'
+DOES_NOT_EXIST_OR_HAS_INVALID_FORMAT = '{0}: {1}, Does not exist or has invalid format, {2}'
+DOES_NOT_MATCH = 'Does not match {0}'
+DOMAIN_NOT_FOUND_IN_DNS = 'Domain not found in DNS!'
+DOMAIN_NOT_VERIFIED_SECONDARY = 'Domain is not a verified secondary domain'
+DONE_GENERATING_PRIVATE_KEY_AND_PUBLIC_CERTIFICATE = 'Done generating private key and public certificate'
+DO_NOT_EXIST = 'Do not exist'
+DOWNLOADING_AGAIN_AND_OVER_WRITING = 'Downloading again and over-writing...'
+DUPLICATE = 'Duplicate'
+DUPLICATE_ALREADY_A_ROLE = 'Duplicate, already a {0}'
+DYNAMIC_GROUP_MEMBERSHIP_CANNOT_BE_MODIFIED = 'Dynamic group membership cannot be modified'
+EITHER = 'Either'
+EMAIL_ADDRESS_IS_UNMANAGED_ACCOUNT = 'The email address is an unmanaged account'
+ENABLE_PROJECT_APIS_AUTOMATICALLY_OR_MANUALLY = 'Do you want to enable project APIs [a]utomatically or [m]anually? (a/m): '
+ENTER_GSUITE_ADMIN_EMAIL_ADDRESS = '\nEnter your Google Workspace admin email address? '
+ENTER_MANAGE_GCP_PROJECT_EMAIL_ADDRESS = '\nEnter your Google Workspace admin or GCP project manager email address authorized to manage project(s): {0}? '
+ENTER_VERIFICATION_CODE_OR_URL = 'Enter verification code or paste "Unable to connect" URL from other computer (only URL data up to &scope required): '
+ENTITY_DOES_NOT_EXIST = '{0} does not exist'
+ENTITY_NAME_NOT_VALID = 'Entity Name Not Valid'
+ERROR = 'error'
+ERRORS = 'errors'
+EVENT_IS_CANCELED = 'Event is canceled'
+EXECUTE_GAM_OAUTH_CREATE = '\nPlease run\n\ngam oauth delete\ngam oauth create\n\n'
+EXISTS = 'Exists'
+EXPECTED = 'Expected'
+EXPORT_NOT_COMPLETE = 'Export needs to be complete before downloading, current status is: {0}'
+EXTRACTING_PUBLIC_CERTIFICATE = 'Extracting public certificate'
+FAILED_PRECONDITION = 'Failed precondition'
+FAILED_TO_PARSE_AS_JSON = 'Failed to parse as JSON'
+FAILED_TO_PARSE_AS_LIST = 'Failed to parse as list'
+FIELD_NOT_FOUND_IN_SCHEMA = 'Field {0} not found in schema {1}'
+FILE_NOT_FOUND = 'File {0} not found'
+FINISHED = 'Finished'
+FILTER_CAN_ONLY_CONTAIN_ONE_CATEGORY_LABEL = 'Filter can only contain one CATEGORY label'
+FILTER_CAN_ONLY_CONTAIN_ONE_USER_LABEL = 'Filter can only contain one USER label'
+FOR = 'for'
+FORBIDDEN = 'Forbidden'
+FORMAT_NOT_AVAILABLE = 'Format ({0}) not available'
+FORMAT_NOT_DOWNLOADABLE = 'Format not downloadable'
+FROM = 'From'
+FROM_LC = 'from'
+FULL_PATH_MUST_START_WITH_DRIVE = 'fullpath must start with {0} or {1}'
+GAM_BATCH_FILE_WRITTEN = 'GAM batch file {0} written\n'
+GAM_LATEST_VERSION_NOT_AVAILABLE = 'GAM Latest Version information not available'
+GAM_OUT_OF_MEMORY = 'GAM has run out of memory. If this is a large Google Workspace instance, you should use a 64-bit version of GAM on Windows or a 64-bit version of Python on other systems.'
+GENERATING_NEW_PRIVATE_KEY = 'Generating new private key'
+GETTING = 'Getting'
+GETTING_ALL = 'Getting all'
+GOOGLE_DELEGATION_ERROR = 'Google delegation error, delegator and delegate both exist and are valid for delegation'
+GOT = 'Got'
+GROUP_MAPS_TO_MULTIPLE_OUS = 'File: {0}, Group: {1} references multiple OUs: {2}'
+GROUP_MAPS_TO_OU_INVALID_ROW = 'File: {0}, Invalid row, must contain non-blank <EmailAddress> and <OrgUnitPath>: <{1}> <{2}>'
+GUARDIAN_INVITATION_STATUS_NOT_PENDING = 'Guardian invitation status is not PENDING'
+HAS_CHILD_ORGS = 'Has child {0}'
+HAS_INVALID_FORMAT = '{0}: {1}, Has invalid format'
+HAS_RIGHTS_TO_ROTATE_OWN_PRIVATE_KEY = 'Giving account {0} rights to rotate {1} private key'
+HEADER_NOT_FOUND_IN_CSV_HEADERS = 'Header "{0}" not found in CSV headers of "{1}".'
+HELP_SYNTAX = 'Help: Syntax in file {0}\n'
+HELP_WIKI = 'Help: Documentation is at {0}\n'
+IGNORED = 'Ignored'
+INSTRUCTIONS_CLIENT_SECRETS_JSON = 'Please run\n\ngam create|use project\ngam oauth create\n\nto create and authorize a Client account.\n'
+INSTRUCTIONS_OAUTH2SERVICE_JSON = 'Please run\n\ngam create|use project\ngam user <user> check serviceaccount\n\nto create and authorize a Service account.\n'
+INSUFFICIENT_PERMISSIONS_TO_PERFORM_TASK = 'Insufficient permissions to perform this task'
+INTER_BATCH_WAIT_INCREASED = 'inter_batch_wait increased to {0:.2f}'
+INVALID = 'Invalid'
+INVALID_ALIAS = 'Invalid Alias'
+INVALID_ATTENDEE_CHANGE = 'Invalid attendee change "{0}"'
+INVALID_CHARSET = 'Invalid charset "{0}"'
+INVALID_DATE_TIME_RANGE = '{0} {1} must be greater than/equal to {2} {3}'
+INVALID_ENTITY = 'Invalid {0}, {1}'
+INVALID_FILE_SELECTION_WITH_ADMIN_ACCESS = 'Invalid file selection with adminaccess|asadmin'
+INVALID_GROUP = 'Invalid Group'
+INVALID_HTTP_HEADER = 'Invalid http header data: {0}'
+INVALID_JSON_INFORMATION = 'Google API reported Invalid JSON Information'
+INVALID_JSON_SETTING = 'Invalid JSON setting'
+INVALID_LIST = 'Invalid list'
+INVALID_MEMBER = 'Invalid Member address'
+INVALID_MESSAGE_ID = 'Invalid message id(s)'
+INVALID_MIMETYPE = 'Invalid mimeType {0}, must be {1}'
+INVALID_NUMBER_OF_CHAT_SPACE_MEMBERS = '{0} type {1} number of members, {2}, must be in range {3} to {4}'
+INVALID_ORGUNIT = 'Invalid Organizational Unit'
+INVALID_PATH = 'Invalid Path'
+INVALID_PERMISSION_ATTRIBUTE_TYPE = 'permission attribute {0} not allowed with type {1}'
+INVALID_REGION = 'See: https://github.com/taers232c/GAMADV-XTD3/wiki/Context-Aware-Access-Levels#caa-region-codes'
+INVALID_QUERY = 'Invalid Query'
+INVALID_RE = 'Invalid RE'
+INVALID_REQUEST = 'Invalid Request'
+INVALID_RESELLER_CUSTOMER_NAME = 'name must be: accounts/<ResellerID>/customers/<ChannelCustomerID>'
+INVALID_ROLE = 'Invalid subkeyfield Role, must be one of: {0}'
+INVALID_SCHEMA_VALUE = 'Invalid Schema Value'
+INVALID_SCOPE = 'Invalid Scope'
+INVALID_SITE = 'Invalid Site ({0}), must match pattern ({1})'
+INVALID_TAG_SPECIFICATION = 'Invalid tag, expected field.subfield or field.subfield.subfield.string'
+INVALID_TIMEOFDAY_RANGE = '{0} must be less than/equal to {1}'
+IN_SKIPIDS = 'In skipids'
+IN_THE = ' in the {0}'
+IN_TRASH_AND_EXCLUDE_TRASHED = 'In Trash and excludeTrashed'
+IS_EXPIRED_OR_REVOKED = '{0}: {1}, Is expired or has been revoked'
+IS_NOT_DONE_CHECKING_IN_SECONDS = 'Is not done, checking again in {0} seconds'
+IS_NOT_UNIQUE = 'Is not unique, {0}: {1}'
+IS_REQD_TO_CHG_PWD_NO_DELEGATION = 'Is required to change password at next login. You must change password or clear changepassword flag for delegation.'
+IS_SUSPENDED_NO_BACKUPCODES = 'User is suspended. You must unsuspend to process backupcodes'
+IS_SUSPENDED_NO_DELEGATION = 'Is suspended. You must unsuspend for delegation.'
+IS_YUBIKEY_INSERTED = 'Is YubiKey inserted?'
+JSON_ERROR = 'JSON error "{0}" in file {1}'
+JSON_KEY_NOT_FOUND = 'JSON key "{0}" not found in file {1}'
+KIOSK_MODE_REQUIRED = ' This command ({0}) requires that the ChromeOS device be in Kiosk mode.'
+LESS_THAN_1_SECOND = 'less than 1 second'
+LIST_CHROMEOS_INVALID_INPUT_PAGE_TOKEN_RETRY = 'List ChromeOSdevices Invalid Input: pageToken retry'
+LOGGING_INITIALIZATION_ERROR = 'Logging initialization error: {0}'
+LOOKING_UP_GOOGLE_UNIQUE_ID = 'Looking up Google Unique ID'
+MARKED_AS = 'Marked as'
+MATCHED_THE_FOLLOWING = 'Matched the following'
+MATTER_NOT_OPEN = 'Matter needs to be open, current state is: {0}'
+MAXIMUM_OF = 'maximum of'
+MEMBERSHIP_IS_PENDING_WILL_DELETE_ADD_TO_ACCEPT = 'Membership is pending, will delete and add to accept'
+MIMETYPE_MISMATCH = 'Shortcut target mimeType {0} does not match actual target mimeType {1}'
+MIMETYPE_NOT_PRESENT_IN_ATTACHMENT = 'MIME type not present in attachment'
+MISMATCH_RE_SEARCH_REPLACE_SUBFIELDS = 'The subfield ({2}) in replace "{3}" exceeds the number of subfields ({0}) in search "{1}"'
+MISMATCH_SEARCH_REPLACE_SUBFIELDS = 'The number of subfields ({0}) in search "{1}" does not match the number of subfields ({2}) in replace "{3}"'
+MISSING_FIELDS = 'Missing fields: {0}\n'
+MULTIPLE_BUILDINGS_SAME_NAME = '{0} {1} with the same (case-insensitive) name exist'
+MULTIPLE_ENTITIES_FOUND = 'Multiple {0} ({1}) found, {2}'
+MULTIPLE_ITEMS_SPECIFIED = 'Multiple {0} are specfied, only one is allowed'
+MULTIPLE_ITEMS_MARKED_PRIMARY = 'Multiple {0} are marked primary, only one can be primary'
+MULTIPLE_PARENTS_SPECIFIED = 'Multiple parents ({0}) specified, only one is allowed'
+MULTIPLE_SEARCH_METHODS_SPECIFIED = 'Multiple search methods ({0}) specified, only one is allowed'
+MULTIPLE_SSO_PROFILES_MATCH = 'Multiple SSO profiles match display name {0}:\n'
+MULTIPLE_YUBIKEYS_CONNECTED = 'Multiple YubiKeys connected. Specify yubikey_serial_number and one of {0}\n'
+MUST_BE_NUMERIC = 'Must be numeric'
+NEED_READ_ACCESS = 'Need Read access'
+NEED_READ_WRITE_ACCESS = 'Need Read/Write access'
+NEED_WRITE_ACCESS = 'Need Write access'
+NESTED_LOOP_CMD_NOT_ALLOWED = 'Command can not be nested.'
+NEWUSER_REQUIREMENTS = 'newuser option requires: at least 1 recipient and givenname, familyname and password options'
+NEW_OWNER_MUST_DIFFER_FROM_OLD_OWNER = 'New owner must differ from old owner'
+NO_DATA = 'No data'
+NON_BLANK = 'Non-blank'
+NON_EMPTY = 'Non-empty'
+NOT_A = 'Not a'
+NOT_A_PRIMARY_EMAIL_ADDRESS = 'Not a primary email address'
+NOT_A_MEMBER = 'Not a member'
+NOT_ACTIVE = 'Not Active'
+NOT_ALLOWED = 'Not Allowed'
+NOT_AN_ENTITY = 'Not a {0}'
+NOT_APPROPRIATE = 'Not Appropriate'
+NOT_COMPATIBLE = 'Not Compatible'
+NOT_COPYABLE = 'Not Copyable'
+NOT_COPYABLE_INTO_ITSELF = 'Not copyable into itself'
+NOT_COPYABLE_SAME_NAME_CURRENT_FOLDER_MERGE = 'Not copyable with same name into current folder with duplicatefolders merge'
+NOT_COPYABLE_SAME_NAME_CURRENT_FOLDER_OVERWRITE = 'Not copyable with same name into current folder with duplicatefiles overwriteall|overwriteolder'
+NOT_DELETABLE = 'Not Deletable'
+NOT_FOUND = 'Not Found'
+NOT_MOVABLE = 'Not Movable'
+NOT_MOVABLE_IN_TRASH = 'Not Movable, in Trash'
+NOT_MOVABLE_INTO_ITSELF = 'Not movable into itself'
+NOT_MOVABLE_SAME_NAME_CURRENT_FOLDER_MERGE = 'Not movable with same name into current folder with duplicatefolders merge'
+NOT_MOVABLE_SAME_NAME_CURRENT_FOLDER_OVERWRITE = 'Not movable with same name into current folder with duplicatefiles overwriteall|overwriteolder'
+NOT_OWNED_BY = 'Not owned by {0}'
+NOT_SELECTED = 'Not Selected'
+NOT_WRITABLE = 'Not Writable'
+NOW_THE_PRIMARY_DOMAIN = 'Now the primary domain'
+NO_ACTION_SPECIFIED = 'No action specified'
+NO_AVAILABLE_LICENSES = "There aren't enough available licenses for the specified product-SKU pair(s)"
+NO_CHANGES = 'No changes'
+NO_CLIENT_ACCESS_ALLOWED = 'No Client Access allowed'
+NO_CLIENT_ACCESS_CREATE_UPDATE_ALLOWED = 'No Client Access create/update allowed'
+NO_COLUMNS_SELECTED_WITH_CSV_OUTPUT_HEADER_FILTER = 'No columns selected with {0} and {1}'
+NO_CREDENTIALS_REPLACEMENT = '{0}: {1} has {2} {3}. We only replace if there are 2.\n'
+NO_CSV_DATA_TO_UPLOAD = 'No CSV data to upload'
+NO_CSV_FILE_DATA_FOUND = 'No CSV file data found'
+NO_CSV_FILE_DATA_SAVED = 'No CSV file data saved'
+NO_CSV_FILE_SUBKEYS_SAVED = 'No CSV file subkeys saved'
+NO_DATA_TRANSFER_APP_FOR_PARAMETER = 'No data transfer application for key {0}'
+NO_ENTITIES_FOUND = 'No {0} found'
+NO_ENTITIES_MATCHED = 'No {0} matched'
+NO_FILTER_ACTIONS = 'No {0} actions specified'
+NO_FILTER_CRITERIA = 'No {0} criteria specified'
+NO_LABELS_MATCH = 'No Labels match'
+NO_LABELS_TO_PROCESS = 'No Labels to process'
+NO_MESSAGES_WITH_LABEL = 'No Messages with Label'
+NO_PARENTS_TO_CONVERT_TO_SHORTCUTS = 'No parents to convert to shortcuts'
+NO_REPORT_AVAILABLE = 'No {0} report available.'
+NO_SCOPES_FOR_API = 'There are no scopes authorized for the {0}'
+NO_SERIAL_NUMBERS_SPECIFIED = 'No serial numbers specified'
+NO_SSO_PROFILE_MATCHES = 'No SSO profile matches display name {0}'
+NO_SSO_PROFILE_ASSIGNED = 'No SSO profile assigned to {0} {1}'
+NO_SVCACCT_ACCESS_ALLOWED = 'No Service Account Access allowed'
+NO_TRANSFER_LACK_OF_DISK_SPACE = 'Transfer not performed due to lack of target drive space.'
+NO_USAGE_PARAMETERS_DATA_AVAILABLE = 'No usage parameters data available.'
+NO_USER_COUNTS_DATA_AVAILABLE = 'No User counts data available.'
+OAUTH2_GO_TO_LINK_MESSAGE = """
+Go to the following link in a browser on this computer or on another computer:
+
+    {url}
+
+If you use a browser on another computer, you will get a browser error that the site can't be reached AFTER you
+click the Allow button, paste "Unable to connect" URL from other computer (only URL data up to &scope required):
+"""
+ON_CURRENT_PRIVATE_KEY = ' on current key'
+ON_VAULT_HOLD = 'On Google Vault Hold'
+ONLY_ADMINISTRATORS_CAN_PERFORM_SHARED_DRIVE_QUERIES = 'Only administrators can perform Shared Drive queries'
+ONLY_ADMINISTRATORS_CAN_SPECIFY_SHARED_DRIVE_ORGUNIT = 'Only administrators can specify Shared Drive Org Unit'
+ONLY_ONE_DEVICE_SELECTION_ALLOWED = 'Only one device selection allowed, filter = "{0}"'
+ONLY_ONE_JSON_RANGE_ALLOWED = 'Only one range/json allowed'
+ONLY_ONE_OWNER_ALLOWED = 'Only one owner allowed'
+OR = 'or'
+OU_AND_MOVETOOU_CANNOT_BE_IDENTICAL = 'ou {0} can not be be identical to movetoou {1}'
+OU_SUBOUS_CANNOT_BE_MOVED_TO_MOVETOOU = 'ou {0} sub OUs can not be be moved to movetoou {1}'
+PERMISSION_DENIED = 'The caller does not have permission'
+PLEASE_CORRECT_YOUR_SYSTEM_TIME = 'Please correct your system time.'
+PLEASE_ENTER_A_OR_M = 'Please enter a or m ...\n'
+PLEASE_SELECT_ENTITY_TO_PROCESS = '{0} {1} found, please select the correct one to {2} and specify with {3}'
+PLEASE_SPECIFY_BUILDING_EXACT_CASE_NAME_OR_ID = 'Please specify building by exact case name or ID.'
+PREVIEW_ONLY = 'Preview Only'
+PRIMARY_EMAIL_DID_NOT_MATCH_PATTERN = 'primaryEmail address did not match pattern: {0}'
+PROCESS = 'process'
+PROCESSES = 'processes'
+PROCESSING_ITEM_N = '{0},0,Processing item {1}\n'
+PROCESSING_ITEM_N_OF_M = '{0},0,Processing item {1}/{2}\n'
+PROFILE_PHOTO_NOT_FOUND = 'Profile photo not found'
+PROFILE_PHOTO_IS_DEFAULT = 'Profile photo is default'
+REASON_ONLY_VALID_WITH_CONTENTRESTRICTIONS_READONLY_TRUE = 'reason only valid with contentrestrictions readonly true'
+REAUTHENTICATION_IS_NEEDED = 'Reauthentication is needed, please run\n\ngam oauth create'
+RECOMMEND_RUNNING_GAM_ROTATE_SAKEY = 'Recommend running "gam rotate sakey" to get a new key\n'
+REFUSING_TO_DEPROVISION_DEVICES = 'Refusing to deprovision {0} devices because acknowledge_device_touch_requirement not specified.\nDeprovisioning a device means the device will have to be physically wiped and re-enrolled to be managed by your domain again.\nThis requires physical access to the device and is very time consuming to perform for each device.\nPlease add "acknowledge_device_touch_requirement" to the GAM command if you understand this and wish to proceed with the deprovision.\nPlease also be aware that deprovisioning can have an effect on your device license count.\nSee https://support.google.com/chrome/a/answer/3523633 for full details.'
+REPLY_TO_CUSTOM_REQUIRES_EMAIL_ADDRESS = 'replyto REPLY_TO_CUSTOM requires customReplyTo <EmailAddress>'
+REQUEST_COMPLETED_NO_FILES = 'Request completed but no results/files were returned, try requesting again'
+REQUEST_NOT_COMPLETE = 'Request needs to be completed before downloading, current status is: {0}'
+RESOURCE_CAPACITY_FLOOR_REQUIRED = 'Options "capacity <Number>" (<Number> > 0) and "floor <String>" required'
+RESOURCE_FLOOR_REQUIRED = 'Option "floor <String>" required'
+RESULTS_TOO_LARGE_FOR_GOOGLE_SPREADSHEET = 'Results are too large for Google Spreadsheets. Uploading as a regular CSV file.'
+RETRIES_EXHAUSTED = 'Retries {0} exhausted'
+RETRYING_GOOGLE_SHEET_EXPORT_SLEEPING = 'Retrying Google Sheet export {0}/{1}. Sleeping {2} seconds\n'
+ROLE_MUST_BE_ORGANIZER = 'Role must be organizer'
+ROLE_NOT_IN_SET = 'Role not in set: {0})'
+SCHEMA_WOULD_HAVE_NO_FIELDS = '{0} would have no {1}'
+SELECTED = 'Selected'
+SERVICE_NOT_APPLICABLE = 'Service not applicable/Does not exist'
+SERVICE_NOT_APPLICABLE_THIS_ADDRESS = 'Service not applicable for this address: {0}'
+SERVICE_NOT_ENABLED = '{0} Service/App not enabled'
+SHORTCUT_TARGET_CAPABILITY_IS_FALSE = '{0} capability {1} is False'
+SKU_HAS_NO_MATCHING_ARCHIVED_USER_SKU = 'SKU {0} has no matching Archived User SKU'
+STARTING_THREAD = 'Starting thread'
+STATISTICS_COPY_FILE = 'Total: {0}, Copied: {1}, Shortcut created {2}, Shortcut exists {3}, Duplicate: {4}, Copy Failed: {5}, Not copyable: {6}, In skipids: {7}, Permissions Failed: {8}, Protected Ranges Failed: {9}'
+STATISTICS_COPY_FOLDER = 'Total: {0}, Copied: {1}, Shortcut created {2}, Shortcut exists {3}, Duplicate: {4}, Merged: {5}, Copy Failed: {6}, Not writable: {7}, Permissions Failed: {8}'
+STATISTICS_MOVE_FILE = 'Total: {0}, Moved: {1}, Shortcut created {2}, Shortcut exists {3}, Duplicate: {4}, Move Failed: {5}, Not movable: {6}'
+STATISTICS_MOVE_FOLDER = 'Total: {0}, Moved: {1}, Shortcut created {2}, Shortcut exists {3}, Duplicate: {4}, Merged: {5}, Move Failed: {6}, Not writable: {7}'
+STATISTICS_USER_NOT_ORGANIZER = 'User not organizer: {0}'
+STRING_LENGTH = 'string length'
+SUBKEY_FIELD_MISMATCH = 'subkeyfield {0} does not match saved subkeyfield {1}'
+SUBSCRIPTION_NOT_FOUND = 'Could not find subscription'
+SUFFIX_NOT_ALLOWED_WITH_CUSTOMLANGUAGE = 'Suffix {0} not allowed with customLanguage {1}'
+TASKLIST_TITLE_NOT_FOUND = 'Task list title not found'
+THREAD = 'thread'
+THREADS = 'threads'
+TO = 'To'
+TO_LC = 'to'
+TO_MAXIMUM_OF = 'to maximum of'
+TO_SET_UP_GOOGLE_CHAT = """
+To set up Google Chat for your API project, please go to:
+
+    {0}
+
+and complete all fields.
+"""
+TOTAL_ITEMS_IN_ENTITY = 'Total {0} in {1}'
+TRIMMED_MESSAGE_FROM_LENGTH_TO_MAXIMUM = 'Trimmed message of length {0} to maximum length {1}'
+UNABLE_TO_GET_PERMISSION_ID = 'Unable to get Permission ID for <{0}>'
+UNABLE_TO_CREATE_NOT_FOUND_USER = 'Unable to create not found user, some required field (givenName, familyName, password/notfoundpassword) not present'
+UNAVAILABLE = 'Unavailable'
+UNKNOWN = 'Unknown'
+UNKNOWN_API_OR_VERSION = 'Unknown Google API or version: ({0}), contact {1}'
+UNRECOVERABLE_ERROR = 'Unrecoverable error'
+UPDATE_ATTENDEE_CHANGES = 'Update attendee changes'
+UPDATE_GAM_TO_64BIT = "You're running a 32-bit version of GAM on a 64-bit version of Windows, upgrade to a windows-x86_64 version of GAM"
+UPDATE_USER_PASSWORD_CHANGE_NOTIFY_MESSAGE = 'The account password for #givenname# #familyname#, #user# has been changed to: #password#\n'
+UPDATE_USER_PASSWORD_CHANGE_NOTIFY_SUBJECT = 'Account #user# password has been changed'
+UPLOAD_CSV_FILE_INTERNAL_ERROR = 'Google reported "{0}" but the file was probably uploaded, check that it has {1} rows'
+UPLOADING_NEW_PUBLIC_CERTIFICATE_TO_GOOGLE = 'Uploading new public certificate to Google...\n'
+URL_ERROR = 'URL error: {0}'
+USE_MIMETYPE_TO_SPECIFY_GOOGLE_FORMAT = 'Use "mimetype <MimeType>" to specify Google file format\n'
+USED = 'Used'
+USER_BELONGS_TO_N_GROUPS_THAT_MAP_TO_ORGUNITS = 'User belongs to {0} groups ({1}) that map to OUs'
+USER_CANCELLED = 'User cancelled'
+USER_HAS_MULTIPLE_DIRECT_OR_INHERITED_MEMBERSHIPS_IN_GROUP = 'User has multiple direct or inherited memberships in group'
+USER_IN_OTHER_DOMAIN = '{0}: {1} in other domain.'
+USER_IS_NOT_ORGANIZER = 'User is not organizer, use anyorganizer option to override'
+USER_NOT_IN_MATCHUSERS = 'User not in matchusers'
+USER_SUBS_NOT_ALLOWED_TAG_REPLACEMENT = 'user substitutions not allowed in replace <Tag> <String>'
+USE_DOIT_ARGUMENT_TO_PERFORM_ACTION = 'Use the "doit" argument to perform action'
+USING_N_PROCESSES = '{0},0/{1},Using {2} {3}...\n'
+VALUES_ARE_NOT_CONSISTENT = 'Values are not consistent'
+VERSION_UPDATE_AVAILABLE = 'Version update available'
+WAITING_FOR_DATA_TRANSFER_TO_COMPLETE_SLEEPING = 'Waiting for Data Transfer to complete. Sleeping {0} seconds\n'
+WAITING_FOR_ITEM_CREATION_TO_COMPLETE_SLEEPING = 'Waiting for {0} creation to complete. Sleeping {1} seconds\n'
+WHAT_IS_YOUR_PROJECT_ID = '\nWhat is your project ID? '
+WILL_RERUN_WITH_NO_BROWSER_TRUE = 'Will re-run command with no_browser true\n'
+WITH = 'with'
+WOULD_MAKE_MEMBERSHIP_CYCLE = 'Would make membership cycle'
+WRITER_ACCESS_REQUIRED_TO_BOTH_CALENDARS = 'Writer access required to both calendars'
+WROTE_PRIVATE_KEY_DATA = 'Wrote private key data to {0}\n'
+WROTE_PUBLIC_CERTIFICATE = 'Wrote public certificate to {0}\n'
+YOU_CAN_ADD_DOMAIN_TO_ACCOUNT = 'You can now add: {0} or its subdomains as secondary or domain aliases of the Google Workspace Account: {1}'
+YUBIKEY_GENERATING_NONEXPORTABLE_PRIVATE_KEY = 'YubiKey is generating a non-exportable private key...\n'
+YUBIKEY_PIN_SET_TO = 'YubiKey PIN set to: {0}\n'

src/gam/gamlib/glskus.py

@@ -0,0 +1,246 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2023 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""Google SKUs
+
+"""
+
+# Products/SKUs
+_PRODUCTS = {
+  '101001': 'Cloud Identity Free',
+  '101005': 'Cloud Identity Premium',
+  '101031': 'Google Workspace for Education',
+  '101033': 'Google Voice',
+  '101034': 'Google Workspace Archived User',
+  '101035': 'Cloud Search',
+  '101036': 'Google Meet Global Dialing',
+  '101037': 'Google Workspace for Education',
+  '101038': 'AppSheet',
+  '101039': 'Assured Controls',
+  '101040': 'Chrome Enterprise',
+  '101043': 'Google Workspace Additional Storage',
+  '101047': 'Gemini',
+  '101049': 'Education Endpoint Management',
+  '101050': 'Colab',
+  'Google-Apps': 'Google Workspace',
+  'Google-Chrome-Device-Management': 'Google Chrome Device Management',
+  'Google-Drive-storage': 'Google Drive Storage',
+  'Google-Vault': 'Google Vault',
+  }
+_SKUS = {
+  '1010010001': {
+    'product': '101001', 'aliases': ['identity', 'cloudidentity'], 'displayName': 'Cloud Identity'},
+  '1010050001': {
+    'product': '101005', 'aliases': ['identitypremium', 'cloudidentitypremium'], 'displayName': 'Cloud Identity Premium'},
+  '1010310002': {
+    'product': '101031', 'aliases': ['gsefe', 'e4e', 'gsuiteenterpriseeducation'], 'displayName': 'Google Workspace for Education Plus - Legacy'},
+  '1010310003': {
+    'product': '101031', 'aliases': ['gsefes', 'e4es', 'gsuiteenterpriseeducationstudent'], 'displayName': 'Google Workspace for Education Plus - Legacy (Student)'},
+  '1010310005': {
+    'product': '101031', 'aliases': ['gwes', 'workspaceeducationstandard'], 'displayName': 'Google Workspace for Education Standard'},
+  '1010310006': {
+    'product': '101031', 'aliases': ['gwesstaff', 'workspaceeducationstandardstaff'], 'displayName': 'Google Workspace for Education Standard (Staff)'},
+  '1010310007': {
+    'product': '101031', 'aliases': ['gwesstudent', 'workspaceeducationstandardstudent'], 'displayName': 'Google Workspace for Education Standard (Extra Student)'},
+  '1010310008': {
+    'product': '101031', 'aliases': ['gwep', 'workspaceeducationplus'], 'displayName': 'Google Workspace for Education Plus'},
+  '1010310009': {
+    'product': '101031', 'aliases': ['gwepstaff', 'workspaceeducationplusstaff'], 'displayName': 'Google Workspace for Education Plus (Staff)'},
+  '1010310010': {
+    'product': '101031', 'aliases': ['gwepstudent', 'workspaceeducationplusstudent'], 'displayName': 'Google Workspace for Education Plus (Extra Student)'},
+  '1010330002': {
+    'product': '101033', 'aliases': ['gvpremier', 'voicepremier', 'googlevoicepremier'], 'displayName': 'Google Voice Premier'},
+  '1010330003': {
+    'product': '101033', 'aliases': ['gvstarter', 'voicestarter', 'googlevoicestarter'], 'displayName': 'Google Voice Starter'},
+  '1010330004': {
+    'product': '101033', 'aliases': ['gvstandard', 'voicestandard', 'googlevoicestandard'], 'displayName': 'Google Voice Standard'},
+  '1010350001': {
+    'product': '101035', 'aliases': ['cloudsearch'], 'displayName': 'Cloud Search'},
+  '1010360001': {
+    'product': '101036', 'aliases': ['meetdialing','googlemeetglobaldialing'], 'displayName': 'Google Meet Global Dialing'},
+  '1010370001': {
+    'product': '101037', 'aliases': ['gwetlu', 'workspaceeducationupgrade'], 'displayName': 'Google Workspace for Education: Teaching and Learning Upgrade'},
+  '1010380001': {
+    'product': '101038', 'aliases': ['appsheetcore'], 'displayName': 'AppSheet Core'},
+  '1010380002': {
+    'product': '101038', 'aliases': ['appsheetstandard', 'appsheetenterprisestandard'], 'displayName': 'AppSheet Enterprise Standard'},
+  '1010380003': {
+    'product': '101038', 'aliases': ['appsheetplus', 'appsheetenterpriseplus'], 'displayName': 'AppSheet Enterprise Plus'},
+  '1010390001': {
+    'product': '101039', 'aliases': ['assuredcontrols'], 'displayName': 'Assured Controls'},
+  '1010400001': {
+    'product': '101040', 'aliases': ['beyondcorp', 'beyondcorpenterprise', 'bce', 'cep', 'chromeenterprisepremium'], 'displayName': 'Chrome Enterprise Premium'},
+  '1010430001': {
+    'product': '101043', 'aliases': ['gwas', 'plusstorage'], 'displayName': 'Google Workspace Additional Storage'},
+  '1010470001': {
+    'product': '101047', 'aliases': ['geminient', 'duetai'], 'displayName': 'Gemini Enterprise'},
+  '1010470002': {
+    'product': '101047', 'aliases': ['gwlabs', 'workspacelabs'], 'displayName': 'Google Workspace Labs'},
+  '1010470003': {
+    'product': '101047', 'aliases': ['geminibiz'], 'displayName': 'Gemini Business'},
+  '1010470004': {
+    'product': '101047', 'aliases': ['geminiedu'], 'displayName': 'Gemini Education'},
+  '1010470005': {
+    'product': '101047', 'aliases': ['geminiedupremium'], 'displayName': 'Gemini Education Premium'},
+  '1010470006': {
+    'product': '101047', 'aliases': ['aisecurity'], 'displayName': 'AI Security'},
+  '1010470007': {
+    'product': '101047', 'aliases': ['aimeetingsandmessaging'], 'displayName': 'AI Meetings and Messaging'},
+  '1010490001': {
+    'product': '101049', 'aliases': ['eeu'], 'displayName': 'Endpoint Education Upgrade'},
+  '1010500001': {
+    'product': '101050', 'aliases': ['colabpro'], 'displayName': 'Colab Pro'},
+  '1010500002': {
+    'product': '101050', 'aliases': ['colabpro+', 'colabproplus'], 'displayName': 'Colab Pro+'},
+  'Google-Apps': {
+    'product': 'Google-Apps', 'aliases': ['standard', 'free'], 'displayName': 'G Suite Legacy'},
+  'Google-Apps-For-Business': {
+    'product': 'Google-Apps', 'aliases': ['gafb', 'gafw', 'basic', 'gsuitebasic'], 'displayName': 'G Suite Basic'},
+  'Google-Apps-For-Government': {
+    'product': 'Google-Apps', 'aliases': ['gafg', 'gsuitegovernment', 'gsuitegov'], 'displayName': 'Google Workspace Government'},
+  'Google-Apps-For-Postini': {
+    'product': 'Google-Apps', 'aliases': ['gams', 'postini', 'gsuitegams', 'gsuitepostini', 'gsuitemessagesecurity'], 'displayName': 'Google Apps Message Security'},
+  'Google-Apps-Lite': {
+    'product': 'Google-Apps', 'aliases': ['gal', 'gsl', 'lite', 'gsuitelite'], 'displayName': 'G Suite Lite'},
+  'Google-Apps-Unlimited': {
+    'product': 'Google-Apps', 'aliases': ['gau', 'gsb', 'unlimited', 'gsuitebusiness'], 'displayName': 'G Suite Business'},
+  '1010020020': {
+    'product': 'Google-Apps', 'aliases': ['gae', 'gse', 'enterprise', 'gsuiteenterprise',
+                                          'wsentplus', 'workspaceenterpriseplus'], 'displayName': 'Google Workspace Enterprise Plus'},
+  '1010020025': {
+    'product': 'Google-Apps', 'aliases': ['wsbizplus', 'workspacebusinessplus'], 'displayName': 'Google Workspace Business Plus'},
+  '1010020026': {
+    'product': 'Google-Apps', 'aliases': ['wsentstan', 'workspaceenterprisestandard'], 'displayName': 'Google Workspace Enterprise Standard'},
+  '1010020027': {
+    'product': 'Google-Apps', 'aliases': ['wsbizstart', 'wsbizstarter', 'workspacebusinessstarter'], 'displayName': 'Google Workspace Business Starter'},
+  '1010020028': {
+    'product': 'Google-Apps', 'aliases': ['wsbizstan', 'workspacebusinessstandard'], 'displayName': 'Google Workspace Business Standard'},
+  '1010020029': {
+    'product': 'Google-Apps', 'aliases': ['wes', 'wsentstarter', 'workspaceenterprisestarter'], 'displayName': 'Workspace Enterprise Starter'},
+  '1010020030': {
+    'product': 'Google-Apps', 'aliases': ['wsflw', 'workspacefrontline', 'workspacefrontlineworker'], 'displayName': 'Google Workspace Frontline Starter'},
+  '1010020031': {
+    'product': 'Google-Apps', 'aliases': ['wsflwstan', 'workspacefrontlinestan', 'workspacefrontlineworkerstan'], 'displayName': 'Google Workspace Frontline Standard'},
+  '1010340001': {
+    'product': '101034', 'aliases': ['gseau', 'enterprisearchived', 'gsuiteenterprisearchived'], 'displayName': 'Google Workspace Enterprise Plus - Archived User'},
+  '1010340002': {
+    'product': '101034', 'aliases': ['gsbau', 'businessarchived', 'gsuitebusinessarchived'], 'displayName': 'Google Workspace Business - Archived User'},
+  '1010340003': {
+    'product': '101034', 'aliases': ['wsbizplusarchived', 'workspacebusinessplusarchived'], 'displayName': 'Google Workspace Business Plus - Archived User'},
+  '1010340004': {
+    'product': '101034', 'aliases': ['wsentstanarchived', 'workspaceenterprisestandardarchived'], 'displayName': 'Google Workspace Enterprise Standard - Archived User'},
+  '1010340005': {
+    'product': '101034', 'aliases': ['wsbizstarterarchived', 'workspacebusinessstarterarchived'], 'displayName': 'Google Workspace Business Starter - Archived User'},
+  '1010340006': {
+    'product': '101034', 'aliases': ['wsbizstanarchived', 'workspacebusinessstanarchived'], 'displayName': 'Google Workspace Business Standard - Archived User'},
+  '1010060001': {
+    'product': '101006', 'aliases': ['gsuiteessentials', 'essentials',
+                                     'd4e', 'driveenterprise', 'drive4enterprise',
+                                     'wsess', 'workspaceesentials'], 'displayName': 'Google Workspace Essentials'},
+  '1010060003': {
+    'product': 'Google-Apps', 'aliases': ['wsentess', 'workspaceenterpriseessentials'], 'displayName': 'Google Workspace Enterprise Essentials'},
+  '1010060005': {
+    'product': 'Google-Apps', 'aliases': ['wsessplus', 'workspaceessentialsplus'], 'displayName': 'Google Workspace Essentials Plus'},
+  'Google-Drive-storage-20GB': {
+    'product': 'Google-Drive-storage', 'aliases': ['drive20gb', '20gb', 'googledrivestorage20gb'], 'displayName': 'Google Drive Storage 20GB'},
+  'Google-Drive-storage-50GB': {
+    'product': 'Google-Drive-storage', 'aliases': ['drive50gb', '50gb', 'googledrivestorage50gb'], 'displayName': 'Google Drive Storage 50GB'},
+  'Google-Drive-storage-200GB': {
+    'product': 'Google-Drive-storage', 'aliases': ['drive200gb', '200gb', 'googledrivestorage200gb'], 'displayName': 'Google Drive Storage 200GB'},
+  'Google-Drive-storage-400GB': {
+    'product': 'Google-Drive-storage', 'aliases': ['drive400gb', '400gb', 'googledrivestorage400gb'], 'displayName': 'Google Drive Storage 400GB'},
+  'Google-Drive-storage-1TB': {
+    'product': 'Google-Drive-storage', 'aliases': ['drive1tb', '1tb', 'googledrivestorage1tb'], 'displayName': 'Google Drive Storage 1TB'},
+  'Google-Drive-storage-2TB': {
+    'product': 'Google-Drive-storage', 'aliases': ['drive2tb', '2tb', 'googledrivestorage2tb'], 'displayName': 'Google Drive Storage 2TB'},
+  'Google-Drive-storage-4TB': {
+    'product': 'Google-Drive-storage', 'aliases': ['drive4tb', '4tb', 'googledrivestorage4tb'], 'displayName': 'Google Drive Storage 4TB'},
+  'Google-Drive-storage-8TB': {
+    'product': 'Google-Drive-storage', 'aliases': ['drive8tb', '8tb', 'googledrivestorage8tb'], 'displayName': 'Google Drive Storage 8TB'},
+  'Google-Drive-storage-16TB': {
+    'product': 'Google-Drive-storage', 'aliases': ['drive16tb', '16tb', 'googledrivestorage16tb'], 'displayName': 'Google Drive Storage 16TB'},
+  'Google-Vault': {
+    'product': 'Google-Vault', 'aliases': ['vault', 'googlevault'], 'displayName': 'Google Vault'},
+  'Google-Vault-Former-Employee': {
+    'product': 'Google-Vault', 'aliases': ['vfe', 'googlevaultformeremployee'], 'displayName': 'Google Vault - Former Employee'},
+  'Google-Chrome-Device-Management': {
+    'product': 'Google-Chrome-Device-Management', 'aliases': ['chrome', 'cdm', 'googlechromedevicemanagement'], 'displayName': 'Google Chrome Device Management'}
+  }
+
+ARCHIVABLE_SKUS = {'1010020020', '1010020025', '1010020026', '1010020027', '1010020028', 'Google-Apps-Unlimited'}
+
+def getProductAndSKU(sku):
+  l_sku = sku.lower().replace('-', '').replace(' ', '').replace('"', '').replace("'", '').strip()
+  if l_sku.startswith('nv:'):
+    if ':' in sku[3:]:
+      return sku[3:].split(':', 1)
+    return (None, sku)
+  for a_sku, sku_values in list(_SKUS.items()):
+    if ((l_sku == a_sku.lower().replace('-', '')) or
+        (l_sku in sku_values['aliases']) or
+        (l_sku == sku_values['displayName'].lower().replace(' ', ''))):
+      return (sku_values['product'], a_sku)
+  return (None, sku)
+
+def productIdToDisplayName(productId):
+  return _PRODUCTS.get(productId, productId)
+
+def formatProductIdDisplayName(productId):
+  productIdDisplay = productIdToDisplayName(productId)
+  if productId == productIdDisplay:
+    return productId
+  return f'{productId} ({productIdDisplay})'
+
+def normalizeProductId(product):
+  l_product = product.lower().replace('-', '').replace(' ', '').strip()
+  if l_product.startswith('nv:'):
+    return (True, product[3:])
+  for a_sku, sku_values in list(_SKUS.items()):
+    if ((l_product == sku_values['product'].lower().replace('-', '')) or
+        (l_product == a_sku.lower().replace('-', '')) or
+        (l_product in sku_values['aliases']) or
+        (l_product == sku_values['displayName'].lower().replace(' ', ''))):
+      return (True, sku_values['product'])
+  return (False, product)
+
+def getSortedProductList():
+  return sorted(_PRODUCTS)
+
+def skuIdToDisplayName(skuId):
+  return _SKUS[skuId]['displayName'] if skuId in _SKUS else skuId
+
+def formatSKUIdDisplayName(skuId):
+  skuIdDisplay = skuIdToDisplayName(skuId)
+  if skuId == skuIdDisplay:
+    return skuId
+  return f'{skuId} ({skuIdDisplay})'
+
+def getSortedSKUList():
+  return sorted(_SKUS)
+
+def convertProductListToSKUList(productList):
+  skuList = []
+  for productId in productList:
+    skuList += [(productId, skuId) for skuId in _SKUS if _SKUS[skuId]['product'] == productId]
+  return skuList
+
+def getAllSKUs():
+  return convertProductListToSKUList(sorted(_PRODUCTS))
+
+def getGSuiteSKUs():
+  return convertProductListToSKUList(['Google-Apps', '101031'])

src/gam/gamlib/gluprop.py

@@ -0,0 +1,279 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2023 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""GAM user properties
+
+"""
+
+# notes
+# a|b|c
+# getKeywordAttribute(CUSTOM_TYPE_NOCUSTOM, attrdict)
+
+#CUSTOM_TYPE_NOCUSTOM = {
+#    PTKW_CL_TYPE_KEYWORD: 'type',
+#    PTKW_CL_CUSTOM_KEYWORD: None,
+#    PTKW_ATTR_TYPE_KEYWORD: 'type',
+#    PTKW_ATTR_TYPE_CUSTOM_VALUE: None,
+#    PTKW_ATTR_CUSTOMTYPE_KEYWORD: None,
+#    PTKW_KEYWORD_LIST: ['a', 'b', 'c']
+#    }
+
+# addresses, ims
+# type a|b|c|([custom] <String>)
+# getChoice([CUSTOM_TYPE_CUSTOM[PTKW_CL_TYPE_KEYWORD]])
+# getKeywordAttribute(CUSTOM_TYPE_CUSTOM, attrdict)
+
+# emails, externalids, relations, websites
+# [type] a|b|c|([custom] <String>)
+# getChoice([CUSTOM_TYPE_CUSTOM[PTKW_CL_TYPE_KEYWORD]], defaultChoice=None)
+# getKeywordAttribute(CUSTOM_TYPE_IMPLICIT, attrdict)
+
+# locations, phones
+# type a|b|c|([custom] <String>)
+# if argument == CUSTOM_TYPE_CUSTOM[PTKW_CL_TYPE_KEYWORD]:
+#   getKeywordAttribute(CUSTOM_TYPE_CUSTOM, attrdict)
+
+#CUSTOM_TYPE_CUSTOM = {
+#    PTKW_CL_TYPE_KEYWORD: 'type',
+#    PTKW_CL_CUSTOM_KEYWORD: 'custom',
+#    PTKW_ATTR_TYPE_KEYWORD: 'type',
+#    PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom',
+#    PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+#    PTKW_KEYWORD_LIST: ['custom', 'a', 'b', 'c']
+#    }
+
+# organizations
+# (type a|b|c|([custom] <String>)) | (custom_type <String>)
+# if argument == CUSTOM_TYPE_DIFFERENT_KEYWORD[PTKW_CL_TYPE_KEYWORD]:
+#   getKeywordAttribute(CUSTOM_TYPE_DIFFERENT_KEYWORD, attrdict)
+# elif argument == CUSTOM_TYPE_DIFFERENT_KEYWORD[PTKW_CL_CUSTOM_KEYWORD]:
+#   attrdict[CUSTOM_TYPE_DIFFERENT_KEYWORD[PTKW_ATTR_CUSTOMTYPE_KEYWORD]] = getValue()
+
+#CUSTOM_TYPE_DIFFERENT_KEYWORD = {
+#    PTKW_CL_TYPE_KEYWORD: 'type',
+#    PTKW_CL_CUSTOM_KEYWORD: 'custom',
+#    PTKW_CL_CUSTOMTYPE_KEYWORD: 'custom_type',
+#    PTKW_ATTR_TYPE_KEYWORD: 'type',
+#    PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom',
+#    PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+#    PTKW_KEYWORD_LIST: ['custom', 'a', 'b', 'c']
+#    }
+
+# Keys into USER_PROPERTIES
+CLASS = 'clas'
+TITLE = 'titl'
+TYPE_KEYWORDS = 'tykw'
+PTKW_CL_TYPE_KEYWORD = 'ctkw'
+PTKW_CL_CUSTOM_KEYWORD = 'ccuk'
+PTKW_CL_CUSTOMTYPE_KEYWORD = 'cctk'
+PTKW_ATTR_TYPE_KEYWORD = 'atkw'
+PTKW_ATTR_TYPE_CUSTOM_VALUE = 'atcv'
+PTKW_ATTR_CUSTOMTYPE_KEYWORD = 'actk'
+PTKW_KEYWORD_LIST = 'kwli'
+#
+PC_ADDRESSES = 'addr'
+PC_ALIASES = 'alia'
+PC_ARRAY = 'arry'
+PC_BOOLEAN = 'bool'
+PC_EMAILS = 'emai'
+PC_GENDER = 'gndr'
+PC_IMS = 'ims '
+PC_LANGUAGES = 'lang'
+PC_LOCATIONS = 'loca'
+PC_NAME = 'name'
+PC_NOTES = 'note'
+PC_ORGANIZATIONS = 'orga'
+PC_POSIX = 'posi'
+PC_SCHEMAS = 'schm'
+PC_SSH = 'ssh '
+PC_STRING = 'stri'
+PC_TIME = 'time'
+
+PROPERTIES = {
+  'primaryEmail':
+    {CLASS: PC_STRING, TITLE: 'User',},
+  'name':
+    {CLASS: PC_NAME, TITLE: 'Name',},
+  'givenName':
+    {CLASS: PC_STRING, TITLE: 'First Name',},
+  'familyName':
+    {CLASS: PC_STRING, TITLE: 'Last Name',},
+  'fullName':
+    {CLASS: PC_STRING, TITLE: 'Full Name',},
+  'displayName':
+    {CLASS: PC_STRING, TITLE: 'Display Name',},
+  'languages':
+    {CLASS: PC_LANGUAGES, TITLE: 'Languages',},
+  'languageCode':
+    {CLASS: PC_LANGUAGES, TITLE: 'Languages',},
+  'customLanguage':
+    {CLASS: PC_LANGUAGES, TITLE: 'Custom Languages',},
+  'password':
+    {CLASS: PC_STRING, TITLE: 'Password',},
+  'hashFunction':
+    {CLASS: PC_STRING, TITLE: 'Hash Function',},
+  'isAdmin':
+    {CLASS: PC_BOOLEAN, TITLE: 'Is a Super Admin',},
+  'isDelegatedAdmin':
+    {CLASS: PC_BOOLEAN, TITLE: 'Is Delegated Admin',},
+  'isEnrolledIn2Sv':
+    {CLASS: PC_BOOLEAN, TITLE: '2-step enrolled',},
+  'isEnforcedIn2Sv':
+    {CLASS: PC_BOOLEAN, TITLE: '2-step enforced',},
+  'agreedToTerms':
+    {CLASS: PC_BOOLEAN, TITLE: 'Has Agreed to Terms',},
+  'ipWhitelisted':
+    {CLASS: PC_BOOLEAN, TITLE: 'IP Whitelisted',},
+  'archived':
+    {CLASS: PC_BOOLEAN, TITLE: 'Is Archived',},
+  'suspended':
+    {CLASS: PC_BOOLEAN, TITLE: 'Account Suspended',},
+  'suspensionReason':
+    {CLASS: PC_STRING, TITLE: 'Suspension Reason',},
+  'changePasswordAtNextLogin':
+    {CLASS: PC_BOOLEAN, TITLE: 'Must Change Password',},
+  'recoveryEmail':
+    {CLASS: PC_STRING, TITLE: 'Recovery Email',},
+  'recoveryPhone':
+    {CLASS: PC_STRING, TITLE: 'Recovery Phone',},
+  'id':
+    {CLASS: PC_STRING, TITLE: 'Google Unique ID',},
+  'customerId':
+    {CLASS: PC_STRING, TITLE: 'Customer ID',},
+  'isMailboxSetup':
+    {CLASS: PC_BOOLEAN, TITLE: 'Mailbox is setup',},
+  'includeInGlobalAddressList':
+    {CLASS: PC_BOOLEAN, TITLE: 'Included in GAL',},
+  'creationTime':
+    {CLASS: PC_TIME, TITLE: 'Creation Time',},
+  'lastLoginTime':
+    {CLASS: PC_TIME, TITLE: 'Last login time',},
+  'deletionTime':
+    {CLASS: PC_TIME, TITLE: 'Deletion Time',},
+  'orgUnitPath':
+    {CLASS: PC_STRING, TITLE: 'Google Org Unit Path',},
+  'thumbnailPhotoUrl':
+    {CLASS: PC_STRING, TITLE: 'Photo URL',},
+  'addresses':
+    {CLASS: PC_ADDRESSES, TITLE: 'Addresses',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'custom',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+        PTKW_KEYWORD_LIST: ['custom', 'home', 'other', 'work'],},},
+  'emails':
+    {CLASS: PC_EMAILS, TITLE: 'Other Emails',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'custom',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+        PTKW_KEYWORD_LIST: ['custom', 'home', 'other', 'work'],},},
+  'externalIds':
+    {CLASS: PC_ARRAY, TITLE: 'External IDs',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'custom',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+        PTKW_KEYWORD_LIST: ['custom', 'account', 'customer', 'login_id', 'network', 'organization'],},},
+  'gender':
+    {CLASS: PC_GENDER, TITLE: 'Gender',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'other',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'other', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customGender',
+        PTKW_KEYWORD_LIST: ['male', 'female', 'other', 'unknown'],},},
+  'ims':
+    {CLASS: PC_IMS, TITLE: 'IMs',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'custom',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+        PTKW_KEYWORD_LIST: ['custom', 'home', 'other', 'work'],},},
+  'keywords':
+    {CLASS: PC_ARRAY, TITLE: 'Keywords',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'custom',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+        PTKW_KEYWORD_LIST: ['custom', 'mission', 'occupation', 'outlook'],},},
+  'locations':
+    {CLASS: PC_LOCATIONS, TITLE: 'Locations',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'custom',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+        PTKW_KEYWORD_LIST: ['custom', 'default', 'desk'],},},
+  'notes':
+    {CLASS: PC_NOTES, TITLE: 'Notes',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: None,
+        PTKW_ATTR_TYPE_KEYWORD: 'contentType', PTKW_ATTR_TYPE_CUSTOM_VALUE: None, PTKW_ATTR_CUSTOMTYPE_KEYWORD: None,
+        PTKW_KEYWORD_LIST: ['text_plain', 'text_html'],},},
+  'organizations':
+    {CLASS: PC_ORGANIZATIONS, TITLE: 'Organizations',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'custom', PTKW_CL_CUSTOMTYPE_KEYWORD: 'customtype',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: None, PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+        PTKW_KEYWORD_LIST: ['custom', 'domain_only', 'school', 'unknown', 'work'],},},
+  'phones':
+    {CLASS: PC_ARRAY, TITLE: 'Phones',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'custom',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+        PTKW_KEYWORD_LIST: ['custom', 'home', 'work', 'other',
+                            'home_fax', 'work_fax', 'other_fax',
+                            'mobile', 'pager',
+                            'company_main', 'assistant',
+                            'car', 'radio', 'isdn', 'callback',
+                            'telex', 'tty_tdd', 'work_mobile',
+                            'work_pager', 'main', 'grand_central'],},},
+  'posixAccounts':
+    {CLASS: PC_POSIX, TITLE: 'Posix Accounts',},
+  'relations':
+    {CLASS: PC_ARRAY, TITLE: 'Relations',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'custom',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+        PTKW_KEYWORD_LIST: ['custom', 'spouse', 'child', 'mother',
+                            'father', 'parent', 'brother',
+                            'sister', 'friend', 'relative',
+                            'domestic_partner', 'partner',
+                            'manager', 'dotted_line_manager',
+                            'assistant', 'admin_assistant', 'exec_assistant',
+                            'referred_by'],},},
+  'sshPublicKeys':
+    {CLASS: PC_SSH, TITLE: 'SSH Public Keys',},
+  'websites':
+    {CLASS: PC_ARRAY, TITLE: 'Websites',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'custom',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+        PTKW_KEYWORD_LIST: ['custom', 'home', 'work',
+                            'home_page', 'ftp', 'blog',
+                            'profile', 'other', 'reservations',
+                            'app_install_page', 'resume'],},},
+  'customSchemas':
+    {CLASS: PC_SCHEMAS, TITLE: 'Custom Schemas',
+     TYPE_KEYWORDS:
+       {PTKW_CL_TYPE_KEYWORD: 'type', PTKW_CL_CUSTOM_KEYWORD: 'custom',
+        PTKW_ATTR_TYPE_KEYWORD: 'type', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customType',
+        PTKW_KEYWORD_LIST: ['custom', 'home', 'other', 'work'],},},
+  'aliases': {
+    CLASS: PC_ALIASES, TITLE: 'Email Aliases',},
+  'nonEditableAliases': {
+    CLASS: PC_ALIASES, TITLE: 'NonEditable Aliases',},
+  }
+#
+IM_PROTOCOLS = {
+  PTKW_CL_TYPE_KEYWORD: 'protocol', PTKW_CL_CUSTOM_KEYWORD: 'custom_protocol',
+  PTKW_ATTR_TYPE_KEYWORD: 'protocol', PTKW_ATTR_TYPE_CUSTOM_VALUE: 'custom_protocol', PTKW_ATTR_CUSTOMTYPE_KEYWORD: 'customProtocol',
+  PTKW_KEYWORD_LIST: ['custom_protocol', 'aim', 'gtalk', 'icq', 'jabber', 'msn', 'net_meeting', 'qq', 'skype', 'xmpp', 'yahoo']
+  }

src/gam/gamlib/glverlibs.py

@@ -0,0 +1,33 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2023 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""GAM extended library versions
+
+"""
+
+GAM_VER_LIBS = ['cryptography',
+                'filelock',
+                'google-api-python-client',
+                'google-auth-httplib2',
+                'google-auth-oauthlib',
+                'google-auth',
+                'httplib2',
+                'passlib',
+                'python-dateutil',
+                'yubikey-manager',
+                ]

src/gam/gamlib/yubikey.py

@@ -0,0 +1,202 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2023 Ross Scroggs All Rights Reserved.
+#
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""YubiKey"""
+
+import base64
+import datetime
+from secrets import SystemRandom
+import string
+import sys
+
+from cryptography.hazmat.primitives import hashes, serialization
+from cryptography.hazmat.primitives.asymmetric import padding
+from smartcard.Exceptions import CardConnectionException
+from ykman.device import list_all_devices
+from ykman.piv import generate_self_signed_certificate, generate_chuid
+from yubikit.piv import DEFAULT_MANAGEMENT_KEY, \
+                        InvalidPinError, \
+                        KEY_TYPE, \
+                        MANAGEMENT_KEY_TYPE, \
+                        PIN_POLICY, \
+                        PivSession, \
+                        OBJECT_ID, \
+                        SLOT, \
+                        TOUCH_POLICY
+from yubikit.core.smartcard import ApduError, SmartCardConnection
+
+YUBIKEY_CONNECTION_ERROR_RC = 80
+YUBIKEY_INVALID_KEY_TYPE_RC = 81
+YUBIKEY_INVALID_SLOT_RC = 82
+YUBIKEY_INVALID_PIN_RC = 83
+YUBIKEY_APDU_ERROR_RC = 84
+YUBIKEY_VALUE_ERROR_RC = 85
+YUBIKEY_MULTIPLE_CONNECTED_RC = 86
+YUBIKEY_NOT_FOUND_RC = 87
+
+from gam import mplock
+
+from gam import systemErrorExit
+from gam import readStdin
+from gam import writeStdout
+
+from gam.gamlib import glmsgs as Msg
+
+PIN_PUK_CHARS = string.ascii_letters+string.digits+string.punctuation
+
+class YubiKey():
+
+  def __init__(self, service_account_info=None):
+    self.key_type = None
+    self.slot = None
+    self.serial_number = None
+    self.pin = None
+    self.key_id = None
+    if service_account_info:
+      key_type = service_account_info.get('yubikey_key_type', 'RSA2048')
+      try:
+        self.key_type = getattr(KEY_TYPE, key_type.upper())
+      except AttributeError:
+        systemErrorExit(YUBIKEY_INVALID_KEY_TYPE_RC, f'{key_type} is not a valid value for yubikey_key_type')
+      slot = service_account_info.get('yubikey_slot', 'AUTHENTICATION')
+      try:
+        self.slot = getattr(SLOT, slot.upper())
+      except AttributeError:
+        systemErrorExit(YUBIKEY_INVALID_SLOT_RC, f'{slot} is not a valid value for yubikey_slot')
+      self.serial_number = service_account_info.get('yubikey_serial_number')
+      self.pin = service_account_info.get('yubikey_pin')
+      self.key_id = service_account_info.get('private_key_id')
+
+  def _connect(self):
+    try:
+      devices = list_all_devices()
+      for (device, info) in devices:
+        if info.serial == self.serial_number:
+          return device.open_connection(SmartCardConnection)
+    except CardConnectionException as err:
+      systemErrorExit(YUBIKEY_CONNECTION_ERROR_RC, f'YubiKey - {err}')
+
+  def get_certificate(self):
+    try:
+      conn = self._connect()
+      with conn:
+        session = PivSession(conn)
+        if self.pin:
+          try:
+            session.verify_pin(self.pin)
+          except InvalidPinError as err:
+            systemErrorExit(YUBIKEY_INVALID_PIN_RC, f'YubiKey - {err}')
+        try:
+          cert = session.get_certificate(self.slot)
+        except ApduError as err:
+          systemErrorExit(YUBIKEY_APDU_ERROR_RC, f'YubiKey - {err}')
+      cert_pem = cert.public_bytes(serialization.Encoding.PEM).decode()
+      publicKeyData = base64.b64encode(cert_pem.encode())
+      if isinstance(publicKeyData, bytes):
+        publicKeyData = publicKeyData.decode()
+      return publicKeyData
+    except ValueError as err:
+      systemErrorExit(YUBIKEY_VALUE_ERROR_RC, f'YubiKey - {err}')
+    except TypeError as err:
+      systemErrorExit(YUBIKEY_NOT_FOUND_RC, f'YubiKey - {err} - {Msg.IS_YUBIKEY_INSERTED}')
+
+  def get_serial_number(self):
+    try:
+      devices = list_all_devices()
+      if not devices:
+        systemErrorExit(YUBIKEY_NOT_FOUND_RC, Msg.COULD_NOT_FIND_ANY_YUBIKEY)
+      if self.serial_number:
+        for (_, info) in devices:
+          if info.serial == self.serial_number:
+            return info.serial
+        systemErrorExit(YUBIKEY_NOT_FOUND_RC, Msg.COULD_NOT_FIND_YUBIKEY_WITH_SERIAL.format(self.serial_number))
+      if len(devices) > 1:
+        serials = ', '.join([str(info.serial) for (_, info) in devices])
+        systemErrorExit(YUBIKEY_MULTIPLE_CONNECTED_RC, Msg.MULTIPLE_YUBIKEYS_CONNECTED.format(serials))
+      return devices[0][1].serial
+    except ValueError as err:
+      systemErrorExit(YUBIKEY_VALUE_ERROR_RC, f'YubiKey - {err}')
+
+  def reset_piv(self):
+    '''Resets YubiKey PIV app and generates new key for GAM to use.'''
+    reply = str(readStdin(Msg.CONFIRM_WIPE_YUBIKEY_PIV).lower().strip())
+    if reply != 'y':
+      sys.exit(1)
+    try:
+      conn = self._connect()
+      with conn:
+        piv = PivSession(conn)
+        piv.reset()
+        rnd = SystemRandom()
+        new_puk = ''.join(rnd.choice(PIN_PUK_CHARS) for _ in range(8))
+        new_pin = ''.join(rnd.choice(PIN_PUK_CHARS) for _ in range(8))
+        piv.change_puk('12345678', new_puk)
+        piv.change_pin('123456', new_pin)
+        writeStdout(Msg.YUBIKEY_PIN_SET_TO.format(new_pin))
+        piv.authenticate(MANAGEMENT_KEY_TYPE.TDES, DEFAULT_MANAGEMENT_KEY)
+        piv.verify_pin(new_pin)
+        writeStdout(Msg.YUBIKEY_GENERATING_NONEXPORTABLE_PRIVATE_KEY)
+        pubkey = piv.generate_key(SLOT.AUTHENTICATION,
+                                  KEY_TYPE.RSA2048,
+                                  PIN_POLICY.ALWAYS,
+                                  TOUCH_POLICY.NEVER)
+        now = datetime.datetime.utcnow()
+        valid_to = now + datetime.timedelta(days=36500)
+        subject = 'CN=GAM Created Key'
+        piv.authenticate(MANAGEMENT_KEY_TYPE.TDES, DEFAULT_MANAGEMENT_KEY)
+        piv.verify_pin(new_pin)
+        cert = generate_self_signed_certificate(piv,
+                                                SLOT.AUTHENTICATION,
+                                                pubkey,
+                                                subject,
+                                                now,
+                                                valid_to)
+        piv.put_certificate(SLOT.AUTHENTICATION, cert)
+        piv.put_object(OBJECT_ID.CHUID, generate_chuid())
+    except ValueError as err:
+      systemErrorExit(YUBIKEY_VALUE_ERROR_RC, f'YubiKey - {err}')
+    except TypeError as err:
+      systemErrorExit(YUBIKEY_NOT_FOUND_RC, f'YubiKey - {err} - {Msg.IS_YUBIKEY_INSERTED}')
+
+  def sign(self, message):
+    if mplock is not None:
+      mplock.acquire()
+    try:
+      conn = self._connect()
+      with conn:
+        session = PivSession(conn)
+        if self.pin:
+          try:
+            session.verify_pin(self.pin)
+          except InvalidPinError as err:
+            systemErrorExit(YUBIKEY_INVALID_PIN_RC, f'YubiKey - {err}')
+        try:
+          signed = session.sign(slot=self.slot,
+                                key_type=self.key_type,
+                                message=message,
+                                hash_algorithm=hashes.SHA256(),
+                                padding=padding.PKCS1v15())
+        except ApduError as err:
+          systemErrorExit(YUBIKEY_APDU_ERROR_RC, f'YubiKey - {err}')
+    except ValueError as err:
+      systemErrorExit(YUBIKEY_VALUE_ERROR_RC, f'YubiKey - {err}')
+    except TypeError as err:
+      systemErrorExit(YUBIKEY_NOT_FOUND_RC, f'YubiKey - {err} - {Msg.IS_YUBIKEY_INSERTED}')
+    if mplock is not None:
+      mplock.release()
+    return signed

src/gam/gapi/__init__.py

@@ -1,349 +0,0 @@
-"""Methods related to execution of GAPI requests."""
-
-import sys
-
-import googleapiclient.errors
-import google.auth.exceptions
-import httplib2
-
-from gam import controlflow
-from gam import display
-from gam.gapi import errors
-from gam import transport
-from gam.var import (GM_Globals, GM_CURRENT_API_SCOPES, GM_CURRENT_API_USER,
-                     GM_EXTRA_ARGS_DICT, GM_OAUTH2SERVICE_ACCOUNT_CLIENT_ID,
-                     MAX_RESULTS_API_EXCEPTIONS, MESSAGE_API_ACCESS_CONFIG,
-                     MESSAGE_API_ACCESS_DENIED, MESSAGE_SERVICE_NOT_APPLICABLE)
-
-
-def call(service,
-         function,
-         silent_errors=False,
-         soft_errors=False,
-         throw_reasons=None,
-         retry_reasons=None,
-         **kwargs):
-    """Executes a single request on a Google service function.
-
-  Args:
-    service: A Google service object for the desired API.
-    function: String, The name of a service request method to execute.
-    silent_errors: Bool, If True, error messages are suppressed when
-      encountered.
-    soft_errors: Bool, If True, writes non-fatal errors to stderr.
-    throw_reasons: A list of Google HTTP error reason strings indicating the
-      errors generated by this request should be re-thrown. All other HTTP
-      errors are consumed.
-    retry_reasons: A list of Google HTTP error reason strings indicating which
-      error should be retried, using exponential backoff techniques, when the
-      error reason is encountered.
-    **kwargs: Additional params to pass to the request method.
-
-  Returns:
-    A response object for the corresponding Google API call.
-  """
-    if throw_reasons is None:
-        throw_reasons = []
-    if retry_reasons is None:
-        retry_reasons = []
-
-    method = getattr(service, function)
-    retries = 10
-    parameters = dict(
-        list(kwargs.items()) + list(GM_Globals[GM_EXTRA_ARGS_DICT].items()))
-    for n in range(1, retries + 1):
-        try:
-            return method(**parameters).execute()
-        except googleapiclient.errors.HttpError as e:
-            http_status, reason, message = errors.get_gapi_error_detail(
-                e,
-                soft_errors=soft_errors,
-                silent_errors=silent_errors,
-                retry_on_http_error=n < 3)
-            if http_status == -1:
-                # The error detail indicated that we should retry this request
-                # We'll refresh credentials and make another pass
-                service._http.credentials.refresh(transport.create_http())
-                continue
-            if http_status == 0:
-                return None
-
-            is_known_error_reason = reason in [
-                r.value for r in errors.ErrorReason
-            ]
-            if is_known_error_reason and errors.ErrorReason(
-                    reason) in throw_reasons:
-                if errors.ErrorReason(
-                        reason) in errors.ERROR_REASON_TO_EXCEPTION:
-                    raise errors.ERROR_REASON_TO_EXCEPTION[errors.ErrorReason(
-                        reason)](message)
-                raise e
-            if (n != retries) and (is_known_error_reason and errors.ErrorReason(
-                    reason) in errors.DEFAULT_RETRY_REASONS + retry_reasons):
-                controlflow.wait_on_failure(n, retries, reason)
-                continue
-            if soft_errors:
-                display.print_error(
-                    f'{http_status}: {message} - {reason}{["", ": Giving up."][n > 1]}'
-                )
-                return None
-            controlflow.system_error_exit(
-                int(http_status), f'{http_status}: {message} - {reason}')
-        except google.auth.exceptions.RefreshError as e:
-            handle_oauth_token_error(
-                e, soft_errors or
-                errors.ErrorReason.SERVICE_NOT_AVAILABLE in throw_reasons)
-            if errors.ErrorReason.SERVICE_NOT_AVAILABLE in throw_reasons:
-                raise errors.GapiServiceNotAvailableError(str(e))
-            display.print_error(
-                f'User {GM_Globals[GM_CURRENT_API_USER]}: {str(e)}')
-            return None
-        except ValueError as e:
-            if hasattr(service._http,
-                       'cache') and service._http.cache is not None:
-                service._http.cache = None
-                continue
-            controlflow.system_error_exit(4, str(e))
-        except (httplib2.ServerNotFoundError, RuntimeError) as e:
-            if n != retries:
-                service._http.connections = {}
-                controlflow.wait_on_failure(n, retries, str(e))
-                continue
-            controlflow.system_error_exit(4, str(e))
-        except TypeError as e:
-            controlflow.system_error_exit(4, str(e))
-
-
-def get_items(service,
-              function,
-              items='items',
-              throw_reasons=None,
-              retry_reasons=None,
-              **kwargs):
-    """Gets a single page of items from a Google service function that is paged.
-
-  Args:
-    service: A Google service object for the desired API.
-    function: String, The name of a service request method to execute.
-    items: String, the name of the resulting "items" field within the service
-      method's response object.
-    throw_reasons: A list of Google HTTP error reason strings indicating the
-      errors generated by this request should be re-thrown. All other HTTP
-      errors are consumed.
-    retry_reasons: A list of Google HTTP error reason strings indicating which
-      error should be retried, using exponential backoff techniques, when the
-      error reason is encountered.
-    **kwargs: Additional params to pass to the request method.
-
-  Returns:
-    The list of items in the first page of a response.
-  """
-    results = call(service,
-                   function,
-                   throw_reasons=throw_reasons,
-                   retry_reasons=retry_reasons,
-                   **kwargs)
-    if results:
-        return results.get(items, [])
-    return []
-
-
-def _get_max_page_size_for_api_call(service, function, **kwargs):
-    """Gets the maximum number of results supported for a single API call.
-
-  Args:
-    service: A Google service object for the desired API.
-    function: String, The name of the service method to check for max page size.
-    **kwargs: Additional params that will be passed to the request method.
-
-  Returns:
-    Int, A value from discovery if it exists, otherwise value from
-        MAX_RESULTS_API_EXCEPTIONS, otherwise None
-  """
-    method = getattr(service, function)
-    api_id = method(**kwargs).methodId
-    for resource in service._rootDesc.get('resources', {}).values():
-        for a_method in resource.get('methods', {}).values():
-            if a_method.get('id') == api_id:
-                if not a_method.get('parameters') or a_method['parameters'].get(
-                        'pageSize'
-                ) or not a_method['parameters'].get('maxResults'):
-                    # Make sure API call supports maxResults. For now we don't care to
-                    # set pageSize since all known pageSize API calls have
-                    # default pageSize == max pageSize.
-                    return None
-                known_api_max = MAX_RESULTS_API_EXCEPTIONS.get(api_id)
-                max_results = a_method['parameters']['maxResults'].get(
-                    'maximum', known_api_max)
-                return {'maxResults': max_results}
-
-    return None
-
-
-TOTAL_ITEMS_MARKER = '%%total_items%%'
-FIRST_ITEM_MARKER = '%%first_item%%'
-LAST_ITEM_MARKER = '%%last_item%%'
-
-
-def got_total_items_msg(items, eol):
-    """Format a page_message to be used by get_all_pages
-
-  The page message indicates the number of items returned
-
-  Args:
-    items: String, the description of the items being returned by get_all_pages
-    eol: String, the line terminator
-       Values used: '', '...', '\n', '...\n'
-
-  Returns:
-    The formatted page_message
-  """
-
-    return f'Got {TOTAL_ITEMS_MARKER} {items}{eol}'
-
-
-def got_total_items_first_last_msg(items):
-    """Format a page_message to be used by get_all_pages
-
-  The page message indicates the number of items returned and the
-  value of the first and list items
-
-  Args:
-    items: String, the description of the items being returned by get_all_pages
-
-  Returns:
-    The formatted page_message
-  """
-
-    return f'Got {TOTAL_ITEMS_MARKER} {items}: {FIRST_ITEM_MARKER} - {LAST_ITEM_MARKER}' + '\n'
-
-
-def get_all_pages(service,
-                  function,
-                  items='items',
-                  page_message=None,
-                  message_attribute=None,
-                  soft_errors=False,
-                  throw_reasons=None,
-                  retry_reasons=None,
-                  **kwargs):
-    """Aggregates and returns all pages of a Google service function response.
-
-  All pages of items are aggregated and returned as a single list.
-
-  Args:
-    service: A Google service object for the desired API.
-    function: String, The name of a service request method to execute.
-    items: String, the name of the resulting "items" field within the method's
-      response object. The items in this field will be aggregated across all
-      pages and returned.
-    page_message: String, a message to be displayed to the user during paging.
-      Template strings allow for dynamic content to be inserted during paging.
-        Supported template strings:
-          TOTAL_ITEMS_MARKER : The current number of items discovered across all
-            pages.
-          FIRST_ITEM_MARKER  : In conjunction with `message_attribute` arg, will
-            display a unique property of the first item in the current page.
-          LAST_ITEM_MARKER   : In conjunction with `message_attribute` arg, will
-            display a unique property of the last item in the current page.
-    message_attribute: String or list, the name of a signature field within a
-    single returned item which identifies that unique item. This field is used
-    with `page_message` to templatize a paging status message.
-    soft_errors: Bool, If True, writes non-fatal errors to stderr.
-    throw_reasons: A list of Google HTTP error reason strings indicating the
-      errors generated by this request should be re-thrown. All other HTTP
-      errors are consumed.
-    retry_reasons: A list of Google HTTP error reason strings indicating which
-      error should be retried, using exponential backoff techniques, when the
-      error reason is encountered.
-    **kwargs: Additional params to pass to the request method.
-
-  Returns:
-    A list of all items received from all paged responses.
-  """
-    if 'maxResults' not in kwargs and 'pageSize' not in kwargs:
-        page_key = _get_max_page_size_for_api_call(service, function, **kwargs)
-        if page_key:
-            kwargs.update(page_key)
-    all_items = []
-    page_token = None
-    total_items = 0
-    while True:
-        page = call(service,
-                    function,
-                    soft_errors=soft_errors,
-                    throw_reasons=throw_reasons,
-                    retry_reasons=retry_reasons,
-                    pageToken=page_token,
-                    **kwargs)
-        if page:
-            page_token = page.get('nextPageToken')
-            page_items = page.get(items, [])
-            num_page_items = len(page_items)
-            total_items += num_page_items
-            all_items.extend(page_items)
-        else:
-            page_token = None
-            num_page_items = 0
-
-        # Show a paging message to the user that indicates paging progress
-        if page_message:
-            show_message = page_message.replace(TOTAL_ITEMS_MARKER,
-                                                str(total_items))
-            if message_attribute:
-                first_item = page_items[0] if num_page_items > 0 else {}
-                last_item = page_items[-1] if num_page_items > 1 else first_item
-                if type(message_attribute) is str:
-                    first_item = str(first_item.get(message_attribute, ''))
-                    last_item = str(last_item.get(message_attribute, ''))
-                else:
-                    for attr in message_attribute:
-                        first_item = first_item.get(attr, {})
-                        last_item = last_item.get(attr, {})
-                    first_item = str(first_item)
-                    last_item = str(last_item)
-                show_message = show_message.replace(FIRST_ITEM_MARKER, first_item)
-                show_message = show_message.replace(LAST_ITEM_MARKER, last_item)
-            sys.stderr.write('\r')
-            sys.stderr.flush()
-            sys.stderr.write(show_message)
-
-        if not page_token:
-            # End the paging status message and return all items.
-            if page_message and (page_message[-1] != '\n'):
-                sys.stderr.write('\r\n')
-                sys.stderr.flush()
-            return all_items
-
-
-# TODO: Make this private once all execution related items that use this method
-# have been brought into this file
-def handle_oauth_token_error(e, soft_errors):
-    """On a token error, exits the application and writes a message to stderr.
-
-  Args:
-    e: google.auth.exceptions.RefreshError, The error to handle.
-    soft_errors: Boolean, if True, suppresses any applicable errors and instead
-      returns to the caller.
-  """
-    token_error = str(e).replace('.', '')
-    if token_error in errors.OAUTH2_TOKEN_ERRORS or e.startswith(
-            'Invalid response'):
-        if soft_errors:
-            return
-        if not GM_Globals[GM_CURRENT_API_USER]:
-            display.print_error(
-                MESSAGE_API_ACCESS_DENIED.format(
-                    GM_Globals[GM_OAUTH2SERVICE_ACCOUNT_CLIENT_ID],
-                    ','.join(GM_Globals[GM_CURRENT_API_SCOPES])))
-            controlflow.system_error_exit(12, MESSAGE_API_ACCESS_CONFIG)
-        else:
-            controlflow.system_error_exit(
-                19,
-                MESSAGE_SERVICE_NOT_APPLICABLE.format(
-                    GM_Globals[GM_CURRENT_API_USER]))
-    controlflow.system_error_exit(18, f'Authentication Token Error - {str(e)}')
-
-
-def get_enum_values_minus_unspecified(values):
-    return [a_type for a_type in values if '_UNSPECIFIED' not in a_type]

src/gam/gapi/__init___test.py

@@ -1,519 +0,0 @@
-"""Tests for gapi."""
-
-import json
-import unittest
-from unittest.mock import MagicMock
-from unittest.mock import patch
-
-from gam import SetGlobalVariables
-import gam.gapi as gapi
-from gam.gapi import errors
-
-
-def create_http_error(status, reason, message):
-    """Creates a HttpError object similar to most Google API Errors.
-
-  Args:
-    status: Int, the error's HTTP response status number.
-    reason: String, a camelCase reason for the HttpError being given.
-    message: String, a general error message describing the error that occurred.
-
-  Returns:
-    googleapiclient.errors.HttpError
-  """
-    response = {
-        'status': status,
-        'content-type': 'application/json',
-    }
-    content = {
-        'error': {
-            'code': status,
-            'errors': [{
-                'reason': str(reason),
-                'message': message,
-            }]
-        }
-    }
-    content_bytes = json.dumps(content).encode('UTF-8')
-    return gapi.googleapiclient.errors.HttpError(response, content_bytes)
-
-
-class GapiTest(unittest.TestCase):
-
-    def setUp(self):
-        SetGlobalVariables()
-        self.mock_service = MagicMock()
-        self.mock_method_name = 'mock_method'
-        self.mock_method = getattr(self.mock_service, self.mock_method_name)
-
-        self.simple_3_page_response = [
-            {
-                'items': [{
-                    'position': 'page1,item1'
-                }, {
-                    'position': 'page1,item2'
-                }, {
-                    'position': 'page1,item3'
-                }],
-                'nextPageToken': 'page2'
-            },
-            {
-                'items': [{
-                    'position': 'page2,item1'
-                }, {
-                    'position': 'page2,item2'
-                }, {
-                    'position': 'page2,item3'
-                }],
-                'nextPageToken': 'page3'
-            },
-            {
-                'items': [{
-                    'position': 'page3,item1'
-                }, {
-                    'position': 'page3,item2'
-                }, {
-                    'position': 'page3,item3'
-                }],
-            },
-        ]
-        self.empty_items_response = {'items': []}
-
-        super(GapiTest, self).setUp()
-
-    def test_call_returns_basic_200_response(self):
-        response = gapi.call(self.mock_service, self.mock_method_name)
-        self.assertEqual(response, self.mock_method().execute.return_value)
-
-    def test_call_passes_target_method_params(self):
-        gapi.call(self.mock_service,
-                  self.mock_method_name,
-                  my_param_1=1,
-                  my_param_2=2)
-        self.assertEqual(self.mock_method.call_count, 1)
-        method_kwargs = self.mock_method.call_args[1]
-        self.assertEqual(method_kwargs.get('my_param_1'), 1)
-        self.assertEqual(method_kwargs.get('my_param_2'), 2)
-
-    @patch.object(gapi.errors, 'get_gapi_error_detail')
-    def test_call_retries_with_soft_errors(self, mock_error_detail):
-        mock_error_detail.return_value = (-1, 'aReason', 'some message')
-
-        # Make the request fail first, then return the proper response on the retry.
-        fake_http_error = create_http_error(403, 'aReason', 'unused message')
-        fake_200_response = MagicMock()
-        self.mock_method.return_value.execute.side_effect = [
-            fake_http_error, fake_200_response
-        ]
-
-        response = gapi.call(self.mock_service,
-                             self.mock_method_name,
-                             soft_errors=True)
-        self.assertEqual(response, fake_200_response)
-        self.assertEqual(self.mock_service._http.credentials.refresh.call_count,
-                         1)
-        self.assertEqual(self.mock_method.return_value.execute.call_count, 2)
-
-    def test_call_throws_for_provided_reason(self):
-        throw_reason = errors.ErrorReason.USER_NOT_FOUND
-        fake_http_error = create_http_error(404, throw_reason, 'forced throw')
-        self.mock_method.return_value.execute.side_effect = fake_http_error
-
-        gam_exception = errors.ERROR_REASON_TO_EXCEPTION[throw_reason]
-        with self.assertRaises(gam_exception):
-            gapi.call(self.mock_service,
-                      self.mock_method_name,
-                      throw_reasons=[throw_reason])
-
-    # Prevent wait_on_failure from performing actual backoff unnecessarily, since
-    # we're not actually testing over a network connection
-    @patch.object(gapi.controlflow, 'wait_on_failure')
-    def test_call_retries_request_for_default_retry_reasons(
-        self, mock_wait_on_failure):
-
-        # Test using one of the default retry reasons
-        default_throw_reason = errors.ErrorReason.BACKEND_ERROR
-        self.assertIn(default_throw_reason, errors.DEFAULT_RETRY_REASONS)
-
-        fake_http_error = create_http_error(404, default_throw_reason,
-                                            'message')
-        fake_200_response = MagicMock()
-        # Fail once, then succeed on retry
-        self.mock_method.return_value.execute.side_effect = [
-            fake_http_error, fake_200_response
-        ]
-
-        response = gapi.call(self.mock_service,
-                             self.mock_method_name,
-                             retry_reasons=[])
-        self.assertEqual(response, fake_200_response)
-        self.assertEqual(self.mock_method.return_value.execute.call_count, 2)
-        # Make sure a backoff technique was used for retry.
-        self.assertEqual(mock_wait_on_failure.call_count, 1)
-
-    # Prevent wait_on_failure from performing actual backoff unnecessarily, since
-    # we're not actually testing over a network connection
-    @patch.object(gapi.controlflow, 'wait_on_failure')
-    def test_call_retries_requests_for_provided_retry_reasons(
-        self, unused_mock_wait_on_failure):
-
-        retry_reason1 = errors.ErrorReason.INTERNAL_ERROR
-        fake_retrieable_error1 = create_http_error(400, retry_reason1,
-                                                   'Forced Error 1')
-        retry_reason2 = errors.ErrorReason.SYSTEM_ERROR
-        fake_retrieable_error2 = create_http_error(400, retry_reason2,
-                                                   'Forced Error 2')
-        non_retriable_reason = errors.ErrorReason.SERVICE_NOT_AVAILABLE
-        fake_non_retriable_error = create_http_error(
-            400, non_retriable_reason,
-            'This error should not cause the request to be retried')
-        # Fail once, then succeed on retry
-        self.mock_method.return_value.execute.side_effect = [
-            fake_retrieable_error1, fake_retrieable_error2,
-            fake_non_retriable_error
-        ]
-
-        with self.assertRaises(SystemExit):
-            # The third call should raise the SystemExit when non_retriable_error is
-            # raised.
-            gapi.call(self.mock_service,
-                      self.mock_method_name,
-                      retry_reasons=[retry_reason1, retry_reason2])
-
-        self.assertEqual(self.mock_method.return_value.execute.call_count, 3)
-
-    def test_call_exits_on_oauth_token_error(self):
-        # An error with any OAUTH2_TOKEN_ERROR
-        fake_token_error = gapi.google.auth.exceptions.RefreshError(
-            errors.OAUTH2_TOKEN_ERRORS[0])
-        self.mock_method.return_value.execute.side_effect = fake_token_error
-
-        with self.assertRaises(SystemExit):
-            gapi.call(self.mock_service, self.mock_method_name)
-
-    def test_call_exits_on_nonretriable_error(self):
-        error_reason = 'unknownReason'
-        fake_http_error = create_http_error(500, error_reason,
-                                            'Testing unretriable errors')
-        self.mock_method.return_value.execute.side_effect = fake_http_error
-
-        with self.assertRaises(SystemExit):
-            gapi.call(self.mock_service, self.mock_method_name)
-
-    def test_call_exits_on_request_valueerror(self):
-        self.mock_method.return_value.execute.side_effect = ValueError()
-
-        with self.assertRaises(SystemExit):
-            gapi.call(self.mock_service, self.mock_method_name)
-
-    def test_call_clears_bad_http_cache_on_request_failure(self):
-        self.mock_service._http.cache = 'something that is not None'
-        fake_200_response = MagicMock()
-        self.mock_method.return_value.execute.side_effect = [
-            ValueError(), fake_200_response
-        ]
-
-        self.assertIsNotNone(self.mock_service._http.cache)
-        response = gapi.call(self.mock_service, self.mock_method_name)
-        self.assertEqual(response, fake_200_response)
-        # Assert the cache was cleared
-        self.assertIsNone(self.mock_service._http.cache)
-
-    # Prevent wait_on_failure from performing actual backoff unnecessarily, since
-    # we're not actually testing over a network connection
-    @patch.object(gapi.controlflow, 'wait_on_failure')
-    def test_call_retries_requests_with_backoff_on_servernotfounderror(
-        self, mock_wait_on_failure):
-        fake_servernotfounderror = gapi.httplib2.ServerNotFoundError()
-        fake_200_response = MagicMock()
-        # Fail once, then succeed on retry
-        self.mock_method.return_value.execute.side_effect = [
-            fake_servernotfounderror, fake_200_response
-        ]
-
-        http_connections = self.mock_service._http.connections
-        response = gapi.call(self.mock_service, self.mock_method_name)
-        self.assertEqual(response, fake_200_response)
-        # HTTP cached connections should be cleared on receiving this error
-        self.assertNotEqual(http_connections,
-                            self.mock_service._http.connections)
-        self.assertEqual(self.mock_method.return_value.execute.call_count, 2)
-        # Make sure a backoff technique was used for retry.
-        self.assertEqual(mock_wait_on_failure.call_count, 1)
-
-    def test_get_items_calls_correct_service_function(self):
-        gapi.get_items(self.mock_service, self.mock_method_name)
-        self.assertTrue(self.mock_method.called)
-
-    def test_get_items_returns_one_page(self):
-        fake_response = {'items': [{}, {}, {}]}
-        self.mock_method.return_value.execute.return_value = fake_response
-        page = gapi.get_items(self.mock_service, self.mock_method_name)
-        self.assertEqual(page, fake_response['items'])
-
-    def test_get_items_non_default_page_field_name(self):
-        field_name = 'things'
-        fake_response = {field_name: [{}, {}, {}]}
-        self.mock_method.return_value.execute.return_value = fake_response
-        page = gapi.get_items(self.mock_service,
-                              self.mock_method_name,
-                              items=field_name)
-        self.assertEqual(page, fake_response[field_name])
-
-    def test_get_items_passes_additional_kwargs_to_service(self):
-        gapi.get_items(self.mock_service,
-                       self.mock_method_name,
-                       my_param_1=1,
-                       my_param_2=2)
-        self.assertEqual(self.mock_method.call_count, 1)
-        method_kwargs = self.mock_method.call_args[1]
-        self.assertEqual(1, method_kwargs.get('my_param_1'))
-        self.assertEqual(2, method_kwargs.get('my_param_2'))
-
-    def test_get_items_returns_empty_list_when_no_items_returned(self):
-        non_items_response = {'noItemsInThisResponse': {}}
-        self.mock_method.return_value.execute.return_value = non_items_response
-        page = gapi.get_items(self.mock_service, self.mock_method_name)
-        self.assertIsInstance(page, list)
-        self.assertEqual(0, len(page))
-
-    def test_get_all_pages_returns_all_items(self):
-        page_1 = {'items': ['1-1', '1-2', '1-3'], 'nextPageToken': '2'}
-        page_2 = {'items': ['2-1', '2-2', '2-3'], 'nextPageToken': '3'}
-        page_3 = {'items': ['3-1', '3-2', '3-3']}
-        self.mock_method.return_value.execute.side_effect = [
-            page_1, page_2, page_3
-        ]
-        response_items = gapi.get_all_pages(self.mock_service,
-                                            self.mock_method_name)
-        self.assertListEqual(
-            response_items, page_1['items'] + page_2['items'] + page_3['items'])
-
-    def test_get_all_pages_includes_next_pagetoken_in_request(self):
-        page_1 = {'items': ['1-1', '1-2', '1-3'], 'nextPageToken': 'someToken'}
-        page_2 = {'items': ['2-1', '2-2', '2-3']}
-        self.mock_method.return_value.execute.side_effect = [page_1, page_2]
-
-        gapi.get_all_pages(self.mock_service,
-                           self.mock_method_name,
-                           pageSize=100)
-        self.assertEqual(self.mock_method.call_count, 2)
-        call_2_kwargs = self.mock_method.call_args_list[1][1]
-        self.assertIn('pageToken', call_2_kwargs)
-        self.assertEqual(call_2_kwargs['pageToken'], page_1['nextPageToken'])
-
-    def test_get_all_pages_uses_default_max_page_size(self):
-        sample_api_id = list(gapi.MAX_RESULTS_API_EXCEPTIONS.keys())[0]
-        sample_api_max_results = gapi.MAX_RESULTS_API_EXCEPTIONS[sample_api_id]
-        self.mock_method.return_value.methodId = sample_api_id
-        self.mock_service._rootDesc = {
-            'resources': {
-                'someResource': {
-                    'methods': {
-                        'someMethod': {
-                            'id': sample_api_id,
-                            'parameters': {
-                                'maxResults': {
-                                    'maximum': sample_api_max_results
-                                }
-                            }
-                        }
-                    }
-                }
-            }
-        }
-        self.mock_method.return_value.execute.return_value = self.empty_items_response
-
-        gapi.get_all_pages(self.mock_service, self.mock_method_name)
-        request_method_kwargs = self.mock_method.call_args[1]
-        self.assertIn('maxResults', request_method_kwargs)
-        self.assertEqual(request_method_kwargs['maxResults'],
-                         gapi.MAX_RESULTS_API_EXCEPTIONS.get(sample_api_id))
-
-    def test_get_all_pages_max_page_size_overrided(self):
-        self.mock_method.return_value.execute.return_value = self.empty_items_response
-
-        gapi.get_all_pages(self.mock_service,
-                           self.mock_method_name,
-                           pageSize=123456)
-        request_method_kwargs = self.mock_method.call_args[1]
-        self.assertIn('pageSize', request_method_kwargs)
-        self.assertEqual(123456, request_method_kwargs['pageSize'])
-
-    def test_get_all_pages_prints_paging_message(self):
-        self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
-
-        paging_message = 'A simple string displayed during paging'
-        with patch.object(gapi.sys.stderr, 'write') as mock_write:
-            gapi.get_all_pages(self.mock_service,
-                               self.mock_method_name,
-                               page_message=paging_message)
-        messages_written = [
-            call_args[0][0] for call_args in mock_write.call_args_list
-        ]
-        self.assertIn(paging_message, messages_written)
-
-    def test_get_all_pages_prints_paging_message_inline(self):
-        self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
-
-        paging_message = 'A simple string displayed during paging'
-        with patch.object(gapi.sys.stderr, 'write') as mock_write:
-            gapi.get_all_pages(self.mock_service,
-                               self.mock_method_name,
-                               page_message=paging_message)
-        messages_written = [
-            call_args[0][0] for call_args in mock_write.call_args_list
-        ]
-
-        # Make sure a return carriage was written between two pages
-        paging_message_call_positions = [
-            i for i, message in enumerate(messages_written)
-            if message == paging_message
-        ]
-        self.assertGreater(len(paging_message_call_positions), 1)
-        printed_between_page_messages = messages_written[
-            paging_message_call_positions[0]:paging_message_call_positions[1]]
-        self.assertIn('\r', printed_between_page_messages)
-
-    def test_get_all_pages_ends_paging_message_with_newline(self):
-        self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
-
-        paging_message = 'A simple string displayed during paging'
-        with patch.object(gapi.sys.stderr, 'write') as mock_write:
-            gapi.get_all_pages(self.mock_service,
-                               self.mock_method_name,
-                               page_message=paging_message)
-        messages_written = [
-            call_args[0][0] for call_args in mock_write.call_args_list
-        ]
-        last_page_message_index = len(
-            messages_written) - messages_written[::-1].index(paging_message)
-        last_carriage_return_index = len(
-            messages_written) - messages_written[::-1].index('\r\n')
-        self.assertGreater(last_carriage_return_index, last_page_message_index)
-
-    def test_get_all_pages_prints_attribute_total_items_in_paging_message(self):
-        self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
-
-        paging_message = 'Total number of items discovered: %%total_items%%'
-        with patch.object(gapi.sys.stderr, 'write') as mock_write:
-            gapi.get_all_pages(self.mock_service,
-                               self.mock_method_name,
-                               page_message=paging_message)
-
-        messages_written = [
-            call_args[0][0] for call_args in mock_write.call_args_list
-        ]
-        page_1_item_count = len(self.simple_3_page_response[0]['items'])
-        page_1_message = paging_message.replace('%%total_items%%',
-                                                str(page_1_item_count))
-        self.assertIn(page_1_message, messages_written)
-
-        page_2_item_count = len(self.simple_3_page_response[1]['items'])
-        page_2_message = paging_message.replace(
-            '%%total_items%%', str(page_1_item_count + page_2_item_count))
-        self.assertIn(page_2_message, messages_written)
-
-        page_3_item_count = len(self.simple_3_page_response[2]['items'])
-        page_3_message = paging_message.replace(
-            '%%total_items%%',
-            str(page_1_item_count + page_2_item_count + page_3_item_count))
-        self.assertIn(page_3_message, messages_written)
-
-        # Assert that the template text is always replaced.
-        for message in messages_written:
-            self.assertNotIn('%%total_items', message)
-
-    def test_get_all_pages_prints_attribute_first_item_in_paging_message(self):
-        self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
-
-        paging_message = 'First item in page: %%first_item%%'
-
-        with patch.object(gapi.sys.stderr, 'write') as mock_write:
-            gapi.get_all_pages(self.mock_service,
-                               self.mock_method_name,
-                               page_message=paging_message,
-                               message_attribute='position')
-
-        messages_written = [
-            call_args[0][0] for call_args in mock_write.call_args_list
-        ]
-        page_1_message = paging_message.replace(
-            '%%first_item%%',
-            self.simple_3_page_response[0]['items'][0]['position'])
-        self.assertIn(page_1_message, messages_written)
-
-        page_2_message = paging_message.replace(
-            '%%first_item%%',
-            self.simple_3_page_response[1]['items'][0]['position'])
-        self.assertIn(page_2_message, messages_written)
-
-        # Assert that the template text is always replaced.
-        for message in messages_written:
-            self.assertNotIn('%%first_item', message)
-
-    def test_get_all_pages_prints_attribute_last_item_in_paging_message(self):
-        self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
-
-        paging_message = 'Last item in page: %%last_item%%'
-        with patch.object(gapi.sys.stderr, 'write') as mock_write:
-            gapi.get_all_pages(self.mock_service,
-                               self.mock_method_name,
-                               page_message=paging_message,
-                               message_attribute='position')
-
-        messages_written = [
-            call_args[0][0] for call_args in mock_write.call_args_list
-        ]
-        page_1_message = paging_message.replace(
-            '%%last_item%%',
-            self.simple_3_page_response[0]['items'][-1]['position'])
-        self.assertIn(page_1_message, messages_written)
-
-        page_2_message = paging_message.replace(
-            '%%last_item%%',
-            self.simple_3_page_response[1]['items'][-1]['position'])
-        self.assertIn(page_2_message, messages_written)
-
-        # Assert that the template text is always replaced.
-        for message in messages_written:
-            self.assertNotIn('%%last_item', message)
-
-    def test_get_all_pages_prints_all_attributes_in_paging_message(self):
-        pass
-
-    def test_get_all_pages_passes_additional_kwargs_to_service_method(self):
-        self.mock_method.return_value.execute.return_value = self.empty_items_response
-        gapi.get_all_pages(self.mock_service,
-                           self.mock_method_name,
-                           my_param_1=1,
-                           my_param_2=2)
-        method_kwargs = self.mock_method.call_args[1]
-        self.assertEqual(method_kwargs.get('my_param_1'), 1)
-        self.assertEqual(method_kwargs.get('my_param_2'), 2)
-
-    @patch.object(gapi, 'call')
-    def test_get_all_pages_passes_throw_and_retry_reasons(self, mock_call):
-        throw_for = MagicMock()
-        retry_for = MagicMock()
-        mock_call.return_value = self.empty_items_response
-        gapi.get_all_pages(self.mock_service,
-                           self.mock_method_name,
-                           throw_reasons=throw_for,
-                           retry_reasons=retry_for)
-        method_kwargs = mock_call.call_args[1]
-        self.assertEqual(method_kwargs.get('throw_reasons'), throw_for)
-        self.assertEqual(method_kwargs.get('retry_reasons'), retry_for)
-
-    def test_get_all_pages_non_default_items_field_name(self):
-        field_name = 'things'
-        fake_response = {field_name: [{}, {}, {}]}
-        self.mock_method.return_value.execute.return_value = fake_response
-        page = gapi.get_all_pages(self.mock_service,
-                                  self.mock_method_name,
-                                  items=field_name)
-        self.assertEqual(page, fake_response[field_name])
-
-
-if __name__ == '__main__':
-    unittest.main()

src/gam/gapi/calendar.py

@@ -1,984 +0,0 @@
-import csv
-import sys
-import uuid
-
-# TODO: get rid of these hacks
-import gam
-from gam.var import *
-
-from gam import controlflow
-from gam import display
-from gam import fileutils
-from gam import gapi
-from gam import utils
-
-
-def normalizeCalendarId(calname, checkPrimary=False):
-    if checkPrimary and calname.lower() == 'primary':
-        return calname
-    if not GC_Values[GC_DOMAIN]:
-        GC_Values[GC_DOMAIN] = gam._getValueFromOAuth('hd')
-    return gam.convertUIDtoEmailAddress(calname,
-                                        email_types=['user', 'resource'])
-
-
-def buildCalendarGAPIObject(calname):
-    calendarId = normalizeCalendarId(calname)
-    return (calendarId, gam.buildGAPIServiceObject('calendar', calendarId))
-
-
-def buildCalendarDataGAPIObject(calname):
-    calendarId = normalizeCalendarId(calname)
-
-    # Try to impersonate the calendar owner. If we fail, fall back to using
-    # admin for authentication. Resource calendars cannot be impersonated,
-    # so we need to access them as the admin.
-    cal = None
-    if not calname.endswith('.calendar.google.com'):
-        cal = gam.buildGAPIServiceObject('calendar', calendarId, False)
-    if cal is None:
-        _, cal = buildCalendarGAPIObject(gam._get_admin_email())
-    return (calendarId, cal)
-
-
-def printShowACLs(csvFormat):
-    calendarId, cal = buildCalendarDataGAPIObject(sys.argv[2])
-    if not cal:
-        return
-    toDrive = False
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if csvFormat and myarg == 'todrive':
-            toDrive = True
-            i += 1
-        else:
-            action = ['showacl', 'printacl'][csvFormat]
-            message = f'gam calendar <email> {action}'
-            controlflow.invalid_argument_exit(sys.argv[i], message)
-    acls = gapi.get_all_pages(cal.acl(), 'list', 'items', calendarId=calendarId)
-    i = 0
-    if csvFormat:
-        titles = []
-        rows = []
-    else:
-        count = len(acls)
-    for rule in acls:
-        i += 1
-        if csvFormat:
-            row = utils.flatten_json(rule, None)
-            for key in row:
-                if key not in titles:
-                    titles.append(key)
-            rows.append(row)
-        else:
-            formatted_acl = formatACLRule(rule)
-            current_count = display.current_count(i, count)
-            print(
-                f'Calendar: {calendarId}, ACL: {formatted_acl}{current_count}')
-    if csvFormat:
-        display.write_csv_file(rows, titles, f'{calendarId} Calendar ACLs',
-                               toDrive)
-
-
-def _getCalendarACLScope(i, body):
-    body['scope'] = {}
-    myarg = sys.argv[i].lower()
-    body['scope']['type'] = myarg
-    i += 1
-    if myarg in ['user', 'group']:
-        body['scope']['value'] = gam.normalizeEmailAddressOrUID(sys.argv[i],
-                                                                noUid=True)
-        i += 1
-    elif myarg == 'domain':
-        if i < len(sys.argv) and \
-           sys.argv[i].lower().replace('_', '') != 'sendnotifications':
-            body['scope']['value'] = sys.argv[i].lower()
-            i += 1
-        else:
-            body['scope']['value'] = GC_Values[GC_DOMAIN]
-    elif myarg != 'default':
-        body['scope']['type'] = 'user'
-        body['scope']['value'] = gam.normalizeEmailAddressOrUID(myarg,
-                                                                noUid=True)
-    return i
-
-
-CALENDAR_ACL_ROLES_MAP = {
-    'editor': 'writer',
-    'freebusy': 'freeBusyReader',
-    'freebusyreader': 'freeBusyReader',
-    'owner': 'owner',
-    'read': 'reader',
-    'reader': 'reader',
-    'writer': 'writer',
-    'none': 'none',
-}
-
-
-def addACL(function):
-    calendarId, cal = buildCalendarDataGAPIObject(sys.argv[2])
-    if not cal:
-        return
-    myarg = sys.argv[4].lower().replace('_', '')
-    if myarg not in CALENDAR_ACL_ROLES_MAP:
-        controlflow.expected_argument_exit('Role',
-                                           ', '.join(CALENDAR_ACL_ROLES_MAP),
-                                           myarg)
-    body = {'role': CALENDAR_ACL_ROLES_MAP[myarg]}
-    i = _getCalendarACLScope(5, body)
-    sendNotifications = True
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'sendnotifications':
-            sendNotifications = gam.getBoolean(sys.argv[i + 1], myarg)
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(
-                sys.argv[i], f'gam calendar <email> {function.lower()}')
-    print(f'Calendar: {calendarId}, {function} ACL: {formatACLRule(body)}')
-    gapi.call(cal.acl(),
-              'insert',
-              calendarId=calendarId,
-              body=body,
-              sendNotifications=sendNotifications)
-
-
-def delACL():
-    calendarId, cal = buildCalendarDataGAPIObject(sys.argv[2])
-    if not cal:
-        return
-    if sys.argv[4].lower() == 'id':
-        ruleId = sys.argv[5]
-        print(f'Removing rights for {ruleId} to {calendarId}')
-        gapi.call(cal.acl(), 'delete', calendarId=calendarId, ruleId=ruleId)
-    else:
-        body = {'role': 'none'}
-        _getCalendarACLScope(5, body)
-        print(f'Calendar: {calendarId}, Delete ACL: {formatACLScope(body)}')
-        gapi.call(cal.acl(),
-                  'insert',
-                  calendarId=calendarId,
-                  body=body,
-                  sendNotifications=False)
-
-
-def wipeData():
-    calendarId, cal = buildCalendarDataGAPIObject(sys.argv[2])
-    if not cal:
-        return
-    gapi.call(cal.calendars(), 'clear', calendarId=calendarId)
-
-
-def printEvents():
-    calendarId, cal = buildCalendarDataGAPIObject(sys.argv[2])
-    if not cal:
-        return
-    q = showDeleted = showHiddenInvitations = timeMin = \
-        timeMax = timeZone = updatedMin = None
-    toDrive = False
-    titles = []
-    csvRows = []
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'query':
-            q = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'includedeleted':
-            showDeleted = True
-            i += 1
-        elif myarg == 'includehidden':
-            showHiddenInvitations = True
-            i += 1
-        elif myarg == 'after':
-            timeMin = utils.get_time_or_delta_from_now(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'before':
-            timeMax = utils.get_time_or_delta_from_now(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'timezone':
-            timeZone = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'updated':
-            updatedMin = utils.get_time_or_delta_from_now(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'todrive':
-            toDrive = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(
-                sys.argv[i], 'gam calendar <email> printevents')
-    page_message = gapi.got_total_items_msg(f'Events for {calendarId}', '')
-    results = gapi.get_all_pages(cal.events(),
-                                 'list',
-                                 'items',
-                                 page_message=page_message,
-                                 calendarId=calendarId,
-                                 q=q,
-                                 showDeleted=showDeleted,
-                                 showHiddenInvitations=showHiddenInvitations,
-                                 timeMin=timeMin,
-                                 timeMax=timeMax,
-                                 timeZone=timeZone,
-                                 updatedMin=updatedMin)
-    for result in results:
-        row = {'calendarId': calendarId}
-        display.add_row_titles_to_csv_file(
-            utils.flatten_json(result, flattened=row), csvRows, titles)
-    display.sort_csv_titles(['calendarId', 'id', 'summary', 'status'], titles)
-    display.write_csv_file(csvRows, titles, 'Calendar Events', toDrive)
-
-
-def formatACLScope(rule):
-    if rule['scope']['type'] != 'default':
-        return f'(Scope: {rule["scope"]["type"]}:{rule["scope"]["value"]})'
-    return f'(Scope: {rule["scope"]["type"]})'
-
-
-def formatACLRule(rule):
-    if rule['scope']['type'] != 'default':
-        return f'(Scope: {rule["scope"]["type"]}:{rule["scope"]["value"]}, ' \
-               f'Role: {rule["role"]})'
-    return f'(Scope: {rule["scope"]["type"]}, Role: {rule["role"]})'
-
-
-def getSendUpdates(myarg, i, cal):
-    if myarg == 'notifyattendees':
-        sendUpdates = 'all'
-        i += 1
-    elif myarg == 'sendnotifications':
-        sendUpdates = 'all' if gam.getBoolean(sys.argv[i +
-                                                       1], myarg) else 'none'
-        i += 2
-    else:  # 'sendupdates':
-        sendUpdatesMap = {}
-        for val in cal._rootDesc['resources']['events']['methods']['delete'][
-                'parameters']['sendUpdates']['enum']:
-            sendUpdatesMap[val.lower()] = val
-        sendUpdates = sendUpdatesMap.get(sys.argv[i + 1].lower(), False)
-        if not sendUpdates:
-            controlflow.expected_argument_exit('sendupdates',
-                                               ', '.join(sendUpdatesMap),
-                                               sys.argv[i + 1])
-        i += 2
-    return (sendUpdates, i)
-
-
-def moveOrDeleteEvent(moveOrDelete):
-    calendarId, cal = buildCalendarDataGAPIObject(sys.argv[2])
-    if not cal:
-        return
-    sendUpdates = 'none'
-    doit = False
-    kwargs = {}
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg in ['notifyattendees', 'sendnotifications', 'sendupdates']:
-            sendUpdates, i = getSendUpdates(myarg, i, cal)
-        elif myarg in ['id', 'eventid']:
-            eventId = sys.argv[i + 1]
-            i += 2
-        elif myarg in ['query', 'eventquery']:
-            controlflow.system_error_exit(
-                2, f'query is no longer supported for {moveOrDelete}event. ' \
-                   f'Use "gam calendar <email> printevents query <query> | ' \
-                   f'gam csv - gam {moveOrDelete}event id ~id" instead.')
-        elif myarg == 'doit':
-            doit = True
-            i += 1
-        elif moveOrDelete == 'move' and myarg == 'destination':
-            kwargs['destination'] = sys.argv[i + 1]
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(
-                sys.argv[i], f'gam calendar <email> {moveOrDelete}event')
-    if doit:
-        print(f' going to {moveOrDelete} eventId {eventId}')
-        gapi.call(cal.events(),
-                  moveOrDelete,
-                  calendarId=calendarId,
-                  eventId=eventId,
-                  sendUpdates=sendUpdates,
-                  **kwargs)
-    else:
-        print(
-            f' would {moveOrDelete} eventId {eventId}. Add doit to command ' \
-            f'to actually {moveOrDelete} event')
-
-
-def infoEvent():
-    calendarId, cal = buildCalendarDataGAPIObject(sys.argv[2])
-    if not cal:
-        return
-    eventId = sys.argv[4]
-    result = gapi.call(cal.events(),
-                       'get',
-                       calendarId=calendarId,
-                       eventId=eventId)
-    display.print_json(result)
-
-
-def addOrUpdateEvent(action):
-    calendarId, cal = buildCalendarDataGAPIObject(sys.argv[2])
-    if not cal:
-        return
-    # only way for non-Google calendars to get updates is via email
-    kwargs = {}
-    body = {}
-    if action == 'add':
-        i = 4
-        func = 'insert'
-    else:
-        eventId = sys.argv[4]
-        kwargs = {'eventId': eventId}
-        i = 5
-        func = 'patch'
-        requires_full_update = [
-            'attendee', 'optionalattendee', 'removeattendee',
-            'replacedescription'
-        ]
-        for arg in sys.argv[i:]:
-            if arg.replace('_', '').lower() in requires_full_update:
-                func = 'update'
-                body = gapi.call(cal.events(),
-                                 'get',
-                                 calendarId=calendarId,
-                                 eventId=eventId)
-                break
-    sendUpdates, body = getEventAttributes(i, calendarId, cal, body, action)
-    result = gapi.call(cal.events(),
-                       func,
-                       conferenceDataVersion=1,
-                       supportsAttachments=True,
-                       calendarId=calendarId,
-                       sendUpdates=sendUpdates,
-                       body=body,
-                       fields='id',
-                       **kwargs)
-    print(f'Event {result["id"]} {action} finished')
-
-
-def _remove_attendee(attendees, remove_email):
-    return [
-        attendee for attendee in attendees
-        if not attendee['email'].lower() == remove_email
-    ]
-
-
-def getEventAttributes(i, calendarId, cal, body, action):
-    # Default to external only so non-Google
-    # calendars are notified of changes
-    sendUpdates = 'externalOnly'
-    action = 'update' if body else 'add'
-    timeZone = None
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg in ['notifyattendees', 'sendnotifications', 'sendupdates']:
-            sendUpdates, i = getSendUpdates(myarg, i, cal)
-        elif myarg == 'attendee':
-            body.setdefault('attendees', [])
-            body['attendees'].append({'email': sys.argv[i + 1]})
-            i += 2
-        elif myarg == 'removeattendee' and action == 'update':
-            remove_email = sys.argv[i + 1].lower()
-            if 'attendees' in body:
-                body['attendees'] = _remove_attendee(body['attendees'],
-                                                     remove_email)
-            i += 2
-        elif myarg == 'optionalattendee':
-            body.setdefault('attendees', [])
-            body['attendees'].append({
-                'email': sys.argv[i + 1],
-                'optional': True
-            })
-            i += 2
-        elif myarg == 'anyonecanaddself':
-            body['anyoneCanAddSelf'] = True
-            i += 1
-        elif myarg == 'description':
-            body['description'] = sys.argv[i + 1].replace('\\n', '\n')
-            i += 2
-        elif myarg == 'replacedescription' and action == 'update':
-            search = sys.argv[i + 1]
-            replace = sys.argv[i + 2]
-            if 'description' in body:
-                body['description'] = re.sub(search, replace,
-                                             body['description'])
-            i += 3
-        elif myarg == 'start':
-            if sys.argv[i + 1].lower() == 'allday':
-                body['start'] = {'date': utils.get_yyyymmdd(sys.argv[i + 2])}
-                i += 3
-            else:
-                start_time = utils.get_time_or_delta_from_now(sys.argv[i + 1])
-                body['start'] = {'dateTime': start_time}
-                i += 2
-        elif myarg == 'end':
-            if sys.argv[i + 1].lower() == 'allday':
-                body['end'] = {'date': utils.get_yyyymmdd(sys.argv[i + 2])}
-                i += 3
-            else:
-                end_time = utils.get_time_or_delta_from_now(sys.argv[i + 1])
-                body['end'] = {'dateTime': end_time}
-                i += 2
-        elif myarg == 'guestscantinviteothers':
-            body['guestsCanInviteOthers'] = False
-            i += 1
-        elif myarg == 'guestscaninviteothers':
-            body['guestsCanInviteTohters'] = gam.getBoolean(
-                sys.argv[i + 1], 'guestscaninviteothers')
-            i += 2
-        elif myarg == 'guestscantseeothers':
-            body['guestsCanSeeOtherGuests'] = False
-            i += 1
-        elif myarg == 'guestscanseeothers':
-            body['guestsCanSeeOtherGuests'] = gam.getBoolean(
-                sys.argv[i + 1], 'guestscanseeothers')
-            i += 2
-        elif myarg == 'guestscanmodify':
-            body['guestsCanModify'] = gam.getBoolean(sys.argv[i + 1],
-                                                     'guestscanmodify')
-            i += 2
-        elif myarg == 'id':
-            if action == 'update':
-                controlflow.invalid_argument_exit(
-                    'id', 'gam calendar <calendar> updateevent')
-            body['id'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'summary':
-            body['summary'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'location':
-            body['location'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'available':
-            body['transparency'] = 'transparent'
-            i += 1
-        elif myarg == 'transparency':
-            validTransparency = ['opaque', 'transparent']
-            if sys.argv[i + 1].lower() in validTransparency:
-                body['transparency'] = sys.argv[i + 1].lower()
-            else:
-                controlflow.expected_argument_exit('transparency',
-                                                   ', '.join(validTransparency),
-                                                   sys.argv[i + 1])
-            i += 2
-        elif myarg == 'visibility':
-            validVisibility = ['default', 'public', 'private']
-            if sys.argv[i + 1].lower() in validVisibility:
-                body['visibility'] = sys.argv[i + 1].lower()
-            else:
-                controlflow.expected_argument_exit('visibility',
-                                                   ', '.join(validVisibility),
-                                                   sys.argv[i + 1])
-            i += 2
-        elif myarg == 'tentative':
-            body['status'] = 'tentative'
-            i += 1
-        elif myarg == 'status':
-            validStatus = ['confirmed', 'tentative', 'cancelled']
-            if sys.argv[i + 1].lower() in validStatus:
-                body['status'] = sys.argv[i + 1].lower()
-            else:
-                controlflow.expected_argument_exit('visibility',
-                                                   ', '.join(validStatus),
-                                                   sys.argv[i + 1])
-            i += 2
-        elif myarg == 'source':
-            body['source'] = {'title': sys.argv[i + 1], 'url': sys.argv[i + 2]}
-            i += 3
-        elif myarg == 'noreminders':
-            body['reminders'] = {'useDefault': False}
-            i += 1
-        elif myarg == 'reminder':
-            minutes = \
-            gam.getInteger(sys.argv[i+1], myarg, minVal=0,
-                                maxVal=CALENDAR_REMINDER_MAX_MINUTES)
-            reminder = {'minutes': minutes, 'method': sys.argv[i + 2]}
-            body.setdefault('reminders', {'overrides': [], 'useDefault': False})
-            body['reminders']['overrides'].append(reminder)
-            i += 3
-        elif myarg == 'recurrence':
-            body.setdefault('recurrence', [])
-            body['recurrence'].append(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'timezone':
-            timeZone = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'privateproperty':
-            if 'extendedProperties' not in body:
-                body['extendedProperties'] = {'private': {}, 'shared': {}}
-            body['extendedProperties']['private'][sys.argv[i +
-                                                           1]] = sys.argv[i + 2]
-            i += 3
-        elif myarg == 'sharedproperty':
-            if 'extendedProperties' not in body:
-                body['extendedProperties'] = {'private': {}, 'shared': {}}
-            body['extendedProperties']['shared'][sys.argv[i + 1]] = sys.argv[i +
-                                                                             2]
-            i += 3
-        elif myarg == 'colorindex':
-            body['colorId'] = gam.getInteger(sys.argv[i + 1], myarg,
-                                             CALENDAR_EVENT_MIN_COLOR_INDEX,
-                                             CALENDAR_EVENT_MAX_COLOR_INDEX)
-            i += 2
-        elif myarg == 'hangoutsmeet':
-            body['conferenceData'] = {
-                'createRequest': {
-                    'requestId': f'{str(uuid.uuid4())}'
-                }
-            }
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(
-                sys.argv[i], f'gam calendar <email> {action}event')
-    if ('recurrence' in body) and (('start' in body) or ('end' in body)):
-        if not timeZone:
-            timeZone = gapi.call(cal.calendars(),
-                                 'get',
-                                 calendarId=calendarId,
-                                 fields='timeZone')['timeZone']
-        if 'start' in body:
-            body['start']['timeZone'] = timeZone
-        if 'end' in body:
-            body['end']['timeZone'] = timeZone
-    return (sendUpdates, body)
-
-
-def modifySettings():
-    calendarId, cal = buildCalendarDataGAPIObject(sys.argv[2])
-    if not cal:
-        return
-    body = {}
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'description':
-            body['description'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'location':
-            body['location'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'summary':
-            body['summary'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'timezone':
-            body['timeZone'] = sys.argv[i + 1]
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam calendar <email> modify')
-    gapi.call(cal.calendars(), 'patch', calendarId=calendarId, body=body)
-
-
-def changeAttendees(users):
-    do_it = True
-    i = 5
-    allevents = False
-    start_date = end_date = None
-    while len(sys.argv) > i:
-        myarg = sys.argv[i].lower()
-        if myarg == 'csv':
-            csv_file = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'dryrun':
-            do_it = False
-            i += 1
-        elif myarg == 'start':
-            start_date = utils.get_time_or_delta_from_now(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'end':
-            end_date = utils.get_time_or_delta_from_now(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'allevents':
-            allevents = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(
-                sys.argv[i], 'gam <users> update calattendees')
-    attendee_map = {}
-    f = fileutils.open_file(csv_file)
-    csvFile = csv.reader(f)
-    for row in csvFile:
-        attendee_map[row[0].lower()] = row[1].lower()
-    fileutils.close_file(f)
-    for user in users:
-        sys.stdout.write(f'Checking user {user}\n')
-        user, cal = buildCalendarGAPIObject(user)
-        if not cal:
-            continue
-        page_token = None
-        while True:
-            events_page = gapi.call(cal.events(),
-                                    'list',
-                                    calendarId=user,
-                                    pageToken=page_token,
-                                    timeMin=start_date,
-                                    timeMax=end_date,
-                                    showDeleted=False,
-                                    showHiddenInvitations=False)
-            print(f'Got {len(events_page.get("items", []))}')
-            for event in events_page.get('items', []):
-                if event['status'] == 'cancelled':
-                    # print u' skipping cancelled event'
-                    continue
-                try:
-                    event_summary = event['summary']
-                except (KeyError, UnicodeEncodeError, UnicodeDecodeError):
-                    event_summary = event['id']
-                try:
-                    organizer = event['organizer']['email'].lower()
-                    if not allevents and organizer != user:
-                        #print(f' skipping not-my-event {event_summary}')
-                        continue
-                except KeyError:
-                    pass  # no email for organizer
-                needs_update = False
-                try:
-                    for attendee in event['attendees']:
-                        try:
-                            if attendee['email'].lower() in attendee_map:
-                                old_email = attendee['email'].lower()
-                                new_email = attendee_map[
-                                    attendee['email'].lower()]
-                                print(f' SWITCHING attendee {old_email} to ' \
-                                    f'{new_email} for {event_summary}')
-                                event['attendees'].remove(attendee)
-                                event['attendees'].append({'email': new_email})
-                                needs_update = True
-                        except KeyError:  # no email for that attendee
-                            pass
-                except KeyError:
-                    continue  # no attendees
-                if needs_update:
-                    body = {}
-                    body['attendees'] = event['attendees']
-                    print(f'UPDATING {event_summary}')
-                    if do_it:
-                        gapi.call(cal.events(),
-                                  'patch',
-                                  calendarId=user,
-                                  eventId=event['id'],
-                                  sendNotifications=False,
-                                  body=body)
-                    else:
-                        print(' not pulling the trigger.')
-                # else:
-                #  print(f' no update needed for {event_summary}')
-            try:
-                page_token = events_page['nextPageToken']
-            except KeyError:
-                break
-
-
-def deleteCalendar(users):
-    calendarId = normalizeCalendarId(sys.argv[5])
-    for user in users:
-        user, cal = buildCalendarGAPIObject(user)
-        if not cal:
-            continue
-        gapi.call(cal.calendarList(),
-                  'delete',
-                  soft_errors=True,
-                  calendarId=calendarId)
-
-
-CALENDAR_REMINDER_MAX_MINUTES = 40320
-
-CALENDAR_MIN_COLOR_INDEX = 1
-CALENDAR_MAX_COLOR_INDEX = 24
-
-CALENDAR_EVENT_MIN_COLOR_INDEX = 1
-CALENDAR_EVENT_MAX_COLOR_INDEX = 11
-
-
-def getCalendarAttributes(i, body, function):
-    colorRgbFormat = False
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'selected':
-            body['selected'] = gam.getBoolean(sys.argv[i + 1], myarg)
-            i += 2
-        elif myarg == 'hidden':
-            body['hidden'] = gam.getBoolean(sys.argv[i + 1], myarg)
-            i += 2
-        elif myarg == 'summary':
-            body['summaryOverride'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'colorindex':
-            body['colorId'] = gam.getInteger(sys.argv[i + 1],
-                                             myarg,
-                                             minVal=CALENDAR_MIN_COLOR_INDEX,
-                                             maxVal=CALENDAR_MAX_COLOR_INDEX)
-            i += 2
-        elif myarg == 'backgroundcolor':
-            body['backgroundColor'] = gam.getColor(sys.argv[i + 1])
-            colorRgbFormat = True
-            i += 2
-        elif myarg == 'foregroundcolor':
-            body['foregroundColor'] = gam.getColor(sys.argv[i + 1])
-            colorRgbFormat = True
-            i += 2
-        elif myarg == 'reminder':
-            body.setdefault('defaultReminders', [])
-            method = sys.argv[i + 1].lower()
-            if method not in CLEAR_NONE_ARGUMENT:
-                if method not in CALENDAR_REMINDER_METHODS:
-                    controlflow.expected_argument_exit(
-                        'Method', ', '.join(CALENDAR_REMINDER_METHODS +
-                                            CLEAR_NONE_ARGUMENT), method)
-                minutes = gam.getInteger(sys.argv[i + 2],
-                                         myarg,
-                                         minVal=0,
-                                         maxVal=CALENDAR_REMINDER_MAX_MINUTES)
-                body['defaultReminders'].append({
-                    'method': method,
-                    'minutes': minutes
-                })
-                i += 3
-            else:
-                i += 2
-        elif myarg == 'notification':
-            body.setdefault('notificationSettings', {'notifications': []})
-            method = sys.argv[i + 1].lower()
-            if method not in CLEAR_NONE_ARGUMENT:
-                if method not in CALENDAR_NOTIFICATION_METHODS:
-                    controlflow.expected_argument_exit(
-                        'Method', ', '.join(CALENDAR_NOTIFICATION_METHODS +
-                                            CLEAR_NONE_ARGUMENT), method)
-                eventType = sys.argv[i + 2].lower()
-                if eventType not in CALENDAR_NOTIFICATION_TYPES_MAP:
-                    controlflow.expected_argument_exit(
-                        'Event', ', '.join(CALENDAR_NOTIFICATION_TYPES_MAP),
-                        eventType)
-                notice = {
-                    'method': method,
-                    'type': CALENDAR_NOTIFICATION_TYPES_MAP[eventType]
-                }
-                body['notificationSettings']['notifications'].append(notice)
-                i += 3
-            else:
-                i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              f'gam {function} calendar')
-    return colorRgbFormat
-
-
-def addCalendar(users):
-    calendarId = normalizeCalendarId(sys.argv[5])
-    body = {'id': calendarId, 'selected': True, 'hidden': False}
-    colorRgbFormat = getCalendarAttributes(6, body, 'add')
-    i = 0
-    count = len(users)
-    for user in users:
-        i += 1
-        user, cal = buildCalendarGAPIObject(user)
-        if not cal:
-            continue
-        current_count = display.current_count(i, count)
-        print(f'Subscribing {user} to calendar {calendarId}{current_count}')
-        gapi.call(cal.calendarList(),
-                  'insert',
-                  soft_errors=True,
-                  body=body,
-                  colorRgbFormat=colorRgbFormat)
-
-
-def updateCalendar(users):
-    calendarId = normalizeCalendarId(sys.argv[5], checkPrimary=True)
-    body = {}
-    colorRgbFormat = getCalendarAttributes(6, body, 'update')
-    i = 0
-    count = len(users)
-    for user in users:
-        i += 1
-        user, cal = buildCalendarGAPIObject(user)
-        if not cal:
-            continue
-        current_count = display.current_count(i, count)
-        print(f"Updating {user}'s subscription to calendar ' \
-              f'{calendarId}{current_count}")
-        calId = calendarId if calendarId != 'primary' else user
-        gapi.call(cal.calendarList(),
-                  'patch',
-                  soft_errors=True,
-                  calendarId=calId,
-                  body=body,
-                  colorRgbFormat=colorRgbFormat)
-
-
-def _showCalendar(userCalendar, j, jcount):
-    current_count = display.current_count(j, jcount)
-    summary = userCalendar.get('summaryOverride', userCalendar['summary'])
-    print(f'  Calendar: {userCalendar["id"]}{current_count}')
-    print(f'    Summary: {summary}')
-    print(f'    Description: {userCalendar.get("description", "")}')
-    print(f'    Access Level: {userCalendar["accessRole"]}')
-    print(f'    Timezone: {userCalendar["timeZone"]}')
-    print(f'    Location: {userCalendar.get("location", "")}')
-    print(f'    Hidden: {userCalendar.get("hidden", "False")}')
-    print(f'    Selected: {userCalendar.get("selected", "False")}')
-    print(f'    Color ID: {userCalendar["colorId"]}, ' \
-          f'Background Color: {userCalendar["backgroundColor"]}, ' \
-          f'Foreground Color: {userCalendar["foregroundColor"]}')
-    print(f'    Default Reminders:')
-    for reminder in userCalendar.get('defaultReminders', []):
-        print(f'      Method: {reminder["method"]}, ' \
-              f'Minutes: {reminder["minutes"]}')
-    print('    Notifications:')
-    if 'notificationSettings' in userCalendar:
-        notifications = userCalendar['notificationSettings'].get(
-            'notifications', [])
-        for notification in notifications:
-            print(f'      Method: {notification["method"]}, ' \
-                  f'Type: {notification["type"]}')
-
-
-def infoCalendar(users):
-    calendarId = normalizeCalendarId(sys.argv[5], checkPrimary=True)
-    i = 0
-    count = len(users)
-    for user in users:
-        i += 1
-        user, cal = buildCalendarGAPIObject(user)
-        if not cal:
-            continue
-        result = gapi.call(cal.calendarList(),
-                           'get',
-                           soft_errors=True,
-                           calendarId=calendarId)
-        if result:
-            print(f'User: {user}, Calendar:{display.current_count(i, count)}')
-            _showCalendar(result, 1, 1)
-
-
-def printShowCalendars(users, csvFormat):
-    if csvFormat:
-        todrive = False
-        titles = []
-        csvRows = []
-    i = 5
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if csvFormat and myarg == 'todrive':
-            todrive = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(
-                myarg, f"gam <users> {['show', 'print'][csvFormat]} calendars")
-    i = 0
-    count = len(users)
-    for user in users:
-        i += 1
-        user, cal = buildCalendarGAPIObject(user)
-        if not cal:
-            continue
-        result = gapi.get_all_pages(cal.calendarList(),
-                                    'list',
-                                    'items',
-                                    soft_errors=True)
-        jcount = len(result)
-        if not csvFormat:
-            print(f'User: {user}, Calendars:{display.current_count(i, count)}')
-            if jcount == 0:
-                continue
-            j = 0
-            for userCalendar in result:
-                j += 1
-                _showCalendar(userCalendar, j, jcount)
-        else:
-            if jcount == 0:
-                continue
-            for userCalendar in result:
-                row = {'primaryEmail': user}
-                display.add_row_titles_to_csv_file(
-                    utils.flatten_json(userCalendar, flattened=row), csvRows,
-                    titles)
-    if csvFormat:
-        display.sort_csv_titles(['primaryEmail', 'id'], titles)
-        display.write_csv_file(csvRows, titles, 'Calendars', todrive)
-
-
-def showCalSettings(users):
-    i = 0
-    count = len(users)
-    for user in users:
-        i += 1
-        user, cal = buildCalendarGAPIObject(user)
-        if not cal:
-            continue
-        feed = gapi.get_all_pages(cal.settings(),
-                                  'list',
-                                  'items',
-                                  soft_errors=True)
-        if feed:
-            current_count = display.current_count(i, count)
-            print(f'User: {user}, Calendar Settings:{current_count}')
-            settings = {}
-            for setting in feed:
-                settings[setting['id']] = setting['value']
-            for attr, value in sorted(settings.items()):
-                print(f'  {attr}: {value}')
-
-
-def transferSecCals(users):
-    target_user = sys.argv[5]
-    remove_source_user = sendNotifications = True
-    i = 6
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'keepuser':
-            remove_source_user = False
-            i += 1
-        elif myarg == 'sendnotifications':
-            sendNotifications = gam.getBoolean(sys.argv[i + 1], myarg)
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam <users> transfer seccals')
-    if remove_source_user:
-        target_user, target_cal = buildCalendarGAPIObject(target_user)
-        if not target_cal:
-            return
-    for user in users:
-        user, source_cal = buildCalendarGAPIObject(user)
-        if not source_cal:
-            continue
-        calendars = gapi.get_all_pages(source_cal.calendarList(),
-                                       'list',
-                                       'items',
-                                       soft_errors=True,
-                                       minAccessRole='owner',
-                                       showHidden=True,
-                                       fields='items(id),nextPageToken')
-        for calendar in calendars:
-            calendarId = calendar['id']
-            if calendarId.find('@group.calendar.google.com') != -1:
-                body = {
-                    'role': 'owner',
-                    'scope': {
-                        'type': 'user',
-                        'value': target_user
-                    }
-                }
-                gapi.call(source_cal.acl(),
-                          'insert',
-                          calendarId=calendarId,
-                          body=body,
-                          sendNotifications=sendNotifications)
-                if remove_source_user:
-                    body = {
-                        'role': 'none',
-                        'scope': {
-                            'type': 'user',
-                            'value': user
-                        }
-                    }
-                    gapi.call(target_cal.acl(),
-                              'insert',
-                              calendarId=calendarId,
-                              body=body,
-                              sendNotifications=sendNotifications)

src/gam/gapi/cloudidentity/__init__.py

@@ -1,9 +0,0 @@
-import gam
-
-
-def build(api='cloudidentity'):
-    return gam.buildGAPIObject(api)
-
-def build_dwd(api='cloudidentity'):
-    admin = gam._get_admin_email()
-    return gam.buildGAPIServiceObject(api, admin, True)

src/gam/gapi/cloudidentity/devices.py

@@ -1,331 +0,0 @@
-import csv
-import sys
-
-import googleapiclient
-
-from gam.var import *
-from gam import controlflow
-from gam import display
-from gam import fileutils
-from gam import gapi
-from gam import utils
-from gam.gapi import errors as gapi_errors
-from gam.gapi import cloudidentity as gapi_cloudidentity
-
-
-def _get_device_customerid():
-  customer = GC_Values[GC_CUSTOMER_ID]
-  if customer.startswith('C'):
-    customer = customer[1:]
-  return f'customers/{customer}'
-
-def create():
-    ci = gapi_cloudidentity.build_dwd()
-    customer = _get_device_customerid()
-    device_types = gapi.get_enum_values_minus_unspecified(
-        ci._rootDesc['schemas']['GoogleAppsCloudidentityDevicesV1Device']['properties']['deviceType']['enum'])
-    body = {'deviceType': '', 'serialNumber': ''}
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'serialnumber':
-            body['serialNumber'] = sys.argv[i+1]
-            i += 2
-        elif myarg == 'devicetype':
-            body['deviceType'] = sys.argv[i+1].upper()
-            if body['deviceType'] not in device_types:
-                controlflow.expected_argument_exit('device_type',
-                                                   ', '.join(device_types),
-                                                   sys.argv[i+1])
-            i += 2
-        elif myarg in {'assettag', 'assetid'}:
-            body['assetTag'] = sys.argv[i+1]
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam create device')
-    if not body['serialNumber'] or not body['deviceType']:
-        controlflow.system_error_exit(
-            3, 'serial_number and device_type are required arguments for "gam create device".')
-    result = gapi.call(ci.devices(), 'create', customer=customer, body=body)
-    print(f'Created device {result["response"]["name"]}')
-
-def _get_device_name():
-    name = sys.argv[3]
-    if name == 'id':
-        name = sys.argv[4]
-    if not name.startswith('devices/'):
-        name = f'devices/{name}'
-    return name
-
-
-def info():
-    ci = gapi_cloudidentity.build_dwd()
-    customer = _get_device_customerid()
-    name = _get_device_name()
-    device = gapi.call(ci.devices(), 'get', name=name, customer=customer)
-    device_users = gapi.get_all_pages(ci.devices().deviceUsers(), 'list',
-        'deviceUsers', parent=name, customer=customer)
-    display.print_json(device)
-    print('Device Users:')
-    display.print_json(device_users)
-
-def _generic_action(action, device_user=False):
-    ci = gapi_cloudidentity.build_dwd()
-    customer = _get_device_customerid()
-    name = _get_device_name()
-
-    # bah, inconsistencies in API
-    if action == 'delete':
-        kwargs = {'customer': customer}
-    else:
-        kwargs = {'body': {'customer': customer}}
-
-    if device_user:
-        endpoint = ci.devices().deviceUsers()
-    else:
-        endpoint = ci.devices()
-    op = gapi.call(endpoint, action, name=name, **kwargs)
-    print(op)
-
-def delete():
-    _generic_action('delete')
-
-def cancel_wipe():
-    _generic_action('cancelWipe')
-
-def wipe():
-    _generic_action('wipe')
-
-def approve_user():
-    _generic_action('approve', True)
-
-def block_user():
-    _generic_action('block', True)
-
-def cancel_wipe_user():
-    _generic_action('cancelWipe', True)
-
-def delete_user():
-    _generic_action('delete', True)
-
-def wipe_user():
-    _generic_action('wipe', True)
-
-def print_():
-    ci = gapi_cloudidentity.build_dwd()
-    customer = _get_device_customerid()
-    parent = 'devices/-'
-    device_filter = None
-    get_device_users = True
-    view = None
-    orderByList = []
-    titles = []
-    csvRows = []
-    todrive = False
-    sortHeaders = False
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg in ['filter', 'query']:
-            device_filter = sys.argv[i+1]
-            i += 2
-        elif myarg == 'company':
-            view = 'COMPANY_INVENTORY'
-            i += 1
-        elif myarg == 'personal':
-            view = 'USER_ASSIGNED_DEVICES'
-            i += 1
-        elif myarg == 'nocompanydevices':
-            view = 'USER_ASSIGNED_DEVICES'
-            i += 1
-        elif myarg == 'nopersonaldevices':
-            view = 'COMPANY_INVENTORY'
-            i += 1
-        elif myarg == 'nodeviceusers':
-            get_device_users = False
-            i += 1
-        elif myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg == 'orderby':
-            fieldName = sys.argv[i + 1].lower()
-            i += 2
-            if fieldName in DEVICE_ORDERBY_CHOICES_MAP:
-                fieldName = DEVICE_ORDERBY_CHOICES_MAP[fieldName]
-                orderBy = ''
-                if i < len(sys.argv):
-                    orderBy = sys.argv[i].lower()
-                    if orderBy in SORTORDER_CHOICES_MAP:
-                        orderBy = SORTORDER_CHOICES_MAP[orderBy]
-                        i += 1
-                if orderBy != 'DESCENDING':
-                    orderByList.append(fieldName)
-                else:
-                    orderByList.append(f'{fieldName} desc')
-            else:
-                controlflow.expected_argument_exit(
-                    'orderby', ', '.join(sorted(DEVICE_ORDERBY_CHOICES_MAP)),
-                    fieldName)
-        elif myarg == 'sortheaders':
-            sortHeaders = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam print devices')
-    view_name_map = {
-      None: 'Devices',
-      'COMPANY_INVENTORY': 'Company Devices',
-      'USER_ASSIGNED_DEVICES': 'Personal Devices',
-      }
-    if orderByList:
-        orderBy = ','.join(orderByList)
-    else:
-        orderBy = None
-    devices = []
-    page_message = gapi.got_total_items_msg(view_name_map[view], '...\n')
-    devices += gapi.get_all_pages(ci.devices(), 'list', 'devices',
-        customer=customer, page_message=page_message,
-        pageSize=100, filter=device_filter, view=view, orderBy=orderBy)
-    if get_device_users:
-        page_message = gapi.got_total_items_msg('Device Users', '...\n')
-        device_users = gapi.get_all_pages(ci.devices().deviceUsers(), 'list',
-            'deviceUsers', customer=customer, parent=parent,
-            page_message=page_message, pageSize=20, filter=device_filter)
-        for device_user in device_users:
-            for device in devices:
-                if device_user.get('name').startswith(device.get('name')):
-                    if 'users' not in device:
-                        device['users'] = []
-                    device['users'].append(device_user)
-                    break
-    for device in devices:
-        device = utils.flatten_json(device)
-        for a_key in device:
-            if a_key not in titles:
-                titles.append(a_key)
-        csvRows.append(device)
-    if sortHeaders:
-        display.sort_csv_titles(['name',], titles)
-    display.write_csv_file(csvRows, titles, 'Devices', todrive)
-
-
-def sync():
-    ci = gapi_cloudidentity.build_dwd()
-    device_types = gapi.get_enum_values_minus_unspecified(
-        ci._rootDesc['schemas']['GoogleAppsCloudidentityDevicesV1Device']['properties']['deviceType']['enum'])
-    customer = _get_device_customerid()
-    device_filter = None
-    csv_file = None
-    serialnumber_column = 'serialNumber'
-    devicetype_column = 'deviceType'
-    static_devicetype = None
-    assettag_column = None
-    unassigned_missing_action = 'delete'
-    assigned_missing_action = 'donothing'
-    missing_actions = ['delete', 'wipe', 'donothing']
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg in ['filter', 'query']:
-          device_filter = sys.argv[i+1]
-          i += 2
-        elif myarg == 'csvfile':
-          csv_file = sys.argv[i+1]
-          i += 2
-        elif myarg == 'serialnumbercolumn':
-          serialnumber_column = sys.argv[i+1]
-          i += 2
-        elif myarg == 'devicetypecolumn':
-          devicetype_column = sys.argv[i+1]
-          i += 2
-        elif myarg == 'staticdevicetype':
-          static_devicetype = sys.argv[i+1].upper()
-          if static_devicetype not in device_types:
-                controlflow.expected_argument_exit('device_type',
-                                                   ', '.join(device_types),
-                                                   sys.argv[i+1])
-          i += 2
-        elif myarg in {'assettagcolumn', 'assetidcolumn'}:
-          assettag_column = sys.argv[i+1]
-          i += 2
-        elif myarg == 'unassignedmissingaction':
-          unassigned_missing_action = sys.argv[i+1].lower().replace('_', '')
-          if unassigned_missing_action not in missing_actions:
-              controlflow.expected_argument_exit('unassigned_missing_action',
-                                                   ', '.join(missing_actions),
-                                                   sys.argv[i+1])
-          i += 2
-        elif myarg == 'assignedmissingaction':
-          assigned_missing_action = sys.argv[i+1].lower().replace('_', '')
-          if assigned_missing_action not in missing_actions:
-              controlflow.expected_argument_exit('assigned_missing_action',
-                                                   ', '.join(missing_actions),
-                                                   sys.argv[i+1])
-          i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam sync devices')
-    if not csv_file:
-        controlflow.system_error_exit(
-            3, 'csvfile is a required argument for "gam sync devices".')
-    f = fileutils.open_file(csv_file)
-    input_file = csv.DictReader(f, restval='')
-    if serialnumber_column not in input_file.fieldnames:
-        controlflow.csv_field_error_exit(serialnumber_column, input_file.fieldnames)
-    if not static_devicetype and devicetype_column not in input_file.fieldnames:
-        controlflow.csv_field_error_exit(devicetype_column, input_file.fieldnames)
-    if assettag_column and assettag_column not in input_file.fieldnames:
-        controlflow.csv_field_error_exit(assettag_column, input_file.fieldnames)
-    local_devices = []
-    for row in input_file:
-        # upper() is very important to comparison since Google
-        # always return uppercase serials
-        local_device = {'serialNumber': row[serialnumber_column].strip().upper()}
-        if static_devicetype:
-            local_device['deviceType'] = static_devicetype
-        else:
-            local_device['deviceType'] = row[devicetype_column].strip()
-        if assettag_column:
-            local_device['assetTag'] = row[assettag_column].strip()
-        local_devices.append(local_device)
-    fileutils.close_file(f)
-    page_message = gapi.got_total_items_msg('Company Devices', '...\n')
-    device_fields = ['serialNumber', 'deviceType', 'lastSyncTime', 'name']
-    if assettag_column:
-       device_fields.append('assetTag')
-    fields = f'nextPageToken,devices({",".join(device_fields)})'
-    remote_devices = gapi.get_all_pages(ci.devices(), 'list', 'devices',
-            customer=customer, page_message=page_message,
-            pageSize=100, filter=device_filter, view='COMPANY_INVENTORY', fields=fields)
-    remote_device_map = {}
-    for remote_device in remote_devices:
-        sn = remote_device['serialNumber']
-        last_sync = remote_device.pop('lastSyncTime', NEVER_TIME_NOMS)
-        name = remote_device.pop('name')
-        remote_device_map[sn] = {'name': name}
-        if last_sync == NEVER_TIME_NOMS:
-            remote_device_map[sn]['unassigned'] = True
-    devices_to_add = [device for device in local_devices if device not in remote_devices]
-    missing_devices = [device for device in remote_devices if device not in local_devices]
-    print(f'Need to add {len(devices_to_add)} and remove {len(missing_devices)} devices...')
-    for add_device in devices_to_add:
-        print(f'Creating {add_device["serialNumber"]}')
-        try:
-            result = gapi.call(ci.devices(), 'create', customer=customer,
-              throw_reasons=[gapi_errors.ErrorReason.FOUR_O_NINE], body=add_device)
-            print(f' created {result["response"]["deviceType"]} device {result["response"]["name"]} with serial {result["response"]["serialNumber"]}')
-        except googleapiclient.errors.HttpError:
-            print(f' {add_device["serialNumber"]} already exists')
-    for missing_device in missing_devices:
-        sn = missing_device['serialNumber']
-        name = remote_device_map[sn]['name']
-        unassigned = remote_device_map[sn].get('unassigned')
-        action = unassigned_missing_action if unassigned else assigned_missing_action
-        if action == 'donothing':
-            pass
-        else:
-            if action == 'delete':
-                kwargs = {'customer': customer}
-            else:
-                kwargs = {'body': {'customer': customer}}
-            gapi.call(ci.devices(), unassigned_missing_action,
-                name=name, **kwargs)
-            print(f'{action}d {sn}')

src/gam/gapi/cloudidentity/groups.py

@@ -1,748 +0,0 @@
-import gam
-from gam.var import *
-from gam import controlflow
-from gam import display
-from gam import gapi
-from gam import utils
-from gam.gapi import errors as gapi_errors
-from gam.gapi import cloudidentity as gapi_cloudidentity
-from gam.gapi.directory import customer as gapi_directory_customer
-
-
-def create():
-    ci = gapi_cloudidentity.build('cloudidentity_beta')
-    initialGroupConfig = 'EMPTY'
-    gapi_directory_customer.setTrueCustomerId()
-    parent = f'customers/{GC_Values[GC_CUSTOMER_ID]}'
-    body = {
-        'groupKey': {
-            'id': gam.normalizeEmailAddressOrUID(sys.argv[3], noUid=True)
-        },
-        'parent': parent,
-        'labels': {
-            'cloudidentity.googleapis.com/groups.discussion_forum': ''
-        },
-    }
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'name':
-            body['displayName'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'description':
-            body['description'] = sys.argv[i + 1]
-            i += 2
-        elif myarg in ['alias', 'aliases']:
-            # As of 2020/06/25 this doesn't work (yet?)
-            aliases = sys.argv[i + 1].split(' ')
-            body['additionalGroupKeys'] = []
-            for alias in aliases:
-                body['additionalGroupKeys'].append({'id': alias})
-            i += 2
-        elif myarg in ['dynamic']:
-            # As of 2020/06/25 this doesn't work (yet?)
-            body['dynamicGroupMetadata'] = {
-                'queries': [{
-                    'query': sys.argv[i + 1],
-                    'resourceType': 'USER'
-                }]
-            }
-            i += 2
-        elif myarg in ['makeowner']:
-            initialGroupConfig = 'WITH_INITIAL_OWNER'
-            i += 1
-        else:
-            print('should not get here')
-            sys.exit(5)
-    print(f'Creating group {body["groupKey"]["id"]}')
-    gapi.call(ci.groups(),
-              'create',
-              initialGroupConfig=initialGroupConfig,
-              body=body)
-
-
-def delete():
-    ci = gapi_cloudidentity.build('cloudidentity_beta')
-    group = sys.argv[3]
-    name = group_email_to_id(ci, group)
-    print(f'Deleting group {group}')
-    gapi.call(ci.groups(), 'delete', name=name)
-
-
-def info():
-    ci = gapi_cloudidentity.build('cloudidentity_beta')
-    group = gam.normalizeEmailAddressOrUID(sys.argv[3])
-    getUsers = True
-    showJoinDate = True
-    showUpdateDate = False
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'nousers':
-            getUsers = False
-            i += 1
-        elif myarg == 'nojoindate':
-            showJoinDate = False
-            i += 1
-        elif myarg == 'showupdatedate':
-            showUpdateDate = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(myarg, 'gam info cigroup')
-    name = group_email_to_id(ci, group)
-    basic_info = gapi.call(ci.groups(), 'get', name=name)
-    display.print_json(basic_info)
-    if getUsers:
-        if not showJoinDate and not showUpdateDate:
-            view = 'BASIC'
-            pageSize = 1000
-        else:
-            view = 'FULL'
-            pageSize = 500
-        members = gapi.get_all_pages(ci.groups().memberships(),
-                                     'list',
-                                     'memberships',
-                                     parent=name,
-                                     fields='*',
-                                     pageSize=pageSize,
-                                     view=view)
-        print('Members:')
-        for member in members:
-            role = get_single_role(member.get('roles', [])).lower()
-            email = member.get('memberKey', {}).get('id')
-            jc_string = ''
-            if showJoinDate:
-                joined = member.get('createTime', 'Unknown')
-                jc_string += f'  joined {joined}'
-            if showUpdateDate:
-                updated = member.get('updateTime', 'Unknown')
-                jc_string += f'  updated {updated}'
-            print(
-                f'{role}: {email}{jc_string}'
-                # f' {member.get("role", ROLE_MEMBER).lower()}: {member.get("email", member["id"])} ({member["type"].lower()})'
-            )
-        print(f'Total {len(members)} users in group')
-
-
-def info_member():
-    ci = gapi_cloudidentity.build('cloudidentity_beta')
-    member = gam.normalizeEmailAddressOrUID(sys.argv[3])
-    group = gam.normalizeEmailAddressOrUID(sys.argv[4])
-    group_name = gapi.call(ci.groups(),
-                           'lookup',
-                           groupKey_id=group,
-                           fields='name').get('name')
-    member_name = gapi.call(ci.groups().memberships(),
-                            'lookup',
-                            parent=group_name,
-                            memberKey_id=member,
-                            fields='name').get('name')
-    member_details = gapi.call(ci.groups().memberships(),
-                               'get',
-                               name=member_name)
-    display.print_json(member_details)
-
-
-UPDATE_GROUP_SUBCMDS = ['add', 'clear', 'delete', 'remove', 'sync', 'update']
-GROUP_ROLES_MAP = {
-    'owner': ROLE_OWNER,
-    'owners': ROLE_OWNER,
-    'manager': ROLE_MANAGER,
-    'managers': ROLE_MANAGER,
-    'member': ROLE_MEMBER,
-    'members': ROLE_MEMBER,
-}
-
-
-def print_():
-    ci = gapi_cloudidentity.build('cloudidentity_beta')
-    i = 3
-    members = membersCountOnly = managers = managersCountOnly = owners = ownersCountOnly = False
-    gapi_directory_customer.setTrueCustomerId()
-    parent = f'customers/{GC_Values[GC_CUSTOMER_ID]}'
-    memberDelimiter = '\n'
-    todrive = False
-    titles = []
-    csvRows = []
-    roles = []
-    sortHeaders = False
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg == 'delimiter':
-            memberDelimiter = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'sortheaders':
-            sortHeaders = True
-            i += 1
-        elif myarg in ['members', 'memberscount']:
-            roles.append(ROLE_MEMBER)
-            members = True
-            if myarg == 'memberscount':
-                membersCountOnly = True
-            i += 1
-        elif myarg in ['owners', 'ownerscount']:
-            roles.append(ROLE_OWNER)
-            owners = True
-            if myarg == 'ownerscount':
-                ownersCountOnly = True
-            i += 1
-        elif myarg in ['managers', 'managerscount']:
-            roles.append(ROLE_MANAGER)
-            managers = True
-            if myarg == 'managerscount':
-                managersCountOnly = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam print cigroups')
-    if roles:
-        if members:
-            display.add_titles_to_csv_file([
-                'MembersCount',
-            ], titles)
-            if not membersCountOnly:
-                display.add_titles_to_csv_file([
-                    'Members',
-                ], titles)
-        if managers:
-            display.add_titles_to_csv_file([
-                'ManagersCount',
-            ], titles)
-            if not managersCountOnly:
-                display.add_titles_to_csv_file([
-                    'Managers',
-                ], titles)
-        if owners:
-            display.add_titles_to_csv_file([
-                'OwnersCount',
-            ], titles)
-            if not ownersCountOnly:
-                display.add_titles_to_csv_file([
-                    'Owners',
-                ], titles)
-    gam.printGettingAllItems('Groups', None)
-    page_message = gapi.got_total_items_first_last_msg('Groups')
-    entityList = gapi.get_all_pages(ci.groups(),
-                                    'list',
-                                    'groups',
-                                    page_message=page_message,
-                                    message_attribute=['groupKey', 'id'],
-                                    parent=parent,
-                                    view='FULL',
-                                    pageSize=500)
-    i = 0
-    count = len(entityList)
-    for groupEntity in entityList:
-        i += 1
-        groupEmail = groupEntity['groupKey']['id']
-        group = utils.flatten_json(groupEntity)
-        for a_key in group:
-            if a_key not in titles:
-                titles.append(a_key)
-        groupKey_id = groupEntity['name']
-        if roles:
-            sys.stderr.write(
-                f' Getting {roles} for {groupEmail}{gam.currentCountNL(i, count)}'
-            )
-            page_message = gapi.got_total_items_first_last_msg('Members')
-            validRoles, _, _ = gam._getRoleVerification(
-                '.'.join(roles), 'nextPageToken,members(email,id,role)')
-            groupMembers = gapi.get_all_pages(ci.groups().memberships(),
-                                              'list',
-                                              'memberships',
-                                              page_message=page_message,
-                                              message_attribute=['memberKey', 'id'],
-                                              soft_errors=True,
-                                              parent=groupKey_id,
-                                              view='BASIC')
-            if members:
-                membersList = []
-                membersCount = 0
-            if managers:
-                managersList = []
-                managersCount = 0
-            if owners:
-                ownersList = []
-                ownersCount = 0
-            for member in groupMembers:
-                member_email = member['memberKey']['id']
-                role = get_single_role(member.get('roles'))
-                if not validRoles or role in validRoles:
-                    if role == ROLE_MEMBER:
-                        if members:
-                            membersCount += 1
-                            if not membersCountOnly:
-                                membersList.append(member_email)
-                    elif role == ROLE_MANAGER:
-                        if managers:
-                            managersCount += 1
-                            if not managersCountOnly:
-                                managersList.append(member_email)
-                    elif role == ROLE_OWNER:
-                        if owners:
-                            ownersCount += 1
-                            if not ownersCountOnly:
-                                ownersList.append(member_email)
-                    elif members:
-                        membersCount += 1
-                        if not membersCountOnly:
-                            membersList.append(member_email)
-            if members:
-                group['MembersCount'] = membersCount
-                if not membersCountOnly:
-                    group['Members'] = memberDelimiter.join(membersList)
-            if managers:
-                group['ManagersCount'] = managersCount
-                if not managersCountOnly:
-                    group['Managers'] = memberDelimiter.join(managersList)
-            if owners:
-                group['OwnersCount'] = ownersCount
-                if not ownersCountOnly:
-                    group['Owners'] = memberDelimiter.join(ownersList)
-        csvRows.append(group)
-    if sortHeaders:
-        display.sort_csv_titles([
-            'Email',
-        ], titles)
-    display.write_csv_file(csvRows, titles, 'Groups', todrive)
-
-
-def print_members():
-    ci = gapi_cloudidentity.build('cloudidentity_beta')
-    todrive = False
-    gapi_directory_customer.setTrueCustomerId()
-    parent = f'customers/{GC_Values[GC_CUSTOMER_ID]}'
-    roles = []
-    titles = ['group']
-    csvRows = []
-    groups_to_get = []
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg in ['role', 'roles']:
-            for role in sys.argv[i + 1].lower().replace(',', ' ').split():
-                if role in GROUP_ROLES_MAP:
-                    roles.append(GROUP_ROLES_MAP[role])
-                else:
-                    controlflow.system_error_exit(
-                        2,
-                        f'{role} is not a valid role for "gam print group-members {myarg}"'
-                    )
-            i += 2
-        elif myarg in ['cigroup', 'cigroups']:
-            group_email = gam.normalizeEmailAddressOrUID(sys.argv[i + 1])
-            groups_to_get = [group_email]
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam print cigroup-members')
-    if not groups_to_get:
-        gam.printGettingAllItems('Groups', None)
-        page_message = gapi.got_total_items_first_last_msg('Groups')
-        groups_to_get = gapi.get_all_pages(
-            ci.groups(),
-            'list',
-            'groups',
-            message_attribute=['groupKey', 'id'],
-            page_message=page_message,
-            parent=parent,
-            view='BASIC',
-            pageSize=1000,
-            fields='nextPageToken,groups(groupKey(id))')
-        groups_to_get = [group['groupKey']['id'] for group in groups_to_get]
-    i = 0
-    count = len(groups_to_get)
-    for group_email in groups_to_get:
-        i += 1
-
-        sys.stderr.write(
-            f'Getting members for {group_email}{gam.currentCountNL(i, count)}')
-        group_id = group_email_to_id(ci, group_email)
-        print(f'Getting members of cigroup {group_email}...')
-        page_message = f' {gapi.got_total_items_first_last_msg("Members")}'
-        group_members = gapi.get_all_pages(
-            ci.groups().memberships(),
-            'list',
-            'memberships',
-            soft_errors=True,
-            parent=group_id,
-            view='FULL',
-            pageSize=500,
-            page_message=page_message,
-            message_attribute=['memberKey', 'id'])
-        #fields='nextPageToken,memberships(memberKey,roles,createTime,updateTime)')
-        if roles:
-            group_members = filter_members_to_roles(group_members, roles)
-        for member in group_members:
-            # reduce role to a single value
-            member['role'] = get_single_role(member.pop('roles'))
-            member = utils.flatten_json(member)
-            for title in member:
-                if title not in titles:
-                    titles.append(title)
-            member['group'] = group_email
-            csvRows.append(member)
-    display.write_csv_file(csvRows, titles, 'Group Members', todrive)
-
-
-def update():
-
-    # Convert foo@googlemail.com to foo@gmail.com; eliminate periods in name for foo.bar@gmail.com
-    def _cleanConsumerAddress(emailAddress, mapCleanToOriginal):
-        atLoc = emailAddress.find('@')
-        if atLoc > 0:
-            if emailAddress[atLoc + 1:] in ['gmail.com', 'googlemail.com']:
-                cleanEmailAddress = emailAddress[:atLoc].replace(
-                    '.', '') + '@gmail.com'
-                if cleanEmailAddress != emailAddress:
-                    mapCleanToOriginal[cleanEmailAddress] = emailAddress
-                    return cleanEmailAddress
-        return emailAddress
-
-    def _getRoleAndUsers():
-        checkSuspended = None
-        role = None
-        i = 5
-        if sys.argv[i].lower() in GROUP_ROLES_MAP:
-            role = GROUP_ROLES_MAP[sys.argv[i].lower()]
-            i += 1
-        if sys.argv[i].lower() in ['suspended', 'notsuspended']:
-            checkSuspended = sys.argv[i].lower() == 'suspended'
-            i += 1
-        if sys.argv[i].lower() in usergroup_types:
-            users_email = gam.getUsersToModify(entity_type=sys.argv[i].lower(),
-                                               entity=sys.argv[i + 1],
-                                               checkSuspended=checkSuspended,
-                                               groupUserMembersOnly=False)
-        else:
-            users_email = [
-                gam.normalizeEmailAddressOrUID(sys.argv[i],
-                                               checkForCustomerId=True)
-            ]
-        return (role, users_email)
-
-    ci = gapi_cloudidentity.build('cloudidentity_beta')
-    group = sys.argv[3]
-    myarg = sys.argv[4].lower()
-    items = []
-    if myarg in UPDATE_GROUP_SUBCMDS:
-        group = gam.normalizeEmailAddressOrUID(group)
-        if group.startswith('groups/'):
-            parent = group
-        else:
-            parent = group_email_to_id(ci, group)
-        if not parent:
-            return
-        if myarg == 'add':
-            role, users_email = _getRoleAndUsers()
-            if not role:
-                role = ROLE_MEMBER
-            if len(users_email) > 1:
-                sys.stderr.write(
-                    f'Group: {group}, Will add {len(users_email)} {role}s.\n')
-                for user_email in users_email:
-                    item = [
-                        'gam', 'update', 'cigroup', f'id:{parent}', 'add', role,
-                        user_email
-                    ]
-                    items.append(item)
-            elif len(users_email) > 0:
-                body = {
-                    'memberKey': {
-                        'id': users_email[0]
-                    },
-                    'roles': [{
-                        'name': ROLE_MEMBER
-                    }]
-                }
-                if role != ROLE_MEMBER:
-                    body['roles'].append({'name': role})
-                add_text = [f'as {role}']
-                for i in range(2):
-                    try:
-                        gapi.call(
-                            ci.groups().memberships(),
-                            'create',
-                            throw_reasons=[
-                                gapi_errors.ErrorReason.FOUR_O_NINE,
-                                gapi_errors.ErrorReason.MEMBER_NOT_FOUND,
-                                gapi_errors.ErrorReason.RESOURCE_NOT_FOUND,
-                                gapi_errors.ErrorReason.INVALID_MEMBER,
-                                gapi_errors.ErrorReason.
-                                CYCLIC_MEMBERSHIPS_NOT_ALLOWED
-                            ],
-                            parent=parent,
-                            body=body)
-                        print(
-                            f' Group: {group}, {users_email[0]} Added {" ".join(add_text)}'
-                        )
-                        break
-                    except (gapi_errors.GapiMemberNotFoundError,
-                            gapi_errors.GapiResourceNotFoundError,
-                            gapi_errors.GapiInvalidMemberError,
-                            gapi_errors.GapiCyclicMembershipsNotAllowedError
-                           ) as e:
-                        print(
-                            f' Group: {group}, {users_email[0]} Add {" ".join(add_text)} Failed: {str(e)}'
-                        )
-                        break
-        elif myarg == 'sync':
-            syncMembersSet = set()
-            syncMembersMap = {}
-            role, users_email = _getRoleAndUsers()
-            for user_email in users_email:
-                if user_email in ('*', GC_Values[GC_CUSTOMER_ID]):
-                    syncMembersSet.add(GC_Values[GC_CUSTOMER_ID])
-                else:
-                    syncMembersSet.add(
-                        _cleanConsumerAddress(user_email.lower(),
-                                              syncMembersMap))
-            currentMembersSet = set()
-            currentMembersMap = {}
-            for current_email in gam.getUsersToModify(
-                    entity_type='cigroup',
-                    entity=group,
-                    member_type=role,
-                    groupUserMembersOnly=False):
-                if current_email == GC_Values[GC_CUSTOMER_ID]:
-                    currentMembersSet.add(current_email)
-                else:
-                    currentMembersSet.add(
-                        _cleanConsumerAddress(current_email.lower(),
-                                              currentMembersMap))
-            to_add = [
-                syncMembersMap.get(emailAddress, emailAddress)
-                for emailAddress in syncMembersSet - currentMembersSet
-            ]
-            to_remove = [
-                currentMembersMap.get(emailAddress, emailAddress)
-                for emailAddress in currentMembersSet - syncMembersSet
-            ]
-            sys.stderr.write(
-                f'Group: {group}, Will add {len(to_add)} and remove {len(to_remove)} {role}s.\n'
-            )
-            for user in to_add:
-                item = [
-                    'gam', 'update', 'cigroup', f'id:{parent}', 'add', role,
-                    user
-                ]
-                items.append(item)
-            for user in to_remove:
-                items.append([
-                    'gam', 'update', 'cigroup', f'id:{parent}', 'remove', user
-                ])
-        elif myarg in ['delete', 'remove']:
-            _, users_email = _getRoleAndUsers()
-            if len(users_email) > 1:
-                sys.stderr.write(
-                    f'Group: {group}, Will remove {len(users_email)} emails.\n')
-                for user_email in users_email:
-                    items.append([
-                        'gam', 'update', 'cigroup', f'id:{parent}', 'remove',
-                        user_email
-                    ])
-            elif len(users_email) == 1:
-                name = membership_email_to_id(ci, parent, users_email[0])
-                try:
-                    gapi.call(ci.groups().memberships(),
-                              'delete',
-                              throw_reasons=[
-                                  gapi_errors.ErrorReason.MEMBER_NOT_FOUND,
-                                  gapi_errors.ErrorReason.INVALID_MEMBER
-                              ],
-                              name=name)
-                    print(f' Group: {group}, {users_email[0]} Removed')
-                except (gapi_errors.GapiMemberNotFoundError,
-                        gapi_errors.GapiInvalidMemberError) as e:
-                    print(
-                        f' Group: {group}, {users_email[0]} Remove Failed: {str(e)}'
-                    )
-        elif myarg == 'update':
-            role, users_email = _getRoleAndUsers()
-            if not role:
-                role = ROLE_MEMBER
-            if len(users_email) > 1:
-                sys.stderr.write(
-                    f'Group: {group}, Will update {len(users_email)} {role}s.\n'
-                )
-                for user_email in users_email:
-                    item = [
-                        'gam', 'update', 'cigroup', f'id:{parent}', 'update',
-                        role, user_email
-                    ]
-                    items.append(item)
-            elif len(users_email) > 0:
-                name = membership_email_to_id(ci, parent, users_email[0])
-                addRoles = []
-                removeRoles = []
-                current_roles = gapi.call(ci.groups().memberships(),
-                                          'get',
-                                          name=name,
-                                          fields='roles').get('roles', [])
-                current_roles = [role['name'] for role in current_roles]
-                for crole in current_roles:
-                    if crole not in {ROLE_MEMBER, role}:
-                        removeRoles.append(crole)
-                if role not in current_roles:
-                    addRoles.append({'name': role})
-                bodys = []
-                if addRoles:
-                    bodys.append({'addRoles': addRoles})
-                if removeRoles:
-                    bodys.append({'removeRoles': removeRoles})
-                for body in bodys:
-                    try:
-                        gapi.call(ci.groups().memberships(),
-                                  'modifyMembershipRoles',
-                                  throw_reasons=[
-                                      gapi_errors.ErrorReason.MEMBER_NOT_FOUND,
-                                      gapi_errors.ErrorReason.INVALID_MEMBER
-                                  ],
-                                  name=name,
-                                  body=body)
-                    except (gapi_errors.GapiMemberNotFoundError,
-                            gapi_errors.GapiInvalidMemberError) as e:
-                        print(
-                            f' Group: {group}, {users_email[0]} Update to {role} Failed: {str(e)}'
-                        )
-                        break
-                print(
-                  f' Group: {group}, {users_email[0]} Updated to {role}'
-                )
-
-        else:  # clear
-            roles = []
-            i = 5
-            while i < len(sys.argv):
-                myarg = sys.argv[i].lower()
-                if myarg.upper() in [ROLE_OWNER, ROLE_MANAGER, ROLE_MEMBER]:
-                    roles.append(myarg.upper())
-                    i += 1
-                else:
-                    controlflow.invalid_argument_exit(
-                        sys.argv[i], 'gam update cigroup clear')
-            if not roles:
-                roles = [ROLE_MEMBER]
-            group = gam.normalizeEmailAddressOrUID(group)
-            member_type_message = f'{",".join(roles).lower()}s'
-            sys.stderr.write(
-                f'Getting {member_type_message} of {group} (may take some time for large groups)...\n'
-            )
-            page_message = gapi.got_total_items_msg(f'{member_type_message}',
-                                                    '...')
-            try:
-                result = gapi.get_all_pages(
-                    ci.groups().memberships(),
-                    'list',
-                    'memberships',
-                    page_message=page_message,
-                    throw_reasons=gapi_errors.MEMBERS_THROW_REASONS,
-                    parent=parent,
-                    fields='nextPageToken,memberships(memberKey,roles)')
-                result = filter_members_to_roles(result, roles)
-                if not result:
-                    print('Group already has 0 members')
-                    return
-                users_email = [member['memberKey']['id'] for member in result]
-                sys.stderr.write(
-                    f'Group: {group}, Will remove {len(users_email)} {", ".join(roles).lower()}s.\n'
-                )
-                for user_email in users_email:
-                    items.append([
-                        'gam', 'update', 'cigroup', group, 'remove', user_email
-                    ])
-            except (gapi_errors.GapiGroupNotFoundError,
-                    gapi_errors.GapiDomainNotFoundError,
-                    gapi_errors.GapiInvalidError,
-                    gapi_errors.GapiForbiddenError):
-                gam.entityUnknownWarning('Group', group, 0, 0)
-        if items:
-            gam.run_batch(items)
-    else:
-        i = 4
-        body = {}
-        while i < len(sys.argv):
-            myarg = sys.argv[i].lower().replace('_', '')
-            if myarg == 'name':
-                body['displayName'] = sys.argv[i + 1]
-                i += 2
-            elif myarg == 'description':
-                body['description'] = sys.argv[i + 1]
-                i += 2
-            elif myarg == 'security':
-                body['labels'] = {
-                    'cloudidentity.googleapis.com/groups.security': '',
-                    'cloudidentity.googleapis.com/groups.discussion_forum': ''
-                }
-                i += 1
-            else:
-                controlflow.invalid_argument_exit(sys.argv[i],
-                                                  'gam update cigroup')
-        updateMask = ','.join(body.keys())
-        name = group_email_to_id(ci, group)
-        print(f'Updating group {group}')
-        gapi.call(ci.groups(),
-                  'patch',
-                  updateMask=updateMask,
-                  name=name,
-                  body=body)
-
-
-def group_email_to_id(ci, group, i=0, count=0):
-    group = gam.normalizeEmailAddressOrUID(group)
-    try:
-        return gapi.call(ci.groups(),
-                         'lookup',
-                         throw_reasons=gapi_errors.GROUP_GET_THROW_REASONS,
-                         retry_reasons=gapi_errors.GROUP_GET_RETRY_REASONS,
-                         groupKey_id=group,
-                         fields='name').get('name')
-    except (gapi_errors.GapiGroupNotFoundError,
-            gapi_errors.GapiDomainNotFoundError,
-            gapi_errors.GapiDomainCannotUseApisError,
-            gapi_errors.GapiForbiddenError, gapi_errors.GapiBadRequestError):
-        gam.entityUnknownWarning('Group', group, i, count)
-        return None
-
-
-def membership_email_to_id(ci, parent, membership, i=0, count=0):
-    membership = gam.normalizeEmailAddressOrUID(membership)
-    try:
-        return gapi.call(ci.groups().memberships(),
-                         'lookup',
-                         throw_reasons=gapi_errors.GROUP_GET_THROW_REASONS,
-                         retry_reasons=gapi_errors.GROUP_GET_RETRY_REASONS,
-                         parent=parent,
-                         memberKey_id=membership,
-                         fields='name').get('name')
-    except (gapi_errors.GapiGroupNotFoundError,
-            gapi_errors.GapiDomainNotFoundError,
-            gapi_errors.GapiDomainCannotUseApisError,
-            gapi_errors.GapiForbiddenError, gapi_errors.GapiBadRequestError):
-        gam.entityUnknownWarning('Membership', membership, i, count)
-        return None
-
-
-def get_single_role(roles):
-    ''' returns the highest role of member '''
-    roles = [role.get('name') for role in roles]
-    if not roles:
-        return ROLE_MEMBER
-    for a_role in [ROLE_OWNER, ROLE_MANAGER, ROLE_MEMBER]:
-        if a_role in roles:
-            return a_role
-    return roles[0]
-
-
-def filter_members_to_roles(members, roles):
-    filtered_members = []
-    for member in members:
-        role = get_single_role(member.get('roles', []))
-        if role in roles:
-            filtered_members.append(member)
-    return filtered_members

src/gam/gapi/directory/__init__.py

@@ -1,5 +0,0 @@
-import gam
-
-
-def build():
-    return gam.buildGAPIObject('directory')

src/gam/gapi/directory/asps.py

@@ -1,57 +0,0 @@
-import sys
-
-from gam.var import *
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-from gam import utils
-
-
-def info(users):
-    cd = gapi_directory.build()
-    for user in users:
-        asps = gapi.get_items(cd.asps(), 'list', 'items', userKey=user)
-        if asps:
-            print(f'Application-Specific Passwords for {user}')
-            for asp in asps:
-                if asp['creationTime'] == '0':
-                    created_date = 'Unknown'
-                else:
-                    created_date = utils.formatTimestampYMDHMS(
-                        asp['creationTime'])
-                if asp['lastTimeUsed'] == '0':
-                    used_date = 'Never'
-                else:
-                    last_used = asp['lastTimeUsed']
-                    used_date = utils.formatTimestampYMDHMS(last_used)
-                print(f' ID: {asp["codeId"]}\n' \
-                      f'  Name: {asp["name"]}\n' \
-                      f'  Created: {created_date}\n' \
-                      f'  Last Used: {used_date}\n')
-        else:
-            print(f' no ASPs for {user}\n')
-
-
-def delete(users, cd=None, codeIdList=None):
-    if not cd:
-        cd = gapi_directory.build()
-    if not codeIdList:
-        codeIdList = sys.argv[5].lower()
-    if codeIdList == 'all':
-        allCodeIds = True
-    else:
-        allCodeIds = False
-        codeIds = codeIdList.replace(',', ' ').split()
-    for user in users:
-        if allCodeIds:
-            print(f'Getting Application Specific Passwords for {user}')
-            asps = gapi.get_items(cd.asps(),
-                                  'list',
-                                  'items',
-                                  userKey=user,
-                                  fields='items/codeId')
-            codeIds = [asp['codeId'] for asp in asps]
-        if not codeIds:
-            print('No ASPs')
-        for codeId in codeIds:
-            gapi.call(cd.asps(), 'delete', userKey=user, codeId=codeId)
-            print(f'deleted ASP {codeId} for {user}')

src/gam/gapi/directory/cros.py

@@ -1,832 +0,0 @@
-import datetime
-
-from gam.var import *
-import gam
-from gam import controlflow
-from gam import display
-from gam import fileutils
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-from gam.gapi.directory import orgunits as gapi_directory_orgunits
-from gam import utils
-
-
-def doUpdateCros():
-    cd = gapi_directory.build()
-    i, devices = getCrOSDeviceEntity(3, cd)
-    update_body = {}
-    action_body = {}
-    orgUnitPath = None
-    ack_wipe = False
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'user':
-            update_body['annotatedUser'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'location':
-            update_body['annotatedLocation'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'notes':
-            update_body['notes'] = sys.argv[i + 1].replace('\\n', '\n')
-            i += 2
-        elif myarg in ['tag', 'asset', 'assetid']:
-            update_body['annotatedAssetId'] = sys.argv[i + 1]
-            i += 2
-        elif myarg in ['ou', 'org']:
-            orgUnitPath = gapi_directory_orgunits.getOrgUnitItem(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'action':
-            action = sys.argv[i + 1].lower().replace('_', '').replace('-', '')
-            deprovisionReason = None
-            if action in [
-                    'deprovisionsamemodelreplace',
-                    'deprovisionsamemodelreplacement'
-            ]:
-                action = 'deprovision'
-                deprovisionReason = 'same_model_replacement'
-            elif action in [
-                    'deprovisiondifferentmodelreplace',
-                    'deprovisiondifferentmodelreplacement'
-            ]:
-                action = 'deprovision'
-                deprovisionReason = 'different_model_replacement'
-            elif action in ['deprovisionretiringdevice']:
-                action = 'deprovision'
-                deprovisionReason = 'retiring_device'
-            elif action == 'deprovisionupgradetransfer':
-                action = 'deprovision'
-                deprovisionReason = 'upgrade_transfer'
-            elif action not in ['disable', 'reenable']:
-                controlflow.system_error_exit(2, f'expected action of ' \
-                    f'deprovision_same_model_replace, ' \
-                    f'deprovision_different_model_replace, ' \
-                    f'deprovision_retiring_device, ' \
-                    f'deprovision_upgrade_transfer, disable or reenable,'
-                    f' got {action}')
-            action_body = {'action': action}
-            if deprovisionReason:
-                action_body['deprovisionReason'] = deprovisionReason
-            i += 2
-        elif myarg == 'acknowledgedevicetouchrequirement':
-            ack_wipe = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam update cros')
-    i = 0
-    count = len(devices)
-    if action_body:
-        if action_body['action'] == 'deprovision' and not ack_wipe:
-            print(f'WARNING: Refusing to deprovision {count} devices because '
-                  'acknowledge_device_touch_requirement not specified. ' \
-                  'Deprovisioning a device means the device will have to ' \
-                  'be physically wiped and re-enrolled to be managed by ' \
-                  'your domain again. This requires physical access to ' \
-                  'the device and is very time consuming to perform for ' \
-                  'each device. Please add ' \
-                  '"acknowledge_device_touch_requirement" to the GAM ' \
-                  'command if you understand this and wish to proceed ' \
-                  'with the deprovision. Please also be aware that ' \
-                  'deprovisioning can have an effect on your device ' \
-                  'license count. See ' \
-                  'https://support.google.com/chrome/a/answer/3523633 '\
-                  'for full details.')
-            sys.exit(3)
-        for deviceId in devices:
-            i += 1
-            cur_count = gam.currentCount(i, count)
-            print(f' performing action {action} for {deviceId}{cur_count}')
-            gapi.call(cd.chromeosdevices(),
-                      function='action',
-                      customerId=GC_Values[GC_CUSTOMER_ID],
-                      resourceId=deviceId,
-                      body=action_body)
-    else:
-        if update_body:
-            for deviceId in devices:
-                i += 1
-                current_count = gam.currentCount(i, count)
-                print(f' updating {deviceId}{current_count}')
-                gapi.call(cd.chromeosdevices(),
-                          'update',
-                          customerId=GC_Values[GC_CUSTOMER_ID],
-                          deviceId=deviceId,
-                          body=update_body)
-        if orgUnitPath:
-            # split moves into max 50 devices per batch
-            for l in range(0, len(devices), 50):
-                move_body = {'deviceIds': devices[l:l + 50]}
-                print(f' moving {len(move_body["deviceIds"])} devices to ' \
-                      f'{orgUnitPath}')
-                gapi.call(cd.chromeosdevices(),
-                          'moveDevicesToOu',
-                          customerId=GC_Values[GC_CUSTOMER_ID],
-                          orgUnitPath=orgUnitPath,
-                          body=move_body)
-
-
-def doGetCrosInfo():
-    cd = gapi_directory.build()
-    i, devices = getCrOSDeviceEntity(3, cd)
-    downloadfile = None
-    targetFolder = GC_Values[GC_DRIVE_DIR]
-    projection = None
-    fieldsList = []
-    noLists = False
-    startDate = endDate = None
-    listLimit = 0
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'nolists':
-            noLists = True
-            i += 1
-        elif myarg == 'listlimit':
-            listLimit = gam.getInteger(sys.argv[i + 1], myarg, minVal=-1)
-            i += 2
-        elif myarg in CROS_START_ARGUMENTS:
-            startDate = _getFilterDate(sys.argv[i + 1])
-            i += 2
-        elif myarg in CROS_END_ARGUMENTS:
-            endDate = _getFilterDate(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'allfields':
-            projection = 'FULL'
-            fieldsList = []
-            i += 1
-        elif myarg in PROJECTION_CHOICES_MAP:
-            projection = PROJECTION_CHOICES_MAP[myarg]
-            if projection == 'FULL':
-                fieldsList = []
-            else:
-                fieldsList = CROS_BASIC_FIELDS_LIST[:]
-            i += 1
-        elif myarg in CROS_ARGUMENT_TO_PROPERTY_MAP:
-            fieldsList.extend(CROS_ARGUMENT_TO_PROPERTY_MAP[myarg])
-            i += 1
-        elif myarg == 'fields':
-            fieldNameList = sys.argv[i + 1]
-            for field in fieldNameList.lower().replace(',', ' ').split():
-                if field in CROS_ARGUMENT_TO_PROPERTY_MAP:
-                    fieldsList.extend(CROS_ARGUMENT_TO_PROPERTY_MAP[field])
-                    if field in CROS_ACTIVE_TIME_RANGES_ARGUMENTS + \
-                                CROS_DEVICE_FILES_ARGUMENTS + \
-                                CROS_RECENT_USERS_ARGUMENTS:
-                        projection = 'FULL'
-                        noLists = False
-                else:
-                    controlflow.invalid_argument_exit(field,
-                                                      'gam info cros fields')
-            i += 2
-        elif myarg == 'downloadfile':
-            downloadfile = sys.argv[i + 1]
-            if downloadfile.lower() == 'latest':
-                downloadfile = downloadfile.lower()
-            i += 2
-        elif myarg == 'targetfolder':
-            targetFolder = os.path.expanduser(sys.argv[i + 1])
-            if not os.path.isdir(targetFolder):
-                os.makedirs(targetFolder)
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam info cros')
-    if fieldsList:
-        fieldsList.append('deviceId')
-        fields = ','.join(set(fieldsList)).replace('.', '/')
-    else:
-        fields = None
-    i = 0
-    device_count = len(devices)
-    for deviceId in devices:
-        i += 1
-        cros = gapi.call(cd.chromeosdevices(),
-                         'get',
-                         customerId=GC_Values[GC_CUSTOMER_ID],
-                         deviceId=deviceId,
-                         projection=projection,
-                         fields=fields)
-        print(f'CrOS Device: {deviceId} ({i} of {device_count})')
-        if 'notes' in cros:
-            cros['notes'] = cros['notes'].replace('\n', '\\n')
-        if 'autoUpdateExpiration' in cros:
-            cros['autoUpdateExpiration'] = utils.formatTimestampYMD(
-                cros['autoUpdateExpiration'])
-        _checkTPMVulnerability(cros)
-        for up in CROS_SCALAR_PROPERTY_PRINT_ORDER:
-            if up in cros:
-                if isinstance(cros[up], str):
-                    print(f'  {up}: {cros[up]}')
-                else:
-                    sys.stdout.write(f'  {up}:')
-                    display.print_json(cros[up], '  ')
-        if not noLists:
-            activeTimeRanges = _filterTimeRanges(
-                cros.get('activeTimeRanges', []), startDate, endDate)
-            lenATR = len(activeTimeRanges)
-            if lenATR:
-                print('  activeTimeRanges')
-                num_ranges = min(lenATR, listLimit or lenATR)
-                for activeTimeRange in activeTimeRanges[:num_ranges]:
-                    active_date = activeTimeRange['date']
-                    active_time = activeTimeRange['activeTime']
-                    duration = utils.formatMilliSeconds(active_time)
-                    minutes = active_time // 60000
-                    print(f'    date: {active_date}')
-                    print(f'      activeTime: {active_time}')
-                    print(f'      duration: {duration}')
-                    print(f'      minutes: {minutes}')
-            recentUsers = cros.get('recentUsers', [])
-            lenRU = len(recentUsers)
-            if lenRU:
-                print('  recentUsers')
-                num_ranges = min(lenRU, listLimit or lenRU)
-                for recentUser in recentUsers[:num_ranges]:
-                    useremail = recentUser.get('email')
-                    if not useremail:
-                        if recentUser['type'] == 'USER_TYPE_UNMANAGED':
-                            useremail = 'UnmanagedUser'
-                        else:
-                            useremail = 'Unknown'
-                    print(f'    type: {recentUser["type"]}')
-                    print(f'      email: {useremail}')
-            deviceFiles = _filterCreateReportTime(cros.get('deviceFiles',
-                                                           []), 'createTime',
-                                                  startDate, endDate)
-            lenDF = len(deviceFiles)
-            if lenDF:
-                num_ranges = min(lenDF, listLimit or lenDF)
-                print('  deviceFiles')
-                for deviceFile in deviceFiles[:num_ranges]:
-                    device_type = deviceFile['type']
-                    create_time = deviceFile['createTime']
-                    print(f'    {device_type}: {create_time}')
-            if downloadfile:
-                deviceFiles = cros.get('deviceFiles', [])
-                lenDF = len(deviceFiles)
-                if lenDF:
-                    if downloadfile == 'latest':
-                        deviceFile = deviceFiles[-1]
-                    else:
-                        for deviceFile in deviceFiles:
-                            if deviceFile['createTime'] == downloadfile:
-                                break
-                        else:
-                            print(f'ERROR: file {downloadfile} not ' \
-                                  f'available to download.')
-                            deviceFile = None
-                    if deviceFile:
-                        created = deviceFile['createTime']
-                        downloadfile = f'cros-logs-{deviceId}-{created}.zip'
-                        downloadfilename = os.path.join(targetFolder,
-                                                        downloadfile)
-                        dl_url = deviceFile['downloadUrl']
-                        _, content = cd._http.request(dl_url)
-                        fileutils.write_file(downloadfilename,
-                                             content,
-                                             mode='wb',
-                                             continue_on_error=True)
-                        print(f'Downloaded: {downloadfilename}')
-                elif downloadfile:
-                    print('ERROR: no files to download.')
-            cpuStatusReports = _filterCreateReportTime(
-                cros.get('cpuStatusReports', []), 'reportTime', startDate,
-                endDate)
-            lenCSR = len(cpuStatusReports)
-            if lenCSR:
-                print('  cpuStatusReports')
-                num_ranges = min(lenCSR, listLimit or lenCSR)
-                for cpuStatusReport in cpuStatusReports[:num_ranges]:
-                    print(f'    reportTime: {cpuStatusReport["reportTime"]}')
-                    print('      cpuTemperatureInfo')
-                    tempInfos = cpuStatusReport.get('cpuTemperatureInfo', [])
-                    for tempInfo in tempInfos:
-                        temp_label = tempInfo['label'].strip()
-                        temperature = tempInfo['temperature']
-                        print(f'        {temp_label}: {temperature}')
-                    pct_info = cpuStatusReport['cpuUtilizationPercentageInfo']
-                    util = ','.join([str(x) for x in pct_info])
-                    print(f'      cpuUtilizationPercentageInfo: {util}')
-            diskVolumeReports = cros.get('diskVolumeReports', [])
-            lenDVR = len(diskVolumeReports)
-            if lenDVR:
-                print('  diskVolumeReports')
-                print('    volumeInfo')
-                num_ranges = min(lenDVR, listLimit or lenDVR)
-                for diskVolumeReport in diskVolumeReports[:num_ranges]:
-                    volumeInfo = diskVolumeReport['volumeInfo']
-                    for volume in volumeInfo:
-                        vid = volume['volumeId']
-                        vstorage_free = volume['storageFree']
-                        vstorage_total = volume['storageTotal']
-                        print(f'      volumeId: {vid}')
-                        print(f'        storageFree: {vstorage_free}')
-                        print(f'        storageTotal: {vstorage_total}')
-            systemRamFreeReports = _filterCreateReportTime(
-                cros.get('systemRamFreeReports', []), 'reportTime', startDate,
-                endDate)
-            lenSRFR = len(systemRamFreeReports)
-            if lenSRFR:
-                print('  systemRamFreeReports')
-                num_ranges = min(lenSRFR, listLimit or lenSRFR)
-                for systemRamFreeReport in systemRamFreeReports[:num_ranges]:
-                    report_time = systemRamFreeReport['reportTime']
-                    free_info = systemRamFreeReport['systemRamFreeInfo']
-                    free_ram = ','.join(free_info)
-                    print(f'    reportTime: {report_time}')
-                    print(f'      systemRamFreeInfo: {free_ram}')
-
-
-def doPrintCrosActivity():
-    cd = gapi_directory.build()
-    todrive = False
-    titles = [
-        'deviceId', 'annotatedAssetId', 'annotatedLocation', 'serialNumber',
-        'orgUnitPath'
-    ]
-    csvRows = []
-    fieldsList = [
-        'deviceId', 'annotatedAssetId', 'annotatedLocation', 'serialNumber',
-        'orgUnitPath'
-    ]
-    startDate = endDate = None
-    selectActiveTimeRanges = selectDeviceFiles = selectRecentUsers = False
-    listLimit = 0
-    delimiter = ','
-    orgUnitPath = None
-    queries = [None]
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg in ['query', 'queries']:
-            queries = gam.getQueries(myarg, sys.argv[i + 1])
-            i += 2
-        elif myarg == 'limittoou':
-            orgUnitPath = gapi_directory_orgunits.getOrgUnitItem(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg in CROS_ACTIVE_TIME_RANGES_ARGUMENTS:
-            selectActiveTimeRanges = True
-            i += 1
-        elif myarg in CROS_DEVICE_FILES_ARGUMENTS:
-            selectDeviceFiles = True
-            i += 1
-        elif myarg in CROS_RECENT_USERS_ARGUMENTS:
-            selectRecentUsers = True
-            i += 1
-        elif myarg == 'both':
-            selectActiveTimeRanges = selectRecentUsers = True
-            i += 1
-        elif myarg == 'all':
-            selectActiveTimeRanges = selectDeviceFiles = True
-            selectRecentUsers = True
-            i += 1
-        elif myarg in CROS_START_ARGUMENTS:
-            startDate = _getFilterDate(sys.argv[i + 1])
-            i += 2
-        elif myarg in CROS_END_ARGUMENTS:
-            endDate = _getFilterDate(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'listlimit':
-            listLimit = gam.getInteger(sys.argv[i + 1], myarg, minVal=0)
-            i += 2
-        elif myarg == 'delimiter':
-            delimiter = sys.argv[i + 1]
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam print crosactivity')
-    if not selectActiveTimeRanges and \
-       not selectDeviceFiles and \
-       not selectRecentUsers:
-        selectActiveTimeRanges = selectRecentUsers = True
-    if selectRecentUsers:
-        fieldsList.append('recentUsers')
-        display.add_titles_to_csv_file([
-            'recentUsers.email',
-        ], titles)
-    if selectActiveTimeRanges:
-        fieldsList.append('activeTimeRanges')
-        titles_to_add = [
-            'activeTimeRanges.date', 'activeTimeRanges.duration',
-            'activeTimeRanges.minutes'
-        ]
-        display.add_titles_to_csv_file(titles_to_add, titles)
-    if selectDeviceFiles:
-        fieldsList.append('deviceFiles')
-        titles_to_add = ['deviceFiles.type', 'deviceFiles.createTime']
-        display.add_titles_to_csv_file(titles_to_add, titles)
-    fields = f'nextPageToken,chromeosdevices({",".join(fieldsList)})'
-    for query in queries:
-        gam.printGettingAllItems('CrOS Devices', query)
-        page_message = gapi.got_total_items_msg('CrOS Devices', '...\n')
-        all_cros = gapi.get_all_pages(cd.chromeosdevices(),
-                                      'list',
-                                      'chromeosdevices',
-                                      page_message=page_message,
-                                      query=query,
-                                      customerId=GC_Values[GC_CUSTOMER_ID],
-                                      projection='FULL',
-                                      fields=fields,
-                                      orgUnitPath=orgUnitPath)
-        for cros in all_cros:
-            row = {}
-            skip_attribs = ['recentUsers', 'activeTimeRanges', 'deviceFiles']
-            for attrib in cros:
-                if attrib not in skip_attribs:
-                    row[attrib] = cros[attrib]
-            if selectActiveTimeRanges:
-                activeTimeRanges = _filterTimeRanges(
-                    cros.get('activeTimeRanges', []), startDate, endDate)
-                lenATR = len(activeTimeRanges)
-                num_ranges = min(lenATR, listLimit or lenATR)
-                for activeTimeRange in activeTimeRanges[:num_ranges]:
-                    newrow = row.copy()
-                    newrow['activeTimeRanges.date'] = activeTimeRange['date']
-                    active_time = activeTimeRange['activeTime']
-                    newrow['activeTimeRanges.duration'] = \
-                        utils.formatMilliSeconds(active_time)
-                    newrow['activeTimeRanges.minutes'] = \
-                        activeTimeRange['activeTime']//60000
-                    csvRows.append(newrow)
-            if selectRecentUsers:
-                recentUsers = cros.get('recentUsers', [])
-                lenRU = len(recentUsers)
-                num_ranges = min(lenRU, listLimit or lenRU)
-                recent_users = []
-                for recentUser in recentUsers[:num_ranges]:
-                    useremail = recentUser.get('email')
-                    if not useremail:
-                        if recentUser['type'] == 'USER_TYPE_UNMANAGED':
-                            useremail = 'UnmanagedUser'
-                        else:
-                            useremail = 'Unknown'
-                    recent_users.append(useremail)
-                row['recentUsers.email'] = delimiter.join(recent_users)
-                csvRows.append(row)
-            if selectDeviceFiles:
-                deviceFiles = _filterCreateReportTime(
-                    cros.get('deviceFiles', []), 'createTime', startDate,
-                    endDate)
-                lenDF = len(deviceFiles)
-                num_ranges = min(lenDF, listLimit or lenDF)
-                for deviceFile in deviceFiles[:num_ranges]:
-                    newrow = row.copy()
-                    newrow['deviceFiles.type'] = deviceFile['type']
-                    create_time = deviceFile['createTime']
-                    newrow['deviceFiles.createTime'] = create_time
-                    csvRows.append(newrow)
-    display.write_csv_file(csvRows, titles, 'CrOS Activity', todrive)
-
-
-def _checkTPMVulnerability(cros):
-    if 'tpmVersionInfo' in cros and \
-       'firmwareVersion' in cros['tpmVersionInfo']:
-        firmware_version = cros['tpmVersionInfo']['firmwareVersion']
-        if firmware_version in CROS_TPM_VULN_VERSIONS:
-            cros['tpmVersionInfo']['tpmVulnerability'] = 'VULNERABLE'
-        elif firmware_version in CROS_TPM_FIXED_VERSIONS:
-            cros['tpmVersionInfo']['tpmVulnerability'] = 'UPDATED'
-        else:
-            cros['tpmVersionInfo']['tpmVulnerability'] = 'NOT IMPACTED'
-
-
-def doPrintCrosDevices():
-
-    def _getSelectedLists(myarg):
-        if myarg in CROS_ACTIVE_TIME_RANGES_ARGUMENTS:
-            selectedLists['activeTimeRanges'] = True
-        elif myarg in CROS_RECENT_USERS_ARGUMENTS:
-            selectedLists['recentUsers'] = True
-        elif myarg in CROS_DEVICE_FILES_ARGUMENTS:
-            selectedLists['deviceFiles'] = True
-        elif myarg in CROS_CPU_STATUS_REPORTS_ARGUMENTS:
-            selectedLists['cpuStatusReports'] = True
-        elif myarg in CROS_DISK_VOLUME_REPORTS_ARGUMENTS:
-            selectedLists['diskVolumeReports'] = True
-        elif myarg in CROS_SYSTEM_RAM_FREE_REPORTS_ARGUMENTS:
-            selectedLists['systemRamFreeReports'] = True
-
-    cd = gapi_directory.build()
-    todrive = False
-    fieldsList = []
-    fieldsTitles = {}
-    titles = []
-    csvRows = []
-    display.add_field_to_csv_file('deviceid', CROS_ARGUMENT_TO_PROPERTY_MAP,
-                                  fieldsList, fieldsTitles, titles)
-    projection = orderBy = sortOrder = orgUnitPath = None
-    queries = [None]
-    noLists = sortHeaders = False
-    selectedLists = {}
-    startDate = endDate = None
-    listLimit = 0
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg in ['query', 'queries']:
-            queries = gam.getQueries(myarg, sys.argv[i + 1])
-            i += 2
-        elif myarg == 'limittoou':
-            orgUnitPath = gapi_directory_orgunits.getOrgUnitItem(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg == 'nolists':
-            noLists = True
-            selectedLists = {}
-            i += 1
-        elif myarg == 'listlimit':
-            listLimit = gam.getInteger(sys.argv[i + 1], myarg, minVal=0)
-            i += 2
-        elif myarg in CROS_START_ARGUMENTS:
-            startDate = _getFilterDate(sys.argv[i + 1])
-            i += 2
-        elif myarg in CROS_END_ARGUMENTS:
-            endDate = _getFilterDate(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'orderby':
-            orderBy = sys.argv[i + 1].lower().replace('_', '')
-            validOrderBy = [
-                'location', 'user', 'lastsync', 'notes', 'serialnumber',
-                'status', 'supportenddate'
-            ]
-            if orderBy not in validOrderBy:
-                controlflow.expected_argument_exit('orderby',
-                                                   ', '.join(validOrderBy),
-                                                   orderBy)
-            if orderBy == 'location':
-                orderBy = 'annotatedLocation'
-            elif orderBy == 'user':
-                orderBy = 'annotatedUser'
-            elif orderBy == 'lastsync':
-                orderBy = 'lastSync'
-            elif orderBy == 'serialnumber':
-                orderBy = 'serialNumber'
-            elif orderBy == 'supportenddate':
-                orderBy = 'supportEndDate'
-            i += 2
-        elif myarg in SORTORDER_CHOICES_MAP:
-            sortOrder = SORTORDER_CHOICES_MAP[myarg]
-            i += 1
-        elif myarg in PROJECTION_CHOICES_MAP:
-            projection = PROJECTION_CHOICES_MAP[myarg]
-            sortHeaders = True
-            if projection == 'FULL':
-                fieldsList = []
-            else:
-                fieldsList = CROS_BASIC_FIELDS_LIST[:]
-            i += 1
-        elif myarg == 'allfields':
-            projection = 'FULL'
-            sortHeaders = True
-            fieldsList = []
-            i += 1
-        elif myarg == 'sortheaders':
-            sortHeaders = True
-            i += 1
-        elif myarg in CROS_LISTS_ARGUMENTS:
-            _getSelectedLists(myarg)
-            i += 1
-        elif myarg in CROS_ARGUMENT_TO_PROPERTY_MAP:
-            display.add_field_to_fields_list(myarg,
-                                             CROS_ARGUMENT_TO_PROPERTY_MAP,
-                                             fieldsList)
-            i += 1
-        elif myarg == 'fields':
-            fieldNameList = sys.argv[i + 1]
-            for field in fieldNameList.lower().replace(',', ' ').split():
-                if field in CROS_LISTS_ARGUMENTS:
-                    _getSelectedLists(field)
-                elif field in CROS_ARGUMENT_TO_PROPERTY_MAP:
-                    display.add_field_to_fields_list(
-                        field, CROS_ARGUMENT_TO_PROPERTY_MAP, fieldsList)
-                else:
-                    controlflow.invalid_argument_exit(field,
-                                                      'gam print cros fields')
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam print cros')
-    if selectedLists:
-        noLists = False
-        projection = 'FULL'
-        for selectList in selectedLists:
-            display.add_field_to_fields_list(selectList,
-                                             CROS_ARGUMENT_TO_PROPERTY_MAP,
-                                             fieldsList)
-    if fieldsList:
-        fieldsList.append('deviceId')
-        fields = f'nextPageToken,chromeosdevices({",".join(set(fieldsList))})'.replace(
-            '.', '/')
-    else:
-        fields = None
-    for query in queries:
-        gam.printGettingAllItems('CrOS Devices', query)
-        page_message = gapi.got_total_items_msg('CrOS Devices', '...\n')
-        all_cros = gapi.get_all_pages(cd.chromeosdevices(),
-                                      'list',
-                                      'chromeosdevices',
-                                      page_message=page_message,
-                                      query=query,
-                                      customerId=GC_Values[GC_CUSTOMER_ID],
-                                      projection=projection,
-                                      orgUnitPath=orgUnitPath,
-                                      orderBy=orderBy,
-                                      sortOrder=sortOrder,
-                                      fields=fields)
-        for cros in all_cros:
-            _checkTPMVulnerability(cros)
-        if not noLists and not selectedLists:
-            for cros in all_cros:
-                if 'notes' in cros:
-                    cros['notes'] = cros['notes'].replace('\n', '\\n')
-                if 'autoUpdateExpiration' in cros:
-                    cros['autoUpdateExpiration'] = utils.formatTimestampYMD(
-                        cros['autoUpdateExpiration'])
-                for cpuStatusReport in cros.get('cpuStatusReports', []):
-                    tempInfos = cpuStatusReport.get('cpuTemperatureInfo', [])
-                    for tempInfo in tempInfos:
-                        tempInfo['label'] = tempInfo['label'].strip()
-                display.add_row_titles_to_csv_file(
-                    utils.flatten_json(cros, listLimit=listLimit), csvRows,
-                    titles)
-            continue
-        for cros in all_cros:
-            if 'notes' in cros:
-                cros['notes'] = cros['notes'].replace('\n', '\\n')
-            if 'autoUpdateExpiration' in cros:
-                cros['autoUpdateExpiration'] = utils.formatTimestampYMD(
-                    cros['autoUpdateExpiration'])
-            row = {}
-            for attrib in cros:
-                if attrib not in set([
-                        'kind', 'etag', 'tpmVersionInfo', 'recentUsers',
-                        'activeTimeRanges', 'deviceFiles', 'cpuStatusReports',
-                        'diskVolumeReports', 'systemRamFreeReports'
-                ]):
-                    row[attrib] = cros[attrib]
-            if selectedLists.get('activeTimeRanges'):
-                timergs = cros.get('activeTimeRanges', [])
-            else:
-                timergs = []
-            activeTimeRanges = _filterTimeRanges(timergs, startDate, endDate)
-            if selectedLists.get('recentUsers'):
-                recentUsers = cros.get('recentUsers', [])
-            else:
-                recentUsers = []
-            if selectedLists.get('deviceFiles'):
-                device_files = cros.get('deviceFiles', [])
-            else:
-                device_files = []
-            deviceFiles = _filterCreateReportTime(device_files, 'createTime',
-                                                  startDate, endDate)
-            if selectedLists.get('cpuStatusReports'):
-                cpu_reports = cros.get('cpuStatusReports', [])
-            else:
-                cpu_reports = []
-            cpuStatusReports = _filterCreateReportTime(cpu_reports,
-                                                       'reportTime', startDate,
-                                                       endDate)
-            if selectedLists.get('diskVolumeReports'):
-                diskVolumeReports = cros.get('diskVolumeReports', [])
-            else:
-                diskVolumeReports = []
-            if selectedLists.get('systemRamFreeReports'):
-                ram_reports = cros.get('systemRamFreeReports', [])
-            else:
-                ram_reports = []
-            systemRamFreeReports = _filterCreateReportTime(
-                ram_reports, 'reportTime', startDate, endDate)
-            if noLists or (not activeTimeRanges and \
-                           not recentUsers and \
-                           not deviceFiles and \
-                           not cpuStatusReports and \
-                           not diskVolumeReports and \
-                           not systemRamFreeReports):
-                display.add_row_titles_to_csv_file(row, csvRows, titles)
-                continue
-            lenATR = len(activeTimeRanges)
-            lenRU = len(recentUsers)
-            lenDF = len(deviceFiles)
-            lenCSR = len(cpuStatusReports)
-            lenDVR = len(diskVolumeReports)
-            lenSRFR = len(systemRamFreeReports)
-            max_len = max(lenATR, lenRU, lenDF, lenCSR, lenDVR, lenSRFR)
-            for i in range(min(max_len, listLimit or max_len)):
-                nrow = row.copy()
-                if i < lenATR:
-                    nrow['activeTimeRanges.date'] = \
-                        activeTimeRanges[i]['date']
-                    nrow['activeTimeRanges.activeTime'] = \
-                        str(activeTimeRanges[i]['activeTime'])
-                    active_time = activeTimeRanges[i]['activeTime']
-                    nrow['activeTimeRanges.duration'] = \
-                        utils.formatMilliSeconds(active_time)
-                    nrow['activeTimeRanges.minutes'] = active_time // 60000
-                if i < lenRU:
-                    nrow['recentUsers.type'] = recentUsers[i]['type']
-                    nrow['recentUsers.email'] = recentUsers[i].get('email')
-                    if not nrow['recentUsers.email']:
-                        if nrow['recentUsers.type'] == 'USER_TYPE_UNMANAGED':
-                            nrow['recentUsers.email'] = 'UnmanagedUser'
-                        else:
-                            nrow['recentUsers.email'] = 'Unknown'
-                if i < lenDF:
-                    nrow['deviceFiles.type'] = deviceFiles[i]['type']
-                    nrow['deviceFiles.createTime'] = \
-                        deviceFiles[i]['createTime']
-                if i < lenCSR:
-                    nrow['cpuStatusReports.reportTime'] = \
-                        cpuStatusReports[i]['reportTime']
-                    tempInfos = cpuStatusReports[i].get('cpuTemperatureInfo',
-                                                        [])
-                    for tempInfo in tempInfos:
-                        label = tempInfo['label'].strip()
-                        base = 'cpuStatusReports.cpuTemperatureInfo.'
-                        nrow[f'{base}{label}'] = tempInfo['temperature']
-                    cpu_field = 'cpuUtilizationPercentageInfo'
-                    cpu_reports = cpuStatusReports[i][cpu_field]
-                    cpu_pcts = [str(x) for x in cpu_reports]
-                    nrow[f'cpuStatusReports.{cpu_field}'] = ','.join(cpu_pcts)
-                if i < lenDVR:
-                    volumeInfo = diskVolumeReports[i]['volumeInfo']
-                    j = 0
-                    vfield = 'diskVolumeReports.volumeInfo.'
-                    for volume in volumeInfo:
-                        nrow[f'{vfield}{j}.volumeId'] = \
-                            volume['volumeId']
-                        nrow[f'{vfield}{j}.storageFree'] = \
-                            volume['storageFree']
-                        nrow[f'{vfield}{j}.storageTotal'] = \
-                            volume['storageTotal']
-                        j += 1
-                if i < lenSRFR:
-                    nrow['systemRamFreeReports.reportTime'] = \
-                        systemRamFreeReports[i]['reportTime']
-                    ram_reports = systemRamFreeReports[i]['systemRamFreeInfo']
-                    ram_info = [str(x) for x in ram_reports]
-                    nrow['systenRamFreeReports.systemRamFreeInfo'] = \
-                        ','.join(ram_info)
-                display.add_row_titles_to_csv_file(nrow, csvRows, titles)
-    if sortHeaders:
-        display.sort_csv_titles([
-            'deviceId',
-        ], titles)
-    display.write_csv_file(csvRows, titles, 'CrOS', todrive)
-
-
-def getCrOSDeviceEntity(i, cd):
-    myarg = sys.argv[i].lower()
-    if myarg == 'cros_sn':
-        return i + 2, gam.getUsersToModify('cros_sn', sys.argv[i + 1])
-    if myarg == 'query':
-        return i + 2, gam.getUsersToModify('crosquery', sys.argv[i + 1])
-    if myarg[:6] == 'query:':
-        query = sys.argv[i][6:]
-        if query[:12].lower() == 'orgunitpath:':
-            kwargs = {'orgUnitPath': query[12:]}
-        else:
-            kwargs = {'query': query}
-        fields = 'nextPageToken,chromeosdevices(deviceId)'
-        devices = gapi.get_all_pages(cd.chromeosdevices(),
-                                     'list',
-                                     'chromeosdevices',
-                                     customerId=GC_Values[GC_CUSTOMER_ID],
-                                     fields=fields,
-                                     **kwargs)
-        return i + 1, [device['deviceId'] for device in devices]
-    return i + 1, sys.argv[i].replace(',', ' ').split()
-
-
-def _getFilterDate(dateStr):
-    return datetime.datetime.strptime(dateStr, YYYYMMDD_FORMAT)
-
-
-def _filterTimeRanges(activeTimeRanges, startDate, endDate):
-    if startDate is None and endDate is None:
-        return activeTimeRanges
-    filteredTimeRanges = []
-    for timeRange in activeTimeRanges:
-        activityDate = datetime.datetime.strptime(timeRange['date'],
-                                                  YYYYMMDD_FORMAT)
-        if ((startDate is None) or \
-            (activityDate >= startDate)) and \
-           ((endDate is None) or \
-            (activityDate <= endDate)):
-            filteredTimeRanges.append(timeRange)
-    return filteredTimeRanges
-
-
-def _filterCreateReportTime(items, timeField, startTime, endTime):
-    if startTime is None and endTime is None:
-        return items
-    filteredItems = []
-    time_format = '%Y-%m-%dT%H:%M:%S.%fZ'
-    for item in items:
-        timeValue = datetime.datetime.strptime(item[timeField], time_format)
-        if ((startTime is None) or \
-            (timeValue >= startTime)) and \
-           ((endTime is None) or \
-            (timeValue <= endTime)):
-            filteredItems.append(item)
-    return filteredItems

src/gam/gapi/directory/customer.py

@@ -1,149 +0,0 @@
-import datetime
-
-import gam
-from gam.var import *
-from gam import controlflow
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-from gam.gapi import reports as gapi_reports
-
-
-def doGetCustomerInfo():
-    cd = gapi_directory.build()
-    customer_info = gapi.call(cd.customers(),
-                              'get',
-                              customerKey=GC_Values[GC_CUSTOMER_ID])
-    print(f'Customer ID: {customer_info["id"]}')
-    print(f'Primary Domain: {customer_info["customerDomain"]}')
-    try:
-        result = gapi.call(
-            cd.domains(),
-            'get',
-            customer=customer_info['id'],
-            domainName=customer_info['customerDomain'],
-            fields='verified',
-            throw_reasons=[gapi.errors.ErrorReason.DOMAIN_NOT_FOUND])
-    except gapi.errors.GapiDomainNotFoundError:
-        result = {'verified': False}
-    print(f'Primary Domain Verified: {result["verified"]}')
-    # If customer has changed primary domain customerCreationTime is date
-    # of current primary being added, not customer create date.
-    # We should also get all domains and use oldest date
-    customer_creation = customer_info['customerCreationTime']
-    date_format = '%Y-%m-%dT%H:%M:%S.%fZ'
-    oldest = datetime.datetime.strptime(customer_creation, date_format)
-    domains = gapi.get_items(cd.domains(),
-                             'list',
-                             'domains',
-                             customer=GC_Values[GC_CUSTOMER_ID],
-                             fields='domains(creationTime)')
-    for domain in domains:
-        creation_timestamp = int(domain['creationTime']) / 1000
-        domain_creation = datetime.datetime.fromtimestamp(creation_timestamp)
-        if domain_creation < oldest:
-            oldest = domain_creation
-    print(f'Customer Creation Time: {oldest.strftime(date_format)}')
-    customer_language = customer_info.get('language', 'Unset (defaults to en)')
-    print(f'Default Language: {customer_language}')
-    if 'postalAddress' in customer_info:
-        print('Address:')
-        for field in ADDRESS_FIELDS_PRINT_ORDER:
-            if field in customer_info['postalAddress']:
-                print(f' {field}: {customer_info["postalAddress"][field]}')
-    if 'phoneNumber' in customer_info:
-        print(f'Phone: {customer_info["phoneNumber"]}')
-    print(f'Admin Secondary Email: {customer_info["alternateEmail"]}')
-    user_counts_map = {
-        'accounts:num_users': 'Total Users',
-        'accounts:gsuite_basic_total_licenses': 'G Suite Basic Licenses',
-        'accounts:gsuite_basic_used_licenses': 'G Suite Basic Users',
-        'accounts:gsuite_enterprise_total_licenses': 'G Suite Enterprise ' \
-        'Licenses',
-        'accounts:gsuite_enterprise_used_licenses': 'G Suite Enterprise ' \
-        'Users',
-        'accounts:gsuite_unlimited_total_licenses': 'G Suite Business ' \
-        'Licenses',
-        'accounts:gsuite_unlimited_used_licenses': 'G Suite Business Users'
-    }
-    parameters = ','.join(list(user_counts_map))
-    tryDate = datetime.date.today().strftime(YYYYMMDD_FORMAT)
-    customerId = GC_Values[GC_CUSTOMER_ID]
-    if customerId == MY_CUSTOMER:
-        customerId = None
-    rep = gapi_reports.build()
-    usage = None
-    throw_reasons = [
-        gapi.errors.ErrorReason.INVALID, gapi.errors.ErrorReason.FORBIDDEN
-    ]
-    while True:
-        try:
-            result = gapi.call(rep.customerUsageReports(),
-                               'get',
-                               throw_reasons=throw_reasons,
-                               customerId=customerId,
-                               date=tryDate,
-                               parameters=parameters)
-        except gapi.errors.GapiInvalidError as e:
-            tryDate = gapi_reports._adjust_date(str(e))
-            continue
-        except gapi.errors.GapiForbiddenError:
-            return
-        warnings = result.get('warnings', [])
-        fullDataRequired = ['accounts']
-        usage = result.get('usageReports')
-        has_reports = bool(usage)
-        fullData, tryDate = gapi_reports._check_full_data_available(
-            warnings, tryDate, fullDataRequired, has_reports)
-        if fullData < 0:
-            print('No user report available.')
-            sys.exit(1)
-        if fullData == 0:
-            continue
-        break
-    print(f'User counts as of {tryDate}:')
-    for item in usage[0]['parameters']:
-        api_name = user_counts_map.get(item['name'])
-        api_value = int(item.get('intValue', 0))
-        if api_name and api_value:
-            print(f'  {api_name}: {api_value:,}')
-
-
-def doUpdateCustomer():
-    cd = gapi_directory.build()
-    body = {}
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg in ADDRESS_FIELDS_ARGUMENT_MAP:
-            body.setdefault('postalAddress', {})
-            arg = ADDRESS_FIELDS_ARGUMENT_MAP[myarg]
-            body['postalAddress'][arg] = sys.argv[i + 1]
-            i += 2
-        elif myarg in ['adminsecondaryemail', 'alternateemail']:
-            body['alternateEmail'] = sys.argv[i + 1]
-            i += 2
-        elif myarg in ['phone', 'phonenumber']:
-            body['phoneNumber'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'language':
-            body['language'] = sys.argv[i + 1]
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(myarg, 'gam update customer')
-    if not body:
-        controlflow.system_error_exit(
-            2, 'no arguments specified for "gam '
-            'update customer"')
-    gapi.call(cd.customers(),
-              'patch',
-              customerKey=GC_Values[GC_CUSTOMER_ID],
-              body=body)
-    print('Updated customer')
-
-
-def setTrueCustomerId():
-    if GC_Values[GC_CUSTOMER_ID] == MY_CUSTOMER:
-        cd = gapi_directory.build()
-        GC_Values[GC_CUSTOMER_ID] = gapi.call(cd.customers(), 'get',
-            customerKey=GC_Values[GC_CUSTOMER_ID],
-            fields='id').get('id', GC_Values[GC_CUSTOMER_ID])

src/gam/gapi/directory/domainaliases.py

@@ -1,76 +0,0 @@
-import sys
-
-from gam.var import *
-from gam import controlflow
-from gam import display
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-from gam import utils
-
-
-def create():
-    cd = gapi_directory.build()
-    body = {'domainAliasName': sys.argv[3], 'parentDomainName': sys.argv[4]}
-    print(f'Adding {body["domainAliasName"]} alias for ' \
-       f'{body["parentDomainName"]}')
-    gapi.call(cd.domainAliases(),
-              'insert',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              body=body)
-
-
-def delete():
-    cd = gapi_directory.build()
-    domainAliasName = sys.argv[3]
-    print(f'Deleting domain alias {domainAliasName}')
-    gapi.call(cd.domainAliases(),
-              'delete',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              domainAliasName=domainAliasName)
-
-
-def info():
-    cd = gapi_directory.build()
-    alias = sys.argv[3]
-    result = gapi.call(cd.domainAliases(),
-                       'get',
-                       customer=GC_Values[GC_CUSTOMER_ID],
-                       domainAliasName=alias)
-    if 'creationTime' in result:
-        result['creationTime'] = utils.formatTimestampYMDHMSF(
-            result['creationTime'])
-    display.print_json(result)
-
-
-def print_():
-    cd = gapi_directory.build()
-    todrive = False
-    titles = [
-        'domainAliasName',
-    ]
-    csvRows = []
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam print domainaliases')
-    results = gapi.call(cd.domainAliases(),
-                        'list',
-                        customer=GC_Values[GC_CUSTOMER_ID])
-    for domainAlias in results['domainAliases']:
-        domainAlias_attributes = {}
-        for attr in domainAlias:
-            if attr in ['kind', 'etag']:
-                continue
-            if attr == 'creationTime':
-                domainAlias[attr] = utils.formatTimestampYMDHMSF(
-                    domainAlias[attr])
-            if attr not in titles:
-                titles.append(attr)
-            domainAlias_attributes[attr] = domainAlias[attr]
-        csvRows.append(domainAlias_attributes)
-    display.write_csv_file(csvRows, titles, 'Domains', todrive)

src/gam/gapi/directory/domains.py

@@ -1,124 +0,0 @@
-import sys
-
-from gam.var import *
-from gam import controlflow
-from gam import display
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-from gam.gapi.directory import customer as gapi_directory_customer
-from gam import utils
-
-
-def create():
-    cd = gapi_directory.build()
-    domain_name = sys.argv[3]
-    body = {'domainName': domain_name}
-    gapi.call(cd.domains(),
-              'insert',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              body=body)
-    print(f'Added domain {domain_name}')
-
-
-def info():
-    if (len(sys.argv) < 4) or (sys.argv[3] == 'logo'):
-        gapi_directory_customer.doGetCustomerInfo()
-        return
-    cd = gapi_directory.build()
-    domainName = sys.argv[3]
-    result = gapi.call(cd.domains(),
-                       'get',
-                       customer=GC_Values[GC_CUSTOMER_ID],
-                       domainName=domainName)
-    if 'creationTime' in result:
-        result['creationTime'] = utils.formatTimestampYMDHMSF(
-            result['creationTime'])
-    if 'domainAliases' in result:
-        for i in range(0, len(result['domainAliases'])):
-            if 'creationTime' in result['domainAliases'][i]:
-                result['domainAliases'][i][
-                    'creationTime'] = utils.formatTimestampYMDHMSF(
-                        result['domainAliases'][i]['creationTime'])
-    display.print_json(result)
-
-
-def update():
-    cd = gapi_directory.build()
-    domain_name = sys.argv[3]
-    i = 4
-    body = {}
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'primary':
-            body['customerDomain'] = domain_name
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam update domain')
-    gapi.call(cd.customers(),
-              'update',
-              customerKey=GC_Values[GC_CUSTOMER_ID],
-              body=body)
-    print(f'{domain_name} is now the primary domain.')
-
-
-def delete():
-    cd = gapi_directory.build()
-    domainName = sys.argv[3]
-    print(f'Deleting domain {domainName}')
-    gapi.call(cd.domains(),
-              'delete',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              domainName=domainName)
-
-
-def print_():
-    cd = gapi_directory.build()
-    todrive = False
-    titles = [
-        'domainName',
-    ]
-    csvRows = []
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam print domains')
-    results = gapi.call(cd.domains(),
-                        'list',
-                        customer=GC_Values[GC_CUSTOMER_ID])
-    for domain in results.get('domains', []):
-        domain_attributes = {}
-        domain['type'] = ['secondary', 'primary'][domain['isPrimary']]
-        for attr in domain:
-            if attr in ['kind', 'etag', 'domainAliases', 'isPrimary']:
-                continue
-            if attr in [
-                    'creationTime',
-            ]:
-                domain[attr] = utils.formatTimestampYMDHMSF(domain[attr])
-            if attr not in titles:
-                titles.append(attr)
-            domain_attributes[attr] = domain[attr]
-        csvRows.append(domain_attributes)
-        if 'domainAliases' in domain:
-            for aliasdomain in domain['domainAliases']:
-                aliasdomain['domainName'] = aliasdomain['domainAliasName']
-                del aliasdomain['domainAliasName']
-                aliasdomain['type'] = 'alias'
-                aliasdomain_attributes = {}
-                for attr in aliasdomain:
-                    if attr in ['kind', 'etag']:
-                        continue
-                    if attr in [
-                            'creationTime',
-                    ]:
-                        aliasdomain[attr] = utils.formatTimestampYMDHMSF(
-                            aliasdomain[attr])
-                    if attr not in titles:
-                        titles.append(attr)
-                    aliasdomain_attributes[attr] = aliasdomain[attr]
-                csvRows.append(aliasdomain_attributes)
-    display.write_csv_file(csvRows, titles, 'Domains', todrive)

src/gam/gapi/directory/mobiledevices.py

@@ -1,237 +0,0 @@
-import sys
-import uuid
-
-import gam
-from gam.var import *
-from gam import controlflow
-from gam import display
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-from gam import utils
-
-
-def delete():
-    cd = gapi_directory.build()
-    resourceId = sys.argv[3]
-    gapi.call(cd.mobiledevices(),
-              'delete',
-              resourceId=resourceId,
-              customerId=GC_Values[GC_CUSTOMER_ID])
-
-
-
-def info():
-    cd = gapi_directory.build()
-    resourceId = sys.argv[3]
-    device_info = gapi.call(cd.mobiledevices(),
-                            'get',
-                            customerId=GC_Values[GC_CUSTOMER_ID],
-                            resourceId=resourceId)
-    if 'deviceId' in info:
-        device_info['deviceId'] = device_info['deviceId'].encode('unicode-escape').decode(
-            UTF8)
-    attrib = 'securityPatchLevel'
-    if attrib in info and int(device_info[attrib]):
-        device_info[attrib] = utils.formatTimestampYMDHMS(device_info[attrib])
-    display.print_json(info)
-
-
-
-def print_():
-    cd = gapi_directory.build()
-    todrive = False
-    titles = []
-    csvRows = []
-    fields = None
-    projection = orderBy = sortOrder = None
-    queries = [None]
-    delimiter = ' '
-    listLimit = 1
-    appsLimit = -1
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg in ['query', 'queries']:
-            queries = gam.getQueries(myarg, sys.argv[i + 1])
-            i += 2
-        elif myarg == 'delimiter':
-            delimiter = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'listlimit':
-            listLimit = gam.getInteger(sys.argv[i + 1], myarg, minVal=-1)
-            i += 2
-        elif myarg == 'appslimit':
-            appsLimit = gam.getInteger(sys.argv[i + 1], myarg, minVal=-1)
-            i += 2
-        elif myarg == 'fields':
-            fields = f'nextPageToken,mobiledevices({sys.argv[i+1]})'
-            i += 2
-        elif myarg == 'orderby':
-            orderBy = sys.argv[i + 1].lower()
-            validOrderBy = [
-                'deviceid', 'email', 'lastsync', 'model', 'name', 'os',
-                'status', 'type'
-            ]
-            if orderBy not in validOrderBy:
-                controlflow.expected_argument_exit('orderby',
-                                                   ', '.join(validOrderBy),
-                                                   orderBy)
-            if orderBy == 'lastsync':
-                orderBy = 'lastSync'
-            elif orderBy == 'deviceid':
-                orderBy = 'deviceId'
-            i += 2
-        elif myarg in SORTORDER_CHOICES_MAP:
-            sortOrder = SORTORDER_CHOICES_MAP[myarg]
-            i += 1
-        elif myarg in PROJECTION_CHOICES_MAP:
-            projection = PROJECTION_CHOICES_MAP[myarg]
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam print mobile')
-    for query in queries:
-        gam.printGettingAllItems('Mobile Devices', query)
-        page_message = gapi.got_total_items_msg('Mobile Devices', '...\n')
-        all_mobile = gapi.get_all_pages(cd.mobiledevices(),
-                                        'list',
-                                        'mobiledevices',
-                                        page_message=page_message,
-                                        customerId=GC_Values[GC_CUSTOMER_ID],
-                                        query=query,
-                                        projection=projection,
-                                        fields=fields,
-                                        orderBy=orderBy,
-                                        sortOrder=sortOrder)
-        for mobile in all_mobile:
-            row = {}
-            for attrib in mobile:
-                if attrib in ['kind', 'etag']:
-                    continue
-                if attrib in ['name', 'email', 'otherAccountsInfo']:
-                    if attrib not in titles:
-                        titles.append(attrib)
-                    if listLimit > 0:
-                        row[attrib] = delimiter.join(
-                            mobile[attrib][0:listLimit])
-                    elif listLimit == 0:
-                        row[attrib] = delimiter.join(mobile[attrib])
-                elif attrib == 'applications':
-                    if appsLimit >= 0:
-                        if attrib not in titles:
-                            titles.append(attrib)
-                        applications = []
-                        j = 0
-                        for app in mobile[attrib]:
-                            j += 1
-                            if appsLimit and (j > appsLimit):
-                                break
-                            appDetails = []
-                            for field in [
-                                    'displayName', 'packageName', 'versionName'
-                            ]:
-                                appDetails.append(app.get(field, '<None>'))
-                            appDetails.append(
-                                str(app.get('versionCode', '<None>')))
-                            permissions = app.get('permission', [])
-                            if permissions:
-                                appDetails.append('/'.join(permissions))
-                            else:
-                                appDetails.append('<None>')
-                            applications.append('-'.join(appDetails))
-                        row[attrib] = delimiter.join(applications)
-                else:
-                    if attrib not in titles:
-                        titles.append(attrib)
-                    if attrib == 'deviceId':
-                        row[attrib] = mobile[attrib].encode(
-                            'unicode-escape').decode(UTF8)
-                    elif attrib == 'securityPatchLevel' and int(mobile[attrib]):
-                        row[attrib] = utils.formatTimestampYMDHMS(
-                            mobile[attrib])
-                    else:
-                        row[attrib] = mobile[attrib]
-            csvRows.append(row)
-    display.sort_csv_titles(
-        ['resourceId', 'deviceId', 'serialNumber', 'name', 'email', 'status'],
-        titles)
-    display.write_csv_file(csvRows, titles, 'Mobile', todrive)
-
-
-def update():
-    cd = gapi_directory.build()
-    resourceIds = sys.argv[3]
-    match_users = None
-    doit = False
-    if resourceIds[:6] == 'query:':
-        query = resourceIds[6:]
-        fields = 'nextPageToken,mobiledevices(resourceId,email)'
-        page_message = gapi.got_total_items_msg('Mobile Devices', '...\n')
-        devices = gapi.get_all_pages(cd.mobiledevices(),
-                                     'list',
-                                     page_message=page_message,
-                                     customerId=GC_Values[GC_CUSTOMER_ID],
-                                     items='mobiledevices',
-                                     query=query,
-                                     fields=fields)
-    else:
-        devices = [{'resourceId': resourceIds, 'email': ['not set']}]
-        doit = True
-    i = 4
-    body = {}
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'action':
-            body['action'] = sys.argv[i + 1].lower()
-            validActions = [
-                'wipe', 'wipeaccount', 'accountwipe', 'wipe_account',
-                'account_wipe', 'approve', 'block',
-                'cancel_remote_wipe_then_activate',
-                'cancel_remote_wipe_then_block'
-            ]
-            if body['action'] not in validActions:
-                controlflow.expected_argument_exit('action',
-                                                   ', '.join(validActions),
-                                                   body['action'])
-            if body['action'] == 'wipe':
-                body['action'] = 'admin_remote_wipe'
-            elif body['action'].replace('_',
-                                        '') in ['accountwipe', 'wipeaccount']:
-                body['action'] = 'admin_account_wipe'
-            i += 2
-        elif myarg in ['ifusers', 'matchusers']:
-            match_users = gam.getUsersToModify(entity_type=sys.argv[i + 1].lower(),
-                                           entity=sys.argv[i + 2])
-            i += 3
-        elif myarg == 'doit':
-            doit = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam update mobile')
-    if body:
-        if doit:
-            print(f'Updating {len(devices)} devices')
-            describe_as = 'Performing'
-        else:
-            print(
-                f'Showing {len(devices)} changes that would be made, not actually making changes because doit argument not specified'
-            )
-            describe_as = 'Would perform'
-        for device in devices:
-            device_user = device.get('email', [''])[0]
-            if match_users and device_user not in match_users:
-                print(
-                    f'Skipping device for user {device_user} that did not match match_users argument'
-                )
-            else:
-                print(
-                    f'{describe_as} {body["action"]} on user {device_user} device {device["resourceId"]}'
-                )
-                if doit:
-                    gapi.call(cd.mobiledevices(),
-                              'action',
-                              resourceId=device['resourceId'],
-                              body=body,
-                              customerId=GC_Values[GC_CUSTOMER_ID])

src/gam/gapi/directory/orgunits.py

@@ -1,421 +0,0 @@
-import sys
-
-import gam
-from gam.var import *
-from gam import controlflow
-from gam import display
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-from gam.gapi import errors as gapi_errors
-
-
-def create():
-    cd = gapi_directory.build()
-    name = getOrgUnitItem(sys.argv[3], pathOnly=True, absolutePath=False)
-    parent = ''
-    body = {}
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'description':
-            body['description'] = sys.argv[i + 1].replace('\\n', '\n')
-            i += 2
-        elif myarg == 'parent':
-            parent = getOrgUnitItem(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'noinherit':
-            body['blockInheritance'] = True
-            i += 1
-        elif myarg == 'inherit':
-            body['blockInheritance'] = False
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam create org')
-    if parent.startswith('id:'):
-        parent = gapi.call(cd.orgunits(),
-                           'get',
-                           customerId=GC_Values[GC_CUSTOMER_ID],
-                           orgUnitPath=parent,
-                           fields='orgUnitPath')['orgUnitPath']
-    if parent == '/':
-        orgUnitPath = parent + name
-    else:
-        orgUnitPath = parent + '/' + name
-    if orgUnitPath.count('/') > 1:
-        body['parentOrgUnitPath'], body['name'] = orgUnitPath.rsplit('/', 1)
-    else:
-        body['parentOrgUnitPath'] = '/'
-        body['name'] = orgUnitPath[1:]
-    parent = body['parentOrgUnitPath']
-    gapi.call(cd.orgunits(),
-              'insert',
-              customerId=GC_Values[GC_CUSTOMER_ID],
-              body=body,
-              retry_reasons=[gapi_errors.ErrorReason.DAILY_LIMIT_EXCEEDED])
-    print(f'Created OrgUnit {body["name"]}')
-
-
-def delete():
-    cd = gapi_directory.build()
-    name = getOrgUnitItem(sys.argv[3])
-    print(f'Deleting organization {name}')
-    gapi.call(cd.orgunits(),
-              'delete',
-              customerId=GC_Values[GC_CUSTOMER_ID],
-              orgUnitPath=encodeOrgUnitPath(makeOrgUnitPathRelative(name)))
-
-
-def info(name=None, return_attrib=None):
-    cd = gapi_directory.build()
-    checkSuspended = None
-    if not name:
-        name = getOrgUnitItem(sys.argv[3])
-        get_users = True
-        show_children = False
-        i = 4
-        while i < len(sys.argv):
-            myarg = sys.argv[i].lower()
-            if myarg == 'nousers':
-                get_users = False
-                i += 1
-            elif myarg in ['children', 'child']:
-                show_children = True
-                i += 1
-            elif myarg in ['suspended', 'notsuspended']:
-                checkSuspended = myarg == 'suspended'
-                i += 1
-            else:
-                controlflow.invalid_argument_exit(sys.argv[i], 'gam info org')
-    if name == '/':
-        orgs = gapi.call(cd.orgunits(),
-                         'list',
-                         customerId=GC_Values[GC_CUSTOMER_ID],
-                         type='children',
-                         fields='organizationUnits/parentOrgUnitId')
-        if 'organizationUnits' in orgs and orgs['organizationUnits']:
-            name = orgs['organizationUnits'][0]['parentOrgUnitId']
-        else:
-            topLevelOrgId = getTopLevelOrgId(cd, '/')
-            if topLevelOrgId:
-                name = topLevelOrgId
-    else:
-        name = makeOrgUnitPathRelative(name)
-    result = gapi.call(cd.orgunits(),
-                       'get',
-                       customerId=GC_Values[GC_CUSTOMER_ID],
-                       orgUnitPath=encodeOrgUnitPath(name))
-    if return_attrib:
-        return result[return_attrib]
-    display.print_json(result)
-    if get_users:
-        name = result['orgUnitPath']
-        page_message = gapi.got_total_items_first_last_msg('Users')
-        users = gapi.get_all_pages(
-            cd.users(),
-            'list',
-            'users',
-            page_message=page_message,
-            message_attribute='primaryEmail',
-            customer=GC_Values[GC_CUSTOMER_ID],
-            query=orgUnitPathQuery(name, checkSuspended),
-            fields='users(primaryEmail,orgUnitPath),nextPageToken')
-        if checkSuspended is None:
-            print('Users:')
-        elif not checkSuspended:
-            print('Users (Not suspended):')
-        else:
-            print('Users (Suspended):')
-        for user in users:
-            if show_children or (name.lower() == user['orgUnitPath'].lower()):
-                sys.stdout.write(f' {user["primaryEmail"]}')
-                if name.lower() != user['orgUnitPath'].lower():
-                    print(' (child)')
-                else:
-                    print('')
-
-
-def print_():
-    print_order = [
-        'orgUnitPath', 'orgUnitId', 'name', 'description', 'parentOrgUnitPath',
-        'parentOrgUnitId', 'blockInheritance'
-    ]
-    cd = gapi_directory.build()
-    listType = 'all'
-    orgUnitPath = '/'
-    todrive = False
-    fields = ['orgUnitPath', 'name', 'orgUnitId', 'parentOrgUnitId']
-    titles = []
-    csvRows = []
-    parentOrgIds = []
-    retrievedOrgIds = []
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg == 'toplevelonly':
-            listType = 'children'
-            i += 1
-        elif myarg == 'fromparent':
-            orgUnitPath = getOrgUnitItem(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'allfields':
-            fields = None
-            i += 1
-        elif myarg == 'fields':
-            fields += sys.argv[i + 1].split(',')
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam print orgs')
-    gam.printGettingAllItems('Organizational Units', None)
-    if fields:
-        get_fields = ','.join(fields)
-        list_fields = f'organizationUnits({get_fields})'
-    else:
-        list_fields = None
-        get_fields = None
-    orgs = gapi.call(cd.orgunits(),
-                     'list',
-                     customerId=GC_Values[GC_CUSTOMER_ID],
-                     type=listType,
-                     orgUnitPath=orgUnitPath,
-                     fields=list_fields)
-    if not 'organizationUnits' in orgs:
-        topLevelOrgId = getTopLevelOrgId(cd, orgUnitPath)
-        if topLevelOrgId:
-            parentOrgIds.append(topLevelOrgId)
-        orgunits = []
-    else:
-        orgunits = orgs['organizationUnits']
-    for row in orgunits:
-        retrievedOrgIds.append(row['orgUnitId'])
-        if row['parentOrgUnitId'] not in parentOrgIds:
-            parentOrgIds.append(row['parentOrgUnitId'])
-    missing_parents = set(parentOrgIds) - set(retrievedOrgIds)
-    for missing_parent in missing_parents:
-        try:
-            result = gapi.call(cd.orgunits(),
-                               'get',
-                               throw_reasons=['required'],
-                               customerId=GC_Values[GC_CUSTOMER_ID],
-                               orgUnitPath=missing_parent,
-                               fields=get_fields)
-            orgunits.append(result)
-        except:
-            pass
-    for row in orgunits:
-        orgEntity = {}
-        for key, value in list(row.items()):
-            if key in ['kind', 'etag', 'etags']:
-                continue
-            if key not in titles:
-                titles.append(key)
-            orgEntity[key] = value
-        csvRows.append(orgEntity)
-    for title in titles:
-        if title not in print_order:
-            print_order.append(title)
-    titles = sorted(titles, key=print_order.index)
-    # sort results similar to how they list in admin console
-    csvRows.sort(key=lambda x: x['orgUnitPath'].lower(), reverse=False)
-    display.write_csv_file(csvRows, titles, 'Orgs', todrive)
-
-
-def update():
-    cd = gapi_directory.build()
-    orgUnitPath = getOrgUnitItem(sys.argv[3])
-    if sys.argv[4].lower() in ['move', 'add']:
-        entity_type = sys.argv[5].lower()
-        if entity_type in usergroup_types:
-            users = gam.getUsersToModify(entity_type=entity_type,
-                                         entity=sys.argv[6])
-        else:
-            entity_type = 'users'
-            users = gam.getUsersToModify(entity_type=entity_type,
-                                         entity=sys.argv[5])
-        if (entity_type.startswith('cros')) or (
-            (entity_type == 'all') and (sys.argv[6].lower() == 'cros')):
-            for l in range(0, len(users), 50):
-                move_body = {'deviceIds': users[l:l + 50]}
-                print(
-                    f' moving {len(move_body["deviceIds"])} devices to {orgUnitPath}'
-                )
-                gapi.call(cd.chromeosdevices(),
-                          'moveDevicesToOu',
-                          customerId=GC_Values[GC_CUSTOMER_ID],
-                          orgUnitPath=orgUnitPath,
-                          body=move_body)
-        else:
-            i = 0
-            count = len(users)
-            for user in users:
-                i += 1
-                sys.stderr.write(
-                    f' moving {user} to {orgUnitPath}{gam.currentCountNL(i, count)}'
-                )
-                try:
-                    gapi.call(cd.users(),
-                              'update',
-                              throw_reasons=[
-                                  gapi_errors.ErrorReason.CONDITION_NOT_MET
-                              ],
-                              userKey=user,
-                              body={'orgUnitPath': orgUnitPath})
-                except gapi_errors.GapiConditionNotMetError:
-                    pass
-    else:
-        body = {}
-        i = 4
-        while i < len(sys.argv):
-            myarg = sys.argv[i].lower()
-            if myarg == 'name':
-                body['name'] = sys.argv[i + 1]
-                i += 2
-            elif myarg == 'description':
-                body['description'] = sys.argv[i + 1].replace('\\n', '\n')
-                i += 2
-            elif myarg == 'parent':
-                parent = getOrgUnitItem(sys.argv[i + 1])
-                if parent.startswith('id:'):
-                    body['parentOrgUnitId'] = parent
-                else:
-                    body['parentOrgUnitPath'] = parent
-                i += 2
-            elif myarg == 'noinherit':
-                body['blockInheritance'] = True
-                i += 1
-            elif myarg == 'inherit':
-                body['blockInheritance'] = False
-                i += 1
-            else:
-                controlflow.invalid_argument_exit(sys.argv[i], 'gam update org')
-        gapi.call(cd.orgunits(),
-                  'update',
-                  customerId=GC_Values[GC_CUSTOMER_ID],
-                  orgUnitPath=encodeOrgUnitPath(
-                      makeOrgUnitPathRelative(orgUnitPath)),
-                  body=body)
-
-
-def orgUnitPathQuery(path, checkSuspended):
-    query = "orgUnitPath='{0}'".format(path.replace(
-        "'", "\\'")) if path != '/' else ''
-    if checkSuspended is not None:
-        query += f' isSuspended={checkSuspended}'
-    return query
-
-
-def makeOrgUnitPathAbsolute(path):
-    if path == '/':
-        return path
-    if path.startswith('/'):
-        return path.rstrip('/')
-    if path.startswith('id:'):
-        return path
-    if path.startswith('uid:'):
-        return path[1:]
-    return '/' + path.rstrip('/')
-
-
-def makeOrgUnitPathRelative(path):
-    if path == '/':
-        return path
-    if path.startswith('/'):
-        return path[1:].rstrip('/')
-    if path.startswith('id:'):
-        return path
-    if path.startswith('uid:'):
-        return path[1:]
-    return path.rstrip('/')
-
-
-def encodeOrgUnitPath(path):
-    if path.find('+') == -1 and path.find('%') == -1:
-        return path
-    encpath = ''
-    for c in path:
-        if c == '+':
-            encpath += '%2B'
-        elif c == '%':
-            encpath += '%25'
-        else:
-            encpath += c
-    return encpath
-
-
-def getOrgUnitItem(orgUnit, pathOnly=False, absolutePath=True):
-    if pathOnly and (orgUnit.startswith('id:') or orgUnit.startswith('uid:')):
-        controlflow.system_error_exit(
-            2, f'{orgUnit} is not valid in this context')
-    if absolutePath:
-        return makeOrgUnitPathAbsolute(orgUnit)
-    return makeOrgUnitPathRelative(orgUnit)
-
-
-def getTopLevelOrgId(cd, orgUnitPath):
-    try:
-        # create a temp org so we can learn what the top level org ID is (sigh)
-        temp_org = gapi.call(cd.orgunits(),
-                             'insert',
-                             customerId=GC_Values[GC_CUSTOMER_ID],
-                             body={
-                                 'name': 'temp-delete-me',
-                                 'parentOrgUnitPath': orgUnitPath
-                             },
-                             fields='parentOrgUnitId,orgUnitId')
-        gapi.call(cd.orgunits(),
-                  'delete',
-                  customerId=GC_Values[GC_CUSTOMER_ID],
-                  orgUnitPath=temp_org['orgUnitId'])
-        return temp_org['parentOrgUnitId']
-    except:
-        pass
-    return None
-
-
-def getOrgUnitId(orgUnit, cd=None):
-    if cd is None:
-        cd = gapi_directory.build()
-    orgUnit = getOrgUnitItem(orgUnit)
-    if orgUnit[:3] == 'id:':
-        return (orgUnit, orgUnit)
-    if orgUnit == '/':
-        result = gapi.call(cd.orgunits(),
-                           'list',
-                           customerId=GC_Values[GC_CUSTOMER_ID],
-                           orgUnitPath='/',
-                           type='children',
-                           fields='organizationUnits(parentOrgUnitId)')
-        if result.get('organizationUnits', []):
-            return (orgUnit, result['organizationUnits'][0]['parentOrgUnitId'])
-        topLevelOrgId = getTopLevelOrgId(cd, '/')
-        if topLevelOrgId:
-            return (orgUnit, topLevelOrgId)
-        return (orgUnit, '/')  #Bogus but should never happen
-    result = gapi.call(cd.orgunits(),
-                       'get',
-                       customerId=GC_Values[GC_CUSTOMER_ID],
-                       orgUnitPath=encodeOrgUnitPath(
-                           makeOrgUnitPathRelative(orgUnit)),
-                       fields='orgUnitId')
-    return (orgUnit, result['orgUnitId'])
-
-
-def buildOrgUnitIdToNameMap():
-    cd = gapi_directory.build()
-    result = gapi.call(cd.orgunits(),
-                       'list',
-                       customerId=GC_Values[GC_CUSTOMER_ID],
-                       fields='organizationUnits(orgUnitPath,orgUnitId)',
-                       type='all')
-    GM_Globals[GM_MAP_ORGUNIT_ID_TO_NAME] = {}
-    for orgUnit in result['organizationUnits']:
-        GM_Globals[GM_MAP_ORGUNIT_ID_TO_NAME][
-            orgUnit['orgUnitId']] = orgUnit['orgUnitPath']
-
-
-def orgunit_from_orgunitid(orgunitid):
-    if not GM_Globals[GM_MAP_ORGUNIT_ID_TO_NAME]:
-        buildOrgUnitIdToNameMap()
-    return GM_Globals[GM_MAP_ORGUNIT_ID_TO_NAME].get(orgunitid, orgunitid)

src/gam/gapi/directory/privileges.py

@@ -1,32 +0,0 @@
-from gam.var import GC_Values, GC_CUSTOMER_ID
-from gam import display
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-
-
-def flatten_privilege_list(privs, parent=None):
-    flat_privs = []
-    for priv in privs:
-        children = []
-        if parent:
-            priv['parent'] = parent
-        if priv.get('childPrivileges'):
-            children = flatten_privilege_list(priv['childPrivileges'],
-                                              parent=priv['privilegeName'])
-            priv['children'] = ' '.join(
-                [child['privilegeName'] for child in children])
-            del priv['childPrivileges']
-        flat_privs = flat_privs + children
-        flat_privs.append(priv)
-    return flat_privs
-
-
-def print_(return_only=False):
-    cd = gapi_directory.build()
-    privs = gapi.call(cd.privileges(),
-                      'list',
-                      customer=GC_Values[GC_CUSTOMER_ID])
-    privs = flatten_privilege_list(privs.get('items', []))
-    if return_only:
-        return privs
-    display.print_json(privs)

src/gam/gapi/directory/resource.py

@@ -1,534 +0,0 @@
-import sys
-import uuid
-
-import gam
-from gam.var import *
-from gam import controlflow
-from gam import display
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-from gam import utils
-
-
-def printBuildings():
-    to_drive = False
-    cd = gapi_directory.build()
-    titles = []
-    csvRows = []
-    fieldsList = ['buildingId']
-    # buildings.list() currently doesn't support paging
-    # but should soon, attempt to use it now so we
-    # won't break when it's turned on.
-    fields = 'nextPageToken,buildings(%s)'
-    possible_fields = {}
-    for pfield in cd._rootDesc['schemas']['Building']['properties']:
-        possible_fields[pfield.lower()] = pfield
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'todrive':
-            to_drive = True
-            i += 1
-        elif myarg == 'allfields':
-            fields = None
-            i += 1
-        elif myarg in possible_fields:
-            fieldsList.append(possible_fields[myarg])
-            i += 1
-        # Allows shorter arguments like "name" instead of "buildingname"
-        elif 'building' + myarg in possible_fields:
-            fieldsList.append(possible_fields['building' + myarg])
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam print buildings')
-    if fields:
-        fields = fields % ','.join(fieldsList)
-    buildings = gapi.get_all_pages(cd.resources().buildings(),
-                                   'list',
-                                   'buildings',
-                                   customer=GC_Values[GC_CUSTOMER_ID],
-                                   fields=fields)
-    for building in buildings:
-        building.pop('etags', None)
-        building.pop('etag', None)
-        building.pop('kind', None)
-        if 'buildingId' in building:
-            building['buildingId'] = f'id:{building["buildingId"]}'
-        if 'floorNames' in building:
-            building['floorNames'] = ','.join(building['floorNames'])
-        building = utils.flatten_json(building)
-        for item in building:
-            if item not in titles:
-                titles.append(item)
-        csvRows.append(building)
-    display.sort_csv_titles('buildingId', titles)
-    display.write_csv_file(csvRows, titles, 'Buildings', to_drive)
-
-
-def printResourceCalendars():
-    cd = gapi_directory.build()
-    todrive = False
-    fieldsList = []
-    fieldsTitles = {}
-    titles = []
-    csvRows = []
-    query = None
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg == 'query':
-            query = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'allfields':
-            fieldsList = []
-            fieldsTitles = {}
-            titles = []
-            for field in RESCAL_ALLFIELDS:
-                display.add_field_to_csv_file(field,
-                                              RESCAL_ARGUMENT_TO_PROPERTY_MAP,
-                                              fieldsList, fieldsTitles, titles)
-            i += 1
-        elif myarg in RESCAL_ARGUMENT_TO_PROPERTY_MAP:
-            display.add_field_to_csv_file(myarg,
-                                          RESCAL_ARGUMENT_TO_PROPERTY_MAP,
-                                          fieldsList, fieldsTitles, titles)
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam print resources')
-    if not fieldsList:
-        for field in RESCAL_DFLTFIELDS:
-            display.add_field_to_csv_file(field,
-                                          RESCAL_ARGUMENT_TO_PROPERTY_MAP,
-                                          fieldsList, fieldsTitles, titles)
-    fields = f'nextPageToken,items({",".join(set(fieldsList))})'
-    if 'buildingId' in fieldsList:
-        display.add_field_to_csv_file('buildingName',
-                                      {'buildingName': ['buildingName',]},
-                                      fieldsList, fieldsTitles, titles)
-    gam.printGettingAllItems('Resource Calendars', None)
-    page_message = gapi.got_total_items_first_last_msg('Resource Calendars')
-    resources = gapi.get_all_pages(cd.resources().calendars(),
-                                   'list',
-                                   'items',
-                                   page_message=page_message,
-                                   message_attribute='resourceId',
-                                   customer=GC_Values[GC_CUSTOMER_ID],
-                                   query=query,
-                                   fields=fields)
-    for resource in resources:
-        if 'featureInstances' in resource:
-            features = [a_feature['feature']['name'] for \
-                a_feature in resource['featureInstances']]
-            resource['featureInstances'] = ','.join(features)
-        if 'buildingId' in resource:
-            resource['buildingName'] = getBuildingNameById(
-                cd, resource['buildingId'])
-            resource['buildingId'] = f'id:{resource["buildingId"]}'
-        resUnit = {}
-        for field in fieldsList:
-            resUnit[fieldsTitles[field]] = resource.get(field, '')
-        csvRows.append(resUnit)
-    display.sort_csv_titles(['resourceId', 'resourceName', 'resourceEmail'],
-                            titles)
-    display.write_csv_file(csvRows, titles, 'Resources', todrive)
-
-
-RESCAL_DFLTFIELDS = [
-    'id',
-    'name',
-    'email',
-]
-RESCAL_ALLFIELDS = [
-    'id',
-    'name',
-    'email',
-    'description',
-    'type',
-    'buildingid',
-    'category',
-    'capacity',
-    'features',
-    'floor',
-    'floorsection',
-    'generatedresourcename',
-    'uservisibledescription',
-]
-
-RESCAL_ARGUMENT_TO_PROPERTY_MAP = {
-    'description': ['resourceDescription'],
-    'building': ['buildingId',],
-    'buildingid': ['buildingId',],
-    'capacity': ['capacity',],
-    'category': ['resourceCategory',],
-    'email': ['resourceEmail'],
-    'feature': ['featureInstances',],
-    'features': ['featureInstances',],
-    'floor': ['floorName',],
-    'floorname': ['floorName',],
-    'floorsection': ['floorSection',],
-    'generatedresourcename': ['generatedResourceName',],
-    'id': ['resourceId'],
-    'name': ['resourceName'],
-    'type': ['resourceType'],
-    'userdescription': ['userVisibleDescription',],
-    'uservisibledescription': ['userVisibleDescription',],
-}
-
-
-def printFeatures():
-    to_drive = False
-    cd = gapi_directory.build()
-    titles = []
-    csvRows = []
-    fieldsList = ['name']
-    fields = 'nextPageToken,features(%s)'
-    possible_fields = {}
-    for pfield in cd._rootDesc['schemas']['Feature']['properties']:
-        possible_fields[pfield.lower()] = pfield
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'todrive':
-            to_drive = True
-            i += 1
-        elif myarg == 'allfields':
-            fields = None
-            i += 1
-        elif myarg in possible_fields:
-            fieldsList.append(possible_fields[myarg])
-            i += 1
-        elif 'feature' + myarg in possible_fields:
-            fieldsList.append(possible_fields['feature' + myarg])
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam print features')
-    if fields:
-        fields = fields % ','.join(fieldsList)
-    features = gapi.get_all_pages(cd.resources().features(),
-                                  'list',
-                                  'features',
-                                  customer=GC_Values[GC_CUSTOMER_ID],
-                                  fields=fields)
-    for feature in features:
-        feature.pop('etags', None)
-        feature.pop('etag', None)
-        feature.pop('kind', None)
-        feature = utils.flatten_json(feature)
-        for item in feature:
-            if item not in titles:
-                titles.append(item)
-        csvRows.append(feature)
-    display.sort_csv_titles('name', titles)
-    display.write_csv_file(csvRows, titles, 'Features', to_drive)
-
-
-def _getBuildingAttributes(args, body={}):
-    i = 0
-    while i < len(args):
-        myarg = args[i].lower().replace('_', '')
-        if myarg == 'id':
-            body['buildingId'] = args[i + 1]
-            i += 2
-        elif myarg == 'name':
-            body['buildingName'] = args[i + 1]
-            i += 2
-        elif myarg in ['lat', 'latitude']:
-            if 'coordinates' not in body:
-                body['coordinates'] = {}
-            body['coordinates']['latitude'] = args[i + 1]
-            i += 2
-        elif myarg in ['long', 'lng', 'longitude']:
-            if 'coordinates' not in body:
-                body['coordinates'] = {}
-            body['coordinates']['longitude'] = args[i + 1]
-            i += 2
-        elif myarg == 'description':
-            body['description'] = args[i + 1]
-            i += 2
-        elif myarg == 'floors':
-            body['floorNames'] = args[i + 1].split(',')
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(myarg,
-                                              'gam create|update building')
-    return body
-
-
-def createBuilding():
-    cd = gapi_directory.build()
-    body = {
-        'floorNames': ['1'],
-        'buildingId': str(uuid.uuid4()),
-        'buildingName': sys.argv[3]
-    }
-    body = _getBuildingAttributes(sys.argv[4:], body)
-    print(f'Creating building {body["buildingId"]}...')
-    gapi.call(cd.resources().buildings(),
-              'insert',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              body=body)
-
-
-def _makeBuildingIdNameMap(cd):
-    fields = 'nextPageToken,buildings(buildingId,buildingName)'
-    buildings = gapi.get_all_pages(cd.resources().buildings(),
-                                   'list',
-                                   'buildings',
-                                   customer=GC_Values[GC_CUSTOMER_ID],
-                                   fields=fields)
-    GM_Globals[GM_MAP_BUILDING_ID_TO_NAME] = {}
-    GM_Globals[GM_MAP_BUILDING_NAME_TO_ID] = {}
-    for building in buildings:
-        GM_Globals[GM_MAP_BUILDING_ID_TO_NAME][
-            building['buildingId']] = building['buildingName']
-        GM_Globals[GM_MAP_BUILDING_NAME_TO_ID][
-            building['buildingName']] = building['buildingId']
-
-
-def getBuildingByNameOrId(cd, which_building, minLen=1):
-    if not which_building or \
-       (minLen == 0 and which_building in ['id:', 'uid:']):
-        if minLen == 0:
-            return ''
-        controlflow.system_error_exit(3, 'Building id/name is empty')
-    cg = UID_PATTERN.match(which_building)
-    if cg:
-        return cg.group(1)
-    if GM_Globals[GM_MAP_BUILDING_NAME_TO_ID] is None:
-        _makeBuildingIdNameMap(cd)
-    # Exact name match, return ID
-    if which_building in GM_Globals[GM_MAP_BUILDING_NAME_TO_ID]:
-        return GM_Globals[GM_MAP_BUILDING_NAME_TO_ID][which_building]
-    # No exact name match, check for case insensitive name matches
-    which_building_lower = which_building.lower()
-    ci_matches = []
-    for buildingName, buildingId in GM_Globals[
-            GM_MAP_BUILDING_NAME_TO_ID].items():
-        if buildingName.lower() == which_building_lower:
-            ci_matches.append({
-                'buildingName': buildingName,
-                'buildingId': buildingId
-            })
-    # One match, return ID
-    if len(ci_matches) == 1:
-        return ci_matches[0]['buildingId']
-    # No or multiple name matches, try ID
-    # Exact ID match, return ID
-    if which_building in GM_Globals[GM_MAP_BUILDING_ID_TO_NAME]:
-        return which_building
-    # No exact ID match, check for case insensitive id match
-    for buildingId in GM_Globals[GM_MAP_BUILDING_ID_TO_NAME]:
-        # Match, return ID
-        if buildingId.lower() == which_building_lower:
-            return buildingId
-    # Multiple name  matches
-    if len(ci_matches) > 1:
-        message = 'Multiple buildings with same name:\n'
-        for building in ci_matches:
-            message += f'  Name:{building["buildingName"]}  ' \
-                       f'id:{building["buildingId"]}\n'
-        message += '\nPlease specify building name by exact case or by id.'
-        controlflow.system_error_exit(3, message)
-    # No matches
-    else:
-        controlflow.system_error_exit(3, f'No such building {which_building}')
-
-
-def getBuildingNameById(cd, buildingId):
-    if GM_Globals[GM_MAP_BUILDING_ID_TO_NAME] is None:
-        _makeBuildingIdNameMap(cd)
-    return GM_Globals[GM_MAP_BUILDING_ID_TO_NAME].get(buildingId, 'UNKNOWN')
-
-
-def updateBuilding():
-    cd = gapi_directory.build()
-    buildingId = getBuildingByNameOrId(cd, sys.argv[3])
-    body = _getBuildingAttributes(sys.argv[4:])
-    print(f'Updating building {buildingId}...')
-    gapi.call(cd.resources().buildings(),
-              'patch',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              buildingId=buildingId,
-              body=body)
-
-
-def getBuildingInfo():
-    cd = gapi_directory.build()
-    buildingId = getBuildingByNameOrId(cd, sys.argv[3])
-    building = gapi.call(cd.resources().buildings(),
-                         'get',
-                         customer=GC_Values[GC_CUSTOMER_ID],
-                         buildingId=buildingId)
-    if 'buildingId' in building:
-        building['buildingId'] = f'id:{building["buildingId"]}'
-    if 'floorNames' in building:
-        building['floorNames'] = ','.join(building['floorNames'])
-    if 'buildingName' in building:
-        sys.stdout.write(building.pop('buildingName'))
-    display.print_json(building)
-
-
-def deleteBuilding():
-    cd = gapi_directory.build()
-    buildingId = getBuildingByNameOrId(cd, sys.argv[3])
-    print(f'Deleting building {buildingId}...')
-    gapi.call(cd.resources().buildings(),
-              'delete',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              buildingId=buildingId)
-
-
-def _getFeatureAttributes(args, body={}):
-    i = 0
-    while i < len(args):
-        myarg = args[i].lower().replace('_', '')
-        if myarg == 'name':
-            body['name'] = args[i + 1]
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(myarg,
-                                              'gam create|update feature')
-    return body
-
-
-def createFeature():
-    cd = gapi_directory.build()
-    body = _getFeatureAttributes(sys.argv[3:])
-    print(f'Creating feature {body["name"]}...')
-    gapi.call(cd.resources().features(),
-              'insert',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              body=body)
-
-
-def updateFeature():
-    # update does not work for name and name is only field to be updated
-    # if additional writable fields are added to feature in the future
-    # we'll add support for update as well as rename
-    cd = gapi_directory.build()
-    oldName = sys.argv[3]
-    body = {'newName': sys.argv[5:]}
-    print(f'Updating feature {oldName}...')
-    gapi.call(cd.resources().features(),
-              'rename',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              oldName=oldName,
-              body=body)
-
-
-def deleteFeature():
-    cd = gapi_directory.build()
-    featureKey = sys.argv[3]
-    print(f'Deleting feature {featureKey}...')
-    gapi.call(cd.resources().features(),
-              'delete',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              featureKey=featureKey)
-
-
-def _getResourceCalendarAttributes(cd, args, body={}):
-    i = 0
-    while i < len(args):
-        myarg = args[i].lower().replace('_', '')
-        if myarg == 'name':
-            body['resourceName'] = args[i + 1]
-            i += 2
-        elif myarg == 'description':
-            body['resourceDescription'] = args[i + 1].replace('\\n', '\n')
-            i += 2
-        elif myarg == 'type':
-            body['resourceType'] = args[i + 1]
-            i += 2
-        elif myarg in ['building', 'buildingid']:
-            body['buildingId'] = getBuildingByNameOrId(cd,
-                                                       args[i + 1],
-                                                       minLen=0)
-            i += 2
-        elif myarg in ['capacity']:
-            body['capacity'] = gam.getInteger(args[i + 1], myarg, minVal=0)
-            i += 2
-        elif myarg in ['feature', 'features']:
-            features = args[i + 1].split(',')
-            body['featureInstances'] = []
-            for feature in features:
-                instance = {'feature': {'name': feature}}
-                body['featureInstances'].append(instance)
-            i += 2
-        elif myarg in ['floor', 'floorname']:
-            body['floorName'] = args[i + 1]
-            i += 2
-        elif myarg in ['floorsection']:
-            body['floorSection'] = args[i + 1]
-            i += 2
-        elif myarg in ['category']:
-            body['resourceCategory'] = args[i + 1].upper()
-            if body['resourceCategory'] == 'ROOM':
-                body['resourceCategory'] = 'CONFERENCE_ROOM'
-            i += 2
-        elif myarg in ['uservisibledescription', 'userdescription']:
-            body['userVisibleDescription'] = args[i + 1]
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(args[i],
-                                              'gam create|update resource')
-    return body
-
-
-def createResourceCalendar():
-    cd = gapi_directory.build()
-    body = {'resourceId': sys.argv[3], 'resourceName': sys.argv[4]}
-    body = _getResourceCalendarAttributes(cd, sys.argv[5:], body)
-    print(f'Creating resource {body["resourceId"]}...')
-    gapi.call(cd.resources().calendars(),
-              'insert',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              body=body)
-
-
-def updateResourceCalendar():
-    cd = gapi_directory.build()
-    resId = sys.argv[3]
-    body = _getResourceCalendarAttributes(cd, sys.argv[4:])
-    # Use patch since it seems to work better.
-    # update requires name to be set.
-    gapi.call(cd.resources().calendars(),
-              'patch',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              calendarResourceId=resId,
-              body=body,
-              fields='')
-    print(f'updated resource {resId}')
-
-
-def getResourceCalendarInfo():
-    cd = gapi_directory.build()
-    resId = sys.argv[3]
-    resource = gapi.call(cd.resources().calendars(),
-                         'get',
-                         customer=GC_Values[GC_CUSTOMER_ID],
-                         calendarResourceId=resId)
-    if 'featureInstances' in resource:
-        features = []
-        for a_feature in resource.pop('featureInstances'):
-            features.append(a_feature['feature']['name'])
-        resource['features'] = ', '.join(features)
-    if 'buildingId' in resource:
-        resource['buildingName'] = getBuildingNameById(cd,
-                                                       resource['buildingId'])
-        resource['buildingId'] = f'id:{resource["buildingId"]}'
-    display.print_json(resource)
-
-
-def deleteResourceCalendar():
-    resId = sys.argv[3]
-    cd = gapi_directory.build()
-    print(f'Deleting resource calendar {resId}')
-    gapi.call(cd.resources().calendars(),
-              'delete',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              calendarResourceId=resId)

src/gam/gapi/directory/roles.py

@@ -1,124 +0,0 @@
-import sys
-
-from gam.var import GC_Values, GC_CUSTOMER_ID
-import gam
-from gam import controlflow
-from gam import display
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-from gam.gapi.directory import privileges as gapi_directory_privileges
-
-
-def getPrivileges(body, privs, action):
-    all_privileges = gapi_directory_privileges.print_(return_only=True)
-    if privs == 'ALL':
-        body['rolePrivileges'] = [
-            {'privilegeName': p['privilegeName'], 'serviceId': p['serviceId']} for p in all_privileges
-            ]
-    elif privs == 'ALL_OU':
-        body['rolePrivileges'] = [
-            {'privilegeName': p['privilegeName'], 'serviceId': p['serviceId']} for p in all_privileges if p.get('isOuScopable')
-            ]
-    else:
-      body.setdefault('rolePrivileges', [])
-      for priv in privs.split(','):
-          for p in all_privileges:
-              if priv == p['privilegeName']:
-                body['rolePrivileges'].append({'privilegeName': p['privilegeName'], 'serviceId': p['serviceId']})
-                break
-          else:
-              controlflow.invalid_argument_exit(priv,
-                                                f'gam {action} adminrole privileges')
-
-def create():
-    cd = gapi_directory.build()
-    body = {'roleName': sys.argv[3]}
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'privileges':
-            getPrivileges(body, sys.argv[i + 1].upper(), 'create')
-            i += 2
-        elif myarg == 'description':
-            body['roleDescription'] = sys.argv[i + 1]
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam create adminrole')
-
-    if not body.get('rolePrivileges'):
-        controlflow.missing_argument_exit('privileges',
-                                          'gam create adminrole')
-    print(f'Creating role {body["roleName"]}')
-    gapi.call(cd.roles(),
-              'insert',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              body=body)
-
-def update():
-    cd = gapi_directory.build()
-    body = {}
-    roleId = gam.getRoleId(sys.argv[3])
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'privileges':
-            getPrivileges(body, sys.argv[i + 1].upper(), 'update')
-            i += 2
-        elif myarg == 'description':
-            body['roleDescription'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'name':
-            body['roleName'] = sys.argv[i + 1]
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam update adminrole')
-
-    print(f'Updating role {roleId}')
-    gapi.call(cd.roles(),
-              'patch',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              roleId=roleId,
-              body=body)
-
-
-def delete():
-    cd = gapi_directory.build()
-    roleId = gam.getRoleId(sys.argv[3])
-    print(f'Deleting role {roleId}')
-    gapi.call(cd.roles(),
-              'delete',
-              customer=GC_Values[GC_CUSTOMER_ID],
-              roleId=roleId)
-
-
-def print_():
-    cd = gapi_directory.build()
-    todrive = False
-    titles = [
-        'roleId', 'roleName', 'roleDescription', 'isSuperAdminRole',
-        'isSystemRole'
-    ]
-    fields = f'nextPageToken,items({",".join(titles)})'
-    csvRows = []
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam print adminroles')
-    roles = gapi.get_all_pages(cd.roles(),
-                               'list',
-                               'items',
-                               customer=GC_Values[GC_CUSTOMER_ID],
-                               fields=fields)
-    for role in roles:
-        role_attrib = {}
-        for key, value in list(role.items()):
-            role_attrib[key] = value
-        csvRows.append(role_attrib)
-    display.write_csv_file(csvRows, titles, 'Admin Roles', todrive)

src/gam/gapi/directory/users.py

@@ -1,30 +0,0 @@
-import gam
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-
-def signout(users):
-    cd = gapi_directory.build()
-    i = 0
-    count = len(users)
-    for user in users:
-        i += 1
-        user = gam.normalizeEmailAddressOrUID(user)
-        print(f'Signing Out {user}{gam.currentCount(i, count)}')
-        gapi.call(cd.users(),
-                  'signOut',
-                  soft_errors=True,
-                  userKey=user)
-
-
-def turn_off_2sv(users):
-    cd = gapi_directory.build()
-    i = 0
-    count = len(users)
-    for user in users:
-        i += 1
-        user = gam.normalizeEmailAddressOrUID(user)
-        print(f'Turning Off 2-Step Verification for {user}{gam.currentCount(i, count)}')
-        gapi.call(cd.twoStepVerification(),
-                  'turnOff',
-                  soft_errors=True,
-                  userKey=user)

src/gam/gapi/errors.py

@@ -1,380 +0,0 @@
-"""GAPI and OAuth Token related errors methods."""
-
-from enum import Enum
-import json
-
-from gam import controlflow
-from gam import display
-from gam.var import UTF8
-
-
-class GapiAbortedError(Exception):
-    pass
-
-
-class GapiAuthErrorError(Exception):
-    pass
-
-
-class GapiBadGatewayError(Exception):
-    pass
-
-
-class GapiBadRequestError(Exception):
-    pass
-
-
-class GapiConditionNotMetError(Exception):
-    pass
-
-
-class GapiCyclicMembershipsNotAllowedError(Exception):
-    pass
-
-
-class GapiDomainCannotUseApisError(Exception):
-    pass
-
-
-class GapiDomainNotFoundError(Exception):
-    pass
-
-
-class GapiDuplicateError(Exception):
-    pass
-
-
-class GapiFailedPreconditionError(Exception):
-    pass
-
-
-class GapiForbiddenError(Exception):
-    pass
-
-
-class GapiGatewayTimeoutError(Exception):
-    pass
-
-
-class GapiGroupNotFoundError(Exception):
-    pass
-
-
-class GapiInvalidError(Exception):
-    pass
-
-
-class GapiInvalidArgumentError(Exception):
-    pass
-
-
-class GapiInvalidMemberError(Exception):
-    pass
-
-
-class GapiMemberNotFoundError(Exception):
-    pass
-
-
-class GapiNotFoundError(Exception):
-    pass
-
-
-class GapiNotImplementedError(Exception):
-    pass
-
-
-class GapiPermissionDeniedError(Exception):
-    pass
-
-
-class GapiResourceNotFoundError(Exception):
-    pass
-
-
-class GapiServiceNotAvailableError(Exception):
-    pass
-
-
-class GapiUserNotFoundError(Exception):
-    pass
-
-
-# GAPI Error Reasons
-class ErrorReason(Enum):
-    """The reason why a non-200 HTTP response was returned from a GAPI."""
-    ABORTED = 'aborted'
-    AUTH_ERROR = 'authError'
-    BACKEND_ERROR = 'backendError'
-    BAD_GATEWAY = 'badGateway'
-    BAD_REQUEST = 'badRequest'
-    CONDITION_NOT_MET = 'conditionNotMet'
-    CYCLIC_MEMBERSHIPS_NOT_ALLOWED = 'cyclicMembershipsNotAllowed'
-    DAILY_LIMIT_EXCEEDED = 'dailyLimitExceeded'
-    DOMAIN_CANNOT_USE_APIS = 'domainCannotUseApis'
-    DOMAIN_NOT_FOUND = 'domainNotFound'
-    DUPLICATE = 'duplicate'
-    FAILED_PRECONDITION = 'failedPrecondition'
-    FORBIDDEN = 'forbidden'
-    FOUR_O_NINE = '409'
-    FOUR_O_O = '400'
-    FOUR_O_THREE = '403'
-    FOUR_TWO_NINE = '429'
-    GATEWAY_TIMEOUT = 'gatewayTimeout'
-    GROUP_NOT_FOUND = 'groupNotFound'
-    INTERNAL_ERROR = 'internalError'
-    INVALID = 'invalid'
-    INVALID_ARGUMENT = 'invalidArgument'
-    INVALID_MEMBER = 'invalidMember'
-    MEMBER_NOT_FOUND = 'memberNotFound'
-    NOT_FOUND = 'notFound'
-    NOT_IMPLEMENTED = 'notImplemented'
-    PERMISSION_DENIED = 'permissionDenied'
-    QUOTA_EXCEEDED = 'quotaExceeded'
-    RATE_LIMIT_EXCEEDED = 'rateLimitExceeded'
-    RESOURCE_NOT_FOUND = 'resourceNotFound'
-    SERVICE_NOT_AVAILABLE = 'serviceNotAvailable'
-    SERVICE_LIMIT = 'serviceLimit'
-    SYSTEM_ERROR = 'systemError'
-    USER_NOT_FOUND = 'userNotFound'
-    USER_RATE_LIMIT_EXCEEDED = 'userRateLimitExceeded'
-
-    def __str__(self):
-        return str(self.value)
-
-
-# Common sets of GAPI error reasons
-DEFAULT_RETRY_REASONS = [
-    ErrorReason.QUOTA_EXCEEDED,
-    ErrorReason.RATE_LIMIT_EXCEEDED,
-    ErrorReason.USER_RATE_LIMIT_EXCEEDED,
-    ErrorReason.BACKEND_ERROR,
-    ErrorReason.BAD_GATEWAY,
-    ErrorReason.GATEWAY_TIMEOUT,
-    ErrorReason.INTERNAL_ERROR,
-    ErrorReason.FOUR_TWO_NINE,
-]
-GMAIL_THROW_REASONS = [ErrorReason.SERVICE_NOT_AVAILABLE]
-GROUP_GET_THROW_REASONS = [
-    ErrorReason.GROUP_NOT_FOUND, ErrorReason.DOMAIN_NOT_FOUND,
-    ErrorReason.DOMAIN_CANNOT_USE_APIS, ErrorReason.FORBIDDEN,
-    ErrorReason.BAD_REQUEST
-]
-GROUP_GET_RETRY_REASONS = [ErrorReason.INVALID, ErrorReason.SYSTEM_ERROR]
-MEMBERS_THROW_REASONS = [
-    ErrorReason.GROUP_NOT_FOUND, ErrorReason.DOMAIN_NOT_FOUND,
-    ErrorReason.DOMAIN_CANNOT_USE_APIS, ErrorReason.INVALID,
-    ErrorReason.FORBIDDEN
-]
-MEMBERS_RETRY_REASONS = [ErrorReason.SYSTEM_ERROR]
-
-# A map of GAPI error reasons to the corresponding GAM Python Exception
-ERROR_REASON_TO_EXCEPTION = {
-    ErrorReason.ABORTED:
-        GapiAbortedError,
-    ErrorReason.AUTH_ERROR:
-        GapiAuthErrorError,
-    ErrorReason.BAD_GATEWAY:
-        GapiBadGatewayError,
-    ErrorReason.BAD_REQUEST:
-        GapiBadRequestError,
-    ErrorReason.CONDITION_NOT_MET:
-        GapiConditionNotMetError,
-    ErrorReason.CYCLIC_MEMBERSHIPS_NOT_ALLOWED:
-        GapiCyclicMembershipsNotAllowedError,
-    ErrorReason.DOMAIN_CANNOT_USE_APIS:
-        GapiDomainCannotUseApisError,
-    ErrorReason.DOMAIN_NOT_FOUND:
-        GapiDomainNotFoundError,
-    ErrorReason.DUPLICATE:
-        GapiDuplicateError,
-    ErrorReason.FAILED_PRECONDITION:
-        GapiFailedPreconditionError,
-    ErrorReason.FORBIDDEN:
-        GapiForbiddenError,
-    ErrorReason.GATEWAY_TIMEOUT:
-        GapiGatewayTimeoutError,
-    ErrorReason.GROUP_NOT_FOUND:
-        GapiGroupNotFoundError,
-    ErrorReason.INVALID:
-        GapiInvalidError,
-    ErrorReason.INVALID_ARGUMENT:
-        GapiInvalidArgumentError,
-    ErrorReason.INVALID_MEMBER:
-        GapiInvalidMemberError,
-    ErrorReason.MEMBER_NOT_FOUND:
-        GapiMemberNotFoundError,
-    ErrorReason.NOT_FOUND:
-        GapiNotFoundError,
-    ErrorReason.NOT_IMPLEMENTED:
-        GapiNotImplementedError,
-    ErrorReason.PERMISSION_DENIED:
-        GapiPermissionDeniedError,
-    ErrorReason.RESOURCE_NOT_FOUND:
-        GapiResourceNotFoundError,
-    ErrorReason.SERVICE_NOT_AVAILABLE:
-        GapiServiceNotAvailableError,
-    ErrorReason.USER_NOT_FOUND:
-        GapiUserNotFoundError,
-}
-
-# OAuth Token Errors
-OAUTH2_TOKEN_ERRORS = [
-    'access_denied',
-    'access_denied: Requested client not authorized',
-    'internal_failure: Backend Error',
-    'internal_failure: None',
-    'invalid_grant',
-    'invalid_grant: Bad Request',
-    'invalid_grant: Invalid email or User ID',
-    'invalid_grant: Not a valid email',
-    'invalid_grant: Invalid JWT: No valid verifier found for issuer',
-    'invalid_grant: The account has been deleted',
-    'invalid_grant: reauth related error (invalid_rapt)',
-    'invalid_request: Invalid impersonation prn email address',
-    'invalid_request: Invalid impersonation "sub" field',
-    'unauthorized_client: Client is unauthorized to retrieve access tokens '
-    'using this method',
-    'unauthorized_client: Client is unauthorized to retrieve access tokens '
-    'using this method, or client not authorized for any of the scopes '
-    'requested',
-    'unauthorized_client: Unauthorized client or scope in request',
-]
-
-
-def _create_http_error_dict(status_code, reason, message):
-    """Creates a basic error dict similar to most Google API Errors.
-
-  Args:
-    status_code: Int, the error's HTTP response status code.
-    reason: String, a camelCase reason for the HttpError being given.
-    message: String, a general error message describing the error that occurred.
-
-  Returns:
-    dict
-  """
-    return {
-        'error': {
-            'code': status_code,
-            'errors': [{
-                'reason': str(reason),
-                'message': message,
-            }]
-        }
-    }
-
-
-def get_gapi_error_detail(e,
-                          soft_errors=False,
-                          silent_errors=False,
-                          retry_on_http_error=False):
-    """Extracts error detail from a non-200 GAPI Response.
-
-  Args:
-    e: googleapiclient.HttpError, The HTTP Error received.
-    soft_errors: Boolean, If true, causes error messages to be surpressed,
-      rather than sending them to stderr.
-    silent_errors: Boolean, If true, suppresses and ignores any errors from
-      being displayed
-    retry_on_http_error: Boolean, If true, will return -1 as the HTTP Response
-      code, indicating that the request can be retried. TODO: Remove this param,
-        as it seems to be outside the scope of this method.
-
-  Returns:
-    A tuple containing the HTTP Response code, GAPI error reason, and error
-        message.
-  """
-    try:
-        error = json.loads(e.content.decode(UTF8))
-    except ValueError:
-        error_content = e.content.decode(UTF8) if isinstance(
-            e.content, bytes) else e.content
-        if (e.resp['status'] == '503') and (
-                error_content == 'Quota exceeded for the current request'):
-            return (e.resp['status'], ErrorReason.QUOTA_EXCEEDED.value,
-                    error_content)
-        if (e.resp['status'] == '403') and (error_content.startswith(
-                'Request rate higher than configured')):
-            return (e.resp['status'], ErrorReason.QUOTA_EXCEEDED.value,
-                    error_content)
-        if (e.resp['status'] == '502') and ('Bad Gateway' in error_content):
-            return (e.resp['status'], ErrorReason.BAD_GATEWAY.value,
-                    error_content)
-        if (e.resp['status'] == '504') and ('Gateway Timeout' in error_content):
-            return (e.resp['status'], ErrorReason.GATEWAY_TIMEOUT.value,
-                    error_content)
-        if (e.resp['status'] == '403') and ('Invalid domain.' in error_content):
-            error = _create_http_error_dict(403, ErrorReason.NOT_FOUND.value,
-                                            'Domain not found')
-        elif (e.resp['status'] == '400') and (
-                'InvalidSsoSigningKey' in error_content):
-            error = _create_http_error_dict(400, ErrorReason.INVALID.value,
-                                            'InvalidSsoSigningKey')
-        elif (e.resp['status'] == '400') and ('UnknownError' in error_content):
-            error = _create_http_error_dict(400, ErrorReason.INVALID.value,
-                                            'UnknownError')
-        elif retry_on_http_error:
-            return (-1, None, None)
-        elif soft_errors:
-            if not silent_errors:
-                display.print_error(error_content)
-            return (0, None, None)
-        else:
-            controlflow.system_error_exit(5, error_content)
-        # END: ValueError catch
-
-    if 'error' in error:
-        http_status = error['error']['code']
-        try:
-            message = error['error']['errors'][0]['message']
-        except KeyError:
-            message = error['error']['message']
-        if http_status == 404:
-            if 'Requested entity was not found' in message or 'does not exist' in message:
-                error = _create_http_error_dict(404, ErrorReason.NOT_FOUND.value,
-                                                message)
-    else:
-        if 'error_description' in error:
-            if error['error_description'] == 'Invalid Value':
-                message = error['error_description']
-                http_status = 400
-                error = _create_http_error_dict(400, ErrorReason.INVALID.value,
-                                                message)
-            else:
-                controlflow.system_error_exit(4, str(error))
-        else:
-            controlflow.system_error_exit(4, str(error))
-
-    # Extract the error reason
-    try:
-        reason = error['error']['errors'][0]['reason']
-        if reason == 'notFound':
-            if 'userKey' in message:
-                reason = ErrorReason.USER_NOT_FOUND.value
-            elif 'groupKey' in message:
-                reason = ErrorReason.GROUP_NOT_FOUND.value
-            elif 'memberKey' in message:
-                reason = ErrorReason.MEMBER_NOT_FOUND.value
-            elif 'Domain not found' in message:
-                reason = ErrorReason.DOMAIN_NOT_FOUND.value
-            elif 'Resource Not Found' in message:
-                reason = ErrorReason.RESOURCE_NOT_FOUND.value
-        elif reason == 'invalid':
-            if 'userId' in message:
-                reason = ErrorReason.USER_NOT_FOUND.value
-            elif 'memberKey' in message:
-                reason = ErrorReason.INVALID_MEMBER.value
-        elif reason == 'failedPrecondition':
-            if 'Bad Request' in message:
-                reason = ErrorReason.BAD_REQUEST.value
-            elif 'Mail service not enabled' in message:
-                reason = ErrorReason.SERVICE_NOT_AVAILABLE.value
-        elif reason == 'required':
-            if 'memberKey' in message:
-                reason = ErrorReason.MEMBER_NOT_FOUND.value
-        elif reason == 'conditionNotMet':
-            if 'Cyclic memberships not allowed' in message:
-                reason = ErrorReason.CYCLIC_MEMBERSHIPS_NOT_ALLOWED.value
-    except KeyError:
-        reason = f'{http_status}'
-    return (http_status, reason, message)

src/gam/gapi/errors_test.py

@@ -1,210 +0,0 @@
-"""Python unit tests for gapi.errors"""
-
-import json
-
-import unittest
-from unittest.mock import patch
-
-import googleapiclient.errors
-from gam.gapi import errors
-
-
-def create_simple_http_error(status, reason, message):
-    content = errors._create_http_error_dict(status, reason, message)
-    return create_http_error(status, content)
-
-
-def create_http_error(status, content):
-    response = {
-        'status': status,
-        'content-type': 'application/json',
-    }
-    content_as_bytes = json.dumps(content).encode('UTF-8')
-    return googleapiclient.errors.HttpError(response, content_as_bytes)
-
-
-class ErrorsTest(unittest.TestCase):
-
-    def test_get_gapi_error_detail_quota_exceeded(self):
-        # TODO: Add test logic once the opening ValueError exception case has a
-        # repro case (i.e. an Exception type/format that will cause it to raise).
-        pass
-
-    def test_get_gapi_error_detail_invalid_domain(self):
-        # TODO: Add test logic once the opening ValueError exception case has a
-        # repro case (i.e. an Exception type/format that will cause it to raise).
-        pass
-
-    def test_get_gapi_error_detail_invalid_signing_key(self):
-        # TODO: Add test logic once the opening ValueError exception case has a
-        # repro case (i.e. an Exception type/format that will cause it to raise).
-        pass
-
-    def test_get_gapi_error_detail_unknown_error(self):
-        # TODO: Add test logic once the opening ValueError exception case has a
-        # repro case (i.e. an Exception type/format that will cause it to raise).
-        pass
-
-    def test_get_gapi_error_retry_http_error(self):
-        # TODO: Add test logic once the opening ValueError exception case has a
-        # repro case (i.e. an Exception type/format that will cause it to raise).
-        pass
-
-    def test_get_gapi_error_prints_soft_errors(self):
-        # TODO: Add test logic once the opening ValueError exception case has a
-        # repro case (i.e. an Exception type/format that will cause it to raise).
-        pass
-
-    def test_get_gapi_error_exits_on_unrecoverable_errors(self):
-        # TODO: Add test logic once the opening ValueError exception case has a
-        # repro case (i.e. an Exception type/format that will cause it to raise).
-        pass
-
-    def test_get_gapi_error_quota_exceeded_for_current_request(self):
-        # TODO: Add test logic once the opening ValueError exception case has a
-        # repro case (i.e. an Exception type/format that will cause it to raise).
-        pass
-
-    def test_get_gapi_error_quota_exceeded_high_request_rate(self):
-        # TODO: Add test logic once the opening ValueError exception case has a
-        # repro case (i.e. an Exception type/format that will cause it to raise).
-        pass
-
-    def test_get_gapi_error_extracts_user_not_found(self):
-        err = create_simple_http_error(404, 'notFound',
-                                       'Resource Not Found: userKey.')
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, 404)
-        self.assertEqual(reason, errors.ErrorReason.USER_NOT_FOUND.value)
-        self.assertEqual(message, 'Resource Not Found: userKey.')
-
-    def test_get_gapi_error_extracts_group_not_found(self):
-        err = create_simple_http_error(404, 'notFound',
-                                       'Resource Not Found: groupKey.')
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, 404)
-        self.assertEqual(reason, errors.ErrorReason.GROUP_NOT_FOUND.value)
-        self.assertEqual(message, 'Resource Not Found: groupKey.')
-
-    def test_get_gapi_error_extracts_member_not_found(self):
-        err = create_simple_http_error(404, 'notFound',
-                                       'Resource Not Found: memberKey.')
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, 404)
-        self.assertEqual(reason, errors.ErrorReason.MEMBER_NOT_FOUND.value)
-        self.assertEqual(message, 'Resource Not Found: memberKey.')
-
-    def test_get_gapi_error_extracts_domain_not_found(self):
-        err = create_simple_http_error(404, 'notFound', 'Domain not found.')
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, 404)
-        self.assertEqual(reason, errors.ErrorReason.DOMAIN_NOT_FOUND.value)
-        self.assertEqual(message, 'Domain not found.')
-
-    def test_get_gapi_error_extracts_generic_resource_not_found(self):
-        err = create_simple_http_error(404, 'notFound',
-                                       'Resource Not Found: unknownResource.')
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, 404)
-        self.assertEqual(reason, errors.ErrorReason.RESOURCE_NOT_FOUND.value)
-        self.assertEqual(message, 'Resource Not Found: unknownResource.')
-
-    def test_get_gapi_error_extracts_invalid_userid(self):
-        err = create_simple_http_error(400, 'invalid', 'Invalid Input: userId')
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, 400)
-        self.assertEqual(reason, errors.ErrorReason.USER_NOT_FOUND.value)
-        self.assertEqual(message, 'Invalid Input: userId')
-
-    def test_get_gapi_error_extracts_invalid_member(self):
-        err = create_simple_http_error(400, 'invalid',
-                                       'Invalid Input: memberKey')
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, 400)
-        self.assertEqual(reason, errors.ErrorReason.INVALID_MEMBER.value)
-        self.assertEqual(message, 'Invalid Input: memberKey')
-
-    def test_get_gapi_error_extracts_bad_request(self):
-        err = create_simple_http_error(400, 'failedPrecondition', 'Bad Request')
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, 400)
-        self.assertEqual(reason, errors.ErrorReason.BAD_REQUEST.value)
-        self.assertEqual(message, 'Bad Request')
-
-    def test_get_gapi_error_extracts_service_not_available(self):
-        err = create_simple_http_error(400, 'failedPrecondition',
-                                       'Mail service not enabled')
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, 400)
-        self.assertEqual(reason, errors.ErrorReason.SERVICE_NOT_AVAILABLE.value)
-        self.assertEqual(message, 'Mail service not enabled')
-
-    def test_get_gapi_error_extracts_required_member_not_found(self):
-        err = create_simple_http_error(400, 'required',
-                                       'Missing required field: memberKey')
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, 400)
-        self.assertEqual(reason, errors.ErrorReason.MEMBER_NOT_FOUND.value)
-        self.assertEqual(message, 'Missing required field: memberKey')
-
-    def test_get_gapi_error_extracts_cyclic_memberships_error(self):
-        err = create_simple_http_error(400, 'conditionNotMet',
-                                       'Cyclic memberships not allowed')
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, 400)
-        self.assertEqual(
-            reason, errors.ErrorReason.CYCLIC_MEMBERSHIPS_NOT_ALLOWED.value)
-        self.assertEqual(message, 'Cyclic memberships not allowed')
-
-    def test_get_gapi_error_extracts_single_error_with_message(self):
-        status_code = 999
-        response = {'status': status_code}
-        # This error does not have an "errors" key describing each error.
-        content = {'error': {'code': status_code, 'message': 'unknown error'}}
-        content_as_bytes = json.dumps(content).encode('UTF-8')
-        err = googleapiclient.errors.HttpError(response, content_as_bytes)
-
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, status_code)
-        self.assertEqual(reason, str(status_code))
-        self.assertEqual(message, content['error']['message'])
-
-    def test_get_gapi_error_exits_code_4_on_malformed_error_with_unknown_description(
-        self):
-        status_code = 999
-        response = {'status': status_code}
-        # This error only has an error_description_field and an unknown description.
-        content = {'error_description': 'something errored'}
-        content_as_bytes = json.dumps(content).encode('UTF-8')
-        err = googleapiclient.errors.HttpError(response, content_as_bytes)
-
-        with self.assertRaises(SystemExit) as context:
-            errors.get_gapi_error_detail(err)
-        self.assertEqual(4, context.exception.code)
-
-    def test_get_gapi_error_exits_on_invalid_error_description(self):
-        status_code = 400
-        response = {'status': status_code}
-        content = {'error_description': 'Invalid Value'}
-        content_as_bytes = json.dumps(content).encode('UTF-8')
-        err = googleapiclient.errors.HttpError(response, content_as_bytes)
-
-        http_status, reason, message = errors.get_gapi_error_detail(err)
-        self.assertEqual(http_status, status_code)
-        self.assertEqual(reason, errors.ErrorReason.INVALID.value)
-        self.assertEqual(message, 'Invalid Value')
-
-    def test_get_gapi_error_exits_code_4_on_unexpected_error_contents(self):
-        status_code = 900
-        response = {'status': status_code}
-        content = {'notErrorContentThatIsExpected': 'foo'}
-        content_as_bytes = json.dumps(content).encode('UTF-8')
-        err = googleapiclient.errors.HttpError(response, content_as_bytes)
-
-        with self.assertRaises(SystemExit) as context:
-            errors.get_gapi_error_detail(err)
-        self.assertEqual(4, context.exception.code)
-
-
-if __name__ == '__main__':
-    unittest.main()

src/gam/gapi/licensing.py

@@ -1,299 +0,0 @@
-import re
-import sys
-
-import gam
-from gam.var import *
-from gam import controlflow
-from gam import display
-from gam import gapi
-from gam.gapi import errors as gapi_errors
-
-
-def build():
-    return gam.buildGAPIObject('licensing')
-
-
-def getProductAndSKU(sku):
-    l_sku = sku.lower().replace('-', '').replace(' ', '')
-    for a_sku, sku_values in list(SKUS.items()):
-        if l_sku == a_sku.lower().replace(
-                '-',
-                '') or l_sku in sku_values['aliases'] or l_sku == sku_values[
-                    'displayName'].lower().replace(' ', ''):
-            return (sku_values['product'], a_sku)
-    try:
-        product = re.search('^([A-Z,a-z]*-[A-Z,a-z]*)', sku).group(1)
-    except AttributeError:
-        product = sku
-    return (product, sku)
-
-
-def user_lic_result(request_id, response, exception):
-    if exception:
-        http_status, reason, message = gapi_errors.get_gapi_error_detail(
-                exception,
-                soft_errors=True)
-        print(f'ERROR: {request_id}: {http_status} - {reason} {message}')
-
-
-def create(users, sku=None):
-    lic = build()
-    if not sku:
-        sku = sys.argv[5]
-    productId, skuId = getProductAndSKU(sku)
-    sku_name = _formatSKUIdDisplayName(skuId)
-    i = 6
-    if len(sys.argv) > 6 and sys.argv[i].lower() in ['product', 'productid']:
-        productId = sys.argv[i+1]
-        i += 2
-    for user in users:
-        print(f'Adding license {sku_name} from to {user}')
-        gapi.call(lic.licenseAssignments(),
-                  'insert',
-                  soft_errors=True,
-                  productId=productId,
-                  skuId=skuId,
-                  body={'userId': user})
-
-
-def delete(users, sku=None):
-    lic = build()
-    if not sku:
-        sku = sys.argv[5]
-    productId, skuId = getProductAndSKU(sku)
-    sku_name = _formatSKUIdDisplayName(skuId)
-    i = 6
-    if len(sys.argv) > 6 and sys.argv[i].lower() in ['product', 'productid']:
-        productId = sys.argv[i+1]
-        i += 2
-    for user in users:
-         print(f'Removing license {sku_name} from user {user}')
-         gapi.call(lic.licenseAssignments(),
-                   'delete',
-                   soft_errors=True,
-                   productId=productId,
-                   skuId=skuId,
-                   userId=user)
-
-
-def sync(users):
-    sku = sys.argv[5]
-    current_licenses = gam.getUsersToModify(entity_type='license',
-        entity=sku)
-    users_to_license = [user for user in users if user not in current_licenses]
-    users_to_unlicense = [user for user in current_licenses if user not in users]
-    print(f'Need to remove license from {len(users_to_unlicense)} and add to ' \
-          f'{len(users_to_license)} users...')
-    # do the remove first to free up seats
-    delete(users_to_unlicense, sku)
-    create(users_to_license, sku)
-
-
-def update(users, sku=None, old_sku=None):
-    lic = build()
-    if not sku:
-        sku = sys.argv[5]
-    productId, skuId = getProductAndSKU(sku)
-    sku_name = _formatSKUIdDisplayName(skuId)
-    i = 6
-    if len(sys.argv) > 6 and sys.argv[i].lower() in ['product', 'productid']:
-        productId = sys.argv[i+1]
-        i += 2
-    if not old_sku:
-        try:
-            old_sku = sys.argv[i]
-            if old_sku.lower() == 'from':
-                old_sku = sys.argv[i + 1]
-        except KeyError:
-            controlflow.system_error_exit(
-                2,
-                'You need to specify the user\'s old SKU as the last argument'
-            )
-        _, old_sku = getProductAndSKU(old_sku)
-    old_sku_name = _formatSKUIdDisplayName(old_sku)
-    for user in users:
-        print(f'Changing user {user} from license {old_sku_name} to {sku_name}')
-        gapi.call(lic.licenseAssignments(),
-                  'patch',
-                  soft_errors=True,
-                  productId=productId,
-                  skuId=old_sku,
-                  userId=user,
-                  body={'skuId': skuId})
-
-
-def print_(returnFields=None,
-                    skus=None,
-                    countsOnly=False,
-                    returnCounts=False):
-    lic = build()
-    products = []
-    licenses = []
-    licenseCounts = []
-    if not returnFields:
-        csvRows = []
-        todrive = False
-        i = 3
-        while i < len(sys.argv):
-            myarg = sys.argv[i].lower()
-            if not returnCounts and myarg == 'todrive':
-                todrive = True
-                i += 1
-            elif myarg in ['products', 'product']:
-                products = sys.argv[i + 1].split(',')
-                i += 2
-            elif myarg in ['sku', 'skus']:
-                skus = sys.argv[i + 1].split(',')
-                i += 2
-            elif myarg == 'allskus':
-                skus = sorted(SKUS)
-                products = []
-                i += 1
-            elif myarg == 'gsuite':
-                skus = [
-                    skuId for skuId in SKUS
-                    if SKUS[skuId]['product'] in ['Google-Apps', '101031']
-                ]
-                products = []
-                i += 1
-            elif myarg == 'countsonly':
-                countsOnly = True
-                i += 1
-            else:
-                controlflow.invalid_argument_exit(sys.argv[i],
-                                                  'gam print licenses')
-        if not countsOnly:
-            fields = 'nextPageToken,items(productId,skuId,userId)'
-            titles = ['userId', 'productId', 'skuId']
-        else:
-            fields = 'nextPageToken,items(userId)'
-            if not returnCounts:
-                if skus:
-                    titles = ['productId', 'skuId', 'licenses']
-                else:
-                    titles = ['productId', 'licenses']
-    else:
-        fields = f'nextPageToken,items({returnFields})'
-    if skus:
-        for sku in skus:
-            if not products:
-                product, sku = getProductAndSKU(sku)
-            else:
-                product = products[0]
-            page_message = gapi.got_total_items_msg(
-                f'Licenses for {SKUS.get(sku, {"displayName": sku})["displayName"]}',
-                '...\n')
-            try:
-                licenses += gapi.get_all_pages(
-                    lic.licenseAssignments(),
-                    'listForProductAndSku',
-                    'items',
-                    throw_reasons=[
-                        gapi_errors.ErrorReason.INVALID,
-                        gapi_errors.ErrorReason.FORBIDDEN
-                    ],
-                    page_message=page_message,
-                    customerId=GC_Values[GC_DOMAIN],
-                    productId=product,
-                    skuId=sku,
-                    fields=fields)
-                if countsOnly:
-                    licenseCounts.append([
-                        'Product', product, 'SKU', sku, 'Licenses',
-                        len(licenses)
-                    ])
-                    licenses = []
-            except (gapi_errors.GapiInvalidError,
-                    gapi_errors.GapiForbiddenError):
-                pass
-    else:
-        if not products:
-            products = sorted(PRODUCTID_NAME_MAPPINGS)
-        for productId in products:
-            page_message = gapi.got_total_items_msg(
-                f'Licenses for {PRODUCTID_NAME_MAPPINGS.get(productId, productId)}',
-                '...\n')
-            try:
-                licenses += gapi.get_all_pages(
-                    lic.licenseAssignments(),
-                    'listForProduct',
-                    'items',
-                    throw_reasons=[
-                        gapi_errors.ErrorReason.INVALID,
-                        gapi_errors.ErrorReason.FORBIDDEN
-                    ],
-                    page_message=page_message,
-                    customerId=GC_Values[GC_DOMAIN],
-                    productId=productId,
-                    fields=fields)
-                if countsOnly:
-                    licenseCounts.append(
-                        ['Product', productId, 'Licenses',
-                         len(licenses)])
-                    licenses = []
-            except (gapi_errors.GapiInvalidError,
-                    gapi_errors.GapiForbiddenError):
-                pass
-    if countsOnly:
-        if returnCounts:
-            return licenseCounts
-        if skus:
-            for u_license in licenseCounts:
-                csvRows.append({
-                    'productId': u_license[1],
-                    'skuId': u_license[3],
-                    'licenses': u_license[5]
-                })
-        else:
-            for u_license in licenseCounts:
-                csvRows.append({
-                    'productId': u_license[1],
-                    'licenses': u_license[3]
-                })
-        display.write_csv_file(csvRows, titles, 'Licenses', todrive)
-        return
-    if returnFields:
-        if returnFields == 'userId':
-            userIds = []
-            for u_license in licenses:
-                userId = u_license.get('userId', '').lower()
-                if userId:
-                    userIds.append(userId)
-            return userIds
-        userSkuIds = {}
-        for u_license in licenses:
-            userId = u_license.get('userId', '').lower()
-            skuId = u_license.get('skuId')
-            if userId and skuId:
-                userSkuIds.setdefault(userId, [])
-                userSkuIds[userId].append(skuId)
-        return userSkuIds
-    for u_license in licenses:
-        userId = u_license.get('userId', '').lower()
-        skuId = u_license.get('skuId', '')
-        csvRows.append({
-            'userId': userId,
-            'productId': u_license.get('productId', ''),
-            'skuId': _skuIdToDisplayName(skuId)
-        })
-    display.write_csv_file(csvRows, titles, 'Licenses', todrive)
-
-
-def show():
-    licenseCounts = print_(countsOnly=True, returnCounts=True)
-    for u_license in licenseCounts:
-        line = ''
-        for i in range(0, len(u_license), 2):
-            line += f'{u_license[i]}: {u_license[i+1]}, '
-        print(line[:-2])
-
-
-def _skuIdToDisplayName(skuId):
-    return SKUS[skuId]['displayName'] if skuId in SKUS else skuId
-
-
-def _formatSKUIdDisplayName(skuId):
-    skuIdDisplay = _skuIdToDisplayName(skuId)
-    if skuId == skuIdDisplay:
-        return skuId
-    return f'{skuId} ({skuIdDisplay})'

src/gam/gapi/reports.py

@@ -1,585 +0,0 @@
-import calendar
-import datetime
-import re
-import sys
-
-from dateutil.relativedelta import relativedelta
-
-import gam
-from gam.var import *
-from gam import controlflow
-from gam import display
-from gam import gapi
-from gam import utils
-from gam.gapi.directory import orgunits as gapi_directory_orgunits
-
-
-def build():
-    return gam.buildGAPIObject('reports')
-
-
-REPORT_CHOICE_MAP = {
-    'access': 'access_transparency',
-    'accesstransparency': 'access_transparency',
-    'calendars': 'calendar',
-    'customers': 'customer',
-    'doc': 'drive',
-    'docs': 'drive',
-    'domain': 'customer',
-    'enterprisegroups': 'groups_enterprise',
-    'google+': 'gplus',
-    'group': 'groups',
-    'groupsenterprise': 'groups_enterprise',
-    'hangoutsmeet': 'meet',
-    'logins': 'login',
-    'oauthtoken': 'token',
-    'tokens': 'token',
-    'usage': 'usage',
-    'usageparameters': 'usageparameters',
-    'users': 'user',
-    'useraccounts': 'user_accounts',
-}
-
-
-def showUsageParameters():
-    rep = build()
-    throw_reasons = [
-        gapi.errors.ErrorReason.INVALID, gapi.errors.ErrorReason.BAD_REQUEST
-    ]
-    todrive = False
-    if len(sys.argv) == 3:
-        controlflow.missing_argument_exit('user or customer',
-                                          'report usageparameters')
-    report = sys.argv[3].lower()
-    titles = ['parameter']
-    if report == 'customer':
-        endpoint = rep.customerUsageReports()
-        kwargs = {}
-    elif report == 'user':
-        endpoint = rep.userUsageReport()
-        kwargs = {'userKey': gam._get_admin_email()}
-    else:
-        controlflow.expected_argument_exit('usageparameters',
-                                           ['user', 'customer'], report)
-    customerId = GC_Values[GC_CUSTOMER_ID]
-    if customerId == MY_CUSTOMER:
-        customerId = None
-    tryDate = datetime.date.today().strftime(YYYYMMDD_FORMAT)
-    all_parameters = set()
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam report usageparameters')
-    fullDataRequired = ['all']
-    while True:
-        try:
-            result = gapi.call(endpoint,
-                               'get',
-                               throw_reasons=throw_reasons,
-                               date=tryDate,
-                               customerId=customerId,
-                               fields='warnings,usageReports(parameters(name))',
-                               **kwargs)
-            warnings = result.get('warnings', [])
-            usage = result.get('usageReports')
-            has_reports = bool(usage)
-            fullData, tryDate = _check_full_data_available(
-                warnings, tryDate, fullDataRequired, has_reports)
-            if fullData < 0:
-                print('No usage parameters available.')
-                sys.exit(1)
-            if has_reports:
-                for parameter in usage[0]['parameters']:
-                    name = parameter.get('name')
-                    if name:
-                        all_parameters.add(name)
-            if fullData == 1:
-                break
-        except gapi.errors.GapiInvalidError as e:
-            tryDate = _adjust_date(str(e))
-    csvRows = []
-    for parameter in sorted(all_parameters):
-        csvRows.append({'parameter': parameter})
-    display.write_csv_file(csvRows, titles,
-                           f'{report.capitalize()} Report Usage Parameters',
-                           todrive)
-
-
-REPORTS_PARAMETERS_SIMPLE_TYPES = [
-    'intValue', 'boolValue', 'datetimeValue', 'stringValue'
-]
-
-
-def showUsage():
-    rep = build()
-    throw_reasons = [
-        gapi.errors.ErrorReason.INVALID, gapi.errors.ErrorReason.BAD_REQUEST
-    ]
-    todrive = False
-    if len(sys.argv) == 3:
-        controlflow.missing_argument_exit('user or customer', 'report usage')
-    report = sys.argv[3].lower()
-    titles = ['date']
-    if report == 'customer':
-        endpoint = rep.customerUsageReports()
-        kwargs = [{}]
-    elif report == 'user':
-        endpoint = rep.userUsageReport()
-        kwargs = [{'userKey': 'all'}]
-        titles.append('user')
-    else:
-        controlflow.expected_argument_exit('usage', ['user', 'customer'],
-                                           report)
-    customerId = GC_Values[GC_CUSTOMER_ID]
-    if customerId == MY_CUSTOMER:
-        customerId = None
-    parameters = []
-    start_date = end_date = orgUnitId = None
-    skip_day_numbers = []
-    skip_dates = set()
-    one_day = datetime.timedelta(days=1)
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'startdate':
-            start_date = utils.get_yyyymmdd(sys.argv[i + 1],
-                                            returnDateTime=True)
-            i += 2
-        elif myarg == 'enddate':
-            end_date = utils.get_yyyymmdd(sys.argv[i + 1], returnDateTime=True)
-            i += 2
-        elif myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg in ['fields', 'parameters']:
-            parameters = sys.argv[i + 1].split(',')
-            i += 2
-        elif myarg == 'skipdates':
-            for skip in sys.argv[i + 1].split(','):
-                if skip.find(':') == -1:
-                    skip_dates.add(utils.get_yyyymmdd(skip,
-                                                      returnDateTime=True))
-                else:
-                    skip_start, skip_end = skip.split(':', 1)
-                    skip_start = utils.get_yyyymmdd(skip_start,
-                                                    returnDateTime=True)
-                    skip_end = utils.get_yyyymmdd(skip_end, returnDateTime=True)
-                    while skip_start <= skip_end:
-                        skip_dates.add(skip_start)
-                        skip_start += one_day
-            i += 2
-        elif myarg == 'skipdaysofweek':
-            skipdaynames = sys.argv[i + 1].split(',')
-            dow = [d.lower() for d in calendar.day_abbr]
-            skip_day_numbers = [dow.index(d) for d in skipdaynames if d in dow]
-            i += 2
-        elif report == 'user' and myarg in ['orgunit', 'org', 'ou']:
-            _, orgUnitId = gapi_directory_orgunits.getOrgUnitId(sys.argv[i + 1])
-            i += 2
-        elif report == 'user' and myarg in usergroup_types:
-            users = gam.getUsersToModify(myarg, sys.argv[i + 1])
-            kwargs = [{'userKey': user} for user in users]
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              f'gam report usage {report}')
-    if parameters:
-        titles.extend(parameters)
-        parameters = ','.join(parameters)
-    else:
-        parameters = None
-    if not end_date:
-        end_date = datetime.datetime.now()
-    if not start_date:
-        start_date = end_date + relativedelta(months=-1)
-    if orgUnitId:
-        for kw in kwargs:
-            kw['orgUnitID'] = orgUnitId
-    usage_on_date = start_date
-    start_date = usage_on_date.strftime(YYYYMMDD_FORMAT)
-    usage_end_date = end_date
-    end_date = end_date.strftime(YYYYMMDD_FORMAT)
-    start_use_date = end_use_date = None
-    csvRows = []
-    while usage_on_date <= usage_end_date:
-        if usage_on_date.weekday() in skip_day_numbers or \
-           usage_on_date in skip_dates:
-            usage_on_date += one_day
-            continue
-        use_date = usage_on_date.strftime(YYYYMMDD_FORMAT)
-        usage_on_date += one_day
-        try:
-            for kwarg in kwargs:
-                try:
-                    usage = gapi.get_all_pages(endpoint,
-                                               'get',
-                                               'usageReports',
-                                               throw_reasons=throw_reasons,
-                                               customerId=customerId,
-                                               date=use_date,
-                                               parameters=parameters,
-                                               **kwarg)
-                except gapi.errors.GapiBadRequestError:
-                    continue
-                for entity in usage:
-                    row = {'date': use_date}
-                    if 'userEmail' in entity['entity']:
-                        row['user'] = entity['entity']['userEmail']
-                    for item in entity.get('parameters', []):
-                        if 'name' not in item:
-                            continue
-                        name = item['name']
-                        if name == 'cros:device_version_distribution':
-                            for cros_ver in item['msgValue']:
-                                v = cros_ver['version_number']
-                                column_name = f'cros:num_devices_chrome_{v}'
-                                if column_name not in titles:
-                                    titles.append(column_name)
-                                row[column_name] = cros_ver['num_devices']
-                        else:
-                            if not name in titles:
-                                titles.append(name)
-                            for ptype in REPORTS_PARAMETERS_SIMPLE_TYPES:
-                                if ptype in item:
-                                    row[name] = item[ptype]
-                                    break
-                            else:
-                                row[name] = ''
-                    if not start_use_date:
-                        start_use_date = use_date
-                    end_use_date = use_date
-                    csvRows.append(row)
-        except gapi.errors.GapiInvalidError as e:
-            display.print_warning(str(e))
-            break
-    if start_use_date:
-        report_name = f'{report.capitalize()} Usage Report - {start_use_date}:{end_use_date}'
-    else:
-        report_name = f'{report.capitalize()} Usage Report - {start_date}:{end_date} - No Data'
-    display.write_csv_file(csvRows, titles, report_name, todrive)
-
-
-def showReport():
-    rep = build()
-    throw_reasons = [gapi.errors.ErrorReason.INVALID]
-    report = sys.argv[2].lower()
-    report = REPORT_CHOICE_MAP.get(report.replace('_', ''), report)
-    if report == 'usage':
-        showUsage()
-        return
-    if report == 'usageparameters':
-        showUsageParameters()
-        return
-    valid_apps = gapi.get_enum_values_minus_unspecified(
-        rep._rootDesc['resources']['activities']['methods']['list']
-        ['parameters']['applicationName']['enum']) + ['customer', 'user']
-    if report not in valid_apps:
-        controlflow.expected_argument_exit('report',
-                                           ', '.join(sorted(valid_apps)),
-                                           report)
-    customerId = GC_Values[GC_CUSTOMER_ID]
-    if customerId == MY_CUSTOMER:
-        customerId = None
-    filters = parameters = actorIpAddress = startTime = endTime = eventName = orgUnitId = None
-    tryDate = datetime.date.today().strftime(YYYYMMDD_FORMAT)
-    to_drive = False
-    userKey = 'all'
-    fullDataRequired = None
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower()
-        if myarg == 'date':
-            tryDate = utils.get_yyyymmdd(sys.argv[i + 1])
-            i += 2
-        elif myarg in ['orgunit', 'org', 'ou']:
-            _, orgUnitId = gapi_directory_orgunits.getOrgUnitId(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'fulldatarequired':
-            fullDataRequired = []
-            fdr = sys.argv[i + 1].lower()
-            if fdr and fdr == 'all':
-                fullDataRequired = 'all'
-            else:
-                fullDataRequired = fdr.replace(',', ' ').split()
-            i += 2
-        elif myarg == 'start':
-            startTime = utils.get_time_or_delta_from_now(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'end':
-            endTime = utils.get_time_or_delta_from_now(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'event':
-            eventName = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'user':
-            userKey = sys.argv[i + 1].lower()
-            if userKey != 'all':
-                userKey = gam.normalizeEmailAddressOrUID(sys.argv[i + 1])
-            i += 2
-        elif myarg in ['filter', 'filters']:
-            filters = sys.argv[i + 1]
-            i += 2
-        elif myarg in ['fields', 'parameters']:
-            parameters = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'ip':
-            actorIpAddress = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'todrive':
-            to_drive = True
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam report')
-    if report == 'user':
-        while True:
-            try:
-                one_page = gapi.call(rep.userUsageReport(),
-                                     'get',
-                                     throw_reasons=throw_reasons,
-                                     date=tryDate,
-                                     userKey=userKey,
-                                     customerId=customerId,
-                                     orgUnitID=orgUnitId,
-                                     fields='warnings,usageReports',
-                                     maxResults=1)
-                warnings = one_page.get('warnings', [])
-                has_reports = bool(one_page.get('usageReports'))
-                fullData, tryDate = _check_full_data_available(
-                    warnings, tryDate, fullDataRequired, has_reports)
-                if fullData < 0:
-                    print('No user report available.')
-                    sys.exit(1)
-                if fullData == 0:
-                    continue
-                page_message = gapi.got_total_items_msg('Users', '...\n')
-                usage = gapi.get_all_pages(rep.userUsageReport(),
-                                           'get',
-                                           'usageReports',
-                                           page_message=page_message,
-                                           throw_reasons=throw_reasons,
-                                           date=tryDate,
-                                           userKey=userKey,
-                                           customerId=customerId,
-                                           orgUnitID=orgUnitId,
-                                           filters=filters,
-                                           parameters=parameters)
-                break
-            except gapi.errors.GapiInvalidError as e:
-                tryDate = _adjust_date(str(e))
-        if not usage:
-            print('No user report available.')
-            sys.exit(1)
-        titles = ['email', 'date']
-        csvRows = []
-        for user_report in usage:
-            if 'entity' not in user_report:
-                continue
-            row = {'email': user_report['entity']['userEmail'], 'date': tryDate}
-            for item in user_report.get('parameters', []):
-                if 'name' not in item:
-                    continue
-                name = item['name']
-                if not name in titles:
-                    titles.append(name)
-                for ptype in REPORTS_PARAMETERS_SIMPLE_TYPES:
-                    if ptype in item:
-                        row[name] = item[ptype]
-                        break
-                else:
-                    row[name] = ''
-            csvRows.append(row)
-        display.write_csv_file(csvRows, titles, f'User Reports - {tryDate}',
-                               to_drive)
-    elif report == 'customer':
-        while True:
-            try:
-                first_page = gapi.call(rep.customerUsageReports(),
-                                       'get',
-                                       throw_reasons=throw_reasons,
-                                       customerId=customerId,
-                                       date=tryDate,
-                                       fields='warnings,usageReports')
-                warnings = first_page.get('warnings', [])
-                has_reports = bool(first_page.get('usageReports'))
-                fullData, tryDate = _check_full_data_available(
-                    warnings, tryDate, fullDataRequired, has_reports)
-                if fullData < 0:
-                    print('No customer report available.')
-                    sys.exit(1)
-                if fullData == 0:
-                    continue
-                usage = gapi.get_all_pages(rep.customerUsageReports(),
-                                           'get',
-                                           'usageReports',
-                                           throw_reasons=throw_reasons,
-                                           customerId=customerId,
-                                           date=tryDate,
-                                           parameters=parameters)
-                break
-            except gapi.errors.GapiInvalidError as e:
-                tryDate = _adjust_date(str(e))
-        if not usage:
-            print('No customer report available.')
-            sys.exit(1)
-        titles = ['name', 'value', 'client_id']
-        csvRows = []
-        auth_apps = list()
-        for item in usage[0]['parameters']:
-            if 'name' not in item:
-                continue
-            name = item['name']
-            if 'intValue' in item:
-                value = item['intValue']
-            elif 'msgValue' in item:
-                if name == 'accounts:authorized_apps':
-                    for subitem in item['msgValue']:
-                        app = {}
-                        for an_item in subitem:
-                            if an_item == 'client_name':
-                                app['name'] = 'App: ' + \
-                                    subitem[an_item].replace('\n', '\\n')
-                            elif an_item == 'num_users':
-                                app['value'] = f'{subitem[an_item]} users'
-                            elif an_item == 'client_id':
-                                app['client_id'] = subitem[an_item]
-                        auth_apps.append(app)
-                    continue
-                values = []
-                for subitem in item['msgValue']:
-                    if 'count' in subitem:
-                        mycount = myvalue = None
-                        for key, value in list(subitem.items()):
-                            if key == 'count':
-                                mycount = value
-                            else:
-                                myvalue = value
-                            if mycount and myvalue:
-                                values.append(f'{myvalue}:{mycount}')
-                        value = ' '.join(values)
-                    elif 'version_number' in subitem \
-                         and 'num_devices' in subitem:
-                        values.append(f'{subitem["version_number"]}:'
-                                      f'{subitem["num_devices"]}')
-                    else:
-                        continue
-                    value = ' '.join(sorted(values, reverse=True))
-            csvRows.append({'name': name, 'value': value})
-        for app in auth_apps:  # put apps at bottom
-            csvRows.append(app)
-        display.write_csv_file(csvRows,
-                               titles,
-                               f'Customer Report - {tryDate}',
-                               todrive=to_drive)
-    else:
-        page_message = gapi.got_total_items_msg('Activities', '...\n')
-        activities = gapi.get_all_pages(rep.activities(),
-                                        'list',
-                                        'items',
-                                        page_message=page_message,
-                                        applicationName=report,
-                                        userKey=userKey,
-                                        customerId=customerId,
-                                        actorIpAddress=actorIpAddress,
-                                        startTime=startTime,
-                                        endTime=endTime,
-                                        eventName=eventName,
-                                        filters=filters,
-                                        orgUnitID=orgUnitId)
-        if activities:
-            titles = ['name']
-            csvRows = []
-            for activity in activities:
-                events = activity['events']
-                del activity['events']
-                activity_row = utils.flatten_json(activity)
-                purge_parameters = True
-                for event in events:
-                    for item in event.get('parameters', []):
-                        if set(item) == set(['value', 'name']):
-                            event[item['name']] = item['value']
-                        elif set(item) == set(['intValue', 'name']):
-                            if item['name'] in ['start_time', 'end_time']:
-                                val = item.get('intValue')
-                                if val is not None:
-                                    val = int(val)
-                                    if val >= 62135683200:
-                                        event[item['name']] = \
-                                            datetime.datetime.fromtimestamp(
-                                                val-62135683200).isoformat()
-                            else:
-                                event[item['name']] = item['intValue']
-                        elif set(item) == set(['boolValue', 'name']):
-                            event[item['name']] = item['boolValue']
-                        elif set(item) == set(['multiValue', 'name']):
-                            event[item['name']] = ' '.join(item['multiValue'])
-                        elif item['name'] == 'scope_data':
-                            parts = {}
-                            for message in item['multiMessageValue']:
-                                for mess in message['parameter']:
-                                    value = mess.get(
-                                        'value',
-                                        ' '.join(mess.get('multiValue', [])))
-                                    parts[mess['name']] = parts.get(
-                                        mess['name'], []) + [value]
-                            for part, v in parts.items():
-                                if part == 'scope_name':
-                                    part = 'scope'
-                                event[part] = ' '.join(v)
-                        else:
-                            purge_parameters = False
-                    if purge_parameters:
-                        event.pop('parameters', None)
-                    row = utils.flatten_json(event)
-                    row.update(activity_row)
-                    for item in row:
-                        if item not in titles:
-                            titles.append(item)
-                    csvRows.append(row)
-            display.sort_csv_titles([
-                'name',
-            ], titles)
-            display.write_csv_file(csvRows, titles,
-                                   f'{report.capitalize()} Activity Report',
-                                   to_drive)
-
-
-def _adjust_date(errMsg):
-    match_date = re.match(
-        'Data for dates later than (.*) is not yet '
-        'available. Please check back later', errMsg)
-    if not match_date:
-        match_date = re.match('Start date can not be later than (.*)', errMsg)
-    if not match_date:
-        controlflow.system_error_exit(4, errMsg)
-    return str(match_date.group(1))
-
-
-def _check_full_data_available(warnings, tryDate, fullDataRequired,
-                               has_reports):
-    one_day = datetime.timedelta(days=1)
-    tryDateTime = datetime.datetime.strptime(tryDate, YYYYMMDD_FORMAT)
-    # move to day before if we don't have at least one usageReport
-    if not has_reports:
-        tryDateTime -= one_day
-        return (0, tryDateTime.strftime(YYYYMMDD_FORMAT))
-    for warning in warnings:
-        if warning['code'] == 'PARTIAL_DATA_AVAILABLE':
-            for app in warning['data']:
-                if app['key'] == 'application' and \
-                   app['value'] != 'docs' and \
-                   fullDataRequired is not None and \
-                   (fullDataRequired == 'all' or app['value'] in fullDataRequired):
-                    tryDateTime -= one_day
-                    return (0, tryDateTime.strftime(YYYYMMDD_FORMAT))
-        elif warning['code'] == 'DATA_NOT_AVAILABLE':
-            for app in warning['data']:
-                if app['key'] == 'application' and \
-                   app['value'] != 'docs' and \
-                   (not fullDataRequired or app['value'] in fullDataRequired):
-                    return (-1, tryDate)
-    return (1, tryDate)

src/gam/gapi/siteverification.py

@@ -1,188 +0,0 @@
-import json
-import sys
-from urllib.parse import urlencode
-
-import gam
-from gam.var import *
-from gam import controlflow
-from gam import display
-from gam import fileutils
-from gam import gapi
-from gam.gapi import directory as gapi_directory
-from gam.gapi import errors as gapi_errors
-from gam.gapi.directory import customer as gapi_directory_customer
-from gam import transport
-from gam import utils
-
-import gam
-
-
-def build():
-    return gam.buildGAPIObject('siteVerification')
-
-
-def create():
-    verif = build()
-    a_domain = sys.argv[3]
-    txt_record = gapi.call(verif.webResource(),
-                           'getToken',
-                           body={
-                               'site': {
-                                   'type': 'INET_DOMAIN',
-                                   'identifier': a_domain
-                               },
-                               'verificationMethod': 'DNS_TXT'
-                           })
-    print(f'TXT Record Name:   {a_domain}')
-    print(f'TXT Record Value:  {txt_record["token"]}')
-    print()
-    cname_record = gapi.call(verif.webResource(),
-                             'getToken',
-                             body={
-                                 'site': {
-                                     'type': 'INET_DOMAIN',
-                                     'identifier': a_domain
-                                 },
-                                 'verificationMethod': 'DNS_CNAME'
-                             })
-    cname_token = cname_record['token']
-    cname_list = cname_token.split(' ')
-    cname_subdomain = cname_list[0]
-    cname_value = cname_list[1]
-    print(f'CNAME Record Name:   {cname_subdomain}.{a_domain}')
-    print(f'CNAME Record Value:  {cname_value}')
-    print('')
-    webserver_file_record = gapi.call(
-        verif.webResource(),
-        'getToken',
-        body={
-            'site': {
-                'type': 'SITE',
-                'identifier': f'http://{a_domain}/'
-            },
-            'verificationMethod': 'FILE'
-        })
-    webserver_file_token = webserver_file_record['token']
-    print(f'Saving web server verification file to: {webserver_file_token}')
-    fileutils.write_file(webserver_file_token,
-                         f'google-site-verification: {webserver_file_token}',
-                         continue_on_error=True)
-    print(f'Verification File URL: http://{a_domain}/{webserver_file_token}')
-    print()
-    webserver_meta_record = gapi.call(
-        verif.webResource(),
-        'getToken',
-        body={
-            'site': {
-                'type': 'SITE',
-                'identifier': f'http://{a_domain}/'
-            },
-            'verificationMethod': 'META'
-        })
-    print(f'Meta URL:               http://{a_domain}/')
-    print(f'Meta HTML Header Data:  {webserver_meta_record["token"]}')
-    print()
-
-
-def info():
-    verif = build()
-    sites = gapi.get_items(verif.webResource(), 'list', 'items')
-    if sites:
-        for site in sites:
-            print(f'Site: {site["site"]["identifier"]}')
-            print(f'Type: {site["site"]["type"]}')
-            print('Owners:')
-            for owner in site['owners']:
-                print(f' {owner}')
-            print()
-    else:
-        print('No Sites Verified.')
-
-
-def update():
-    verif = build()
-    a_domain = sys.argv[3]
-    verificationMethod = sys.argv[4].upper()
-    if verificationMethod == 'CNAME':
-        verificationMethod = 'DNS_CNAME'
-    elif verificationMethod in ['TXT', 'TEXT']:
-        verificationMethod = 'DNS_TXT'
-    if verificationMethod in ['DNS_TXT', 'DNS_CNAME']:
-        verify_type = 'INET_DOMAIN'
-        identifier = a_domain
-    else:
-        verify_type = 'SITE'
-        identifier = f'http://{a_domain}/'
-    body = {
-        'site': {
-            'type': verify_type,
-            'identifier': identifier
-        },
-        'verificationMethod': verificationMethod
-    }
-    try:
-        verify_result = gapi.call(
-            verif.webResource(),
-            'insert',
-            throw_reasons=[gapi_errors.ErrorReason.BAD_REQUEST],
-            verificationMethod=verificationMethod,
-            body=body)
-    except gapi_errors.GapiBadRequestError as e:
-        print(f'ERROR: {str(e)}')
-        verify_data = gapi.call(verif.webResource(), 'getToken', body=body)
-        print(f'Method:  {verify_data["method"]}')
-        print(f'Expected Token:      {verify_data["token"]}')
-        if verify_data['method'] in ['DNS_CNAME', 'DNS_TXT']:
-            simplehttp = transport.create_http()
-            base_url = 'https://dns.google/resolve?'
-            query_params = {}
-            if verify_data['method'] == 'DNS_CNAME':
-                cname_token = verify_data['token']
-                cname_list = cname_token.split(' ')
-                cname_subdomain = cname_list[0]
-                query_params['name'] = f'{cname_subdomain}.{a_domain}'
-                query_params['type'] = 'cname'
-            else:
-                query_params['name'] = a_domain
-                query_params['type'] = 'txt'
-            full_url = base_url + urlencode(query_params)
-            (_, c) = simplehttp.request(full_url, 'GET')
-            result = json.loads(c)
-            status = result['Status']
-            if status == 0 and 'Answer' in result:
-                answers = result['Answer']
-                if verify_data['method'] == 'DNS_CNAME':
-                    answer = answers[0]['data']
-                else:
-                    answer = 'no matching record found'
-                    for possible_answer in answers:
-                        possible_answer['data'] = possible_answer['data'].strip(
-                            '"')
-                        if possible_answer['data'].startswith(
-                                'google-site-verification'):
-                            answer = possible_answer['data']
-                            break
-                        print(
-                            f'Unrelated TXT record: {possible_answer["data"]}')
-                print(f'Found DNS Record: {answer}')
-            elif status == 0:
-                controlflow.system_error_exit(1, 'DNS record not found')
-            else:
-                controlflow.system_error_exit(
-                    status,
-                    DNS_ERROR_CODES_MAP.get(status, f'Unknown error {status}'))
-        return
-    print('SUCCESS!')
-    print(f'Verified:  {verify_result["site"]["identifier"]}')
-    print(f'ID:  {verify_result["id"]}')
-    print(f'Type: {verify_result["site"]["type"]}')
-    print('All Owners:')
-    try:
-        for owner in verify_result['owners']:
-            print(f' {owner}')
-    except KeyError:
-        pass
-    print()
-    print(
-        f'You can now add {a_domain} or it\'s subdomains as secondary or domain aliases of the {GC_Values[GC_DOMAIN]} G Suite Account.'
-    )

src/gam/gapi/storage.py

@@ -1,81 +0,0 @@
-import base64
-import os
-import re
-import sys
-
-import googleapiclient
-
-import gam
-from gam.var import *
-from gam import fileutils
-from gam import gapi
-from gam import utils
-
-
-def build_gapi():
-    return gam.buildGAPIObject('storage')
-
-
-def get_cloud_storage_object(s,
-                             bucket,
-                             object_,
-                             local_file=None,
-                             expectedMd5=None):
-    if not local_file:
-        local_file = object_
-    if os.path.exists(local_file):
-        sys.stdout.write(' File already exists. ')
-        sys.stdout.flush()
-        if expectedMd5:
-            sys.stdout.write(f'Verifying {expectedMd5} hash...')
-            sys.stdout.flush()
-            if utils.md5_matches_file(local_file, expectedMd5, False):
-                print('VERIFIED')
-                return
-            print('not verified. Downloading again and over-writing...')
-        else:
-            return  # nothing to verify, just assume we're good.
-    print(f'saving to {local_file}')
-    request = s.objects().get_media(bucket=bucket, object=object_)
-    file_path = os.path.dirname(local_file)
-    if not os.path.exists(file_path):
-        os.makedirs(file_path)
-    f = fileutils.open_file(local_file, 'wb')
-    downloader = googleapiclient.http.MediaIoBaseDownload(f, request)
-    done = False
-    while not done:
-        status, done = downloader.next_chunk()
-        sys.stdout.write(f' Downloaded: {status.progress():>7.2%}\r')
-        sys.stdout.flush()
-    sys.stdout.write('\n Download complete. Flushing to disk...\n')
-    fileutils.close_file(f, True)
-    if expectedMd5:
-        f = fileutils.open_file(local_file, 'rb')
-        sys.stdout.write(f' Verifying file hash is {expectedMd5}...')
-        sys.stdout.flush()
-        utils.md5_matches_file(local_file, expectedMd5, True)
-        print('VERIFIED')
-        fileutils.close_file(f)
-
-
-def download_bucket():
-    bucket = sys.argv[3]
-    s = build_gapi()
-    page_message = gapi.got_total_items_msg('Files', '...')
-    fields = 'nextPageToken,items(name,id,md5Hash)'
-    objects = gapi.get_all_pages(s.objects(),
-                                 'list',
-                                 'items',
-                                 page_message=page_message,
-                                 bucket=bucket,
-                                 projection='noAcl',
-                                 fields=fields)
-    i = 1
-    for object_ in objects:
-        print(f'{i}/{len(objects)}')
-        expectedMd5 = base64.b64decode(object_['md5Hash']).hex()
-        get_cloud_storage_object(s,
-                                 bucket,
-                                 object_['name'],
-                                 expectedMd5=expectedMd5)
-        i += 1

src/gam/gapi/vault.py

@@ -1,844 +0,0 @@
-import datetime
-import json
-import sys
-
-import googleapiclient.http
-
-import gam
-from gam.var import *
-from gam import controlflow
-from gam import display
-from gam import fileutils
-from gam import gapi
-from gam.gapi import storage as gapi_storage
-from gam import utils
-
-
-def buildGAPIObject():
-    return gam.buildGAPIObject('vault')
-
-
-def validateCollaborators(collaboratorList, cd):
-    collaborators = []
-    for collaborator in collaboratorList.split(','):
-        collaborator_id = gam.convertEmailAddressToUID(collaborator, cd)
-        if not collaborator_id:
-            controlflow.system_error_exit(
-                4, f'failed to get a UID for '
-                f'{collaborator}. Please make '
-                f'sure this is a real user.')
-        collaborators.append({'email': collaborator, 'id': collaborator_id})
-    return collaborators
-
-
-def createMatter():
-    v = buildGAPIObject()
-    matter_time = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
-    body = {'name': f'New Matter - {matter_time}'}
-    collaborators = []
-    cd = None
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'name':
-            body['name'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'description':
-            body['description'] = sys.argv[i + 1]
-            i += 2
-        elif myarg in ['collaborator', 'collaborators']:
-            if not cd:
-                cd = gam.buildGAPIObject('directory')
-            collaborators.extend(validateCollaborators(sys.argv[i + 1], cd))
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam create matter')
-    matterId = gapi.call(v.matters(), 'create', body=body,
-                         fields='matterId')['matterId']
-    print(f'Created matter {matterId}')
-    for collaborator in collaborators:
-        print(f' adding collaborator {collaborator["email"]}')
-        body = {
-            'matterPermission': {
-                'role': 'COLLABORATOR',
-                'accountId': collaborator['id']
-            }
-        }
-        gapi.call(v.matters(), 'addPermissions', matterId=matterId, body=body)
-
-
-VAULT_SEARCH_METHODS_MAP = {
-    'account': 'ACCOUNT',
-    'accounts': 'ACCOUNT',
-    'entireorg': 'ENTIRE_ORG',
-    'everyone': 'ENTIRE_ORG',
-    'orgunit': 'ORG_UNIT',
-    'ou': 'ORG_UNIT',
-    'room': 'ROOM',
-    'rooms': 'ROOM',
-    'shareddrive': 'SHARED_DRIVE',
-    'shareddrives': 'SHARED_DRIVE',
-    'teamdrive': 'SHARED_DRIVE',
-    'teamdrives': 'SHARED_DRIVE',
-}
-VAULT_SEARCH_METHODS_LIST = [
-    'accounts', 'orgunit', 'shareddrives', 'rooms', 'everyone'
-]
-
-
-def createExport():
-    v = buildGAPIObject()
-    allowed_corpuses = gapi.get_enum_values_minus_unspecified(
-        v._rootDesc['schemas']['Query']['properties']['corpus']['enum'])
-    allowed_scopes = gapi.get_enum_values_minus_unspecified(
-        v._rootDesc['schemas']['Query']['properties']['dataScope']['enum'])
-    allowed_formats = gapi.get_enum_values_minus_unspecified(
-        v._rootDesc['schemas']['MailExportOptions']['properties']
-        ['exportFormat']['enum'])
-    export_format = 'MBOX'
-    showConfidentialModeContent = None  # default to not even set
-    matterId = None
-    body = {'query': {'dataScope': 'ALL_DATA'}, 'exportOptions': {}}
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'matter':
-            matterId = getMatterItem(v, sys.argv[i + 1])
-            body['matterId'] = matterId
-            i += 2
-        elif myarg == 'name':
-            body['name'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'corpus':
-            body['query']['corpus'] = sys.argv[i + 1].upper()
-            if body['query']['corpus'] not in allowed_corpuses:
-                controlflow.expected_argument_exit('corpus',
-                                                   ', '.join(allowed_corpuses),
-                                                   sys.argv[i + 1])
-            i += 2
-        elif myarg in VAULT_SEARCH_METHODS_MAP:
-            if body['query'].get('searchMethod'):
-                message = f'Multiple search methods ' \
-                          f'({", ".join(VAULT_SEARCH_METHODS_LIST)})' \
-                          f'specified, only one is allowed'
-                controlflow.system_error_exit(3, message)
-            searchMethod = VAULT_SEARCH_METHODS_MAP[myarg]
-            body['query']['searchMethod'] = searchMethod
-            if searchMethod == 'ACCOUNT':
-                body['query']['accountInfo'] = {
-                    'emails': sys.argv[i + 1].split(',')
-                }
-                i += 2
-            elif searchMethod == 'ORG_UNIT':
-                body['query']['orgUnitInfo'] = {
-                    'orgUnitId': gam.getOrgUnitId(sys.argv[i + 1])[1]
-                }
-                i += 2
-            elif searchMethod == 'SHARED_DRIVE':
-                body['query']['sharedDriveInfo'] = {
-                    'sharedDriveIds': sys.argv[i + 1].split(',')
-                }
-                i += 2
-            elif searchMethod == 'ROOM':
-                body['query']['hangoutsChatInfo'] = {
-                    'roomId': sys.argv[i + 1].split(',')
-                }
-                i += 2
-            else:
-                i += 1
-        elif myarg == 'scope':
-            body['query']['dataScope'] = sys.argv[i + 1].upper()
-            if body['query']['dataScope'] not in allowed_scopes:
-                controlflow.expected_argument_exit('scope',
-                                                   ', '.join(allowed_scopes),
-                                                   sys.argv[i + 1])
-            i += 2
-        elif myarg in ['terms']:
-            body['query']['terms'] = sys.argv[i + 1]
-            i += 2
-        elif myarg in ['start', 'starttime']:
-            body['query']['startTime'] = utils.get_date_zero_time_or_full_time(
-                sys.argv[i + 1])
-            i += 2
-        elif myarg in ['end', 'endtime']:
-            body['query']['endTime'] = utils.get_date_zero_time_or_full_time(
-                sys.argv[i + 1])
-            i += 2
-        elif myarg in ['timezone']:
-            body['query']['timeZone'] = sys.argv[i + 1]
-            i += 2
-        elif myarg in ['excludedrafts']:
-            body['query']['mailOptions'] = {
-                'excludeDrafts': gam.getBoolean(sys.argv[i + 1], myarg)
-            }
-            i += 2
-        elif myarg in ['driveversiondate']:
-            body['query'].setdefault('driveOptions', {})['versionDate'] = \
-                utils.get_date_zero_time_or_full_time(sys.argv[i+1])
-            i += 2
-        elif myarg in ['includeshareddrives', 'includeteamdrives']:
-            body['query'].setdefault(
-                'driveOptions', {})['includeSharedDrives'] = gam.getBoolean(
-                    sys.argv[i + 1], myarg)
-            i += 2
-        elif myarg in ['includerooms']:
-            body['query']['hangoutsChatOptions'] = {
-                'includeRooms': gam.getBoolean(sys.argv[i + 1], myarg)
-            }
-            i += 2
-        elif myarg in ['format']:
-            export_format = sys.argv[i + 1].upper()
-            if export_format not in allowed_formats:
-                controlflow.expected_argument_exit('export format',
-                                                   ', '.join(allowed_formats),
-                                                   export_format)
-            i += 2
-        elif myarg in ['showconfidentialmodecontent']:
-            showConfidentialModeContent = gam.getBoolean(sys.argv[i + 1], myarg)
-            i += 2
-        elif myarg in ['region']:
-            allowed_regions = gapi.get_enum_values_minus_unspecified(
-                v._rootDesc['schemas']['ExportOptions']['properties']['region']
-                ['enum'])
-            body['exportOptions']['region'] = sys.argv[i + 1].upper()
-            if body['exportOptions']['region'] not in allowed_regions:
-                controlflow.expected_argument_exit(
-                    'region', ', '.join(allowed_regions),
-                    body['exportOptions']['region'])
-            i += 2
-        elif myarg in ['includeaccessinfo']:
-            body['exportOptions'].setdefault(
-                'driveOptions', {})['includeAccessInfo'] = gam.getBoolean(
-                    sys.argv[i + 1], myarg)
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam create export')
-    if not matterId:
-        controlflow.system_error_exit(
-            3, 'you must specify a matter for the new export.')
-    if 'corpus' not in body['query']:
-        controlflow.system_error_exit(3, f'you must specify a corpus for the ' \
-          f'new export. Choose one of {", ".join(allowed_corpuses)}')
-    if 'searchMethod' not in body['query']:
-        controlflow.system_error_exit(3, f'you must specify a search method ' \
-          'for the new export. Choose one of ' \
-          f'{", ".join(VAULT_SEARCH_METHODS_LIST)}')
-    if 'name' not in body:
-        corpus_name = body['query']['corpus']
-        corpus_date = datetime.datetime.now()
-        body['name'] = f'GAM {corpus_name} export - {corpus_date}'
-    options_field = None
-    if body['query']['corpus'] == 'MAIL':
-        options_field = 'mailOptions'
-    elif body['query']['corpus'] == 'GROUPS':
-        options_field = 'groupsOptions'
-    elif body['query']['corpus'] == 'HANGOUTS_CHAT':
-        options_field = 'hangoutsChatOptions'
-    if options_field:
-        body['exportOptions'].pop('driveOptions', None)
-        body['exportOptions'][options_field] = {'exportFormat': export_format}
-        if showConfidentialModeContent is not None:
-            body['exportOptions'][options_field][
-                'showConfidentialModeContent'] = showConfidentialModeContent
-    results = gapi.call(v.matters().exports(),
-                        'create',
-                        matterId=matterId,
-                        body=body)
-    print(f'Created export {results["id"]}')
-    display.print_json(results)
-
-
-def deleteExport():
-    v = buildGAPIObject()
-    matterId = getMatterItem(v, sys.argv[3])
-    exportId = convertExportNameToID(v, sys.argv[4], matterId)
-    print(f'Deleting export {sys.argv[4]} / {exportId}')
-    gapi.call(v.matters().exports(),
-              'delete',
-              matterId=matterId,
-              exportId=exportId)
-
-
-def getExportInfo():
-    v = buildGAPIObject()
-    matterId = getMatterItem(v, sys.argv[3])
-    exportId = convertExportNameToID(v, sys.argv[4], matterId)
-    export = gapi.call(v.matters().exports(),
-                       'get',
-                       matterId=matterId,
-                       exportId=exportId)
-    display.print_json(export)
-
-
-def createHold():
-    v = buildGAPIObject()
-    allowed_corpuses = gapi.get_enum_values_minus_unspecified(
-        v._rootDesc['schemas']['Hold']['properties']['corpus']['enum'])
-    body = {'query': {}}
-    i = 3
-    query = None
-    start_time = None
-    end_time = None
-    matterId = None
-    accounts = []
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'name':
-            body['name'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'query':
-            query = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'corpus':
-            body['corpus'] = sys.argv[i + 1].upper()
-            if body['corpus'] not in allowed_corpuses:
-                controlflow.expected_argument_exit('corpus',
-                                                   ', '.join(allowed_corpuses),
-                                                   sys.argv[i + 1])
-            i += 2
-        elif myarg in ['accounts', 'users', 'groups']:
-            accounts = sys.argv[i + 1].split(',')
-            i += 2
-        elif myarg in ['orgunit', 'ou']:
-            body['orgUnit'] = {
-                'orgUnitId': gam.getOrgUnitId(sys.argv[i + 1])[1]
-            }
-            i += 2
-        elif myarg in ['start', 'starttime']:
-            start_time = utils.get_date_zero_time_or_full_time(sys.argv[i + 1])
-            i += 2
-        elif myarg in ['end', 'endtime']:
-            end_time = utils.get_date_zero_time_or_full_time(sys.argv[i + 1])
-            i += 2
-        elif myarg == 'matter':
-            matterId = getMatterItem(v, sys.argv[i + 1])
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam create hold')
-    if not matterId:
-        controlflow.system_error_exit(
-            3, 'you must specify a matter for the new hold.')
-    if not body.get('name'):
-        controlflow.system_error_exit(
-            3, 'you must specify a name for the new hold.')
-    if not body.get('corpus'):
-        controlflow.system_error_exit(3, f'you must specify a corpus for ' \
-          f'the new hold. Choose one of {", ".join(allowed_corpuses)}')
-    if body['corpus'] == 'HANGOUTS_CHAT':
-        query_type = 'hangoutsChatQuery'
-    else:
-        query_type = f'{body["corpus"].lower()}Query'
-    body['query'][query_type] = {}
-    if body['corpus'] == 'DRIVE':
-        if query:
-            try:
-                body['query'][query_type] = json.loads(query)
-            except ValueError as e:
-                controlflow.system_error_exit(3, f'{str(e)}, query: {query}')
-    elif body['corpus'] in ['GROUPS', 'MAIL']:
-        if query:
-            body['query'][query_type] = {'terms': query}
-        if start_time:
-            body['query'][query_type]['startTime'] = start_time
-        if end_time:
-            body['query'][query_type]['endTime'] = end_time
-    if accounts:
-        body['accounts'] = []
-        cd = gam.buildGAPIObject('directory')
-        account_type = 'group' if body['corpus'] == 'GROUPS' else 'user'
-        for account in accounts:
-            body['accounts'].append({
-                'accountId':
-                    gam.convertEmailAddressToUID(account, cd, account_type)
-            })
-    holdId = gapi.call(v.matters().holds(),
-                       'create',
-                       matterId=matterId,
-                       body=body,
-                       fields='holdId')
-    print(f'Created hold {holdId["holdId"]}')
-
-
-def deleteHold():
-    v = buildGAPIObject()
-    hold = sys.argv[3]
-    matterId = None
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'matter':
-            matterId = getMatterItem(v, sys.argv[i + 1])
-            holdId = convertHoldNameToID(v, hold, matterId)
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(myarg, 'gam delete hold')
-    if not matterId:
-        controlflow.system_error_exit(
-            3, 'you must specify a matter for the hold.')
-    print(f'Deleting hold {hold} / {holdId}')
-    gapi.call(v.matters().holds(), 'delete', matterId=matterId, holdId=holdId)
-
-
-def getHoldInfo():
-    v = buildGAPIObject()
-    hold = sys.argv[3]
-    matterId = None
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'matter':
-            matterId = getMatterItem(v, sys.argv[i + 1])
-            holdId = convertHoldNameToID(v, hold, matterId)
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(myarg, 'gam info hold')
-    if not matterId:
-        controlflow.system_error_exit(
-            3, 'you must specify a matter for the hold.')
-    results = gapi.call(v.matters().holds(),
-                        'get',
-                        matterId=matterId,
-                        holdId=holdId)
-    cd = gam.buildGAPIObject('directory')
-    if 'accounts' in results:
-        account_type = 'group' if results['corpus'] == 'GROUPS' else 'user'
-        for i in range(0, len(results['accounts'])):
-            uid = f'uid:{results["accounts"][i]["accountId"]}'
-            acct_email = gam.convertUIDtoEmailAddress(uid, cd, [account_type])
-            results['accounts'][i]['email'] = acct_email
-    if 'orgUnit' in results:
-        results['orgUnit']['orgUnitPath'] = gam.doGetOrgInfo(
-            results['orgUnit']['orgUnitId'], return_attrib='orgUnitPath')
-    display.print_json(results)
-
-
-def convertExportNameToID(v, nameOrID, matterId):
-    nameOrID = nameOrID.lower()
-    cg = UID_PATTERN.match(nameOrID)
-    if cg:
-        return cg.group(1)
-    fields = 'exports(id,name),nextPageToken'
-    exports = gapi.get_all_pages(v.matters().exports(),
-                                 'list',
-                                 'exports',
-                                 matterId=matterId,
-                                 fields=fields)
-    for export in exports:
-        if export['name'].lower() == nameOrID:
-            return export['id']
-    controlflow.system_error_exit(
-        4, f'could not find export name {nameOrID} '
-        f'in matter {matterId}')
-
-
-def convertHoldNameToID(v, nameOrID, matterId):
-    nameOrID = nameOrID.lower()
-    cg = UID_PATTERN.match(nameOrID)
-    if cg:
-        return cg.group(1)
-    fields = 'holds(holdId,name),nextPageToken'
-    holds = gapi.get_all_pages(v.matters().holds(),
-                               'list',
-                               'holds',
-                               matterId=matterId,
-                               fields=fields)
-    for hold in holds:
-        if hold['name'].lower() == nameOrID:
-            return hold['holdId']
-    controlflow.system_error_exit(
-        4, f'could not find hold name {nameOrID} '
-        f'in matter {matterId}')
-
-
-def convertMatterNameToID(v, nameOrID):
-    nameOrID = nameOrID.lower()
-    cg = UID_PATTERN.match(nameOrID)
-    if cg:
-        return cg.group(1)
-    fields = 'matters(matterId,name),nextPageToken'
-    matters = gapi.get_all_pages(v.matters(),
-                                 'list',
-                                 'matters',
-                                 view='BASIC',
-                                 fields=fields)
-    for matter in matters:
-        if matter['name'].lower() == nameOrID:
-            return matter['matterId']
-    return None
-
-
-def getMatterItem(v, nameOrID):
-    matterId = convertMatterNameToID(v, nameOrID)
-    if not matterId:
-        controlflow.system_error_exit(4, f'could not find matter {nameOrID}')
-    return matterId
-
-
-def updateHold():
-    v = buildGAPIObject()
-    hold = sys.argv[3]
-    matterId = None
-    body = {}
-    query = None
-    add_accounts = []
-    del_accounts = []
-    start_time = None
-    end_time = None
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'matter':
-            matterId = getMatterItem(v, sys.argv[i + 1])
-            holdId = convertHoldNameToID(v, hold, matterId)
-            i += 2
-        elif myarg == 'query':
-            query = sys.argv[i + 1]
-            i += 2
-        elif myarg in ['orgunit', 'ou']:
-            body['orgUnit'] = {
-                'orgUnitId': gam.getOrgUnitId(sys.argv[i + 1])[1]
-            }
-            i += 2
-        elif myarg in ['start', 'starttime']:
-            start_time = utils.get_date_zero_time_or_full_time(sys.argv[i + 1])
-            i += 2
-        elif myarg in ['end', 'endtime']:
-            end_time = utils.get_date_zero_time_or_full_time(sys.argv[i + 1])
-            i += 2
-        elif myarg in ['addusers', 'addaccounts', 'addgroups']:
-            add_accounts = sys.argv[i + 1].split(',')
-            i += 2
-        elif myarg in ['removeusers', 'removeaccounts', 'removegroups']:
-            del_accounts = sys.argv[i + 1].split(',')
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(myarg, 'gam update hold')
-    if not matterId:
-        controlflow.system_error_exit(
-            3, 'you must specify a matter for the hold.')
-    if query or start_time or end_time or body.get('orgUnit'):
-        fields = 'corpus,query,orgUnit'
-        old_body = gapi.call(v.matters().holds(),
-                             'get',
-                             matterId=matterId,
-                             holdId=holdId,
-                             fields=fields)
-        body['query'] = old_body['query']
-        body['corpus'] = old_body['corpus']
-        if 'orgUnit' in old_body and 'orgUnit' not in body:
-            # bah, API requires this to be sent
-            # on update even when it's not changing
-            body['orgUnit'] = old_body['orgUnit']
-        query_type = f'{body["corpus"].lower()}Query'
-        if body['corpus'] == 'DRIVE':
-            if query:
-                try:
-                    body['query'][query_type] = json.loads(query)
-                except ValueError as e:
-                    message = f'{str(e)}, query: {query}'
-                    controlflow.system_error_exit(3, message)
-        elif body['corpus'] in ['GROUPS', 'MAIL']:
-            if query:
-                body['query'][query_type]['terms'] = query
-            if start_time:
-                body['query'][query_type]['startTime'] = start_time
-            if end_time:
-                body['query'][query_type]['endTime'] = end_time
-    if body:
-        print(f'Updating hold {hold} / {holdId}')
-        gapi.call(v.matters().holds(),
-                  'update',
-                  matterId=matterId,
-                  holdId=holdId,
-                  body=body)
-    if add_accounts or del_accounts:
-        cd = gam.buildGAPIObject('directory')
-        for account in add_accounts:
-            print(f'adding {account} to hold.')
-            add_body = {'accountId': gam.convertEmailAddressToUID(account, cd)}
-            gapi.call(v.matters().holds().accounts(),
-                      'create',
-                      matterId=matterId,
-                      holdId=holdId,
-                      body=add_body)
-        for account in del_accounts:
-            print(f'removing {account} from hold.')
-            accountId = gam.convertEmailAddressToUID(account, cd)
-            gapi.call(v.matters().holds().accounts(),
-                      'delete',
-                      matterId=matterId,
-                      holdId=holdId,
-                      accountId=accountId)
-
-
-def updateMatter(action=None):
-    v = buildGAPIObject()
-    matterId = getMatterItem(v, sys.argv[3])
-    body = {}
-    action_kwargs = {'body': {}}
-    add_collaborators = []
-    remove_collaborators = []
-    cd = None
-    i = 4
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'action':
-            action = sys.argv[i + 1].lower()
-            if action not in VAULT_MATTER_ACTIONS:
-                controlflow.system_error_exit(3, f'allowed actions are ' \
-                    f'{", ".join(VAULT_MATTER_ACTIONS)}, got {action}')
-            i += 2
-        elif myarg == 'name':
-            body['name'] = sys.argv[i + 1]
-            i += 2
-        elif myarg == 'description':
-            body['description'] = sys.argv[i + 1]
-            i += 2
-        elif myarg in ['addcollaborator', 'addcollaborators']:
-            if not cd:
-                cd = gam.buildGAPIObject('directory')
-            add_collaborators.extend(validateCollaborators(sys.argv[i + 1], cd))
-            i += 2
-        elif myarg in ['removecollaborator', 'removecollaborators']:
-            if not cd:
-                cd = gam.buildGAPIObject('directory')
-            remove_collaborators.extend(
-                validateCollaborators(sys.argv[i + 1], cd))
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i], 'gam update matter')
-    if action == 'delete':
-        action_kwargs = {}
-    if body:
-        print(f'Updating matter {sys.argv[3]}...')
-        if 'name' not in body or 'description' not in body:
-            # bah, API requires name/description to be sent
-            # on update even when it's not changing
-            result = gapi.call(v.matters(),
-                               'get',
-                               matterId=matterId,
-                               view='BASIC')
-            body.setdefault('name', result['name'])
-            body.setdefault('description', result.get('description'))
-        gapi.call(v.matters(), 'update', body=body, matterId=matterId)
-    if action:
-        print(f'Performing {action} on matter {sys.argv[3]}')
-        gapi.call(v.matters(), action, matterId=matterId, **action_kwargs)
-    for collaborator in add_collaborators:
-        print(f' adding collaborator {collaborator["email"]}')
-        body = {
-            'matterPermission': {
-                'role': 'COLLABORATOR',
-                'accountId': collaborator['id']
-            }
-        }
-        gapi.call(v.matters(), 'addPermissions', matterId=matterId, body=body)
-    for collaborator in remove_collaborators:
-        print(f' removing collaborator {collaborator["email"]}')
-        gapi.call(v.matters(),
-                  'removePermissions',
-                  matterId=matterId,
-                  body={'accountId': collaborator['id']})
-
-
-def getMatterInfo():
-    v = buildGAPIObject()
-    matterId = getMatterItem(v, sys.argv[3])
-    result = gapi.call(v.matters(), 'get', matterId=matterId, view='FULL')
-    if 'matterPermissions' in result:
-        cd = gam.buildGAPIObject('directory')
-        for i in range(0, len(result['matterPermissions'])):
-            uid = f'uid:{result["matterPermissions"][i]["accountId"]}'
-            user_email = gam.convertUIDtoEmailAddress(uid, cd)
-            result['matterPermissions'][i]['email'] = user_email
-    display.print_json(result)
-
-
-def downloadExport():
-    verifyFiles = True
-    extractFiles = True
-    v = buildGAPIObject()
-    s = gapi_storage.build_gapi()
-    matterId = getMatterItem(v, sys.argv[3])
-    exportId = convertExportNameToID(v, sys.argv[4], matterId)
-    targetFolder = GC_Values[GC_DRIVE_DIR]
-    i = 5
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'targetfolder':
-            targetFolder = os.path.expanduser(sys.argv[i + 1])
-            if not os.path.isdir(targetFolder):
-                os.makedirs(targetFolder)
-            i += 2
-        elif myarg == 'noverify':
-            verifyFiles = False
-            i += 1
-        elif myarg == 'noextract':
-            extractFiles = False
-            i += 1
-        else:
-            controlflow.invalid_argument_exit(sys.argv[i],
-                                              'gam download export')
-    export = gapi.call(v.matters().exports(),
-                       'get',
-                       matterId=matterId,
-                       exportId=exportId)
-    for s_file in export['cloudStorageSink']['files']:
-        bucket = s_file['bucketName']
-        s_object = s_file['objectName']
-        filename = os.path.join(targetFolder, s_object.replace('/', '-'))
-        print(f'saving to {filename}')
-        request = s.objects().get_media(bucket=bucket, object=s_object)
-        f = fileutils.open_file(filename, 'wb')
-        downloader = googleapiclient.http.MediaIoBaseDownload(f, request)
-        done = False
-        while not done:
-            status, done = downloader.next_chunk()
-            sys.stdout.write(' Downloaded: {0:>7.2%}\r'.format(
-                status.progress()))
-            sys.stdout.flush()
-        sys.stdout.write('\n Download complete. Flushing to disk...\n')
-        fileutils.close_file(f, True)
-        if verifyFiles:
-            expected_hash = s_file['md5Hash']
-            sys.stdout.write(f' Verifying file hash is {expected_hash}...')
-            sys.stdout.flush()
-            utils.md5_matches_file(filename, expected_hash, True)
-            print('VERIFIED')
-        if extractFiles and re.search(r'\.zip$', filename):
-            gam.extract_nested_zip(filename, targetFolder)
-
-
-def printMatters():
-    v = buildGAPIObject()
-    todrive = False
-    csvRows = []
-    initialTitles = ['matterId', 'name', 'description', 'state']
-    titles = initialTitles[:]
-    view = 'FULL'
-    state = None
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg in PROJECTION_CHOICES_MAP:
-            view = PROJECTION_CHOICES_MAP[myarg]
-            i += 1
-        elif myarg == 'matterstate':
-            valid_states = gapi.get_enum_values_minus_unspecified(
-                v._rootDesc['schemas']['Matter']['properties']['state']['enum'])
-            state = sys.argv[i + 1].upper()
-            if state not in valid_states:
-                controlflow.expected_argument_exit('state',
-                                                   ', '.join(valid_states),
-                                                   state)
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(myarg, 'gam print matters')
-    gam.printGettingAllItems('Vault Matters', None)
-    page_message = gapi.got_total_items_msg('Vault Matters', '...\n')
-    matters = gapi.get_all_pages(v.matters(),
-                                 'list',
-                                 'matters',
-                                 page_message=page_message,
-                                 view=view,
-                                 state=state)
-    for matter in matters:
-        display.add_row_titles_to_csv_file(utils.flatten_json(matter), csvRows,
-                                           titles)
-    display.sort_csv_titles(initialTitles, titles)
-    display.write_csv_file(csvRows, titles, 'Vault Matters', todrive)
-
-
-def printExports():
-    v = buildGAPIObject()
-    todrive = False
-    csvRows = []
-    initialTitles = ['matterId', 'id', 'name', 'createTime', 'status']
-    titles = initialTitles[:]
-    matters = []
-    matterIds = []
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg in ['matter', 'matters']:
-            matters = sys.argv[i + 1].split(',')
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(myarg, 'gam print exports')
-    if not matters:
-        fields = 'matters(matterId),nextPageToken'
-        matters_results = gapi.get_all_pages(v.matters(),
-                                             'list',
-                                             'matters',
-                                             view='BASIC',
-                                             state='OPEN',
-                                             fields=fields)
-        for matter in matters_results:
-            matterIds.append(matter['matterId'])
-    else:
-        for matter in matters:
-            matterIds.append(getMatterItem(v, matter))
-    for matterId in matterIds:
-        sys.stderr.write(f'Retrieving exports for matter {matterId}\n')
-        exports = gapi.get_all_pages(v.matters().exports(),
-                                     'list',
-                                     'exports',
-                                     matterId=matterId)
-        for export in exports:
-            display.add_row_titles_to_csv_file(
-                utils.flatten_json(export, flattened={'matterId': matterId}),
-                csvRows, titles)
-    display.sort_csv_titles(initialTitles, titles)
-    display.write_csv_file(csvRows, titles, 'Vault Exports', todrive)
-
-
-def printHolds():
-    v = buildGAPIObject()
-    todrive = False
-    csvRows = []
-    initialTitles = ['matterId', 'holdId', 'name', 'corpus', 'updateTime']
-    titles = initialTitles[:]
-    matters = []
-    matterIds = []
-    i = 3
-    while i < len(sys.argv):
-        myarg = sys.argv[i].lower().replace('_', '')
-        if myarg == 'todrive':
-            todrive = True
-            i += 1
-        elif myarg in ['matter', 'matters']:
-            matters = sys.argv[i + 1].split(',')
-            i += 2
-        else:
-            controlflow.invalid_argument_exit(myarg, 'gam print holds')
-    if not matters:
-        fields = 'matters(matterId),nextPageToken'
-        matters_results = gapi.get_all_pages(v.matters(),
-                                             'list',
-                                             'matters',
-                                             view='BASIC',
-                                             state='OPEN',
-                                             fields=fields)
-        for matter in matters_results:
-            matterIds.append(matter['matterId'])
-    else:
-        for matter in matters:
-            matterIds.append(getMatterItem(v, matter))
-    for matterId in matterIds:
-        sys.stderr.write(f'Retrieving holds for matter {matterId}\n')
-        holds = gapi.get_all_pages(v.matters().holds(),
-                                   'list',
-                                   'holds',
-                                   matterId=matterId)
-        for hold in holds:
-            display.add_row_titles_to_csv_file(
-                utils.flatten_json(hold, flattened={'matterId': matterId}),
-                csvRows, titles)
-    display.sort_csv_titles(initialTitles, titles)
-    display.write_csv_file(csvRows, titles, 'Vault Holds', todrive)

src/gam/gdata/__init__.py

@@ -0,0 +1,825 @@
+#!/usr/bin/python
+#
+# Copyright (C) 2006 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+"""Contains classes representing Google Data elements.
+
+  Extends Atom classes to add Google Data specific elements.
+"""
+
+
+__author__ = 'j.s@google.com (Jeffrey Scudder)'
+
+import os
+import atom
+import lxml.etree as ElementTree
+
+# XML namespaces which are often used in GData entities.
+GDATA_NAMESPACE = 'http://schemas.google.com/g/2005'
+GDATA_TEMPLATE = '{http://schemas.google.com/g/2005}%s'
+OPENSEARCH_NAMESPACE = 'http://a9.com/-/spec/opensearchrss/1.0/'
+OPENSEARCH_TEMPLATE = '{http://a9.com/-/spec/opensearchrss/1.0/}%s'
+BATCH_NAMESPACE = 'http://schemas.google.com/gdata/batch'
+GACL_NAMESPACE = 'http://schemas.google.com/acl/2007'
+GACL_TEMPLATE = '{http://schemas.google.com/acl/2007}%s'
+
+
+# Labels used in batch request entries to specify the desired CRUD operation.
+BATCH_INSERT = 'insert'
+BATCH_UPDATE = 'update'
+BATCH_DELETE = 'delete'
+BATCH_QUERY = 'query'
+
+class Error(Exception):
+  pass
+
+
+class MissingRequiredParameters(Error):
+  pass
+
+
+class MediaSource(object):
+  """GData Entries can refer to media sources, so this class provides a
+  place to store references to these objects along with some metadata.
+  """
+
+  def __init__(self, file_handle=None, content_type=None, content_length=None,
+      file_path=None, file_name=None):
+    """Creates an object of type MediaSource.
+
+    Args:
+      file_handle: A file handle pointing to the file to be encapsulated in the
+                   MediaSource
+      content_type: string The MIME type of the file. Required if a file_handle
+                    is given.
+      content_length: int The size of the file. Required if a file_handle is
+                      given.
+      file_path: string (optional) A full path name to the file. Used in
+                    place of a file_handle.
+      file_name: string The name of the file without any path information.
+                 Required if a file_handle is given.
+    """
+    self.file_handle = file_handle
+    self.content_type = content_type
+    self.content_length = content_length
+    self.file_name = file_name
+
+    if (file_handle is None and content_type is not None and
+        file_path is not None):
+      self.setFile(file_path, content_type)
+
+  def setFile(self, file_name, content_type):
+    """A helper function which can create a file handle from a given filename
+    and set the content type and length all at once.
+
+    Args:
+      file_name: string The path and file name to the file containing the media
+      content_type: string A MIME type representing the type of the media
+    """
+
+    self.file_handle = open(file_name, 'rb')
+    self.content_type = content_type
+    self.content_length = os.path.getsize(file_name)
+    self.file_name = os.path.basename(file_name)
+
+
+class LinkFinder(atom.LinkFinder):
+  """An "interface" providing methods to find link elements
+
+  GData Entry elements often contain multiple links which differ in the rel
+  attribute or content type. Often, developers are interested in a specific
+  type of link so this class provides methods to find specific classes of
+  links.
+
+  This class is used as a mixin in GData entries.
+  """
+
+  def GetSelfLink(self):
+    """Find the first link with rel set to 'self'
+
+    Returns:
+      An atom.Link or none if none of the links had rel equal to 'self'
+    """
+
+    for a_link in self.link:
+      if a_link.rel == 'self':
+        return a_link
+    return None
+
+  def GetEditLink(self):
+    for a_link in self.link:
+      if a_link.rel == 'edit':
+        return a_link
+    return None
+
+  def GetEditMediaLink(self):
+    """The Picasa API mistakenly returns media-edit rather than edit-media, but
+    this may change soon.
+    """
+    for a_link in self.link:
+      if a_link.rel == 'edit-media':
+        return a_link
+      if a_link.rel == 'media-edit':
+        return a_link
+    return None
+
+  def GetHtmlLink(self):
+    """Find the first link with rel of alternate and type of text/html
+
+    Returns:
+      An atom.Link or None if no links matched
+    """
+    for a_link in self.link:
+      if a_link.rel == 'alternate' and a_link.type == 'text/html':
+        return a_link
+    return None
+
+  def GetPostLink(self):
+    """Get a link containing the POST target URL.
+
+    The POST target URL is used to insert new entries.
+
+    Returns:
+      A link object with a rel matching the POST type.
+    """
+    for a_link in self.link:
+      if a_link.rel == 'http://schemas.google.com/g/2005#post':
+        return a_link
+    return None
+
+  def GetAclLink(self):
+    for a_link in self.link:
+      if a_link.rel == 'http://schemas.google.com/acl/2007#accessControlList':
+        return a_link
+    return None
+
+  def GetFeedLink(self):
+    for a_link in self.link:
+      if a_link.rel == 'http://schemas.google.com/g/2005#feed':
+        return a_link
+    return None
+
+  def GetNextLink(self):
+    for a_link in self.link:
+      if a_link.rel == 'next':
+        return a_link
+    return None
+
+  def GetPrevLink(self):
+    for a_link in self.link:
+      if a_link.rel == 'previous':
+        return a_link
+    return None
+
+
+class TotalResults(atom.AtomBase):
+  """opensearch:TotalResults for a GData feed"""
+
+  _tag = 'totalResults'
+  _namespace = OPENSEARCH_NAMESPACE
+  _children = atom.AtomBase._children.copy()
+  _attributes = atom.AtomBase._attributes.copy()
+
+  def __init__(self, extension_elements=None,
+     extension_attributes=None, text=None):
+    self.text = text
+    self.extension_elements = extension_elements or []
+    self.extension_attributes = extension_attributes or {}
+
+
+def TotalResultsFromString(xml_string):
+  return atom.CreateClassFromXMLString(TotalResults, xml_string)
+
+
+class StartIndex(atom.AtomBase):
+  """The opensearch:startIndex element in GData feed"""
+
+  _tag = 'startIndex'
+  _namespace = OPENSEARCH_NAMESPACE
+  _children = atom.AtomBase._children.copy()
+  _attributes = atom.AtomBase._attributes.copy()
+
+  def __init__(self, extension_elements=None,
+      extension_attributes=None, text=None):
+    self.text = text
+    self.extension_elements = extension_elements or []
+    self.extension_attributes = extension_attributes or {}
+
+
+def StartIndexFromString(xml_string):
+  return atom.CreateClassFromXMLString(StartIndex, xml_string)
+
+
+class ItemsPerPage(atom.AtomBase):
+  """The opensearch:itemsPerPage element in GData feed"""
+
+  _tag = 'itemsPerPage'
+  _namespace = OPENSEARCH_NAMESPACE
+  _children = atom.AtomBase._children.copy()
+  _attributes = atom.AtomBase._attributes.copy()
+
+  def __init__(self, extension_elements=None,
+      extension_attributes=None, text=None):
+    self.text = text
+    self.extension_elements = extension_elements or []
+    self.extension_attributes = extension_attributes or {}
+
+
+def ItemsPerPageFromString(xml_string):
+  return atom.CreateClassFromXMLString(ItemsPerPage, xml_string)
+
+
+class ExtendedProperty(atom.AtomBase):
+  """The Google Data extendedProperty element.
+
+  Used to store arbitrary key-value information specific to your
+  application. The value can either be a text string stored as an XML
+  attribute (.value), or an XML node (XmlBlob) as a child element.
+
+  This element is used in the Google Calendar data API and the Google
+  Contacts data API.
+  """
+
+  _tag = 'extendedProperty'
+  _namespace = GDATA_NAMESPACE
+  _children = atom.AtomBase._children.copy()
+  _attributes = atom.AtomBase._attributes.copy()
+  _attributes['name'] = 'name'
+  _attributes['value'] = 'value'
+
+  def __init__(self, name=None, value=None, extension_elements=None,
+      extension_attributes=None, text=None):
+    self.name = name
+    self.value = value
+    self.text = text
+    self.extension_elements = extension_elements or []
+    self.extension_attributes = extension_attributes or {}
+
+  def GetXmlBlobExtensionElement(self):
+    """Returns the XML blob as an atom.ExtensionElement.
+
+    Returns:
+      An atom.ExtensionElement representing the blob's XML, or None if no
+      blob was set.
+    """
+    if len(self.extension_elements) < 1:
+      return None
+    else:
+      return self.extension_elements[0]
+
+  def GetXmlBlobString(self):
+    """Returns the XML blob as a string.
+
+    Returns:
+      A string containing the blob's XML, or None if no blob was set.
+    """
+    blob = self.GetXmlBlobExtensionElement()
+    if blob:
+      return blob.ToString()
+    return None
+
+  def SetXmlBlob(self, blob):
+    """Sets the contents of the extendedProperty to XML as a child node.
+
+    Since the extendedProperty is only allowed one child element as an XML
+    blob, setting the XML blob will erase any preexisting extension elements
+    in this object.
+
+    Args:
+      blob: str, ElementTree Element or atom.ExtensionElement representing
+            the XML blob stored in the extendedProperty.
+    """
+    # Erase any existing extension_elements, clears the child nodes from the
+    # extendedProperty.
+    self.extension_elements = []
+    if isinstance(blob, atom.ExtensionElement):
+      self.extension_elements.append(blob)
+    elif ElementTree.iselement(blob):
+      self.extension_elements.append(atom._ExtensionElementFromElementTree(
+          blob))
+    else:
+      self.extension_elements.append(atom.ExtensionElementFromString(blob))
+
+
+def ExtendedPropertyFromString(xml_string):
+  return atom.CreateClassFromXMLString(ExtendedProperty, xml_string)
+
+
+class GDataEntry(atom.Entry, LinkFinder):
+  """Extends Atom Entry to provide data processing"""
+
+  _tag = atom.Entry._tag
+  _namespace = atom.Entry._namespace
+  _children = atom.Entry._children.copy()
+  _attributes = atom.Entry._attributes.copy()
+
+  def __GetId(self):
+    return self.__id
+
+  # This method was created to strip the unwanted whitespace from the id's
+  # text node.
+  def __SetId(self, id):
+    self.__id = id
+    if id is not None and id.text is not None:
+      self.__id.text = id.text.strip()
+
+  id = property(__GetId, __SetId)
+
+  def IsMedia(self):
+    """Determines whether or not an entry is a GData Media entry.
+    """
+    if (self.GetEditMediaLink()):
+      return True
+    else:
+      return False
+
+  def GetMediaURL(self):
+    """Returns the URL to the media content, if the entry is a media entry.
+    Otherwise returns None.
+    """
+    if not self.IsMedia():
+      return None
+    else:
+      return self.content.src
+
+
+def GDataEntryFromString(xml_string):
+  """Creates a new GDataEntry instance given a string of XML."""
+  return atom.CreateClassFromXMLString(GDataEntry, xml_string)
+
+
+class GDataFeed(atom.Feed, LinkFinder):
+  """A Feed from a GData service"""
+
+  _tag = 'feed'
+  _namespace = atom.ATOM_NAMESPACE
+  _children = atom.Feed._children.copy()
+  _attributes = atom.Feed._attributes.copy()
+  _children['{%s}totalResults' % OPENSEARCH_NAMESPACE] = ('total_results',
+                                                          TotalResults)
+  _children['{%s}startIndex' % OPENSEARCH_NAMESPACE] = ('start_index',
+                                                        StartIndex)
+  _children['{%s}itemsPerPage' % OPENSEARCH_NAMESPACE] = ('items_per_page',
+                                                          ItemsPerPage)
+               # Add a conversion rule for atom:entry to make it into a GData
+               # Entry.
+  _children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', [GDataEntry])
+
+  def __GetId(self):
+    return self.__id
+
+  def __SetId(self, id):
+    self.__id = id
+    if id is not None and id.text is not None:
+      self.__id.text = id.text.strip()
+
+  id = property(__GetId, __SetId)
+
+  def __GetGenerator(self):
+    return self.__generator
+
+  def __SetGenerator(self, generator):
+    self.__generator = generator
+    if generator is not None:
+      self.__generator.text = generator.text.strip()
+
+  generator = property(__GetGenerator, __SetGenerator)
+
+  def __init__(self, author=None, category=None, contributor=None,
+      generator=None, icon=None, atom_id=None, link=None, logo=None,
+      rights=None, subtitle=None, title=None, updated=None, entry=None,
+      total_results=None, start_index=None, items_per_page=None,
+      extension_elements=None, extension_attributes=None, text=None):
+    """Constructor for Source
+
+    Args:
+      author: list (optional) A list of Author instances which belong to this
+          class.
+      category: list (optional) A list of Category instances
+      contributor: list (optional) A list on Contributor instances
+      generator: Generator (optional)
+      icon: Icon (optional)
+      id: Id (optional) The entry's Id element
+      link: list (optional) A list of Link instances
+      logo: Logo (optional)
+      rights: Rights (optional) The entry's Rights element
+      subtitle: Subtitle (optional) The entry's subtitle element
+      title: Title (optional) the entry's title element
+      updated: Updated (optional) the entry's updated element
+      entry: list (optional) A list of the Entry instances contained in the
+          feed.
+      text: String (optional) The text contents of the element. This is the
+          contents of the Entry's XML text node.
+          (Example: <foo>This is the text</foo>)
+      extension_elements: list (optional) A list of ExtensionElement instances
+          which are children of this element.
+      extension_attributes: dict (optional) A dictionary of strings which are
+          the values for additional XML attributes of this element.
+    """
+
+    self.author = author or []
+    self.category = category or []
+    self.contributor = contributor or []
+    self.generator = generator
+    self.icon = icon
+    self.id = atom_id
+    self.link = link or []
+    self.logo = logo
+    self.rights = rights
+    self.subtitle = subtitle
+    self.title = title
+    self.updated = updated
+    self.entry = entry or []
+    self.total_results = total_results
+    self.start_index = start_index
+    self.items_per_page = items_per_page
+    self.text = text
+    self.extension_elements = extension_elements or []
+    self.extension_attributes = extension_attributes or {}
+
+
+def GDataFeedFromString(xml_string):
+  return atom.CreateClassFromXMLString(GDataFeed, xml_string)
+
+
+class BatchId(atom.AtomBase):
+  _tag = 'id'
+  _namespace = BATCH_NAMESPACE
+  _children = atom.AtomBase._children.copy()
+  _attributes = atom.AtomBase._attributes.copy()
+
+
+def BatchIdFromString(xml_string):
+  return atom.CreateClassFromXMLString(BatchId, xml_string)
+
+
+class BatchOperation(atom.AtomBase):
+  _tag = 'operation'
+  _namespace = BATCH_NAMESPACE
+  _children = atom.AtomBase._children.copy()
+  _attributes = atom.AtomBase._attributes.copy()
+  _attributes['type'] = 'type'
+
+  def __init__(self, op_type=None, extension_elements=None,
+               extension_attributes=None,
+               text=None):
+    self.type = op_type
+    atom.AtomBase.__init__(self,
+                           extension_elements=extension_elements,
+                           extension_attributes=extension_attributes,
+                           text=text)
+
+
+def BatchOperationFromString(xml_string):
+  return atom.CreateClassFromXMLString(BatchOperation, xml_string)
+
+
+class BatchStatus(atom.AtomBase):
+  """The batch:status element present in a batch response entry.
+
+  A status element contains the code (HTTP response code) and
+  reason as elements. In a single request these fields would
+  be part of the HTTP response, but in a batch request each
+  Entry operation has a corresponding Entry in the response
+  feed which includes status information.
+
+  See http://code.google.com/apis/gdata/batch.html#Handling_Errors
+  """
+
+  _tag = 'status'
+  _namespace = BATCH_NAMESPACE
+  _children = atom.AtomBase._children.copy()
+  _attributes = atom.AtomBase._attributes.copy()
+  _attributes['code'] = 'code'
+  _attributes['reason'] = 'reason'
+  _attributes['content-type'] = 'content_type'
+
+  def __init__(self, code=None, reason=None, content_type=None,
+               extension_elements=None, extension_attributes=None, text=None):
+    self.code = code
+    self.reason = reason
+    self.content_type = content_type
+    atom.AtomBase.__init__(self, extension_elements=extension_elements,
+                           extension_attributes=extension_attributes,
+                           text=text)
+
+
+def BatchStatusFromString(xml_string):
+  return atom.CreateClassFromXMLString(BatchStatus, xml_string)
+
+
+class BatchEntry(GDataEntry):
+  """An atom:entry for use in batch requests.
+
+  The BatchEntry contains additional members to specify the operation to be
+  performed on this entry and a batch ID so that the server can reference
+  individual operations in the response feed. For more information, see:
+  http://code.google.com/apis/gdata/batch.html
+  """
+
+  _tag = GDataEntry._tag
+  _namespace = GDataEntry._namespace
+  _children = GDataEntry._children.copy()
+  _children['{%s}operation' % BATCH_NAMESPACE] = ('batch_operation', BatchOperation)
+  _children['{%s}id' % BATCH_NAMESPACE] = ('batch_id', BatchId)
+  _children['{%s}status' % BATCH_NAMESPACE] = ('batch_status', BatchStatus)
+  _attributes = GDataEntry._attributes.copy()
+
+  def __init__(self, author=None, category=None, content=None,
+      contributor=None, atom_id=None, link=None, published=None, rights=None,
+      source=None, summary=None, control=None, title=None, updated=None,
+      batch_operation=None, batch_id=None, batch_status=None,
+      extension_elements=None, extension_attributes=None, text=None):
+    self.batch_operation = batch_operation
+    self.batch_id = batch_id
+    self.batch_status = batch_status
+    GDataEntry.__init__(self, author=author, category=category,
+        content=content, contributor=contributor, atom_id=atom_id, link=link,
+        published=published, rights=rights, source=source, summary=summary,
+        control=control, title=title, updated=updated,
+        extension_elements=extension_elements,
+        extension_attributes=extension_attributes, text=text)
+
+
+def BatchEntryFromString(xml_string):
+  return atom.CreateClassFromXMLString(BatchEntry, xml_string)
+
+
+class BatchInterrupted(atom.AtomBase):
+  """The batch:interrupted element sent if batch request was interrupted.
+
+  Only appears in a feed if some of the batch entries could not be processed.
+  See: http://code.google.com/apis/gdata/batch.html#Handling_Errors
+  """
+
+  _tag = 'interrupted'
+  _namespace = BATCH_NAMESPACE
+  _children = atom.AtomBase._children.copy()
+  _attributes = atom.AtomBase._attributes.copy()
+  _attributes['reason'] = 'reason'
+  _attributes['success'] = 'success'
+  _attributes['failures'] = 'failures'
+  _attributes['parsed'] = 'parsed'
+
+  def __init__(self, reason=None, success=None, failures=None, parsed=None,
+               extension_elements=None, extension_attributes=None, text=None):
+    self.reason = reason
+    self.success = success
+    self.failures = failures
+    self.parsed = parsed
+    atom.AtomBase.__init__(self, extension_elements=extension_elements,
+                           extension_attributes=extension_attributes,
+                           text=text)
+
+
+def BatchInterruptedFromString(xml_string):
+  return atom.CreateClassFromXMLString(BatchInterrupted, xml_string)
+
+
+class BatchFeed(GDataFeed):
+  """A feed containing a list of batch request entries."""
+
+  _tag = GDataFeed._tag
+  _namespace = GDataFeed._namespace
+  _children = GDataFeed._children.copy()
+  _attributes = GDataFeed._attributes.copy()
+  _children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', [BatchEntry])
+  _children['{%s}interrupted' % BATCH_NAMESPACE] = ('interrupted', BatchInterrupted)
+
+  def __init__(self, author=None, category=None, contributor=None,
+      generator=None, icon=None, atom_id=None, link=None, logo=None,
+      rights=None, subtitle=None, title=None, updated=None, entry=None,
+      total_results=None, start_index=None, items_per_page=None,
+      interrupted=None,
+      extension_elements=None, extension_attributes=None, text=None):
+    self.interrupted = interrupted
+    GDataFeed.__init__(self, author=author, category=category,
+                       contributor=contributor, generator=generator,
+                       icon=icon, atom_id=atom_id, link=link,
+                       logo=logo, rights=rights, subtitle=subtitle,
+                       title=title, updated=updated, entry=entry,
+                       total_results=total_results, start_index=start_index,
+                       items_per_page=items_per_page,
+                       extension_elements=extension_elements,
+                       extension_attributes=extension_attributes,
+                       text=text)
+
+  def AddBatchEntry(self, entry=None, id_url_string=None,
+                     batch_id_string=None, operation_string=None):
+    """Logic for populating members of a BatchEntry and adding to the feed.
+
+
+    If the entry is not a BatchEntry, it is converted to a BatchEntry so
+    that the batch specific members will be present.
+
+    The id_url_string can be used in place of an entry if the batch operation
+    applies to a URL. For example query and delete operations require just
+    the URL of an entry, no body is sent in the HTTP request. If an
+    id_url_string is sent instead of an entry, a BatchEntry is created and
+    added to the feed.
+
+    This method also assigns the desired batch id to the entry so that it
+    can be referenced in the server's response. If the batch_id_string is
+    None, this method will assign a batch_id to be the index at which this
+    entry will be in the feed's entry list.
+
+    Args:
+      entry: BatchEntry, atom.Entry, or another Entry flavor (optional) The
+          entry which will be sent to the server as part of the batch request.
+          The item must have a valid atom id so that the server knows which
+          entry this request references.
+      id_url_string: str (optional) The URL of the entry to be acted on. You
+          can find this URL in the text member of the atom id for an entry.
+          If an entry is not sent, this id will be used to construct a new
+          BatchEntry which will be added to the request feed.
+      batch_id_string: str (optional) The batch ID to be used to reference
+          this batch operation in the results feed. If this parameter is None,
+          the current length of the feed's entry array will be used as a
+          count. Note that batch_ids should either always be specified or
+          never, mixing could potentially result in duplicate batch ids.
+      operation_string: str (optional) The desired batch operation which will
+          set the batch_operation.type member of the entry. Options are
+          'insert', 'update', 'delete', and 'query'
+
+    Raises:
+      MissingRequiredParameters: Raised if neither an id_ url_string nor an
+          entry are provided in the request.
+
+    Returns:
+      The added entry.
+    """
+    if entry is None and id_url_string is None:
+      raise MissingRequiredParameters('supply either an entry or URL string')
+    if entry is None and id_url_string is not None:
+      entry = BatchEntry(atom_id=atom.Id(text=id_url_string))
+    # TODO: handle cases in which the entry lacks batch_... members.
+    #if not isinstance(entry, BatchEntry):
+      # Convert the entry to a batch entry.
+    if batch_id_string is not None:
+      entry.batch_id = BatchId(text=batch_id_string)
+    elif entry.batch_id is None or entry.batch_id.text is None:
+      entry.batch_id = BatchId(text=str(len(self.entry)))
+    if operation_string is not None:
+      entry.batch_operation = BatchOperation(op_type=operation_string)
+    self.entry.append(entry)
+    return entry
+
+  def AddInsert(self, entry, batch_id_string=None):
+    """Add an insert request to the operations in this batch request feed.
+
+    If the entry doesn't yet have an operation or a batch id, these will
+    be set to the insert operation and a batch_id specified as a parameter.
+
+    Args:
+      entry: BatchEntry The entry which will be sent in the batch feed as an
+          insert request.
+      batch_id_string: str (optional) The batch ID to be used to reference
+          this batch operation in the results feed. If this parameter is None,
+          the current length of the feed's entry array will be used as a
+          count. Note that batch_ids should either always be specified or
+          never, mixing could potentially result in duplicate batch ids.
+    """
+    entry = self.AddBatchEntry(entry=entry, batch_id_string=batch_id_string,
+                               operation_string=BATCH_INSERT)
+
+  def AddUpdate(self, entry, batch_id_string=None):
+    """Add an update request to the list of batch operations in this feed.
+
+    Sets the operation type of the entry to insert if it is not already set
+    and assigns the desired batch id to the entry so that it can be
+    referenced in the server's response.
+
+    Args:
+      entry: BatchEntry The entry which will be sent to the server as an
+          update (HTTP PUT) request. The item must have a valid atom id
+          so that the server knows which entry to replace.
+      batch_id_string: str (optional) The batch ID to be used to reference
+          this batch operation in the results feed. If this parameter is None,
+          the current length of the feed's entry array will be used as a
+          count. See also comments for AddInsert.
+    """
+    entry = self.AddBatchEntry(entry=entry, batch_id_string=batch_id_string,
+                               operation_string=BATCH_UPDATE)
+
+  def AddDelete(self, url_string=None, entry=None, batch_id_string=None):
+    """Adds a delete request to the batch request feed.
+
+    This method takes either the url_string which is the atom id of the item
+    to be deleted, or the entry itself. The atom id of the entry must be
+    present so that the server knows which entry should be deleted.
+
+    Args:
+      url_string: str (optional) The URL of the entry to be deleted. You can
+         find this URL in the text member of the atom id for an entry.
+      entry: BatchEntry (optional) The entry to be deleted.
+      batch_id_string: str (optional)
+
+    Raises:
+      MissingRequiredParameters: Raised if neither a url_string nor an entry
+          are provided in the request.
+    """
+    entry = self.AddBatchEntry(entry=entry, id_url_string=url_string,
+                               batch_id_string=batch_id_string,
+                               operation_string=BATCH_DELETE)
+
+  def AddQuery(self, url_string=None, entry=None, batch_id_string=None):
+    """Adds a query request to the batch request feed.
+
+    This method takes either the url_string which is the query URL
+    whose results will be added to the result feed. The query URL will
+    be encapsulated in a BatchEntry, and you may pass in the BatchEntry
+    with a query URL instead of sending a url_string.
+
+    Args:
+      url_string: str (optional)
+      entry: BatchEntry (optional)
+      batch_id_string: str (optional)
+
+    Raises:
+      MissingRequiredParameters
+    """
+    entry = self.AddBatchEntry(entry=entry, id_url_string=url_string,
+                               batch_id_string=batch_id_string,
+                               operation_string=BATCH_QUERY)
+
+  def GetBatchLink(self):
+    for link in self.link:
+      if link.rel == 'http://schemas.google.com/g/2005#batch':
+        return link
+    return None
+
+
+def BatchFeedFromString(xml_string):
+  return atom.CreateClassFromXMLString(BatchFeed, xml_string)
+
+
+class EntryLink(atom.AtomBase):
+  """The gd:entryLink element"""
+
+  _tag = 'entryLink'
+  _namespace = GDATA_NAMESPACE
+  _children = atom.AtomBase._children.copy()
+  _attributes = atom.AtomBase._attributes.copy()
+  # The entry used to be an atom.Entry, now it is a GDataEntry.
+  _children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', GDataEntry)
+  _attributes['rel'] = 'rel'
+  _attributes['readOnly'] = 'read_only'
+  _attributes['href'] = 'href'
+
+  def __init__(self, href=None, read_only=None, rel=None,
+      entry=None, extension_elements=None,
+      extension_attributes=None, text=None):
+    self.href = href
+    self.read_only = read_only
+    self.rel = rel
+    self.entry = entry
+    self.text = text
+    self.extension_elements = extension_elements or []
+    self.extension_attributes = extension_attributes or {}
+
+
+def EntryLinkFromString(xml_string):
+  return atom.CreateClassFromXMLString(EntryLink, xml_string)
+
+
+class FeedLink(atom.AtomBase):
+  """The gd:feedLink element"""
+
+  _tag = 'feedLink'
+  _namespace = GDATA_NAMESPACE
+  _children = atom.AtomBase._children.copy()
+  _attributes = atom.AtomBase._attributes.copy()
+  _children['{%s}feed' % atom.ATOM_NAMESPACE] = ('feed', GDataFeed)
+  _attributes['rel'] = 'rel'
+  _attributes['readOnly'] = 'read_only'
+  _attributes['countHint'] = 'count_hint'
+  _attributes['href'] = 'href'
+
+  def __init__(self, count_hint=None, href=None, read_only=None, rel=None,
+      feed=None, extension_elements=None, extension_attributes=None,
+      text=None):
+    self.count_hint = count_hint
+    self.href = href
+    self.read_only = read_only
+    self.rel = rel
+    self.feed = feed
+    self.text = text
+    self.extension_elements = extension_elements or []
+    self.extension_attributes = extension_attributes or {}
+
+
+def FeedLinkFromString(xml_string):
+  return atom.CreateClassFromXMLString(FeedLink, xml_string)

src/gam/gdata/alt/__init__.py

@@ -0,0 +1,20 @@
+#!/usr/bin/python
+#
+# Copyright (C) 2008 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""This package's modules adapt the gdata library to run in other environments
+
+The first example is the appengine module which contains functions and
+classes which modify a GDataService object to run on Google App Engine.
+"""

src/gam/gdata/alt/app_engine.py

@@ -0,0 +1,101 @@
+#!/usr/bin/python
+#
+# Copyright (C) 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+"""Provides functions to persist serialized auth tokens in the datastore.
+
+The get_token and set_token functions should be used in conjunction with
+gdata.gauth's token_from_blob and token_to_blob to allow auth token objects
+to be reused across requests. It is up to your own code to ensure that the
+token key's are unique.
+"""
+
+__author__ = 'j.s@google.com (Jeff Scudder)'
+
+
+from google.appengine.ext import db
+from google.appengine.api import memcache
+
+
+class Token(db.Model):
+  """Datastore Model which stores a serialized auth token."""
+  t = db.BlobProperty()
+
+
+def get_token(unique_key):
+  """Searches for a stored token with the desired key.
+
+  Checks memcache and then the datastore if required.
+
+  Args:
+    unique_key: str which uniquely identifies the desired auth token.
+
+  Returns:
+    A string encoding the auth token data. Use gdata.gauth.token_from_blob to
+    convert back into a usable token object. None if the token was not found
+    in memcache or the datastore.
+  """
+  token_string = memcache.get(unique_key)
+  if token_string is None:
+    # The token wasn't in memcache, so look in the datastore.
+    token = Token.get_by_key_name(unique_key)
+    if token is None:
+      return None
+    return token.t
+  return token_string
+
+
+def set_token(unique_key, token_str):
+  """Saves the serialized auth token in the datastore.
+
+  The token is also stored in memcache to speed up retrieval on a cache hit.
+
+  Args:
+    unique_key: The unique name for this token as a string. It is up to your
+        code to ensure that this token value is unique in your application.
+        Previous values will be silently overwitten.
+    token_str: A serialized auth token as a string. I expect that this string
+        will be generated by gdata.gauth.token_to_blob.
+
+  Returns:
+    True if the token was stored sucessfully, False if the token could not be
+    safely cached (if an old value could not be cleared). If the token was
+    set in memcache, but not in the datastore, this function will return None.
+    However, in that situation an exception will likely be raised.
+
+  Raises:
+    Datastore exceptions may be raised from the App Engine SDK in the event of
+    failure.
+  """
+  # First try to save in memcache.
+  result = memcache.set(unique_key, token_str)
+  # If memcache fails to save the value, clear the cached value.
+  if not result:
+    result = memcache.delete(unique_key)
+    # If we could not clear the cached value for this token, refuse to save.
+    if result == 0:
+      return False
+  # Save to the datastore.
+  if Token(key_name=unique_key, t=token_str).put():
+    return True
+  return None
+
+
+def delete_token(unique_key):
+  # Clear from memcache.
+  memcache.delete(unique_key)
+  # Clear from the datastore.
+  Token(key_name=unique_key).delete()