[actions] bump actions for OpenSSL 3.2.1

Added `permissionidlist <PermissionIDList>` to `<PermissionMatch>`

Added option `exportlinkeddrivefiles <Boolean>` to `gam create vaultexport`

.github/workflows/build.yml

@@ -114,7 +114,7 @@ jobs:
           path: |
             bin.tar.xz
             src/cpython
-          key: gam-${{ matrix.jid }}-202311118
+          key: gam-${{ matrix.jid }}-20240130
 
       - name: Untar Cache archive
         if: matrix.goal == 'build' && steps.cache-python-ssl.outputs.cache-hit == 'true'

docs/Aliases.md

@@ -55,13 +55,18 @@ gam create alias bob[@yourdomain.com] user robert[@yourdomain.com]
 The existing alias is deleted and a new alias is created.
 ```
 gam update alias|aliases <EmailAddressEntity> user|group|target <UniqueID>|<EmailAddress>
-        [notargetverify]
+        [notargetverify] [waitafterdelete <Integer>]
 ```
 `<EmailAddressEntity>` are the aliases, `<EmailAddress>` is the target.
 
 By default, GAM makes additional API calls to verify that the target email address exists before updating the alias;
 if you know that the target exists, you can suppress the verification with `notargetverify.
 
+GAM updates an alias to point to a new target by deleting the alias and then recreates the alias pointing to the new target.
+Unfortunately, if these commands are executed back-to-back; Google generates the `Update Failed: Duplicate` error.
+Now, GAM waits 2 seconds between the delete and the insert which seems to eliminate the problem. If the problem persists,
+use the option `waitafterdelete <Integer>` to increase the wait time to a maximum of 10 seconds.
+
 ## Delete an alias regardless of the target
 ```
 gam delete alias|aliases [user|group|target] <EmailAddressEntity>
@@ -80,7 +85,7 @@ gam <UserTypeEntity> delete aliases
 ```
 
 ## Display aliases
-Display a specific alise.
+Display a specific alias.
 ```
 gam info alias|aliases <EmailAddressEntity>
 ```
@@ -124,6 +129,9 @@ By default, the aliases in a list are separated by the `csv_output_field_delimit
 
 Specifying both `onerowpertarget` and `suppressnoaliasrows` causes GAM to not display any targets that have no aliases.
 
+Add additional columns of data from the command line to the output
+* `addcsvdata <FieldName> <String>`
+
 When multiple domains are specified and a query/queries are specified, an API call is made for each domain/query combination.
 ```
 $ gam print aliases domains school.org,students.school.org queries "'email:admin*','email:test*'"

docs/Basic-Items.md

@@ -456,12 +456,6 @@
 <Marker> ::= <String>
 <MatterItem> ::= <UniqueID>|<String>
 <MatterState> ::= open|closed|deleted
-<MessageContent> ::=
-        (message|textmessage|htmlmessage <String>)|
-        (file|textfile|htmlfile <FileName> [charset <Charset>])|
-        (gdoc|ghtml <UserGoogleDoc>)|
-        (gcsdoc|gcshtml <StorageBucketObjectName>)|
-        (emlfile <FileName>)
 <MessageID> ::= <String>
 <Namespace> ::= <String>
 <NotesName> ::= notes/<String>

docs/Chat-Bot.md

@@ -36,7 +36,7 @@ This Wiki page was built directly from Jay Lee's Wiki page; my sincere thanks fo
 
 <ChatContent> ::=
         ((text <String>)|
-         (textfile <FileName> [charset <CharSet>])|
+         (textfile <FileName> [charset <Charset>])|
          (gdoc <UserGoogleDoc>)|
          (gcsdoc <StorageBucketObjectName>))
 
@@ -162,7 +162,7 @@ gam create chatmessage space <ChatSpace>
 ```
 Specify the text of the message: `<ChatContent>`
 * `text <String>` - The message is `<String>`
-* `textfile <FileName> [charset <CharSet>]` - The message is read from a local file
+* `textfile <FileName> [charset <Charset>]` - The message is read from a local file
 * `gdoc <UserGoogleDoc>` - The message is read from a Google Doc.
 * `gcsdoc <StorageBucketObjectName>` - The message is read from a Google Cloud Storage file.
 
@@ -207,7 +207,7 @@ gam update chatmessage name <ChatMessage>
 ```
 Specify the source of the message:
 * `text <String>` - The message is `<String>`
-* `textfile <FileName> [charset <CharSet>]` - The message is read from a local file
+* `textfile <FileName> [charset <Charset>]` - The message is read from a local file
 * `gdoc <UserGoogleDoc>` - The message is read from a Google Doc.
 * `gcsdoc <StorageBucketObjectName>` - The message is read from a Google Cloud Storage file.
 

docs/ChromeOS-Devices.md

@@ -23,7 +23,7 @@
     - [Print a header row and fields for selected CrOS devices](#print-a-header-row-and-fields-for-selected-cros-devices)
     - [Print a header row and fields for specified CrOS devices](#print-a-header-row-and-fields-for-specified-cros-devices)
     - [Display Examples](#display-examples)
-    - [Display CrOS device count](#display-cros-device-count)
+    - [Display CrOS device counts](#display-cros-device-counts)
   - [Print ChromeOS device activity](#print-chromeos-device-activity)
     - [Print a header row and activity for selected CrOS devices](#print-a-header-row-and-activity-for-selected-cros-devices)
     - [Print a header row and activity for specified CrOS devices](#print-a-header-row-and-activity-for-specified-cros-devices)
@@ -160,8 +160,10 @@ The second form is backwards compatible with Standard GAM and selection with `<C
 ```
 <CrOSAction> ::=
         deprovision_different_model_replace|
+        deprovision_different_model_replacement|
         deprovision_retiring_device|
         deprovision_same_model_replace|
+        deprovision_same_model_replacement|
         deprovision_upgrade_transfer|
         disable|
         reenable|
@@ -403,13 +405,15 @@ gam update ou csvkmd cros.csv keyfield OU datafield deviceId add croscsvdata dev
         deprovision_same_model_replace|
         deprovision_upgrade_transfer|
         disable|
-        reenable|
-        pre_provisioned_disable|
-        pre_provisioned_reenable
+        reenable
 
 gam <CrOSTypeEntity> update action <CrOSAction> [acknowledge_device_touch_requirement]
+        [actionbatchsize <Integer>]
 gam update cros <CrOSEntity> action <CrOSAction> [acknowledge_device_touch_requirement]
+        [actionbatchsize <Integer>]
 ```
+As of GAM version `6.67.00`, the new API function `batchChangeStatus` replaces the old API function `action`; ChromeOS devices are now processed in batches.
+The batch size defaults to 10, the `actionbatchsize <Integer>` option can be used to set a batch size between 10 and 250.
 
 As deprovisioning ChromeOS devices is not reversible, you must enter `acknowledge_device_touch_requirement`
 when `<CrOSAction>` is `deprovision_same_model_replace`, `deprovision_different_model_replace`,
@@ -679,10 +683,34 @@ Print information about CrOS devices synced between 45 days ago and 30 days ago:
 gam print cros query "sync:#querytime1#..#querytime2#" querytime1 -45d querytime2 -30d
 ```
 
-## Display CrOS device count
+## Display CrOS device counts
 Display the number of CrOS devices in an entity.
 ```
 gam <CrOSTypeEntity> show count
+gam <CrOSTypeEntity> print cros showitemcountonly
+gam print cros select <CrOSTypeEntity> showitemcountonly
+gam print cros
+        [(query <QueryCrOS>)|(queries <QueryCrOSList>) [querytime<String> <Time>]
+         [(limittoou|cros_ou <OrgUnitItem>)|(cros_ou_and_children <OrgUnitItem>)|
+          (cros_ous <OrgUnitList>)|(cros_ous_and_children <OrgUnitList>)]]
+        showitemcountonly
+```
+Example
+```
+$ gam print cros query "sync:..2020-01-01" showitemcountonly
+Getting all CrOS Devices that match query (sync:..2020-01-01) for /, may take some time on a large Organizational Unit...
+Got 77 CrOS Devices that matched query (sync:..2020-01-01) for /...
+Got 77 CrOS Devices that matched query (sync:..2020-01-01)
+77
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print cros query "sync:..2020-01-01" showitemcountonly)
+Windows PowerShell
+count = & gam print cros query "sync:..2020-01-01" showitemcountonly
 ```
 
 ## Print ChromeOS device activity

docs/Classroom-Courses.md

@@ -10,6 +10,7 @@
 - [Manage course aliases](#manage-course-aliases)
 - [Manage course topics](#manage-course-topics)
 - [Display courses](#display-courses)
+- [Display course counts](#display-course-counts)
 - [Display course announcements](#display-course-announcements)
 - [Display course materials](#display-course-materials)
 - [Display course topics](#display-course-topics)
@@ -432,6 +433,33 @@ When using the `formatjson` option, double quotes are used extensively in the da
 The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
 `quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
 
+## Display course counts
+Display the number of courses.
+```
+gam print courses
+        (course|class <CourseEntity>)*|([teacher <UserItem>] [student <UserItem>] [states <CourseStateList>])
+        [owneremailmatchpattern <RegularExpression>]
+        showitemcountonly
+```
+Example
+```
+$ gam print courses states active showitemcountonly
+Getting all Courses that match query (Course State: ACTIVE), may take some time on a large Google Workspace Account...
+Got 268 Courses...
+Got 272 Courses...
+Got 272 Courses...
+272
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print courses states active showitemcountonly)
+Windows PowerShell
+count = & gam print courses states active showitemcountonly
+```
+
 ## Display course announcements
 ```
 gam print course-announcements [todrive <ToDriveAttribute>*]

docs/Classroom-Membership.md

@@ -6,6 +6,7 @@
 - [Legacy manage membership](#legacy-manage-membership)
 - [Bulk membership changes](#bulk-membership-changes)
 - [Display course membership](#display-course-membership)
+- [Display course membership counts](#display-course-membership-counts)
 
 ## API documentation
 * https://developers.google.com/classroom/reference/rest/
@@ -131,3 +132,35 @@ the quote character itself, the column delimiter (comma by default) and new-line
 When using the `formatjson` option, double quotes are used extensively in the data resulting in hard to read/process output.
 The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
 `quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
+
+## Display course membership counts
+Display the number of course participants.
+```
+gam print course-participants
+        (course|class <CourseID>)*|([teacher <UserItem>] [student <UserItem>]) [states <CourseStateList>]
+        [show all|students|teachers]
+        showitemcountonly
+```
+Example
+```
+$ gam print course-participants teacher asmith states active show students showitemcountonly
+Getting all Courses that match query (Teacher: asmith@domain.com, Course State: ACTIVE), may take some time on a large Google Workspace Account...
+Got 3 Courses...
+Getting Students for Course: 636981507234 (1/3)
+Got 30 Students...
+Got 43 Students...
+Getting Students for Course: 589346784341 (2/3)
+Got 22 Students...
+Getting Students for Course: 589345535881 (3/3)
+Got 23 Students...
+88
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print course-participants teacher asmith states active show students showitemcountonly)
+Windows PowerShell
+count = & gam print course-participants teacher asmith states active show students showitemcountonly
+```

docs/Cloud-Identity-Devices.md

@@ -9,11 +9,13 @@
 - [Synchronize devices](#synchronize-devices)
 - [Display devices](#display-devices)
 - [Print devices](#print-devices)
+- [Display device counts](#display-device-counts)
 - [Approve or block device users](#approve-or-block-device-users)
 - [Delete device users](#delete-device-users)
 - [Wipe device users](#wipe-device-users)
 - [Perform device user actions](#perform-device-user-actions)
 - [Display device users](#display-device-users)
+- [Display device user counts](#display-device-user-counts)
 - [Print device users](#print-device-users)
 - [Display device user client state](#display-device-user-client-state)
 - [Update device user client state](#update-device-user-client-state)
@@ -225,6 +227,37 @@ When using the `formatjson` option, double quotes are used extensively in the da
 The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
 `quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
 
+## Display device counts
+Display the number of devices.
+```
+gam print devices
+        [(query <QueryDevice>)|(queries <QueryDeviceList>) (querytime<String> <Time>)*]
+        [all|company|personal|nocompanydevices|nopersonaldevices]
+        showitemcountonly
+```
+Example
+```
+$ gam print devices queries "'model:Mac'" showitemcountonly
+Getting all Devices that match query (model:Mac), may take some time on a large Google Workspace Account...
+Got 100 Devices...
+Got 200 Devices...
+Got 300 Devices...
+...
+Got 900 Devices...
+Got 995 Devices...
+Got 995 Devices...
+995
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print devices queries "'model:Mac'" showitemcountonly)
+Windows PowerShell
+count = & gam print devices queries "'model:Mac'" showitemcountonly
+```
+
 ## Approve or block device users
 Approve or block user profiles on a device.
 ```
@@ -285,6 +318,38 @@ When using the `formatjson` option, double quotes are used extensively in the da
 The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
 `quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
 
+## Display device user counts
+Display the number of device users.
+```
+gam print deviceusers [todrive <ToDriveAttribute>*]
+        [select <DeviceID>]
+        [(query <QueryDevice>)|(queries <QueryDeviceList>) (querytime<String> <Time>)*]
+        showitemcountonly
+```
+Example
+```
+$ gam print deviceusers queries "'model:Mac'" showitemcountonly
+Getting all Device Users that match query (model:Mac), may take some time on a large Google Workspace Account...
+Got 20 Device Users...
+Got 40 Device Users...
+Got 60 Device Users...
+...
+Got 980 Device Users...
+Got 995 Device Users...
+Got 995 Device Users...
+995
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print deviceusers queries "'model:Mac'" showitemcountonly)
+Windows PowerShell
+count = & gam print deviceusers queries "'model:Mac'" showitemcountonly
+```
+
+
 ## Display device user client state
 ```
 gam info deviceuserstate <DeviceUserEntity> [clientid <String>] 

docs/Cloud-Identity-Groups.md

@@ -9,6 +9,7 @@
 - [Manage groups](#manage-groups)
 - [Display information about individual groups](#display-information-about-individual-groups)
 - [Display information about multiple groups](#display-information-about-multiple-groups)
+- [Display group counts](#display-group-counts)
 
 ## API documentation
 * https://developers.google.com/admin-sdk/directory/reference/rest/v1/groups
@@ -376,3 +377,29 @@ gam print cigroups query "'cloudidentity.googleapis.com/groups.dynamic' in label
 ```
 gam print cigroups query "'cloudidentity.googleapis.com/groups.security' in labels"
 ```
+
+## Display group counts
+Display the number of groups.
+```
+gam print cigroups
+        [(cimember|showownedby <UserItem>)|(select <GroupEntity>)|(query <String>)]
+        [emailmatchpattern [not] <RegularExpression>] [namematchpattern [not] <RegularExpression>]
+        [descriptionmatchpattern [not] <RegularExpression>]
+        showitemcountonly
+```
+Example
+```
+$ gam print cigroups showitemcountonly
+Getting all Cloud Identity Groups, may take some time on a large Google Workspace Account...
+Got 242 Cloud Identity Groups: td.current@domain.com - postmaster@domain.com
+242
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print cigroups showitemcountonly)
+Windows PowerShell
+count = & gam print cidgroups showitemcountonly
+```

docs/Domain-SharedContacts-GAL.md

@@ -35,7 +35,7 @@
 
 <NoteContent> ::=
         ((<String>)|
-         (file <FileName> [charset <CharSet>])|
+         (file <FileName> [charset <Charset>])|
          (gdoc <UserGoogleDoc>)|
          (gcsdoc <StorageBucketObjectName>))
 

docs/GamUpdates.md

@@ -1,3 +1,4 @@
+
 # Update GAMADV-XTD3 to latest version
 Automatic update to the latest version on Linux/Mac OS/Google Cloud Shell/Raspberry Pi/ChromeOS:
 - Do not create project or authorizations, default path `$HOME/bin`
@@ -10,6 +11,318 @@ Add the `-s` option to the end of the above commands to suppress creating the `g
 
 See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation
 
+### 6.67.24
+
+Fixed bug that caused HTML password notification email messages to be displayed in raw form.
+
+### 6.67.23
+
+Use local copy of `googleapiclient` to remove static discovery documents to improve performance.
+
+### 6.67.22
+
+Added `permissionidlist <PermissionIDList>` to `<PermissionMatch>` that allows matching any permission ID in a list.
+
+Added option `exportlinkeddrivefiles <Boolean>` to `gam create vaultexport` that is used with `corpus mail`.
+
+### 6.67.21
+
+Updated `gam remove aliases <EmailAddress> user|group <EmailAddressEntity>` to give a more informative
+error message when the target/alias combination does not exist.
+```
+Old: User: testsimple@rdschool.org, User Alias: tsalias@rdschool.org, Remove Failed: Invalid Input: resource_id
+New: User: testsimple@rdschool.org, User Alias: tsalias@rdschool.org, Remove Failed: Does not exist
+```
+
+### 6.67.20
+
+Added option `onelicenseperrow|onelicenceperrow` to `gam print users ... licenses` that causes GAM to print
+a seperate user information row for each license a user is assigned. This makes processing
+the licenses in a script possible and allows better sorting in a CSV File.
+
+By default, all licenses for a user are displayed in a list on one row:
+```
+primaryEmail,LicensesCount,Licenses,LicensesDisplay
+user@domain.com,2,1010020020 1010330004,Google Workspace Enterprise Plus Google Voice Standard
+```
+With `onelicenseperrow|onelicenceperrow`, each license is on a separate row:
+```
+primaryEmail,License,LicenseDisplay
+user@domain.com,1010020020,Google Workspace Enterprise Plus
+user@domain.com 1010330004,Google Voice Standard
+```
+
+### 6.67.19
+
+Updated `gam create|update user ... notify` to encode the characters `<>&` in the password
+so that they display correctly when the notify message content is HTML.
+
+### 6.67.18
+
+Cleaned up `Getting/Got` messages for `gam print courses|course-participants`.
+
+### 6.67.17
+
+Added option `showitemcountonly` to various commands that causes GAM to display the
+item count on stdout; no CSV file is written.
+
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Cloud-Identity-Groups#display-group-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Classroom-Courses#display-course-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Classroom-Membership#display-course-membership-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/ChromeOS-Devices#display-cros-device-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Cloud-Identity-Devices#display-device-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Cloud-Identity-Devices#display-device-user-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Groups#display-group-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Mobile-Devices#display-mobile-device-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Organizational-Units#display-organizational-unit-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Resources#display-resource-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Users#display-user-counts
+
+### 6.67.16
+
+By default, `gam print group-members membernames` displays `Unknown` for members whose names can not be determined.
+Added option `unknownname <String>` that let's you specify an alternative value.
+
+Further improved performance of `gam print group-members membernames cachememberinfo`.
+
+### 6.67.15
+
+Update `gam print group-members membernames` to handle the following error:
+```
+ERROR: 400: failedPrecondition - Precondition check failed.
+```
+
+Added option `cachememberinfo [Boolean]` to `gam print group-members` that causes GAM to cache member info
+so that only one API call is made to get information for each user/group. This consumes
+more memory but dramatically reduces the number of API calls.
+
+### 6.67.14
+
+Updated reseller commands to handle the following error:
+```
+ERROR: 400: invalid - Customer domain [domain.com] is linked to one or more email verified customers, please provide a customer id.
+```
+
+### 6.67.13
+
+Updated `gam create domain <DomainName>` to handle the following error:
+```
+ERROR: 409: conflict - Domain in request is in use by an email verified customer.
+```
+
+### 6.67.12
+
+Added option `addcsvdata <FieldName> <String>` to `gam print datatransfers` that adds
+additional columns of data to the CSV file output.
+
+### 6.67.11
+
+Updated various Gmail related commands to handle this error:
+```
+ERROR: 403: permissionDenied - Insufficient Permission
+```
+when the following service account scopes are selected:
+
+```
+[ ] 23)  Gmail API - Basic Settings (Filters,IMAP, Language, POP, Vacation) - read/write, Sharing Settings (Delegates, Forwarding, SendAs) - read
+[ ] 24)  Gmail API - Full Access (Labels, Messages)
+[ ] 25)  Gmail API - Full Access (Labels, Messages) except delete message
+[*] 26)  Gmail API - Full Access - read only
+[ ] 27)  Gmail API - Send Messages - including todrive
+[ ] 28)  Gmail API - Sharing Settings (Delegates, Forwarding, SendAs) - write
+```
+
+### 6.67.10
+
+Fixed bug that caused a trap when optional argument `charset <Charset>` was used with `emlfile <FileName>` in `gam <UserTypeEntity> draft|import|insert message`.
+
+### 6.67.09
+
+Added option `maxevents <Number>` to `gam report <ActivityApplictionName>` that limits
+the number of events displayed for each activity; the default is 0, no limit.
+Setting options `maxactivities 1 maxevents 1 maxresults 1` can be used to as efficiently as possible
+show the most recent activity/event; this can be useful when reporting drive activity for individual drive files.
+
+### 6.67.08
+
+Added optional argument `charset <Charset>` to `emlfile <FileName>` in `gam <UserTypeEntity> draft|import|insert message`;
+the default value is `ascii`.
+
+### 6.67.07
+
+Updated `gam <UserTypeEntity> delete message` to handle this error:
+```
+ERROR: 403: permissionDenied - Insufficient Permission
+```
+when the following service account scopes are selected:
+
+```
+[ ] 24)  Gmail API - Full Access (Labels, Messages)
+[*] 25)  Gmail API - Full Access (Labels, Messages) except delete message
+```
+
+### 6.67.06
+
+Updated commands that create ACLs to handle the following error:
+```
+ERROR: 400: abusiveContentRestriction - Bad Request. User message: "You cannot share this item because it has been flagged as inappropriate."
+```
+
+### 6.67.05
+
+Updated the following commands:
+```
+gam <UserTypeEntity> create|delete|update delegate
+gam <UserTypeEntity> forward
+gam <UserTypeEntity> create|delete forwardingaddresses
+gam <UserTypeEntity> create|delete sendas
+```
+to handle this error:
+```
+ERROR: 403: permissionDenied - Insufficient Permission
+```
+when the following serice account scope is not enabled:
+```
+[ ] 28)  Gmail API - Sharing Settings (Delegates, Forwarding, SendAs) - write
+```
+
+### 6.67.04
+
+Updated user attribute `replace <Tag> <UserReplacement>` to allow `field:photourl` which allows
+embedding a link to a user's photo in their signature. Formatting the signature HTML
+to properly display the photo is left to the GAM admin.
+
+### 6.67.03
+
+Fixed bug introduced in 6.67.02  in `gam <UserTypeEntity> claim ownership` that caused a trap.
+
+### 6.67.02
+
+Added option `skipids <DriveFileEntity>` to `gam <UserTypeEntity> transfer drive` that handles special cases
+where you want to prevent ownership from being transferred for selected files/folders.
+
+Added option `skipids <DriveFileEntity>` to `gam <UserTypeEntity> copy drivefile` that handles special cases
+where you want to prevent selected files/folders from being copied.
+
+Updated commands that create files/folders on Shared Drives to handle the following errors:
+```
+storageQuotaExceeded
+teamDriveFileLimitExceeded
+teamDriveHierarchyTooDeep
+```
+* See: https://support.google.com/a/users/answer/7338880#shared_drives_file_folder_limits
+
+### 6.67.01
+
+Fixed bug in `gam print vaultcounts` that caused a trap.
+
+### 6.67.00
+
+Updated `gam <CrOSTypeEntity> update action <CrOSAction>` to use the new API function `batchChangeStatus`
+that replaces the old API function `action`; ChromeOS devices are now processed in batches.
+The batch size defaults to 10, the `actionbatchsize <Integer>` option can be used to set a batch size between 10 and 250.
+
+Updated `gam create vaultexport matter <MatterItem>` to support `corpus calendar`.
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Vault-Takeout#create-vault-exports
+
+### 6.66.16
+
+Added option `convertcrnl` to `gam update chromepolicy` to properly handle carriage returns (\r) and line feeds (\n)
+in value strings entered on the command line in the `<Field> <Value>` form. 
+```
+gam update chromepolicy convertcrnl chrome.devices.DisabledDeviceReturnInstructions
+    deviceDisabledMessage "Please return device to:\nSchool\n123 Main Street\nAnytown US" ou /Path/to/OU
+```
+
+### 6.66.15
+
+Added option `copysubfilesownedby any|me|others` to `gam <UserTypeEntity> copy drivefile` that allows
+specification of which source folder sub files to copy based on file ownership; the default is `any`.
+This only applies when files are being copied from a 'My Drive'.
+
+### 6.66.14
+
+Updated `gam <UserTypeEntity> modify messages` to recognize the following error:
+```
+ERROR: 400: invalid - Invalid label: SENT
+```
+
+Updated `gam update alias <EmailAddressEntity> user|group|target <EmailAddress>`
+to avoid the following problem.
+```
+$ gam update alias testalias@domain.com user testuser
+User Alias: testalias@domain.com, Deleted
+User Alias: testalias@domain.com, User: testuser@domain.com, Update Failed: Duplicate, Email Address: testalias@domain.com
+```
+GAM updates an alias to point to a new target by deleting the alias and then recreating the alias pointing to the new target.
+Unfortunately, if these commands are executed back-to-back; Google generates the `Update Failed: Duplicate` error.
+Now, GAM waits 2 seconds between the delete and the insert which seems to eliminate the problem. If the problem persists,
+the option `waitafterdelete <Integer>` can be used to increase the wait time to a maximum of 10 seconds.
+
+### 6.66.13
+
+Updated functionality of option `preservefiletimes` in `gam <UserTypeEntity> update drivefile <DriveFileEntity>`.
+
+* Current
+  * `preservefiletimes localfile <FileName>` - `modifiedTime` of `<DriveFileEntity>` is set to that of `localfile <FileName>`
+  * `preservefiletimes` - No effect
+* Updated
+  * `preservefiletimes localfile <FileName>` - `modifiedTime` of `<DriveFileEntity>` is set to that of `localfile <FileName>`
+  * `preservefiletimes` - `modifiedTime` of `<DriveFileEntity>` retains its current value
+
+### 6.66.12
+
+Upgraded to Python 3.12.1 where possible.
+
+Updated all drive commands to handle the following error:
+```
+ERROR: 401: Active session is invalid. Error code: 4 - authError
+```
+This is due to the Drive SDK API being disabled in the user's OU.
+* See: https://support.google.com/a/answer/6105699
+
+### 6.66.11
+
+Fixed/improved handling of shortcuts in `gam <UserTypeEntity> transfer drive`.
+
+### 6.66.10
+
+Updated `gam create datatransfer` to handle the following error:
+```
+ERROR: 401: Active session is invalid. Error code: 4 - authError
+```
+
+### 6.66.09
+
+Fixed bug in `gam <UserTypeEntity> print filelist ... allfields` that caused a trap
+when `gam.cfg` contained `drive_v3_native_names = False`.
+
+### 6.66.08
+
+Added additional columns `isBase` and `baseId` to `gam <UserTypeEntity> print fileparenttree`
+to simplify  processing the output in a script.
+
+### 6.66.07
+
+Fixed bug in `gam <UserTypeEntity> print diskusage` that caused a trap.
+
+### 6.66.06
+
+Added a command the print the parent tree of file/folder.
+```
+gam <UserTypeEntity> print fileparenttree <DriveFileEntity> [todrive <ToDriveAttribute>*]
+        [stripcrsfromname]
+```
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Drive-Files-Display#display-file-parent-tree
+
+### 6.66.05
+
+Added column `space.name` to `gam <UserTypeEntity> print chatmembers`.
+
+### 6.66.04
+
+Updated Chat info|show|print commands to display all time fields in local time if specified in `gam.cfg`.
+
 ### 6.66.03
 
 Fixed bug in `gam <UserTypeEntity> print filelist select <DriveFileEntity>` where `stripcrsfromname` was not being

docs/Google-Data-Transfers.md

@@ -63,6 +63,7 @@ gam show datatransfers|transfers
 gam print datatransfers|transfers [todrive <ToDriveAttribute>*]
         [olduser|oldowner <UserItem>] [newuser|newowner <UserItem>]
         [status completed|failed|inprogress|<String>] [delimiter <Character>]
+        (addcsvdata <FieldName> <String>)*
 ```
 By default, all data transfer operations are printed, use these options to select specific transfers.
 * `olduser|oldowner <UserItem>`
@@ -72,3 +73,5 @@ By default, all data transfer operations are printed, use these options to selec
 By default, the entries in lists of items are separated by the `csv_output_field_delimiter` from `gam.cfg`.
 * `delimiter <Character>` - Separate list items with `<Character>`
 
+Add additional columns of data from the command line to the output
+* `addcsvdata <FieldName> <String>`

docs/Groups-Membership.md

@@ -583,6 +583,7 @@ gam print group-members [todrive <ToDriveAttribute>*]
         [userfields <UserFieldNameList>]
         [(recursive [noduplicates])|includederivedmembership] [nogroupemail]
         [peoplelookup|(peoplelookupuser <EmailAddress>)]
+        [unknownname <String>] [cachememberinfo [Boolean]]
         [formatjson [quotechar <Character>]]
 ```
 By default, the group membership of all groups in the account are displayed, these options allow selection of subsets of groups:
@@ -646,11 +647,17 @@ these options specify which fields to display:
     * `delivery|deliverysettings` - Specify this field to get delivery information; an additional API call per member is required
 * `userfields <UserFieldNameList>` - For members that are users, display these user fields; an additional API call per member is required
 
+The additional API calls can be reduced with the `cachememberinfo` option; a single API call is made for each user/group
+and the data is cached to eliminate to need to repeat the API call; this consumes more memory but dramatically reduces the number of API calls.
+
 If member names are requested, names are not available for users not in the domain; you can request that GAM use the People API to retrieve
 names for these users. Names are not retrieved in all cases and success is dependent on what user is used to perform the retrievals.
 * `peoplelookup` - Use the administrator named in oauth2.txt to perform the retrievals
 * `peoplelookupuser <EmailAddress>` - Use `<EmailAddress>` to perform the retrievals
 
+By default, when `membernames` is specified, GAM displays `Unknown` for members whose names can not be determined.
+Use `unknownname <String>` to specify an alternative value.
+
 By default, the group email address is always shown, you can suppress it with the `nogroupemail` option.
 
 By default, members that are groups are displayed as a single entry of type GROUP; this option recursively expands group members to display their user members.

docs/Groups.md

@@ -16,6 +16,7 @@
 - [Display information about selected groups](#display-information-about-selected-groups)
 - [Display a group and its parents](#Display-a-group-and-its-parents)
 - [Examples](#Examples)
+- [Display group counts](#display-group-counts)
 
 ## API documentation
 * https://developers.google.com/admin-sdk/directory/reference/rest/v1/groups
@@ -563,3 +564,31 @@ Group,Name,ParentsCount,Parents,ParentsName
 testgroup2@domain.com,Test - Group 2,2,testgroup1@domain.com|testgroup@domain.com,Test Group1|Test Group Org
 testgroup2@domain.com,Test - Group 2,1,testgroup@domain.net,Test Group Net
 ```
+## Display group counts
+Display the number of groups.
+```
+gam print groups
+        [([domain|domains <DomainNameEntity>] ([member|showownedby <EmailItem>]|[(query <QueryGroup>)|(queries <QueryGroupList>)]))|
+         (select <GroupEntity>)]
+        [emailmatchpattern [not] <RegularExpression>] [namematchpattern [not] <RegularExpression>]
+        [descriptionmatchpattern [not] <RegularExpression>] (matchsetting [not] <GroupAttribute>)*
+        [admincreatedmatch <Boolean>]
+        showitemcountonly
+```
+Example
+```
+$ gam print groups showitemcountonly
+Getting all Groups, may take some time on a large Google Workspace Account...
+Got 200 Groups: 1aparents@domain.com - students-genderfood@domain.com
+Got 238 Groups: students-worldculture@domain.com - xcarestaff@domain.com
+238
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print groups showitemcountonly)
+Windows PowerShell
+count = & gam print groups showitemcountonly
+```

docs/How-to-Upgrade-from-Standard-GAM.md

@@ -334,10 +334,10 @@ writes the credentials into the file oauth2.txt.
 admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt
 admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version
 WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found
-GAMADV-XTD3 6.66.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.67.24 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
-Python 3.10.8 64-bit final
-MacOS High Sierra 10.13.6 x86_64
+Python 3.12.1 64-bit final
+MacOS Sonoma 14.2.1 x86_64
 Path: /Users/admin/bin/gamadv-xtd3
 Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, customer_id: my_customer, domain.com
 
@@ -1002,9 +1002,9 @@ writes the credentials into the file oauth2.txt.
 C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt
 C:\GAMADV-XTD3>gam version
 WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found
-GAMADV-XTD3 6.66.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.67.24 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
-Python 3.12.0 64-bit final
+Python 3.12.1 64-bit final
 Windows-10-10.0.17134 AMD64
 Path: C:\GAMADV-XTD3
 Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, customer_id: my_customer, domain.com

docs/List-Items.md

@@ -97,6 +97,7 @@
 <TasklistIDTaskIDList> ::= "<TasklistIDTaskID>(,<TasklistIDTaskID>)*"
 <ThreadIDList> ::= "<ThreadID>(,<ThreadID>)*"
 <TimeList> ::= "<Time>(,<Time>)*"
+<URLList> ::= "<URL>(,<URL>)*"
 <UserList> ::= "<UserItem>(,<UserItem>)*"
 <YouTubeChannelIDList> ::= "<YouTubeChannelID>(,<YouTubeChannelID>)*"
 ```

docs/Mobile-Devices.md

@@ -5,6 +5,7 @@
 - [Manage mobile devices](#manage-mobile-devices)
 - [Display mobile devices](#display-mobile-devices)
 - [Print mobile devices](#print-mobile-devices)
+- [Display mobile device counts](#display-mobile-device-counts)
 
 ## API documentation
 * https://developers.google.com/admin-sdk/directory/reference/rest/v1/mobiledevices
@@ -150,3 +151,27 @@ When using the `formatjson` option, double quotes are used extensively in the da
 The `quotechar <Character>` option allows you to choose an alternate quote character, single quote for instance, that makes for readable/processable output.
 `quotechar` defaults to `gam.cfg/csv_output_quote_char`. When uploading CSV files to Google, double quote `"` should be used.
 
+## Display mobile device counts
+Display the number of mobile devices.
+```
+gam print mobile
+        [(query <QueryMobile>)|(queries <QueryMobileList>) (querytime<String> <Time>)*]
+        showitemcountonly
+```
+Example
+```
+$ gam print mobile showitemcountonly
+Getting all Mobile Devices, may take some time on a large Google Workspace Account...
+Got 100 Mobile Devices...
+Got 115 Mobile Devices
+115
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print mobile showitemcountonly)
+Windows PowerShell
+count = & gam print mobile showitemcountonly
+```

docs/Organizational-Units.md

@@ -13,6 +13,7 @@
 - [Synchronize ChromeOS devices with an organizational unit](#synchronize-chromeos-devices-with-an-organizational-unit)
 - [Display organizational units](#display-organizational-units)
 - [Print organizational units](#print-organizational-units)
+- [Display orgamizational unit counts](#display-organizational-unit-counts)
 - [Display indented organizational unit tree](#display-indented-organizational-unit-tree)
 - [Special case handling for large number of organizational units](#special-case-handling-for-large-number-of-organizational-units)
 
@@ -237,6 +238,30 @@ Get file count summaries by OU; top level selector is ou, sub level selectors ar
 gam redirect csv ./TopLevelOUs.csv print ous showparent toplevelonly parentselector ou childselector ou_and_children fields orgunitpath
 gam redirect csv ./FileCounts.csv multiprocess csv ./TopLevelOUs.csv gam "~orgUnitSelector" "~orgUnitPath" print filecounts excludetrashed summary only summaryuser "~orgUnitPath"
 ```
+## Display organizational unit counts
+Display the number of organizational units.
+```
+gam print orgs|ous
+        [fromparent <OrgUnitItem>] [showparent [Boolean>]] [toplevelonly]
+        showitemcountonly
+```
+Example
+```
+$ gam print orgs showitemcountonly     
+Getting all Organizational Units, may take some time on a large Google Workspace Account...
+Got 98 Organizational Units
+98
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print orgs showitemcountonly)
+Windows PowerShell
+count = & gam print orgs showitemcountonly
+```
+
 ## Display indented organizational unit tree
 ```
 gam show orgtree [fromparent <OrgUnitItem>] [batchsuborgs [<Boolean>]]

docs/Permission-Matches.md

@@ -34,6 +34,7 @@
             [type <DriveFileACLType>] [role|notrole <DriveFileACLRole>]
             [allowfilediscovery|withlink <Boolean>]
             [emailaddress <RegularExpression>] [emailaddressList <EmailAddressList>]
+            [permissionidlist <PermissionIDList>
             [name|displayname <String>]
             [domain|notdomain <RegularExpression>] [domainlist|notdomainlist <DomainNameList>]
             [expirationstart <Time>] [expirationend <Time>]
@@ -77,6 +78,7 @@ In the `print/show drivefileacls` and `create/delete permissions` commands you c
 * `allowfilediscovery|withlink <Boolean>` - Whether a link is required or whether the file can be discovered through search.
 * `emailaddress <RegularExpression>` - For types user and group, the required email address.
 * `emailaddresslist <EmailAddressList>` - For types user and group, a list of required email addresses; any one of which must match.
+* `permissionidlist <PermissionIDListList>` - A list of required permission IDs; any one of which must match.
 * `name|displayname <RegularExpression>` - For types domain, user and group, the displayable name.
 * `domain <RegularExpression>` - For type domain, the required domain name. For types user and group, the required domain name in the email address.
 * `notdomain <RegularExpression>` - For type domain, any domain name that doesn't match. For types user and group, any domain name that doesn't match in the email address.

docs/Reports.md

@@ -58,7 +58,7 @@ gam report <ActivityApplicationName> [todrive <ToDriveAttributes>*]
         [filtertime.* <Time>] [filter|filters <String>]
         [event|events <EventNameList>] [ip <String>]
         [groupidfilter <String>]
-        [maxactivities <Number>] [maxresults <Number>]
+        [maxactivities <Number>] [maxevents <Number>] [maxresults <Number>]
         [countsonly [summary] [eventrowfilter]]
         (addcsvdata <FieldName> <String>)* [shownoactivities]
 ```
@@ -100,9 +100,15 @@ Limit to those users that are a member of at least one of a list of groups.
 Limit the total number of activites.
 * `maxactivities <Number>`
 
+Limit the  number of events per activity; this only applies when `countsonly` is False.
+* `maxevents <Number>`
+
 Limit the number of activities downloaded per API call; infrequently used.
 * `maxresults <Number>`
 
+Setting options `maxactivities 1 maxevents 1 maxresults 1` can be used to as efficiently as possible
+show the most recent activity/event; this can be useful when reporting drive activity for individual drive files.
+
 Add additional columns of data from the command line to the output.
 * `addcsvdata <FieldName> <String>`
 
@@ -333,6 +339,9 @@ Select the users for whom information is desired.
   * `showorgunit` - Add a column labelled `orgUnitPath` to the output; an additional API call is made to get the email addresses of the users in `<OrgUnitPath>`
 * `select <UserTypeEntity>` - A selected collection of users, e.g., `select group staff@domain.com`; there is one API call per user
 
+By default, when `user all` is specified (or no user specification in supplied), GAM backs up looking for data with a (basically) random user. If the randaom
+doesn't have any data, the command reports that no data was found. Use `allverifyuser <UserItem>` to specify a specific user to use to search for data.
+
 Specify the report date; the default is today's date.
 * `date <Date>` - A single date; there is one API call
 * `range <Date> <Date>` - A range of dates; there is an API call per date

docs/Reseller.md

@@ -1,6 +1,7 @@
 # Reseller
 - [API documentation](#api-documentation)
 - [Notes](#notes)
+- [Manage Multiple Domains](#manage-multiple-domains)
 - [Definitions](#definitions)
 - [Manage Resold Customers](#manage-resold-customers)
 - [Display Resold Customers](#display-resold-customers)
@@ -25,6 +26,11 @@ Prior to version 6.50.00, this is how the `seats <NumberOfSeats> <MaximumNumberO
 
 Now, you can still use the above option which has been corrected or you can specify `seats <Number>` which will be properly passed in the correct form to the API based on plan name.
 
+## Manage Multiple Domains
+Thanks to Duncan Isaksen-Loxton for a script to help manage multiple domains.
+
+* See: https://gist.github.com/65/b5e9cee9b5812b487b8ae3e8256e262b
+
 ## Definitions
 ```
 <CustomerID> ::= <String>

docs/Resources.md

@@ -8,6 +8,7 @@
 - [Display features](#display-features)
 - [Manage resources](#manage-resources)
 - [Display resources](#display-resources)
+- [Display resource counts](#display-resource-counts)
 - [Manage resource calendar ACLs](#manage-resource-calendar-acls)
 - [Display resource calendar ACLs](#display-resource-calendar-acls)
 
@@ -245,6 +246,30 @@ Print all resources and their owners.
 gam config csv_output_row_filter "role:regex:owner" redirect csv Resource.csv print resources acls
 ```
 
+## Display resource counts
+Display the number of mobile devices.
+```
+gam print resources
+        [query <String>]
+        showitemcountonly
+```
+Example
+```
+$ gam print resources showitemcountonly
+Getting all Resource Calendars, may take some time on a large Google Workspace Account...
+Got 32 Resource Calendars: Back 50 - Video Cameras Class Set
+32
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
+
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print resources showitemcountonly)
+Windows PowerShell
+count = & gam print resources showitemcountonly
+```
+
 ## Manage resource calendar ACLs
 These commands operate on a single resource calendar.
 ```

docs/Send-Email.md

@@ -54,8 +54,7 @@ Added the option `mailbox <EmailAddress>` to `gam sendemail` to allow specifying
         (message|textmessage|htmlmessage <String>)|
         (file|textfile|htmlfile <FileName> [charset <Charset>])|
         (gdoc|ghtml <UserGoogleDoc>)|
-        (gcsdoc|gcshtml <StorageBucketObjectName>)|
-        (emlfile <FileName>)
+        (gcsdoc|gcshtml <StorageBucketObjectName>)
 ```
 ```
 <Time> ::=
@@ -211,7 +210,7 @@ gam sendemail [recipient|to] <RecipientEntity>
         [cc <RecipientEntity>] [bcc <RecipientEntity>] [singlemessage]
         [subject <String>]
         [<MessageContent>] (replace <Tag> <String>)*
-        [html [<Boolean>]] (attach <FileName> [charset <CharSet>])*
+        [html [<Boolean>]] (attach <FileName> [charset <Charset>])*
         (embedimage <FileName> <String>)*
         [newuser <EmailAddress> firstname|givenname <String> lastname|familyname <string> password <Password>]
         (<SMTPDateHeader> <Time>)* (<SMTPHeader> <String>)* (header <String> <String>)*
@@ -267,7 +266,7 @@ gam sendemail [recipient|to] <RecipientEntity> [from <EmailAddress>]
         [cc <RecipientEntity>] [bcc <RecipientEntity>] [singlemessage]
         [subject <String>]
         [<MessageContent>] (replace <Tag> <String>)*
-        [html [<Boolean>]] (attach <FileName> [charset <CharSet>])*
+        [html [<Boolean>]] (attach <FileName> [charset <Charset>])*
         (embedimage <FileName> <String>)*
         [newuser <EmailAddress> firstname|givenname <String> lastname|familyname <string> password <Password>]
         (<SMTPDateHeader> <Time>)* (<SMTPHeader> <String>)* (header <String> <String>)*
@@ -298,7 +297,7 @@ If `message` is not specified, the following value will be used:
 *    `Hello #givenname# #familyname#,\n\nYou have a new account at #domain#\nAccount details:\n\nUsername\n#user#\n\nPassword\n#password#\n\n
     Start using your new account by signing in at\nhttps://www.google.com/accounts/AccountChooser?Email=#user#&continue=https://apps.google.com/user/hub\n`
 
-If you want a language/organization specific message, use a template file: `message file <FileName> [charset <CharSet>]`
+If you want a language/organization specific message, use a template file: `message file <FileName> [charset <Charset>]`
 
 The `<SMTPDateHeader> <Time>` argument  requires `<Time>` values which will be converted to RFC2822 dates. If you have these headers with values that
 are not in `<Time>` format, use the argument `header <SMTPDateHeader> <String>`.
@@ -346,7 +345,7 @@ gam <UserTypeEntity> sendemail recipient|to <RecipientEntity>
         [cc <RecipientEntity>] [bcc <RecipientEntity>] [singlemessage]
         [subject <String>]
         [<MessageContent>] (replace <Tag> <String>)*
-        [html [<Boolean>]] (attach <FileName> [charset <CharSet>])*
+        [html [<Boolean>]] (attach <FileName> [charset <Charset>])*
         (embedimage <FileName> <String>)*
         [newuser <EmailAddress> firstname|givenname <String> lastname|familyname <string> password <Password>]
         (<SMTPDateHeader> <Time>)* (<SMTPHeader> <String>)* (header <String> <String>)*
@@ -386,7 +385,7 @@ gam <UserTypeEntity> sendemail [from <EmailAddress>]
         [cc <RecipientEntity>] [bcc <RecipientEntity>] [singlemessage]
         [subject <String>]
         [<MessageContent>] (replace <Tag> <String>)*
-        [html [<Boolean>]] (attach <FileName> [charset <CharSet>])*
+        [html [<Boolean>]] (attach <FileName> [charset <Charset>])*
         (embedimage <FileName> <String>)*
         [newuser <EmailAddress> firstname|givenname <String> lastname|familyname <string> password <Password>]
         (<SMTPDateHeader> <Time>)* (<SMTPHeader> <String>)* (header <String> <String>)*

docs/Shared-Drives.md

@@ -28,6 +28,7 @@
 * https://developers.google.com/drive/api/v3/reference/teamdrives/list
 * https://support.google.com/a/answer/7374057
 * https://workspaceupdates.googleblog.com/2022/05/shared-drives-in-organizational-units-open-beta.html
+* https://support.google.com/a/users/answer/7338880
 
 ## Query documentation
 * https://developers.google.com/drive/api/v3/search-shareddrives

docs/Tag-Replace.md

@@ -130,8 +130,11 @@
         relation.<RelationSubfieldName>.<RelationSubfieldName>.<String>|
         sshkeys.<SSHkeysSubfieldName>.<SSHkeysSubfieldName>.<String>|
         website.<WebsiteSubfieldName>.<WebsiteSubfieldName>.<String>
+<UserReplacementField> ::=
+        photourl
 <Tag> ::= <String>
 <UserReplacement> ::=
+        (field:<UserReplacementField>)|
         (field:<UserReplacementFieldSubfield>)|
         (field:<UserReplacementFieldSubfieldMatchSubfield>)|
         (schema:<SchemaName>.<FieldName>)|
@@ -145,7 +148,7 @@ This command allows simple text replacement in the message.
 ```
 gam sendemail <EmailAddressEntity> [from <UserItem>] [replyto <EmailAddress>]
         [cc <EmailAddressEntity>] [bcc <EmailAddressEntity>] [singlemessage [<Boolean>]]
-        [subject <String>] [message <String>|(file <FileName> [charset <CharSet>])]
+        [subject <String>] [message <String>|(file <FileName> [charset <Charset>])]
         (replace <Tag> <String>)* [html [<Boolean>]] (attach <FileName>)*
 ```
 * Every instance of `{Tag}` in the message will be replaced by `<String>`.
@@ -162,30 +165,30 @@ These commands allow simple text replacement in the message/signature as well as
 ```
 gam create user <EmailAddress> <UserAttribute>*
         [notify <EmailAddress>] [subject <String>]
-            [message <String>|(file <FileName> [charset <CharSet>])] [html [<Boolean>]]
+            [message <String>|(file <FileName> [charset <Charset>])] [html [<Boolean>]]
             (replace <Tag> <UserReplacement>)*
 gam update user <UserItem> <UserAttribute>
         [updateprimaryemail <RegularExpression> <EmailReplacement>]
-        [updateoufromgroup <FileName> [charset <CharSet>]
+        [updateoufromgroup <FileName> [charset <Charset>]
             [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
             [fields <FieldNameList>] [keyfield <FieldName>] [datafield <FieldName>]]
         [clearschema <SchemaName>] [clearschema <SchemaName>.<FieldName>]
         [createifnotfound] [notify <EmailAddress>] [subject <String>]
-            [message <String>|(file <FileName> [charset <CharSet>])] [html [<Boolean>]]
+            [message <String>|(file <FileName> [charset <Charset>])] [html [<Boolean>]]
             (replace <Tag> <UserReplacement>)*
 
 gam <UserTypeEntity> draft message (<SMTPDateHeader> <Time>)*
         (<SMTPHeader> <String>)* (header <String> <String>)*
         (addlabel <LabelName>)* [labels <LabelNameList>]
-        (textmessage|message <String>)|(textfile|file <FileName> [charset <CharSet>])
-        (htmlmessage <String>)|(htmlfile <FileName> [charset <CharSet>])
-        (replace <Tag> <UserReplacement>)* (attach <FileName> [charset <CharSet>])*
+        (textmessage|message <String>)|(textfile|file <FileName> [charset <Charset>])
+        (htmlmessage <String>)|(htmlfile <FileName> [charset <Charset>])
+        (replace <Tag> <UserReplacement>)* (attach <FileName> [charset <Charset>])*
 gam <UserTypeEntity> import message (<SMTPDateHeader> <Time>)*
         (<SMTPHeader> <String>)*
         (header <String> <String>)*
         (addlabel <LabelName>)*
-        (textmessage <String>)|(textfile <FileName> [charset <CharSet>])
-        (htmlmessage <String>)|(htmlfile <FileName> [charset <CharSet>])
+        (textmessage <String>)|(textfile <FileName> [charset <Charset>])
+        (htmlmessage <String>)|(htmlfile <FileName> [charset <Charset>])
         (replace <Tag> <UserReplacement>)* (attach <FileName>)*
         [deleted [<Boolean>]] [nevermarkspam [<Boolean>]]
         [processforcalendar [<Boolean>]]
@@ -194,19 +197,19 @@ gam <UserTypeEntity> insert message
         (<SMTPHeader> <String>)*
         (header <String> <String>)*
         (addlabel <LabelName>)*
-        (textmessage <String>)|(textfile <FileName> [charset <CharSet>])
-        (htmlmessage <String>)|(htmlfile <FileName> [charset <CharSet>])
+        (textmessage <String>)|(textfile <FileName> [charset <Charset>])
+        (htmlmessage <String>)|(htmlfile <FileName> [charset <Charset>])
         (replace <Tag> <UserReplacement>)* (attach <FileName>)*
         [deleted [<Boolean>]]
 
 gam <UserTypeEntity> [create|add] sendas <EmailAddress> <String>
-        [signature|sig <String>|(file <FileName> [charset <CharSet>])
+        [signature|sig <String>|(file <FileName> [charset <Charset>])
         (replace <Tag> <UserReplacement>)*]
         [html [<Boolean>]] [replyto <EmailAddress>]
         [default] [treatasalias <Boolean>]
 
 gam <UserTypeEntity> update sendas <EmailAddress>
-        [name <String>] [signature|sig <String>|(file <FileName> [charset <CharSet>])
+        [name <String>] [signature|sig <String>|(file <FileName> [charset <Charset>])
         (replace <Tag> <UserReplacement>)*]
         [html [<Boolean>]] [replyto <EmailAddress>] [default] [treatasalias <Boolean>]
 
@@ -216,7 +219,7 @@ gam <UserTypeEntity> signature|sig <String>|(file <FileName> [charset <Charset>]
         [default] [primary] [treatasalias <Boolean>]
 
 gam <UserTypeEntity> vacation <TrueValues> subject <String>
-        [message <String>|(file <FileName> [charset <CharSet>])]
+        [message <String>|(file <FileName> [charset <Charset>])]
         (replace <Tag> <UserReplacement>)*
         [html [<Boolean>]] [contactsonly [<Boolean>]] [domainonly [<Boolean>]]
         [startdate <Date>|Started] [enddate <Date>|NotSpecified]

docs/Users-Calendars-Events.md

@@ -585,13 +585,6 @@ gam <UserTypeEntity> delete events <UserCalendarEntity> [doit] [<EventNotificati
 ```
 No events are deleted unless you specify the `doit` option; omit `doit` to verify that you properly selected the events to delete.
 
-## Move calendar events to another calendar
-Generally you won't move all events from one calendar to another; typically, you'll move events created by the event creator
-using `matchfield creatoremail <RegularExpression>` in conjunction with other `<EventSelectProperty>` and `<EventMatchProperty>` options.
-```
-gam <UserTypeEntity> move events <UserCalendarEntity> [<EventEntity>] destination|to <CalendarItem> [<EventNotificationAttribute>]
-```
-
 ## Empty calendar trash
 A user signed in to Google Calendar can empty the calendar trash but there is no direct API support for this operation.
 To empty the calendar trash a temporary calendar is created, the deleted events are moved to the temporary calendar and then the temporary calendar is deleted.

docs/Users-Chat.md

@@ -59,7 +59,7 @@ Google requires that you have a Chat Bot configured in order to use the Chat API
 
 <ChatContent> ::=
         ((text <String>)|
-         (textfile <FileName> [charset <CharSet>])|
+         (textfile <FileName> [charset <Charset>])|
          (gdoc <UserGoogleDoc>)|
          (gcsdoc <StorageBucketObjectName>))
 
@@ -82,7 +82,7 @@ Google requires that you have a Chat Bot configured in order to use the Chat API
 ```
 gam <UserTypeEntity> create chatspace
         [type <ChatSpaceType>]
-        [externalusersrallowed <Boolean>]
+        [externalusersallowed <Boolean>]
         [members <UserTypeEntity>]
         [displayname <String>]
         [description <String>] [guidelines <String>]
@@ -301,7 +301,7 @@ gam <UserTypeEntity> create chatmessage <ChatSpace>
 ```
 Specify the text of the message: `<ChatContent>`
 * `text <String>` - The message is `<String>`
-* `textfile <FileName> [charset <CharSet>]` - The message is read from a local file
+* `textfile <FileName> [charset <Charset>]` - The message is read from a local file
 * `gdoc <UserGoogleDoc>` - The message is read from a Google Doc.
 * `gcsdoc <StorageBucketObjectName>` - The message is read from a Google Cloud Storage file.
 
@@ -344,7 +344,7 @@ gam <UserTypeEntity> update chatmessage name <ChatMessage>
 ```
 Specify the text of the message: `<ChatContent>`
 * `text <String>` - The message is `<String>`
-* `textfile <FileName> [charset <CharSet>]` - The message is read from a local file
+* `textfile <FileName> [charset <Charset>]` - The message is read from a local file
 * `gdoc <UserGoogleDoc>` - The message is read from a Google Doc.
 * `gcsdoc <StorageBucketObjectName>` - The message is read from a Google Cloud Storage file.
 

docs/Users-Contacts.md

@@ -55,7 +55,7 @@ As of version `6.14.04`, There is now support for managing "Other Contacts".
 
 <NoteContent> ::=
         ((<String>)|
-         (file <FileName> [charset <CharSet>])|
+         (file <FileName> [charset <Charset>])|
          (gdoc <UserGoogleDoc>)|
          (gcsdoc <StorageBucketObjectName>)
 ```

docs/Users-Drive-Copy-Move.md

@@ -16,6 +16,7 @@
 ## API documentation
 * https://developers.google.com/drive/api/v3/reference/files
 * https://support.google.com/a/answer/7374057
+* https://support.google.com/a/users/answer/7338880
 
 ## Definitions
 * [`<DriveFileEntity>`](Drive-File-Selection)
@@ -66,11 +67,13 @@ gam <UserTypeEntity> copy drivefile <DriveFileEntity>
         [summary [<Boolean>]] [showpermissionmessages [<Boolean>]]
         [<DriveFileParentAttribute>]
         [mergewithparent [<Boolean>]] [recursive [depth <Number>]]
+        <DriveFileCopyAttribute>*
+        [skipids <DriveFileEntity>]
         [copysubfiles [<Boolean>]] [filenamematchpattern <RegularExpression>]
              [filemimetype [not] <MimeTypeList>]
+        [copysubfilesownedby any|me|others]
         [copysubfolders [<Boolean>]] [foldernamematchpattern <RegularExpression>]
         [copysubshortcuts [<Boolean>]] [shortcutnamematchpattern <RegularExpression>]
-        <DriveFileCopyAttribute>*
         [duplicatefiles overwriteolder|overwriteall|duplicatename|uniquename|skip]
         [duplicatefolders merge|duplicatename|uniquename|skip]
         [copiedshortcutspointtocopiedfiles [<Boolean>]]
@@ -124,6 +127,9 @@ The `depth <Number>` argument controls which files or folders within the top fol
 * `depth 0` - the files or folders in the top folder are copied, no descendants of folders are copied.
 * `depth N` - the files and folders within the top folder and those files and folders N levels below the top folder are copied.
 
+### This option handles special cases where you want to prevent selected files/folders from being copied.
+* `skipids <DriveFileEntity>` - Do not copy files/folders with the specified IDs.
+
 ### By default, when recursively copying a top folder, all sub files, folders and shortcuts are copied, subject to the `depth` option.
 You can specify whether sub files, folders and shortcuts are copied. If sub folders are not copied, their contents are not copied.
 * `copysubfiles false` - Sub files are not copied
@@ -148,6 +154,11 @@ You can specify `<RegularExpression>` patterns that limit the items copied based
 * `foldernamematchpattern <RegularExpression>` - Only folders whose name matches `<RegularExpression>` are copied
 * `shortcutnamematchpattern <RegularExpression>` - Only shortcuts whose name matches `<RegularExpression>` are copied
 
+### By default, when copying sub files, all files, regardless of ownership, are copied.
+* `copysubfilesownedby any` - All files, regardless of ownership, are copied.
+* `copysubfilesownedby me` - Only files owned by `<UserTypeEntity>`  are copied.
+* `copysubfilesownedby others` - Only files not owned by `<UserTypeEntity>`  are copied.
+
 ### Specify a new name for the file/folder
 * `newfilename <DriveFileName>` - The copied file/folder will be named `<DriveFileName>`
   * If `stripnameprefix <String>` is specified, `<String>` will be stripped from the front of `<DriveFileName>`
@@ -197,6 +208,7 @@ In previous versions, copying shortcuts caused an error because shortcuts can't
 
 If a shortcut in the source structure points to a file/folder that is not in the source structure:
  * The shortcut is re-created to point to the original file/folder.
+
 If a shortcut in the source structure points to a file/folder that is in the source structure:
 * `copiedshortcutspointtocopiedfiles` omitted or `copiedshortcutspointtocopiedfiles true` - The shortcut is re-created to point to the copied file/folder.
 * `copiedshortcutspointtocopiedfiles false` - The shortcut is re-created to point to the original file/folder.

docs/Users-Drive-Files-Display.md

@@ -24,6 +24,7 @@
 - [Display file share counts](#display-file-share-counts)
 - [Display file tree](#display-file-tree)
   - [File selection starting point for Display file tree](#file-selection-starting-point-for-display-file-tree)
+- [Display file parent tree](#display-file-parent-tree)
 - [Display file list](#display-file-list)
   - [File selection by name and entity shortcuts for Display file list](#file-selection-by-name-and-entity-shortcuts-for-display-file-list)
   - [File selection starting point for Display file list](#file-selection-starting-point-for-display-file-list)
@@ -33,6 +34,7 @@
 
 ## API documentation
 * https://developers.google.com/drive/api/v3/reference/files
+* https://support.google.com/a/answer/6105699
 
 ## Definitions
 * [`<DriveFileEntity>`](Drive-File-Selection)
@@ -79,6 +81,7 @@
         canaddfolderfromanotherdrive|
         canaddmydriveparent|
         canchangecopyrequireswriterpermission|
+        canchangecopyrequireswriterpermissionrestriction|
         canchangedomainusersonlyrestriction|
         canchangedrivebackground|
         canchangedrivemembersonlyrestriction|
@@ -96,11 +99,14 @@
         canmanagemembers|
         canmodifycontent|
         canmodifycontentrestriction|
+        canmodifyeditorcontentrestriction|
         canmodifylabels|
+        canmodifyownercontentrestriction|
         canmovechildrenoutofdrive|
         canmovechildrenoutofteamdrive|
         canmovechildrenwithindrive|
         canmovechildrenwithinteamdrive|
+        canmoveitemintodrive|
         canmoveitemintoteamdrive|
         canmoveitemoutofdrive|
         canmoveitemoutofteamdrive|
@@ -112,6 +118,7 @@
         canreadrevisions|
         canreadteamdrive|
         canremovechildren|
+        canremovecontentrestriction|
         canremovemydriveparent|
         canrename|
         canrenamedrive|
@@ -946,6 +953,40 @@ Show file tree starting at the folder named "Middle Folder" and 2 levels deeper
 ```
 gam user testuser show filetree select drivefilename "Middle Folder" depth 2
 ```
+## Display file parent tree
+Print the parent tree of file/folder.
+```
+gam <UserTypeEntity> print fileparenttree <DriveFileEntity> [todrive <ToDriveAttribute>*]
+        [stripcrsfromname]
+```
+### Examples
+```
+# My Drive file
+$ gam user user@domain.com print fileparenttree 1tDGtnaBXc1qx_9NjBSZOUUNZ7FoRc2u6
+User: user@domain.com, Print 1 File Parent Tree
+Owner,id,name,parentId,depth,isRoot
+user@domain.com,1tDGtnaBXc1qx_9NjBSZOUUNZ7FoRc2u6,Bottom Folder,1HvAJtmQ2KZrKJhzY8aeZVScHhZ3HBJLp,4,False
+user@domain.com,1HvAJtmQ2KZrKJhzY8aeZVScHhZ3HBJLp,Middle Folder,1CVqOJJLNQtxX4QEPdpDfbkjiq1oUsxne,3,False
+user@domain.com,1CVqOJJLNQtxX4QEPdpDfbkjiq1oUsxne,TopCopy,0AHYenC8f12ALUk9PVA,2,False
+user@domain.com,0AHYenC8f12ALUk9PVA,My Drive,,1,True
+
+# Shared Drive file
+$ gam user user@domain.com print fileparenttree 1kAHa7Q801KXRF1DfoofNlW05UWDzddhVP_u_L2xGfFQ
+User: user@domain.com, Print 1 File Parent Tree
+Owner,id,name,parentId,depth,isRoot
+user@domain.com,1kAHa7Q801KXRF1DfoofNlW05UWDzddhVP_u_L2xGfFQ,Middle Doc,1DShPJ6iG1TnNsgiBn-Oy1OVE2BahYlPr,4,False
+user@domain.com,1DShPJ6iG1TnNsgiBn-Oy1OVE2BahYlPr,Middle Folder,1s3g64uWfuQrpXRPf82B-bWCB5VuyrOmQ,3,False
+user@domain.com,1s3g64uWfuQrpXRPf82B-bWCB5VuyrOmQ,Top Folder,0AL5LiIe4dqxZUk9PVA,2,False
+user@domain.com,0AL5LiIe4dqxZUk9PVA,TS Shared Drive 1,,1,True
+
+# Shared with Me file
+$ gam user user@domain.com print fileparenttree 1S2D97pyG1vAil4hgNnGGLD2ldCwTOzXUM9D7XbeUv0s
+User: user@domain.com, Print 1 File Parent Tree
+Owner,id,name,parentId,depth,isRoot
+user@domain.com,1S2D97pyG1vAil4hgNnGGLD2ldCwTOzXUM9D7XbeUv0s,GooGoo,0B0NlVEBUkz-hfjVudlF4VHlYYWlmOEdCUUxDaHdLdXhJTF84YWQwbmpRWmZ3Qm0wZnpHSGs,2,False
+user@domain.com,0B0NlVEBUkz-hfjVudlF4VHlYYWlmOEdCUUxDaHdLdXhJTF84YWQwbmpRWmZ3Qm0wZnpHSGs,FooBar,,1,False
+```
+
 ## Display file list
 Display a list of file/folder details in CSV format.
 ```
@@ -1041,6 +1082,12 @@ Use the following option to select a subset of files based on their permissions.
 * `<PermissionMatch>* [<PermissionMatchAction>]` - Use permission matching to select files
 
 ## File selection starting point for Display file list
+You can limit the selection for files on a specific Shared drive.
+Any query will be applied to the Shared drive.
+```
+select <SharedDriveEntity>
+```
+
 You can specify a specific folder from which to select files.
 ```
 select <DriveFileEntity> [selectsubquery <QueryDriveFile>]

docs/Users-Drive-Files-Manage.md

@@ -25,7 +25,9 @@
 * https://developers.google.com/drive/api/v3/ref-single-parent
 * https://developers.google.com/drive/api/v3/shared-drives-diffs
 * https://developers.google.com/drive/api/v3/shortcuts
+* https://support.google.com/a/answer/6105699
 * https://support.google.com/a/answer/7374057
+* https://support.google.com/a/users/answer/7338880
 * https://developers.google.com/docs/api/reference/rest
 
 ## Definitions
@@ -462,7 +464,7 @@ gam <UserTypeEntity> update drivefile <DriveFileEntity> [copy] [returnidonly|ret
         [stripnameprefix <String>]
         <DriveFileUpdateAttribute>*
         [(gsheet|csvsheet <SheetEntity> [clearfilter])|(addsheet <String>)]
-        [charset <CharSet>] [columndelimiter <Character>]
+        [charset <Charset>] [columndelimiter <Character>]
 ```
 By default, an existing file's attributes are updated.
 
@@ -492,7 +494,8 @@ From the Google Drive API documentation.
 By default, Google assigns the current time to the attribute `modifiedTime`; you can assign your own value
 with `modifiedtime <Time>`.
 
-The option `preservefiletimes`, when used with `localfile <FileName>`, will set the `modifiedTime` attribute from the local file.
+* `preservefiletimes localfile <FileName>` - `modifiedTime` of `<DriveFileEntity>` is set to that of `localfile <FileName>`
+* `preservefiletimes` - `modifiedTime` of `<DriveFileEntity>` retains its current value
 
 These are the naming rules when updating from a local file:
 * `update drivefile drivefilename "GoogleFile.csv" localfile "NewLocalFile.csv"` - Google Drive file "GoogleFile.csv" is renamed "NewLocalFile.csv"
@@ -513,7 +516,7 @@ You can update a specific sheet within a Google spreadsheet or add a new sheet t
 * `gsheet|csvsheet id:<Number>` - Specify a sheet by ID in a Google Sheets file to be updated
   * `clearfilter` - When updating a sheet, this option causes GAM to clear the spreadsheet basic filter so hidden data will be overwritten
 * `addsheet <String>` - Specify a sheet name to be added to the Google Sheets file
-* `charset <CharSet>` - Specify the character set of the local file; if not specified, the value of `charset` from `gam.cfg` will be used
+* `charset <Charset>` - Specify the character set of the local file; if not specified, the value of `charset` from `gam.cfg` will be used
 * `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
 If you want the Google spreadsheet to retain its name, specify: `retainname localfile LocalFile.csv`.
 

docs/Users-Drive-Ownership.md

@@ -107,9 +107,9 @@ By default, files in the trash are not transferred.
 Specify order of file processing.
 * `(orderby <DriveOrderByFieldName> [ascending|descending])*`
 
-These options handle special cases where you want to prevent ownership from being transferred for selected files.
-* `skipids <DriveFileEntity>` - Do not transfer ownership for files with the specified IDs.
-* `skipusers <UserTypeEntity>` - Do not transfer ownership for files owned by the specified users.
+These options handle special cases where you want to prevent ownership from being transferred for selected files/folders.
+* `skipids <DriveFileEntity>` - Do not transfer ownership for files/folders with the specified IDs.
+* `skipusers <UserTypeEntity>` - Do not transfer ownership for files/folders owned by the specified users.
 
 By default, only files owned by users in the same domain as the claiming user have their ownership transferred.
 * `subdomains <DomainNameEntity>` - Transfer ownership for files in the selected sub-domains.

docs/Users-Drive-Permissions.md

@@ -7,6 +7,8 @@
 - [Manage file permissions/sharing](#manage-file-permissionssharing)
 - [Display file permissions/sharing](#display-file-permissionssharing)
 - [Delete all ACLs except owner from a file](#delete-all-acls-except-owner-from-a-file)
+- [Change shares to User1 to shares to User2](#change-shares-to-user1-to-shares-to-user2)
+
 ## API documentation
 * https://developers.google.com/drive/api/v3/reference/permissions
 * https://developers.google.com/drive/api/v3/ref-single-parent
@@ -303,3 +305,15 @@ Inspect Permissions.csv, verify that you want to proceed.
 ```
 gam config csv_input_row_drop_filter "permission.role:regex:(owner)|(organizer)" csv ./Permissions.csv gam user "~Owner" delete drivefileacl "~id" "id:~~permission.id~~"
 ```
+
+## Change shares to User1 to shares to User2
+```
+# Get files shared to User1
+gam redirect csv ./FilesSharedWithU1.csv user user1@domain.com print filelist choose sharedwithme fields id,name,mimetype,owners.emailaddress
+# For each of these files, get the sharing settings for U1
+gam redirect csv ./FilesSharedWithU1Settings.csv multiprocess csv FilesSharedWithU1.csv gam user "~owners.0.emailAddress" print drivefileacls "~id" pm emailaddress "~Owner" em
+# For each of these files, delete the share to User1
+gam redirect stdout ./DeleteU1Sharing.txt multiprocess redirect stderr stdout csv FilesSharedWithU1Settings.csv gam user "~Owner" delete drivefileacl "~id" "~permissions.0.emailAddress"
+# For each of these files, add the share to User2 with the same role that User1 had
+gam redirect stdout ./AddUser2Sharing.txt multiprocess redirect stderr stdout csv FilesSharedWithU1Settings.csv gam user "~Owner" create drivefileacl "~id" user user2@domain.com role "~permissions.0.role"
+```

docs/Users-Drive-Transfer.md

@@ -37,8 +37,8 @@ gam <UserTypeEntity> transfer drive <UserItem> [select <DriveFileEntity>]
 By default, all of the source users files will be transferred except those in the trash. If you want to transfer a subset of
 the source users files, use the `select <DriveFileEntity>` option.
 
-This option handles special cases where you want to prevent selected files from being transferred.
-* `skipids <DriveFileEntity>` - Do not transfer files with the specified IDs.
+This option handles special cases where you want to prevent selected files/folders from being transferred.
+* `skipids <DriveFileEntity>` - Do not transfer files/folders with the specified IDs.
 
 You can specify the access that the source user retains to the files that it owns.
 If no option is  specified, the source user retains no access to the transferred files.

docs/Users-Gmail-Messages-Threads.md

@@ -17,7 +17,7 @@
   - [Display a selected set of messages](#display-a-selected-set-of-messages)
   - [Choose information to display](#choose-information-to-display)
     - [Display message content](#display-message-content)
-    - [Display message count](#display-message-count)
+    - [Display message counts](#display-message-counts)
     - [Display label counts](#display-label-counts)
   - [Print only options](#print-only-options)
   - [Show only options](#show-only-options)
@@ -171,7 +171,7 @@
         (file|textfile|htmlfile <FileName> [charset <Charset>])|
         (gdoc|ghtml <UserGoogleDoc>)|
         (gcsdoc|gcshtml <StorageBucketObjectName>)|
-        (emlfile <FileName>)
+        (emlfile <FileName> [charset <Charset>]))
 ```
 ## Message queries with dates
 ```
@@ -204,21 +204,24 @@ You can also replace ` ` with `-` but it doesn't seem to be required.
 
 * `query "label:Foo -Bar-"` - Select messages with label `Foo (Bar)`
 
+You can have GAM do the substitutions for you with the `matchlabel <LabelName>` option.
+* `matchlabel "Foo (Bar)"` is converted to `query "label:Foo -Bar-"`
+
 ## Draft messages
 Add a draft message to a user's mailbox.
 ```
 gam <UserTypeEntity> draft message
-        <MessageContent> (replace <Tag> <UserReplacement>)*
         (<SMTPDateHeader> <Time>)* (<SMTPHeader> <String>)* (header <String> <String>)*
-        (attach <FileName> [charset <CharSet>])*
+        <MessageContent>
+        (attach <FileName> [charset <Charset>])*
         (embedimage <FileName> <String>)*
 ```
 `<MessageContent>` is the message, there are five ways to specify it:
 * `message|textmessage|htmlmessage <String>` - Use `<String>` as the message
-* `file|htmlfile <FileName> [charset <CharSet>]` - Read the message from `<FileName>`
+* `file|htmlfile <FileName> [charset <Charset>]` - Read the message from `<FileName>`
 * `gdoc|ghtml <UserGoogleDoc>` - Read the message from `<UserGoogleDoc>`
 * `gcsdoc|gcshtml <StorageBucketObjectName>` - Read the message from the Google Cloud Storage file `<StorageBucketObjectName>`
-* `emlfile <FileName>` - Read the message from the EML message file `<FileName>`. SMTP headers specified in the command will replace those in the message file.
+* `emlfile <FileName> [charset <Charset>]` - Read the message from the EML message file `<FileName>`. SMTP headers specified in the command will replace those in the message file. The default `chatser` is `ascii`.
 
 The `<SMTPDateHeader> <Time>` argument  requires `<Time>` values which will be converted to RFC2822 dates. If you have these headers with values that
 are not in `<Time>` format, use the argument `header <SMTPDateHeader> <String>`.
@@ -241,20 +244,20 @@ Your command line will have: `embedimage file1.jpg image1` embedimage file2.jpg
 Import a message into a user's mailbox, with standard email delivery scanning and classification similar to receiving via SMTP.
 ```
 gam <UserTypeEntity> import message
-        <MessageContent> (replace <Tag> <UserReplacement>)*
         (<SMTPDateHeader> <Time>)* (<SMTPHeader> <String>)* (header <String> <String>)*
         (addlabel <LabelName>)* [labels <LabelNameList>]
-        (attach <FileName> [charset <CharSet>])*
+        <MessageContent>
+        (attach <FileName> [charset <Charset>])*
         (embedimage <FileName> <String>)*
         [deleted [<Boolean>]] [checkspam [<Boolean>]] [processforcalendar [<Boolean>]]
 ```
 
 `<MessageContent>` is the message, there are five ways to specify it:
 * `message|textmessage|htmlmessage <String>` - Use `<String>` as the message
-* `file|htmlfile <FileName> [charset <CharSet>]` - Read the message from `<FileName>`
+* `file|htmlfile <FileName> [charset <Charset>]` - Read the message from `<FileName>`
 * `gdoc|ghtml <UserGoogleDoc>` - Read the message from `<UserGoogleDoc>`
 * `gcsdoc|gcshtml <StorageBucketObjectName>` - Read the message from the Google Cloud Storage file `<StorageBucketObjectName>`
-* `emlfile <FileName>` - Read the message from the EML message file `<FileName>`. SMTP headers specified in the command will replace those in the message.
+* `emlfile <FileName> [charset <Charset>]` - Read the message from the EML message file `<FileName>`. SMTP headers specified in the command will replace those in the message. The default `chatser` is `ascii`.
 
 When `emlfile` is not specified:
 * If `to` is not specified, it is set to the user email addresses in `<UserTypeEntity>`.
@@ -289,20 +292,20 @@ Your command line will have: `embedimage file1.jpg image1` embedimage file2.jpg
 Insert a message into a user's mailbox similar to IMAP APPEND, bypassing most scanning and classification.
 ```
 gam <UserTypeEntity> insert message
-        <MessageContent> (replace <Tag> <UserReplacement>)*
         (<SMTPDateHeader> <Time>)* (<SMTPHeader> <String>)* (header <String> <String>)*
         (addlabel <LabelName>)* [labels <LabelNameList>]
-        (attach <FileName> [charset <CharSet>])*
+        <MessageContent>
+        (attach <FileName> [charset <Charset>])*
         (embedimage <FileName> <String>)*
         [deleted [<Boolean>]]
 ```
 
 `<MessageContent>` is the message, there are five ways to specify it:
 * `message|textmessage|htmlmessage <String>` - Use `<String>` as the message
-* `file|htmlfile <FileName> [charset <CharSet>]` - Read the message from `<FileName>`
+* `file|htmlfile <FileName> [charset <Charset>]` - Read the message from `<FileName>`
 * `gdoc|ghtml <UserGoogleDoc>` - Read the message from `<UserGoogleDoc>`
 * `gcsdoc|gcshtml <StorageBucketObjectName>` - Read the message from the Google Cloud Storage file `<StorageBucketObjectName>`
-* `emlfile <FileName>` - Read the message from the EML message file `<FileName>`. SMTP headers specified in the command will replace those in the message file.
+* `emlfile <FileName> [charset <Charset>]` - Read the message from the EML message file `<FileName>`. SMTP headers specified in the command will replace those in the message file. The default `chatser` is `ascii`.
 
 When `emlfile` is not specified:
 * If `to` is not specified, it is set to the user email addresses in `<UserTypeEntity>`.
@@ -539,7 +542,7 @@ The `dateheaderconverttimezone [<Boolean>]>` option converts `<SMTPDateHeader>`
 * `showsize` - Display the message size
 * `showsnippet` - Display the message snippet
 
-### Display message count and optionally cumulative message size
+### Display message counts
 * `countsonly` - Display the count of the number of messages
 * `showsize` - Display the cumulative message size
 

docs/Users-Gmail-Send-As-Signature-Vacation.md

@@ -13,6 +13,7 @@
 ## API documentation
 * https://developers.google.com/gmail/api/reference/rest/v1/users.settings.sendAs
 * https://developers.google.com/gmail/api/v1/reference/users/settings
+* https://support.google.com/a/answer/1710338
 
 ## Definitions
 * [`<UserTypeEntity>`](Collections-of-Users)
@@ -81,12 +82,14 @@ of the sendas address.
 
 `<SendAsContent>` is the signature, there are four ways to specify it:
 * `sig|signature|htmlsig <String>` - Use `<String>` as the signature
-* `file|htmlfile <FileName> [charset <CharSet>]` - Read the signature from `<FileName>`
+* `file|htmlfile <FileName> [charset <Charset>]` - Read the signature from `<FileName>`
 * `gdoc|ghtml <UserGoogleDoc>` - Read the signature from `<UserGoogleDoc>`
 * `gcsdoc|gcshtml <StorageBucketObjectName>` - Read the signature from the Google Cloud Storage file `<StorageBucketObjectName>`
 
 The `default` option sets `<EmailAddress>` as the default sendas address for the user.
 
+For `treatasalias`, see: https://support.google.com/a/answer/1710338
+
 You can allow users to send mail through an external SMTP server when configuring a sendas address hosted outside your email domains. You must enable
 this capability in Admin Console/Apps/Google Workspace/Gmail/Advanced settings/End User Access/Allow per-user outbound gateways.
 
@@ -139,12 +142,14 @@ gam <UserTypeEntity> signature|sig
 ```
 `<SignatureContent>` is the signature, there are four ways to specify it:
 * `<String>` - Use `<String>` as the signature
-* `file|htmlfile <FileName> [charset <CharSet>]` - Read the signature from `<FileName>`
+* `file|htmlfile <FileName> [charset <Charset>]` - Read the signature from `<FileName>`
 * `gdoc|ghtml <UserGoogleDoc>` - Read the signature from `<UserGoogleDoc>`
 * `gcsdoc|gcshtml <StorageBucketObjectName>` - Read the signature from the Google Cloud Storage file `<StorageBucketObjectName>`
 
 The `default` option sets `<EmailAddress>` as the default sendas address for the user.
 
+For `treatasalias`, see: https://support.google.com/a/answer/1710338
+
 When `<UserTypeEntity>` specifies an alias, the `primary` option causes the primary
 email address signature rather than the alias signature to be set.
 
@@ -211,7 +216,7 @@ gam <UserTypeEntity> vacation <Boolean> subject <String>
 ```
 `<VacationMessageContent>` is the vacation message, there are four ways to specify it:
 * `message|textmessage|htmlmessage <String>` - Use `<String>` as the vacation message
-* `file|htmlfile <FileName> [charset <CharSet>]` - Read the vacation message from `<FileName>`
+* `file|htmlfile <FileName> [charset <Charset>]` - Read the vacation message from `<FileName>`
 * `gdoc|ghtml <UserGoogleDoc>` - Read the vacation message from `<UserGoogleDoc>`
 * `gcsdoc|gcshtml <StorageBucketObjectName>` - Read the vacation message from the Google Cloud Storage file `<StorageBucketObjectName>`
 

docs/Users-Keep.md

@@ -42,7 +42,7 @@ gam user user@domain.com check serviceaccount
 
 <NoteContent> ::=
         ((text <String>)|
-         (textfile <FileName> [charset <CharSet>])|
+         (textfile <FileName> [charset <Charset>])|
          (gdoc <UserGoogleDoc>)|
          (gcsdoc <StorageBucketObjectName>)|
          <JSONData>)
@@ -90,7 +90,7 @@ gam <UserTypeEntity> create note [title <String>]
 ```
 `<NoteContent>` is the note text, there are four ways to specify it:
 * `message|textmessage|htmlmessage <String>` - Use `<String>` as the note text
-* `file|htmlfile <FileName> [charset <CharSet>]` - Read the note text from `<FileName>`
+* `file|htmlfile <FileName> [charset <Charset>]` - Read the note text from `<FileName>`
 * `gdoc|ghtml <UserGoogleDoc>` - Read the note text from `<UserGoogleDoc>`
 * `gcsdoc|gcshtml <StorageBucketObjectName>` - Read the note text from the Google Cloud Storage file `<StorageBucketObjectName>`
 

docs/Users-Shared-Drives.md

@@ -15,6 +15,7 @@
 - [Display Shared Drive access](#display-shared-drive-access)
   - [Display Shared Drive access for specific Shared Drives](#display-shared-drive-access-for-specific-shared-drives)
   - [Display Shared Drive access for selected Shared Drives](#display-shared-drive-access-for-selected-shared-drives)
+- [Change User1 Shared Drive access to User2](#change-user1-shared-drive-access-to-user2)
 - [Display empty folders on a Shared Drive](#display-empty-folders-on-a-shared-drive)
 - [Delete empty folders on a Shared Drive](#delete-empty-folders-on-a-shared-drive)
 - [Empty the trash on a Shared Drive](#empty-the-trash-on-a-shared-drive)
@@ -25,6 +26,7 @@
 * https://developers.google.com/drive/v3/web/manage-teamdrives#managing_team_drives_for_domain_administrators
 * https://support.google.com/a/answer/7374057
 * https://workspaceupdates.googleblog.com/2022/05/shared-drives-in-organizational-units-open-beta.html
+* https://support.google.com/a/users/answer/7338880
 
 ## Query documentation
 * https://developers.google.com/drive/api/v3/search-shareddrives
@@ -428,6 +430,17 @@ This command must be issued by a user with Shared Drive permission role organize
 gam <UserTypeEntity> print emptydrivefolders [todrive <ToDriveAttribute>*]
         select <SharedDriveEntity>
 ```
+
+## Change User1 Shared Drive access to User2
+```
+# Get Shared Drives for User1
+gam redirect csv ./U1SharedDrives.csv user user1@domain.com print shareddriveacls pm emailaddress user1@domain.com em oneitemperrow
+# For each of those Shared Drives, delete User1 access
+gam redirect stdout ./DeleteU1SharedDriveAccess.txt multiprocess redirect stderr stdout gam delete drivefileacl "~id" "~permission.emailAddress"
+# For each of those Shared Drives, add User2 with the same role that User1 had
+gam redirect stdout ./AddU2SharedDriveAccess.txt multiprocess redirect stderr stdout gam create drivefileacl "~id" user user2@domain.com role "~permission.role"
+```
+
 ## Delete empty folders on a Shared Drive
 This command must be issued by a user with Shared Drive permission role organizer.
 ```

docs/Users.md

@@ -38,7 +38,7 @@
     - [Print domain counts for users in a specific domain and/or selected by a query](#print-domain-counts-for-users-in-a-specific-domain-and-or-selected-by-a-query)
     - [Print domain counts for users specified by `<UserTypeEntity>`](#print-domain-counts-for-users-specified-by-usertypeentity)
 - [Print user list](#print-user-list)
-- [Display user count](#display-user-count)
+- [Display user counts](#display-user-counts)
 
 ## API documentation
 * https://developers.google.com/admin-sdk/directory/reference/rest/v1/users
@@ -385,7 +385,7 @@ If subject is not specified, the following value will be used:
 
 `<NotifyMessageContent>` is the message, there are four ways to specify it:
 * `message|textmessage|htmlmessage <String>` - Use `<String>` as the message
-* `file|htmlfile <FileName> [charset <CharSet>]` - Read the message from `<FileName>`
+* `file|htmlfile <FileName> [charset <Charset>]` - Read the message from `<FileName>`
 * `gdoc|ghtml <UserGoogleDoc>` - Read the message from `<UserGoogleDoc>`
 * `gcsdoc|gcshtml <StorageBucketObjectName>` - Read the message from the Google Cloud Storage file `<StorageBucketObjectName>`
 
@@ -554,6 +554,14 @@ gam redirect stdout CreateUsers.log multiprocess redirect stderr stdout csv Crea
      gam create user "~useremail" firstname "~firstname" lastname "~lastname" ou "~ou" password "~password"
      notify "~~notifyemail~~,helpdesk@domain.com"
 ```
+### Create users in bulk in OU with forced 2FA, notify each user and send a second email with backup codes. Log each step.
+OU needs to be already set with forced 2FA, else you can't create backup codes in step 2.
+These three commands should be run in sequence, as commands two and three are reliant on the previous command being run.
+```
+gam redirect stdout CreateUsers.log multiprocess redirect stderr stdout csv CreateUsers.csv gam create user "~useremail" firstname "~firstname" lastname "~lastname" ou "~ou" password random notify "~~notifyemail"
+gam redirect stdout UpdateUsers.log multiprocess redirect stderr stdout csv CreateUsers.csv gam user ~useremail update backupcodes
+gam redirect stdout SendBackupCodes.log multiprocess redirect stderr stdout csv CreateUsers.csv gam user ~useremail print backupcodes | gam csv - gam sendemail "~notifyemail" subject "Backup codes for 2FA login" message "~verificationCodes"
+```
 
 ## Specify a user's attributes with JSON data
 When creating a user, you may have a set of attributes that you'd like to assign to the user without having to specify
@@ -597,7 +605,7 @@ If the mailbox is setup, a zero return code is returned; if the retries are exha
 gam update user <UserItem> [ignorenullpassword] <UserAttribute>*
         [verifynotinvitable|alwaysevict] [noactionifalias]
         [updateprimaryemail <RegularExpression> <EmailReplacement>]
-        [updateoufromgroup <FileName> [charset <CharSet>]
+        [updateoufromgroup <FileName> [charset <Charset>]
             [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
             [fields <FieldNameList>] [keyfield <FieldName>] [datafield <FieldName>]]
         [clearschema <SchemaName>] [clearschema <SchemaName>.<FieldName>]
@@ -618,7 +626,7 @@ gam update user <UserItem> [ignorenullpassword] <UserAttribute>*
 gam update users <UserTypeEntity> [ignorenullpassword] <UserAttribute>*
         [verifynotinvitable|alwaysevict] [noactionifalias]
         [updateprimaryemail <RegularExpression> <EmailReplacement>]
-        [updateoufromgroup <FileName> [charset <CharSet>]
+        [updateoufromgroup <FileName> [charset <Charset>]
             [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
             [fields <FieldNameList>] [keyfield <FieldName>] [datafield <FieldName>]]
         [clearschema <SchemaName>] [clearschema <SchemaName>.<FieldName>]
@@ -639,7 +647,7 @@ gam update users <UserTypeEntity> [ignorenullpassword] <UserAttribute>*
 gam <UserTypeEntity> update users [ignorenullpassword] <UserAttribute>*
         [verifynotinvitable|alwaysevict] [noactionifalias]
         [updateprimaryemail <RegularExpression> <EmailReplacement>]
-        [updateoufromgroup <FileName> [charset <CharSet>]
+        [updateoufromgroup <FileName> [charset <Charset>]
             [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
             [fields <FieldNameList>] [keyfield <FieldName>] [datafield <FieldName>]]
         [clearschema <SchemaName>] [clearschema <SchemaName>.<FieldName>]
@@ -813,7 +821,7 @@ groupz@domain.com,/Path/To/OUz
 No update is performed if a user does not belong to any group in the CSV file or belongs to multiple groups in the CSV file.
 
 ```
-[updateoufromgroup <FileName> [charset <CharSet>]
+[updateoufromgroup <FileName> [charset <Charset>]
     [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
     [fields <FieldNameList>] [keyfield <FieldName>] [datafield <FieldName>]]
 ```
@@ -972,7 +980,9 @@ gam print users [todrive <ToDriveAttribute>*]
         ([domain|domains <DomainNameEntity>] [(query <QueryUser>)|(queries <QueryUserList>)]
          [limittoou <OrgUnitItem>] [deleted_only|only_deleted])
         [orderby <UserOrderByFieldName> [ascending|descending]]
-        [groups|groupsincolumns] [license|licenses|licence|licences]
+        [groups|groupsincolumns]
+        [license|licenses|licence|licences]
+        [onelicenseperrow|onelicenceperrow]
         [schemas|custom|customschemas all|<SchemaNameList>]
         [emailpart|emailparts|username]
         [userview] [allfields|basic|full|(<UserFieldName>*|fields <UserFieldNameList>)]
@@ -995,6 +1005,7 @@ gam print users [todrive <ToDriveAttribute>*] select <UserTypeEntity>
         [orderby <UserOrderByFieldName> [ascending|descending]]
         [groups|groupsincolumns]
         [license|licenses|licence|licences|licensebyuser|licensesbyuser|licencebyuser|licencesbyuser]
+        [onelicenseperrow|onelicenceperrow]
         [(products|product <ProductIDList>)|(skus|sku <SKUIDList>)]
         [schemas|custom|customschemas all|<SchemaNameList>]
         [emailpart|emailparts|username][schemas|custom all|<SchemaNameList>]
@@ -1006,6 +1017,7 @@ gam <UserTypeEntity> print users [todrive <ToDriveAttribute>*]
         [orderby <UserOrderByFieldName> [ascending|descending]]
         [groups|groupsincolumns]
         [license|licenses|licence|licences|licensebyuser|licensesbyuser|licencebyuser|licencesbyuser]
+        [onelicenseperrow|onelicenceperrow]
         [(products|product <ProductIDList>)|(skus|sku <SKUIDList>)]
         [schemas|custom|customschemas all|<SchemaNameList>]
         [emailpart|emailparts|username]
@@ -1039,7 +1051,7 @@ By default, Gam displays fields that only an adminstrator can view.
 
 By default, Gam displays only the primary email address for each user.
 * `allfields|basic` - Display all non custom schema fields for each user.
-* `full` - Display all non custom schema fields and all custom schema fields for each user.
+* `full` - Display all fields including  all custom schema fields for each user.
 * `<UserFieldName>* [fields <UserFieldNameList>]` - Only display selected fields.
 * `schemas|custom all` - Get custom schema information for all schemas.
 * `schemas|custom <SchemaNameList>` - Get custom schema information for a selected list of schemas.
@@ -1050,6 +1062,17 @@ to limit the display of aliases to those that match `<RegularExpression>`.
 By default, the entries in lists of groups and licenses are separated by the `csv_output_field_delimiter` from `gam.cfg`.
 * `delimiter <Character>` - Separate list items with `<Character>`
 
+By default, all licenses for a user are displayed in a list on one row:
+```
+primaryEmail,LicensesCount,Licenses,LicensesDisplay
+user@domain.com,2,1010020020 1010330004,Google Workspace Enterprise Plus Google Voice Standard
+```
+With `onelicenseperrow|onelicenceperrow`, each license is on a separate row:
+```
+primaryEmail,License,LicenseDisplay
+user@domain.com,1010020020,Google Workspace Enterprise Plus
+user@domain.com 1010330004,Google Voice Standard
+```
 In the output, primaryEmail is the always the first column; these options control the sorting of the remaining columns.
 * `allfields|basic|full` - All other columns are sorted by name.
 * `sortheaders [true]` - All other columns are sorted by name.
@@ -1198,9 +1221,31 @@ $ more UsersList.csv
  ["testuser1@domain.org",  "testuser2@domain.org",  "testuser3@domain.org",  "testuser4@domain.org"] 
 ```
 
-## Display user count
+## Display user counts
 Display the number of users in an entity.
 ```
 gam <UserTypeEntity> show count
+gam <UserTypeEntity> print users showitemcountonly
+gam print users select <UserTypeEntity> showitemcountonly
+gam print users
+        ([domain|domains <DomainNameEntity>] [(query <QueryUser>)|(queries <QueryUserList>)]
+         [limittoou <OrgUnitItem>] [deleted_only|only_deleted])|[select <UserTypeEntity>]
+        [issuspended <Boolean>]
+        showitemcountonly
 ```
+Example
+```
+$ gam print users query "orgUnitPath='/Students/Middle School'" showitemcountonly
+Getting all Users that match query (query="orgUnitPath='/Students/Middle School'"), may take some time on a large Google Workspace Account...
+Got 221 Users: aaron-first@domain.com - zoe-last@domain.com
+221
+```
+The `Getting` and `Got` messages are written to stderr, the count is writtem to stdout.
 
+To retrieve the count with `showitemcountonly`:
+```
+Linux/MacOS
+count=$(gam print users query "orgUnitPath='/Students/Middle School'" showitemcountonly)
+Windows PowerShell
+count = & gam print users query "orgUnitPath='/Students/Middle School'" showitemcountonly
+```

docs/Vault-Takeout.md

@@ -37,6 +37,7 @@
 
 ## Definitions
 ```
+<AttendeeStatus> ::= accepted|declined|needsaction|tentative
 <EmailItem> ::= <EmailAddress>|<UniqueID>|<String>
 <EmailItemList> ::= "<EmailItem>(,<EmailItem>)*"
 <EmailAddressList> ::= "<EmailAddess>(,<EmailAddress>)*"
@@ -52,6 +53,7 @@
 <MatterItem> ::= <UniqueID>|<String>
 <MatterState> ::= open|closed|deleted
 <MatterStateList> ::= "<MatterState>(,<MatterState>)*"
+<URLList> ::= "<URL>(,<URL>)*"
 
 <QueryVaultCorpus> ::= <String>
         See: https://developers.google.com/vault/reference/rest/v1/matters.holds#CorpusQuery
@@ -192,12 +194,9 @@ This command can be useful for discovering legacy former employee accounts which
 gam print vaultcounts [todrive <ToDriveAttributes>*]
         matter <MatterItem> corpus mail|groups
         (accounts <EmailAddressEntity>) | (orgunit|org|ou <OrgUnitPath>) | everyone
-        (shareddrives|teamdrives <SharedDriveIDList>) | (rooms <RoomList>)
         [scope <all_data|held_data|unprocessed_data>]
         [terms <String>] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>] [timezone <TimeZone>]
         [excludedrafts <Boolean>]
-        [includerooms <Boolean>]
-        [includeshareddrives|includeteamdrives <Boolean>] [driveversiondate <Date>|<Time>]
         [wait <Integer>]
 ```
 Check the status of a previous count operation with the name from a previous command.
@@ -210,16 +209,18 @@ gam print vaultcounts [todrive <ToDriveAttributes>*]
 ## Create Vault Exports
 Create a Google Vault export request.
 ```
-gam create vaultexport|export matter <MatterItem> [name <String>] corpus drive|mail|groups|hangouts_chat|voice
+gam create vaultexport|export matter <MatterItem> [name <String>] corpus calendar|drive|mail|groups|hangouts_chat|voice
         (accounts <EmailAddressEntity>) | (orgunit|org|ou <OrgUnitPath>) | everyone
-        (shareddrives|teamdrives <SharedDriveIDList>) | (rooms <RoomList>)
-        [scope <all_data|held_data|unprocessed_data>]
+        (shareddrives|teamdrives <SharedDriveIDList>) | (rooms <RoomList>) | (sitesurl <URLList>)
+        [scope all_data|held_data|unprocessed_data]
         [terms <String>] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>] [timezone <TimeZone>]
-        [excludedrafts <Boolean>] [format mbox|pst]
-        [showconfidentialmodecontent <Boolean>] [usenewexport <Boolean>]
-        [includerooms <Boolean>]
-        [covereddata calllogs|textmessages|voicemails]
+        [locationquery <StringList>] [peoplequery <StringList>] [minuswords <StringList>]
+        [responsestatuses <AttendeeStatus>(,<AttendeeStatus>)*] [calendarversiondate <Date>|<Time>]
         [includeshareddrives <Boolean>] [driveversiondate <Date>|<Time>] [includeaccessinfo <Boolean>]
+        [includerooms <Boolean>]
+        [excludedrafts <Boolean>] [format mbox|pst]
+        [showconfidentialmodecontent <Boolean>] [usenewexport <Boolean>] [exportlinkeddrivefiles <Boolean>]
+        [covereddata calllogs|textmessages|voicemails]
         [region any|europe|us] [showdetails|returnidonly]
 ```
 <MatterItem> specifies the matter name or ID the export should be associated with.
@@ -228,7 +229,8 @@ Specify the name of the export:
 * `name <String>` - The export will be named `<String>`
 * `default` - The export will be named `GAM <corpus> Export - <Time>`
 
-Specify the corpus of data, this option is required::
+Specify the corpus of data, this option is required:
+* `calendar`
 * `drive`
 * `mail`
 * `groups`
@@ -241,17 +243,43 @@ Specify the search method, this option is required:
 * `everyone` - Search for all accounts in the organization
 * `shareddrives|teamdrives <SharedDriveIDList>` - Search for all accounts in the Shared Drives specified in `<SharedDriveIDList>`
 * `rooms <RoomList>` - Search in the Room specified in the chat rooms specified in `<RoomList>`
+* `sitesurl <URLList>` - Search the published site URLs of new Google Sites
 
 Specify the scope of data to include in the export:
 * `all_data` - All available data; this is the default
 * `held_data` - Data on Hold
 * `unprocessed_data` - Data not processed
 
+You can specify search terms to limit the scope of data:
+* `terms <String>` - [Vault search](https://support.google.com/vault/answer/2474474)
+
 Specify time limits on the scope of data:
 * `start|starttime <Date>|<Time>` - The start time range for the search query. These timestamps are in GMT and rounded down to the start of the given date.
 * `end|endtime <Date>|<Time>` - The end time range for the search query. These timestamps are in GMT and rounded down to the start of the given date.
 * `timezone <TimeZone>` - The time zone name. It should be an IANA TZ name, such as "America/Los_Angeles"
 
+For `corpus calendar`, you can specify advanced search options:
+* `locationquery <StringList>`
+  * Matches only those events whose location contains all of the words in the given set.
+  * If the string contains quoted phrases, this method only matches those events whose location contain the exact phrase.
+  * Entries in the set are considered in "and".
+  * Word splitting example: ["New Zealand"] vs ["New","Zealand"] "New Zealand": matched by both "New and better Zealand": only matched by the latter.
+* `peoplequery <StringList>`
+  * Matches only those events whose attendees contain all of the words in the given set.
+  * Entries in the set are considered in "and".
+* `minuswords <StringList>`
+  * Matches only those events that do not contain any of the words in the given set in title, description, location, or attendees.
+  * Entries in the set are considered in "or".
+* `responsestatuses <AttendeeStatus>(,<AttendeeStatus>)*
+  * Matches only events for which the custodian gave one of these responses. If the set is empty, there will be no filtering on responses.
+* `calendarversiondate <Date>|<Time>`
+  * Search the current version of the Calendar event, but export the contents of the last version saved before 12:00 AM UTC on the specified date.
+  * Enter the date in UTC.
+
+For `corpus calendar`, you can specify the format of the exported data:
+* `format ics` - Export in ICS format, this is the default
+* `format pst` - Export in PST format
+
 For `corpus drive`, you can specify advanced search options:
 * `driveversiondate <Date>|<Time>` - Search the versions of the Drive file as of the reference date. These timestamps are in GMT and rounded down to the given date.
 * `includeshareddrives False` - Do not include Shared Drives in the search, this is the default.
@@ -265,9 +293,6 @@ For `corpus hangouts_chat` you can specify advanced search options:
 * `includerooms False` - Do not include rooms, this is the default
 * `includerooms True` - Include rooms
 
-For `corpus mail`, you can specify search terms to limit the scope of data:
-* `terms <String>` - [Vault search](https://support.google.com/vault/answer/2474474)
-
 For `corpus mail`, you can specify whether to exclude draft messages:
 * `excludedrafts False` - Do not exclude drafts, this is the default
 * `excludedrafts True` - Exclude drafts
@@ -280,9 +305,13 @@ For `corpus mail`, you can specify whether to use the new export system:
 * `usenewexport false` - Do not use the new export system
 * `usenewexport true` - Use the new export system
 
+For `corpus mail`, you can specify whether to enable exporting linked Drive files:
+* `exportlinkeddrivefiles false` - Do not export linked Drive files
+* `exportlinkeddrivefiles true` - Export linked Drive files
+
 See: https://support.google.com/vault/answer/4388708#new_gmail_export&zippy=%2Cfebruary-new-gmail-export-system-available
 
-For `corpus mail`, `corpus groups` and `corpus hangouts_chat`, you can specify the format of the exported data:
+For `corpus mail`, `corpus groups`, `corpus hangouts_chat`and `corpus voice`, you can specify the format of the exported data:
 * `format mbox` - Export in MBOX format, this is the default
 * `format pst` - Export in PST format
 

docs/Version-and-Help.md

@@ -1,13 +1,13 @@
-
+\
 # Version and Help
 
 Print the current version of Gam with details
 ```
 gam version
-GAMADV-XTD3 6.66.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.67.24 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
-Python 3.12.0 64-bit final
-MacOS Monterey 12.7 x86_64
+Python 3.12.1 64-bit final
+MacOS Sonoma 14.2.1 x86_64
 Path: /Users/Admin/bin/gamadv-xtd3
 Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, customer_id: my_customer, domain.com
 Time: 2023-06-02T21:10:00-07:00
@@ -16,10 +16,10 @@ Time: 2023-06-02T21:10:00-07:00
 Print the current version of Gam with details and time offset information
 ```
 gam version timeoffset
-GAMADV-XTD3 6.66.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.67.24 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
-Python 3.12.0 64-bit final
-MacOS Monterey 12.7 x86_64
+Python 3.12.1 64-bit final
+MacOS Sonoma 14.2.1 x86_64
 Path: /Users/Admin/bin/gamadv-xtd3
 Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, customer_id: my_customer, domain.com
 Your system time differs from www.googleapis.com by less than 1 second
@@ -28,17 +28,17 @@ Your system time differs from www.googleapis.com by less than 1 second
 Print the current version of Gam with extended details and SSL information
 ```
 gam version extended
-GAMADV-XTD3 6.66.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.67.24 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
-Python 3.12.0 64-bit final
-MacOS Monterey 12.7 x86_64
+Python 3.12.1 64-bit final
+MacOS Sonoma 14.2.1 x86_64
 Path: /Users/Admin/bin/gamadv-xtd3
 Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, customer_id: my_customer, domain.com
 Time: 2023-06-02T21:10:00-07:00
 Your system time differs from admin.googleapis.com by less than 1 second
 OpenSSL 3.1.1 30 May 2023
 cryptography 41.0.1
-filelock 3.12.0
+filelock 3.12.1
 google-api-python-client 2.88.0
 google-auth-httplib2 0.1.0
 google-auth-oauthlib 1.0.0
@@ -65,7 +65,7 @@ MacOS High Sierra 10.13.6 x86_64
 Path: /Users/Admin/bin/gamadv-xtd3
 Version Check:
   Current: 5.35.08
-   Latest: 6.66.03
+   Latest: 6.67.24
 echo $?
 1
 ```
@@ -73,7 +73,7 @@ echo $?
 Print the current version number without details
 ```
 gam version simple
-6.66.03
+6.67.24
 ```
 In Linux/MacOS you can do:
 ```
@@ -83,10 +83,10 @@ echo $VER
 Print the current version of Gam and address of this Wiki
 ```
 gam help
-GAM 6.66.03 - https://github.com/taers232c/GAMADV-XTD3
+GAM 6.67.24 - https://github.com/taers232c/GAMADV-XTD3
 Ross Scroggs <ross.scroggs@gmail.com>
-Python 3.12.0 64-bit final
-MacOS Monterey 12.7 x86_64
+Python 3.12.1 64-bit final
+MacOS Sonoma 14.2.1 x86_64
 Path: /Users/Admin/bin/gamadv-xtd3
 Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, customer_id: my_customer, domain.com
 Time: 2023-06-02T21:10:00-07:00

docs/gam.cfg.md

@@ -1043,7 +1043,7 @@ $gam csv NewGooUsers.csv gam select goo create user ~Email firstname ~FirstName
 ```
 The gam loop command and the select and redirect arguments can be combined to perform powerful operations in a single command line.
 ```
-gam loop (-|<FileName>) [charset <CharSet>] (matchfield|skipfield <FieldName> <RegularExpression>)* gam <GAM argument list>
+gam loop (-|<FileName>) [charset <Charset>] (matchfield|skipfield <FieldName> <RegularExpression>)* gam <GAM argument list>
 ```
 Suppose you have the following CSV file, InfoDomains.csv:
 ```

src/GamCommands.txt

@@ -480,8 +480,7 @@ If an item contains spaces, it should be surrounded by ".
         (message|textmessage|htmlmessage <String>)|
         (file|textfile|htmlfile <FileName> [charset <Charset>])|
         (gdoc|ghtml <UserGoogleDoc>)|
-        (gcsdoc|gcshtml <StorageBucketObjectName>)|
-        (emlfile <FileName>)
+        (gcsdoc|gcshtml <StorageBucketObjectName>)
 <MessageID> ::= <String>
 <Namespace> ::= <String>
 <NotesName> ::= notes/<String>
@@ -728,6 +727,7 @@ If an item contains spaces, it should be surrounded by ".
 <TasklistIDTaskIDList> ::= "<TasklistIDTaskID>(,<TasklistIDTaskID>)*"
 <ThreadIDList> ::= "<ThreadID>(,<ThreadID>)*"
 <TimeList> ::= "<Time>(,<Time>)*"
+<URLList> ::= "<URL>(,<URL>)*"
 <UserList> ::= "<UserItem>(,<UserItem>)*"
 <YouTubeChannelIDList> ::= "<YouTubeChannelID>(,<YouTubeChannelID>)*"
 
@@ -1457,7 +1457,7 @@ gam print alertfeedback [todrive <ToDriveAttribute>*] [alert <AlertID>] [filter
 gam create|add alias|aliases <EmailAddressEntity> user|group|target <UniqueID>|<EmailAddress>
         [verifynotinvitable]
 gam update alias|aliases <EmailAddressEntity> user|group|target <UniqueID>|<EmailAddress>
-        [notargetverify]
+        [notargetverify] [waitafterdelete <Integer>]
 gam delete alias|aliases [user|group|target] <EmailAddressEntity>
 gam remove aliases|nicknames <EmailAddress> user|group <EmailAddressEntity>
 gam <UserTypeEntity> delete alias|aliases
@@ -2164,8 +2164,10 @@ gam print chromehistory releases [todrive <ToDriveAttribute>*]
 
 <CrOSAction> ::=
         deprovision_different_model_replace|
+        deprovision_different_model_replacement|
         deprovision_retiring_device|
         deprovision_same_model_replace|
+        deprovision_same_model_replacement|
         deprovision_upgrade_transfer|
         disable|
         reenable|
@@ -2173,7 +2175,9 @@ gam print chromehistory releases [todrive <ToDriveAttribute>*]
         pre_provisioned_reenable
 
 gam update cros <CrOSEntity> action <CrOSAction> [acknowledge_device_touch_requirement]
+        [actionbatchsize <Integer>]
 gam <CrOSTypeEntity> update action <CrOSAction> [acknowledge_device_touch_requirement]
+        [actionbatchsize <Integer>]
 
 <CrOSCommand>
         reboot|
@@ -2291,6 +2295,7 @@ gam print cros [todrive <ToDriveAttribute>*]
         [timerangeorder ascending|descending] [showdvrsfp]
         [sortheaders]
         [formatjson [quotechar <Character>]]
+        [showitemcountonly]
 
 Print fields for specified CrOS devices.
 
@@ -2303,6 +2308,7 @@ gam print cros [todrive <ToDriveAttribute>*] select <CrOSTypeEntity>
         [timerangeorder ascending|descending] [showdvrsfp]
         [sortheaders]
         [formatjson [quotechar <Character>]]
+        [showitemcountonly]
 
 gam <CrOSTypeEntity> print cros [todrive <ToDriveAttribute>*]
         [orderby <CrOSOrderByFieldName> [ascending|descending]]
@@ -2313,6 +2319,7 @@ gam <CrOSTypeEntity> print cros [todrive <ToDriveAttribute>*]
         [timerangeorder ascending|descending] [showdvrsfp]
         [sortheaders]
         [formatjson [quotechar <Character>]]
+        [showitemcountonly]
 
 The first column will always be deviceId; the remaining field names will be sorted if allfields, basic, full or sortheaders is specified;
 otherwise, the remaining field names will appear in the order specified.
@@ -2475,7 +2482,7 @@ gam print crostelemetry [todrive <ToDriveAttribute>*]
 
 gam create chromepolicyimage <ChromePolicyImageSchemaName> <FileName>
 
-gam update chromepolicy
+gam update chromepolicy [convertcrnl]
         (<SchemaName> ((<Field> <Value>)+ | <JSONData>))+
         ou|org|orgunit <OrgUnitItem> [(printerid <PrinterID>)|(appid <AppID>)]
 gam delete chromepolicy
@@ -2897,13 +2904,16 @@ gam print courses [todrive <ToDriveAttribute>*]
         [owneremail] [owneremailmatchpattern <RegularExpression>]
         [alias|aliases|aliasesincolumns [delimiter <Character>]]
         [show all|students|teachers] [countsonly]
+        [timefilter creationtime|updatetime] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>]
         [fields <CourseFieldNameList>] [skipfields <CourseFieldNameList>]
         [formatjson [quotechar <Character>]]
-        [timefilter creationtime|updatetime] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>]
+        [showitemcountonly]
+
 gam print course-participants [todrive <ToDriveAttribute>*]
         (course|class <CourseEntity>)*|([teacher <UserItem>] [student <UserItem>] [states <CourseStateList>])
         [show all|students|teachers]
         [formatjson [quotechar <Character>]]
+        [showitemcountonly]
 
 <CourseAnnouncementFieldName> ::=
         alternatelink|
@@ -3136,6 +3146,7 @@ gam info datatransfer|transfer <TransferID>
 gam print datatransfers|transfers [todrive <ToDriveAttribute>*]
         [olduser|oldowner <UserItem>] [newuser|newowner <UserItem>]
         [status completed|failed|inprogress] [delimiter <Character>]
+        (addcsvdata <FieldName> <String>)*
 gam show datatransfers|transfers
         [olduser|oldowner <UserItem>] [newuser|newowner <UserItem>]
         [status completed|failed|inprogress] [delimiter <Character>]
@@ -3611,6 +3622,7 @@ gam print groups [todrive <ToDriveAttribute>*]
         [memberemaildisplaypattern|memberemailskippattern <RegularExpression>]
         [convertcrnl] [delimiter <Character>] [sortheaders]
         [formatjson [quotechar <Character>]]
+        [showitemcountonly]
 
 gam print grouptree <GroupEntity> [todrive <ToDriveAttribute>*]
         [showparentsaslist [<Boolean>]] [delimiter <Character>]
@@ -3647,6 +3659,7 @@ gam print group-members [todrive <ToDriveAttribute>*]
         [userfields <UserFieldNameList>]
         [(recursive [noduplicates])|includederivedmembership] [nogroupemail]
         [peoplelookup|(peoplelookupuser <EmailAddress>)]
+        [unknownname <String>] [cachememberinfo [Boolean]]
         [formatjson [quotechar <Character>]]
 gam show group-members
         [([domain|domains <DomainNameEntity>] ([member|showownedby <EmailItem>]|[(query <QueryGroup>)|(queries <QueryGroupList>)]))|
@@ -3730,6 +3743,7 @@ gam print cigroups [todrive <ToDriveAttribute>*]
         [memberemaildisplaypattern|memberemailskippattern <RegularExpression>]
         [convertcrnl] [delimiter <Character>]
         [formatjson [quotechar <Character>]]
+        [showitemcountonly]
 
 <CIGroupMembersFieldName> ::=
         createtime
@@ -3854,6 +3868,7 @@ gam print devices [todrive <ToDriveAttribute>*]
         [all|company|personal|nocompanydevices|nopersonaldevices]
         [nodeviceusers]
         [formatjson [quotechar <Character>]]
+        [showitemcountonly]
 
 <DeviceUserAction> ::=
         approve|
@@ -3877,6 +3892,7 @@ gam print deviceusers [todrive <ToDriveAttribute>*]
         <DeviceUserFieldName>* [fields <DeviceUserFieldNameList>]
         [orderby <DeviceOrderByFieldName> [ascending|descending]]
         [formatjson [quotechar <Character>]]
+        [showitemcountonly]
 
 gam info deviceuserstate <DeviceUserEntity> [clientid <String>]
 gam update deviceuserstate <DeviceUserEntity> [clientid <String>]
@@ -4019,6 +4035,7 @@ gam print mobile [todrive <ToDriveAttribute>*]
         [basic|full|allfields] <MobileFieldName>* [fields <MobileFieldNameList>]
         [delimiter <Character>] [appslimit <Number>] [oneappperrow] [listlimit <Number>]
         [formatjson [quotechar <Character>]]
+        [showitemcountonly]
 
 # Organizational Units
 
@@ -4061,6 +4078,7 @@ gam print orgs|ous [todrive <ToDriveAttribute>*]
         [allfields|<OrgUnitFieldName>*|(fields <OrgUnitFieldNameList>)] [convertcrnl] [batchsuborgs [<Boolean>]]
         [mincroscount <Number>] [maxcroscount <Number>]
         [minusercount <Number>] [maxusercount <Number>]
+        [showitemcountonly]
 gam show orgtree [fromparent <OrgUnitItem>] [batchsuborgs [<Boolean>]]
 
 # Printers
@@ -4162,7 +4180,7 @@ gam report <ActivityApplicationName> [todrive <ToDriveAttribute>*]
         [filtertime.* <Time>] [filter|filters <String>]
         [event|events <EventNameList>] [ip <String>]
         [groupidfilter <String>]
-        [maxactivities <Number>] [maxresults <Number>]
+        [maxactivities <Number>] [maxevents <Number>] [maxresults <Number>]
         [countsonly [summary] [eventrowfilter]]
         (addcsvdata <FieldName> <String>)* [shownoactivities]
 
@@ -4354,6 +4372,7 @@ gam print resources [todrive <ToDriveAttribute>*] [allfields|<ResourceFieldName>
         [query <String>]
         [acls] [noselfowner] [calendar] [convertcrnl]
         [formatjson [quotechar <Character>]]
+        [showitemcountonly]
 
 <CalendarACLRole> ::= editor|freebusy|freebusyreader|owner|reader|writer|none
 <CalendarACLScope> ::= <EmailAddress>|user:<EmailAdress>|group:<EmailAddress>|domain:<DomainName>|domain|default
@@ -4407,6 +4426,12 @@ gam print schema|schemas [todrive <ToDriveAttribute>*]
 
 # Send Email
 
+<MessageContent> ::=
+        (message|textmessage|htmlmessage <String>)|
+        (file|textfile|htmlfile <FileName> [charset <Charset>])|
+        (gdoc|ghtml <UserGoogleDoc>)|
+        (gcsdoc|gcshtml <StorageBucketObjectName>)|
+
 gam sendemail [recipient|to] <RecipientEntity>
         [from <EmailAddress>] [mailbox <EmailAddress>] [replyto <EmailAddress>]
         [cc <RecipientEntity>] [bcc <RecipientEntity>] [singlemessage]
@@ -4853,8 +4878,11 @@ gam <UserTypeEntity> show teamdriveacls
         relation.<RelationSubfieldName>.<RelationSubfieldName>.<String>|
         sshkeys.<SSHkeysSubfieldName>.<SSHkeysSubfieldName>.<String>|
         website.<WebsiteSubfieldName>.<WebsiteSubfieldName>.<String>
+<UserReplacementField> ::=
+        photourl
 <Tag> ::= <String>
 <UserReplacement> ::=
+        (field:<UserReplacementField>)|
         (field:<UserReplacementFieldSubfield>)|
         (field:<UserReplacementFieldSubfieldMatchSubfield>)|
         (schema:<SchemaName>.<FieldName>)|
@@ -4894,25 +4922,25 @@ gam show vaultmatters|matters [matterstate <MatterStateList>]
 gam print vaultcounts [todrive <ToDriveAttributes>*]
         matter <MatterItem> corpus mail|groups
         (accounts <EmailAddressEntity>) | (orgunit|org|ou <OrgUnitPath>) | everyone
-        (shareddrives|teamdrives <SharedDriveIDList>) | (rooms <RoomList>)
         [scope [all_data|held_data|unprocessed_data]]
         [terms <String>] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>] [timezone <TimeZone>]
         [excludedrafts <Boolean>]
-        [includerooms <Boolean>]
-        [includeshareddrives|includeteamdrives <Boolean>] [driveversiondate <Date>|<Time>]
         [wait <Integer>]
 gam print vaultcounts [todrive <ToDriveAttributes>*]
         matter <MatterItem> operation <String> [wait <Integer>]
 
-gam create vaultexport|export matter <MatterItem> [name <String>] corpus drive|mail|groups|hangouts_chat|voice
+gam create vaultexport|export matter <MatterItem> [name <String>] corpus calendar|drive|mail|groups|hangouts_chat|voice
         (accounts <EmailAddressEntity>) | (orgunit|org|ou <OrgUnitPath>) | everyone
-        (shareddrives|teamdrives <SharedDriveIDList>) | (rooms <RoomList>)
-        [scope [all_data|held_data|unprocessed_data]]
+        (shareddrives|teamdrives <SharedDriveIDList>) | (rooms <RoomList>) | (sitesurl <URLList>)
+        [scope all_data|held_data|unprocessed_data]
         [terms <String>] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>] [timezone <TimeZone>]
+        [locationquery <StringList>] [peoplequery <StringList>] [minuswords <StringList>]
+        [responsestatuses <AttendeeStatus>(,<AttendeeStatus>)*] [calendarversiondate <Date>|<Time>]
+        [includeshareddrives <Boolean>] [driveversiondate <Date>|<Time>] [includeaccessinfo <Boolean>]
+        [includerooms <Boolean>]
         [excludedrafts <Boolean>] [format mbox|pst]
-        [showconfidentialmodecontent <Boolean>] [usenewexport <Boolean>]
-        [includerooms <Boolean>] [covereddata calllogs|textmessages|voicemails]
-        [includeshareddrives|includeteamdrives <Boolean>] [driveversiondate <Date>|<Time>] [includeaccessinfo <Boolean>]
+        [showconfidentialmodecontent <Boolean>] [usenewexport <Boolean>] [exportlinkeddrivefiles <Boolean>]
+        [covereddata calllogs|textmessages|voicemails]
         [region any|europe|us] [showdetails|returnidonly]
 gam delete vaultexport|export <ExportItem> matter <MatterItem>
 gam delete vaultexport|export <MatterItem> <ExportItem>
@@ -5320,6 +5348,7 @@ gam print users [todrive <ToDriveAttribute>*]
         [orderby <UserOrderByFieldName> [ascending|descending]]
         [groups|groupsincolumns]
         [license|licenses|licence|licences|licensebyuser|licensesbyuser|licencebyuser|licencesbyuser]
+        [onelicenseperrow|onelicenceperrow]
         [(products|product <ProductIDList>)|(skus|sku <SKUIDList>)]
         [schemas|custom|customschemas all|<SchemaNameList>]
         [emailpart|emailparts|username]
@@ -5327,6 +5356,7 @@ gam print users [todrive <ToDriveAttribute>*]
         [delimiter <Character>] [sortheaders [<Boolean>]] [scalarsfirst [<Boolean>]]
         [formatjson [quotechar <Character>]] [quoteplusphonenumbers]
         [issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
+        [showitemcountonly]
 
 Print fields for specified users.
 
@@ -5334,6 +5364,7 @@ gam print users [todrive <ToDriveAttribute>*] select <UserTypeEntity>
         [orderby <UserOrderByFieldName> [ascending|descending]]
         [groups|groupsincolumns]
         [license|licenses|licence|licences|licensebyuser|licensesbyuser|licencebyuser|licencesbyuser]
+        [onelicenseperrow|onelicenceperrow]
         [(products|product <ProductIDList>)|(skus|sku <SKUIDList>)]
         [schemas|custom|customschemas all|<SchemaNameList>]
         [emailpart|emailparts|username]
@@ -5341,11 +5372,13 @@ gam print users [todrive <ToDriveAttribute>*] select <UserTypeEntity>
         [delimiter <Character>] [sortheaders [<Boolean>]] [scalarsfirst [<Boolean>]]
         [formatjson [quotechar <Character>]] [quoteplusphonenumbers]
         [issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
+        [showitemcountonly]
 
 gam <UserTypeEntity> print users [todrive <ToDriveAttribute>*]
         [orderby <UserOrderByFieldName> [ascending|descending]]
         [groups|groupsincolumns]
         [license|licenses|licence|licences|licensebyuser|licensesbyuser|licencebyuser|licencesbyuser]
+        [onelicenseperrow|onelicenceperrow]
         [(products|product <ProductIDList>)|(skus|sku <SKUIDList>)]
         [schemas|custom|customschemas all|<SchemaNameList>]
         [emailpart|emailparts|username]
@@ -5353,6 +5386,7 @@ gam <UserTypeEntity> print users [todrive <ToDriveAttribute>*]
         [delimiter <Character>] [sortheaders [<Boolean>]] [scalarsfirst [<Boolean>]]
         [formatjson [quotechar <Character>]] [quoteplusphonenumbers]
         [issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
+        [showitemcountonly]
 
 The first column will always be primaryEmail; the remaining field names will be sorted if allfields, basic, full or sortheaders is specified;
 otherwise, the remaining field names will appear in the order specified.
@@ -5747,7 +5781,7 @@ gam <UserTypeEntity> create workinglocation
         (home|
          (custom <String>)|
          (office <String> [building|buildingid <String>] [floor|floorname <String>]
-           		  [section|floorsection <String>] [desk|deskcode <String>]))
+                          [section|floorsection <String>] [desk|deskcode <String>]))
         ((date yyyy-mm-dd)|
          (range yyyy-mm-dd yyyy-mm-dd)|
          (daily yyyy-mm-dd N)|
@@ -6006,10 +6040,13 @@ gam <UserTypeEntity> copy drivefile <DriveFileEntity>
         [summary [<Boolean>]] [showpermissionmessages [<Boolean>]]
         [<DriveFileParentAttribute>]
         [mergewithparent [<Boolean>]] [recursive [depth <Number>]]
-        [copysubfiles [<Boolean>]] [filenamematchpattern <RegularExpression>] [filemimetype [not] <MimeTypeList>]
+        <DriveFileCopyAttribute>*
+        [skipids <DriveFileEntity>]
+        [copysubfiles [<Boolean>]] [filenamematchpattern <RegularExpression>]
+             [filemimetype [not] <MimeTypeList>]
+        [copysubfilesownedby any|me|others]
         [copysubfolders [<Boolean>]] [foldernamematchpattern <RegularExpression>]
         [copysubshortcuts [<Boolean>]] [shortcutnamematchpattern <RegularExpression>]
-        <DriveFileCopyAttribute>*
         [duplicatefiles overwriteolder|overwriteall|duplicatename|uniquename|skip]
         [duplicatefolders merge|duplicatename|uniquename|skip]
         [copiedshortcutspointtocopiedfiles [<Boolean>]]
@@ -6284,6 +6321,7 @@ gam <UserTypeEntity> collect orphans
         canaddfolderfromanotherdrive|
         canaddmydriveparent|
         canchangecopyrequireswriterpermission|
+        canchangecopyrequireswriterpermissionrestriction|
         canchangedomainusersonlyrestriction|
         canchangedrivebackground|
         canchangedrivemembersonlyrestriction|
@@ -6301,11 +6339,14 @@ gam <UserTypeEntity> collect orphans
         canmanagemembers|
         canmodifycontent|
         canmodifycontentrestriction|
+        canmodifyeditorcontentrestriction|
         canmodifylabels|
+        canmodifyownercontentrestriction|
         canmovechildrenoutofdrive|
         canmovechildrenoutofteamdrive|
         canmovechildrenwithindrive|
         canmovechildrenwithinteamdrive|
+        canmoveitemintodrive|
         canmoveitemintoteamdrive|
         canmoveitemoutofdrive|
         canmoveitemoutofteamdrive|
@@ -6317,6 +6358,7 @@ gam <UserTypeEntity> collect orphans
         canreadrevisions|
         canreadteamdrive|
         canremovechildren|
+        canremovecontentrestriction|
         canremovemydriveparent|
         canrename|
         canrenamedrive|
@@ -6608,6 +6650,9 @@ gam <UserTypeEntity> show filetree
         (orderby <DriveFileOrderByFieldName> [ascending|descending])* [delimiter <Character>]
         [stripcrsfromname]
 
+gam <UserTypeEntity> print fileparenttree <DriveFileEntity> [todrive <ToDriveAttribute>*]
+        [stripcrsfromname]
+
 gam <UserTypeEntity> print filelist [todrive <ToDriveAttribute>*]
         [((query <QueryDriveFile>) | (fullquery <QueryDriveFile>) | <DriveFileQueryShortcut>)
             (querytime<String> <Time>)*]
@@ -6913,22 +6958,29 @@ gam <UserTypeEntity> print labels|label [todrive <ToDriveAttribute>*]
         x400-trace
 <SMTPHeaderList> ::= "<SMTPDateHeader>|<SMTPHeader>(,<SMTPDateHeader>|<SMTPHeader>)*"
 
+<MessageContent> ::=
+        (message|textmessage|htmlmessage <String>)|
+        (file|textfile|htmlfile <FileName> [charset <Charset>])|
+        (gdoc|ghtml <UserGoogleDoc>)|
+        (gcsdoc|gcshtml <StorageBucketObjectName>)|
+        (emlfile <FileName> [charset <Charset>])
+
 gam <UserTypeEntity> draft message
-        (<SMTPDateHeader> <Time>)* (<SMTPHeader> <String>)* (header <String> <String>)*
         <MessageContent> (replace <Tag> <UserReplacement>)*
+        (<SMTPDateHeader> <Time>)* (<SMTPHeader> <String>)* (header <String> <String>)*
         (attach <FileName> [charset <CharSet>])*
         (embedimage <FileName> <String>)*
 gam <UserTypeEntity> import message
+        <MessageContent> (replace <Tag> <UserReplacement>)*
         (<SMTPDateHeader> <Time>)* (<SMTPHeader> <String>)* (header <String> <String>)*
         (addlabel <LabelName>)* [labels <LabelNameList>]
-        <MessageContent> (replace <Tag> <UserReplacement>)*
         (attach <FileName> [charset <CharSet>])*
         (embedimage <FileName> <String>)*
         [deleted [<Boolean>]] [checkspam|nevermarkspam [<Boolean>]] [processforcalendar [<Boolean>]]
 gam <UserTypeEntity> insert message
+        <MessageContent> (replace <Tag> <UserReplacement>)*
         (<SMTPDateHeader> <Time>)* (<SMTPHeader> <String>)* (header <String> <String>)*
         (addlabel <LabelName>)* [labels <LabelNameList>]
-        <MessageContent> (replace <Tag> <UserReplacement>)*
         (attach <FileName> [charset <CharSet>])*
         (embedimage <FileName> <String>)*
         [deleted [<Boolean>]]

src/GamUpdate.txt

@@ -2,6 +2,315 @@
 
 Merged GAM-Team version
 
+6.67.24
+
+Fixed bug that caused HTML password notification email messages to be displayed in raw form.
+
+6.67.23
+
+Use local copy of `googleapiclient` to remove static discovery documents to improve performance.
+
+6.67.22
+
+Added `permissionidlist <PermissionIDList>` to `<PermissionMatch>` that allows matching any permission ID in a list.
+
+Added option `exportlinkeddrivefiles <Boolean>` to `gam create vaultexport` that is used with `corpus mail`.
+
+6.67.21
+
+Updated `gam remove aliases <EmailAddress> user|group <EmailAddressEntity>` to give a more informative
+error message when the target/alias combination does not exist.
+```
+Old: User: testsimple@rdschool.org, User Alias: tsalias@rdschool.org, Remove Failed: Invalid Input: resource_id
+New: User: testsimple@rdschool.org, User Alias: tsalias@rdschool.org, Remove Failed: Does not exist
+```
+
+6.67.20
+
+Added option `onelicenseperrow|onelicenceperrow` to `gam print users ... licenses` that causes GAM to print
+a seperate user information row for each license a user is assigned. This makes processing
+the licenses in a script possible and allows better sorting in a CSV File.
+
+By default, all licenses for a user are displayed in a list on one row:
+```
+primaryEmail,LicensesCount,Licenses,LicensesDisplay
+user@domain.com,2,1010020020 1010330004,Google Workspace Enterprise Plus Google Voice Standard
+```
+With `onelicenseperrow|onelicenceperrow`, each license is on a separate row:
+```
+primaryEmail,License,LicenseDisplay
+user@domain.com,1010020020,Google Workspace Enterprise Plus
+user@domain.com 1010330004,Google Voice Standard
+```
+
+6.67.19
+
+Updated `gam create|update user ... notify` to encode the characters `<>&` in the password
+so that they display correctly when the notify message content is HTML.
+
+6.67.18
+
+Cleaned up `Getting/Got` messages for `gam print courses|course-participants`.
+
+6.67.17
+
+Added option `showitemcountonly` to various commands that causes GAM to display the
+item count on stdout; no CSV file is written.
+
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Cloud-Identity-Groups#display-group-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Classroom-Courses#display-course-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Classroom-Membership#display-course-membership-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/ChromeOS-Devices#display-cros-device-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Cloud-Identity-Devices#display-device-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Cloud-Identity-Devices#display-device-user-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Groups#display-group-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Mobile-Devices#display-mobile-device-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Organizational-Units#display-organizational-unit-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Resources#display-resource-counts
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Users#display-user-counts
+
+6.67.16
+
+By default, `gam print group-members membernames` displays `Unknown` for members whose names can not be determined.
+Added option `unknownname <String>` that let's you specify an alternative value.
+
+Further improved performance of `gam print group-members membernames cachememberinfo`.
+
+6.67.15
+
+Update `gam print group-members membernames` to handle the following error:
+```
+ERROR: 400: failedPrecondition - Precondition check failed.
+```
+
+Added option `cachememberinfo [Boolean]` to `gam print group-members` that causes GAM to cache member info
+so that only one API call is made to get information for each user/group. This consumes
+more memory but dramatically reduces the number of API calls.
+
+6.67.14
+
+Updated reseller commands to handle the following error:
+```
+ERROR: 400: invalid - Customer domain [domain.com] is linked to one or more email verified customers, please provide a customer id.
+```
+
+6.67.13
+
+Updated `gam create domain <DomainName>` to handle the following error:
+```
+ERROR: 409: conflict - Domain in request is in use by an email verified customer.
+```
+
+6.67.12
+
+Added option `addcsvdata <FieldName> <String>` to `gam print datatransfers` that adds
+additional columns of data to the CSV file output.
+
+6.67.11
+
+Updated various Gmail related commands to handle this error:
+```
+ERROR: 403: permissionDenied - Insufficient Permission
+```
+when the following service account scopes are selected:
+
+```
+[ ] 23)  Gmail API - Basic Settings (Filters,IMAP, Language, POP, Vacation) - read/write, Sharing Settings (Delegates, Forwarding, SendAs) - read
+[ ] 24)  Gmail API - Full Access (Labels, Messages)
+[ ] 25)  Gmail API - Full Access (Labels, Messages) except delete message
+[*] 26)  Gmail API - Full Access - read only
+[ ] 27)  Gmail API - Send Messages - including todrive
+[ ] 28)  Gmail API - Sharing Settings (Delegates, Forwarding, SendAs) - write
+```
+
+6.67.10
+
+Fixed bug that caused a trap when optional argument `charset <Charset>` was used with `emlfile <FileName>` in `gam <UserTypeEntity> draft|import|insert message`.
+
+6.67.09
+
+Added option `maxevents <Number>` to `gam report <ActivityApplictionName>` that limits
+the number of events displayed for each activity; the default is 0, no limit.
+Setting options `maxactivities 1 maxevents 1 maxresults 1` can be used to as efficiently as possible
+show the most recent activity/event; this can be useful when reporting drive activity for individual drive files.
+
+6.67.08
+
+Added optional argument `charset <Charset>` to `emlfile <FileName>` in `gam <UserTypeEntity> draft|import|insert message`;
+the default value is `ascii`.
+
+6.67.07
+
+Updated `gam <UserTypeEntity> delete message` to handle this error:
+```
+ERROR: 403: permissionDenied - Insufficient Permission
+```
+when the following service account scopes are selected:
+
+```
+[ ] 24)  Gmail API - Full Access (Labels, Messages)
+[*] 25)  Gmail API - Full Access (Labels, Messages) except delete message
+```
+
+6.67.06
+
+Updated commands that create ACLs to handle the following error:
+```
+ERROR: 400: abusiveContentRestriction - Bad Request. User message: "You cannot share this item because it has been flagged as inappropriate."
+```
+
+6.67.05
+
+Updated the following commands:
+```
+gam <UserTypeEntity> create|delete|update delegate
+gam <UserTypeEntity> forward
+gam <UserTypeEntity> create|delete forwardingaddresses
+gam <UserTypeEntity> create|delete sendas
+```
+to handle this error:
+```
+ERROR: 403: permissionDenied - Insufficient Permission
+```
+when the following serice account scope is not enabled:
+```
+[ ] 28)  Gmail API - Sharing Settings (Delegates, Forwarding, SendAs) - write
+```
+
+6.67.04
+
+Updated user attribute `replace <Tag> <UserReplacement>` to allow `field:photourl` which allows
+embedding a link to a user's photo in their signature. Formatting the signature HTML
+to properly display the photo is left to the GAM admin.
+
+6.67.03
+
+Fixed bug introduced in 6.67.02  in `gam <UserTypeEntity> claim ownership` that caused a trap.
+
+6.67.02
+
+Added option `skipids <DriveFileEntity>` to `gam <UserTypeEntity> copy drivefile` that handles special cases
+where you want to prevent selected files/folders from being copied.
+
+Updated commands that create files/folders on Shared Drives to handle the following errors:
+```
+storageQuotaExceeded
+teamDriveFileLimitExceeded
+teamDriveHierarchyTooDeep
+```
+* See: https://support.google.com/a/users/answer/7338880#shared_drives_file_folder_limits
+
+6.67.01
+
+Fixed bug in `gam print vaultcounts` that caused a trap.
+
+6.67.00
+
+Updated `gam <CrOSTypeEntity> update action <CrOSAction>` to use the new API function `batchChangeStatus`
+that replaces the old API function `action`; ChromeOS devices are now processed in batches.
+The batch size defaults to 10, the `actionbatchsize <Integer>` option can be used to set a batch size between 10 and 250.
+
+Updated `gam create vaultexport matter <MatterItem>` to support `corpus calendar`.
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Vault-Takeout#create-vault-exports
+
+6.66.16
+
+Added option `convertcrnl` to `gam update chromepolicy` to properly handle carriage returns (\r) and line feeds (\n)
+in value strings entered on the command line in the `<Field> <Value>` form. 
+```
+gam update chromepolicy convertcrnl chrome.devices.DisabledDeviceReturnInstructions
+    deviceDisabledMessage "Please return device to:\nSchool\n123 Main Street\nAnytown US" ou /Path/to/OU
+```
+
+6.66.15
+
+Added option `copysubfilesownedby any|me|others` to `gam <UserTypeEntity> copy drivefile` that allows
+specification of which source folder sub files to copy based on file ownership; the default is `any`.
+This only applies when files are being copied from a 'My Drive'.
+
+6.66.14
+
+Updated `gam <UserTypeEntity> modify messages` to recognize the following error:
+```
+ERROR: 400: invalid - Invalid label: SENT
+```
+
+Updated `gam update alias <EmailAddressEntity> user|group|target <EmailAddress>`
+to avoid the following problem.
+```
+$ gam update alias testalias@domain.com user testuser
+User Alias: testalias@domain.com, Deleted
+User Alias: testalias@domain.com, User: testuser@domain.com, Update Failed: Duplicate, Email Address: testalias@domain.com
+```
+GAM updates an alias to point to a new target by deleting the alias and then recreating the alias pointing to the new target.
+Unfortunately, if these commands are executed back-to-back; Google generates the `Update Failed: Duplicate` error.
+Now, GAM waits 2 seconds between the delete and the insert which seems to eliminate the problem. If the problem persists,
+the option `waitafterdelete <Integer>` can be used to increase the wait time to a maximum of 10 seconds.
+
+6.66.13
+
+Updated functionality of option `preservefiletimes` in `gam <UserTypeEntity> update drivefile <DriveFileEntity>`.
+
+* Current
+  * `preservefiletimes localfile <FileName>` - `modifiedTime` of `<DriveFileEntity>` is set to that of `localfile <FileName>`
+  * `preservefiletimes` - No effect
+* Updated
+  * `preservefiletimes localfile <FileName>` - `modifiedTime` of `<DriveFileEntity>` is set to that of `localfile <FileName>`
+  * `preservefiletimes` - `modifiedTime` of `<DriveFileEntity>` retains its current value
+
+6.66.12
+
+Upgraded to Python 3.12.1 where possible.
+
+Updated all drive commands to handle the following error:
+```
+ERROR: 401: Active session is invalid. Error code: 4 - authError
+```
+This is due to the Drive SDK API being disabled in the user's OU.
+* See: https://support.google.com/a/answer/6105699
+
+6.66.11
+
+Fixed/improved handling of shortcuts in `gam <UserTypeEntity> transfer drive`.
+
+6.66.10
+
+Updated `gam create datatransfer` to handle the following error:
+```
+ERROR: 401: Active session is invalid. Error code: 4 - authError
+```
+
+6.66.09
+
+Fixed bug in `gam <UserTypeEntity> print filelist ... allfields` that caused a trap
+when `gam.cfg` contained `drive_v3_native_names = False`.
+
+6.66.08
+
+Added additional columns `isBase` and `baseId` to `gam <UserTypeEntity> print fileparenttree`
+to simplify  processing the output in a script.
+
+6.66.07
+
+Fixed bug in `gam <UserTypeEntity> print diskusage` that caused a trap.
+
+6.66.06
+
+Added a command the print the parent tree of file/folder.
+```
+gam <UserTypeEntity> print fileparenttree <DriveFileEntity> [todrive <ToDriveAttribute>*]
+        [stripcrsfromname]
+```
+* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Drive-Files-Display#display-file-parent-tree
+
+6.66.05
+
+Added column `space.name` to `gam <UserTypeEntity> print chatmembers`.
+
+6.66.04
+
+Updated Chat info|show|print commands to display all time fields in local time if specified in `gam.cfg`.
+
 6.66.03
 
 Fixed bug in `gam <UserTypeEntity> print filelist select <DriveFileEntity>` where `stripcrsfromname` was not being

src/gam/gamlib/glaction.py

@@ -49,6 +49,7 @@ class GamAction():
   DELETE = 'dele'
   DELETE_EMPTY = 'delm'
   DELETE_PREVIEW = 'delp'
+  DELETE_SHORTCUT = 'desc'
   DEPROVISION = 'depr'
   DISABLE = 'disa'
   DOWNLOAD = 'down'
@@ -160,11 +161,12 @@ class GamAction():
     COPY_MERGE: ['Copied(Merge)', 'Copy(Merge)'],
     CREATE: ['Created', 'Create'],
     CREATE_PREVIEW: ['Created (Preview)', 'Create (Preview)'],
-    CREATE_SHORTCUT: ['Created Shortcut', 'Create SHORTCUT'],
+    CREATE_SHORTCUT: ['Created Shortcut', 'Create Shortcut'],
     DEDUP: ['Duplicates Deleted', 'Delete Duplicates'],
     DELETE: ['Deleted', 'Delete'],
     DELETE_EMPTY: ['Deleted', 'Delete Empty'],
     DELETE_PREVIEW: ['Deleted (Preview)', 'Delete (Preview)'],
+    DELETE_SHORTCUT: ['Deleted Shortcut', 'Delete Shortcut'],
     DEPROVISION: ['Deprovisioned', 'Deprovision'],
     DISABLE: ['Disabled', 'Disable'],
     DOWNLOAD: ['Downloaded', 'Download'],

src/gam/gamlib/glapi.py

@@ -579,19 +579,19 @@ _SVCACCT_SCOPES = [
    'api': FORMS,
    'subscopes': [],
    'scope': DRIVE_SCOPE},
-  {'name': 'Gmail API - Full Access',
+  {'name': 'Gmail API - Full Access (Labels, Messages)',
    'api': GMAIL,
    'subscopes': [],
    'scope': 'https://mail.google.com/'},
-  {'name': 'Gmail API - Full Access except immediate delete',
+  {'name': 'Gmail API - Full Access (Labels, Messages) except delete message',
    'api': GMAIL,
    'subscopes': [],
    'scope': 'https://www.googleapis.com/auth/gmail.modify'},
-  {'name': 'Gmail API - Basic Settings',
+  {'name': 'Gmail API - Basic Settings (Filters,IMAP, Language, POP, Vacation) - read/write, Sharing Settings (Delegates, Forwarding, SendAs) - read',
    'api': GMAIL,
    'subscopes': [],
    'scope': 'https://www.googleapis.com/auth/gmail.settings.basic'},
-  {'name': 'Gmail API - Settings Sharing (Aliases, Delegates, Forwarding)',
+  {'name': 'Gmail API - Sharing Settings (Delegates, Forwarding, SendAs) - write',
    'api': GMAIL,
    'subscopes': [],
    'scope': 'https://www.googleapis.com/auth/gmail.settings.sharing'},
@@ -691,14 +691,6 @@ DRIVE3_TO_DRIVE2_CAPABILITIES_NAMES_MAP = {
   'canChangeViewersCanCopyContent': 'canChangeRestrictedDownload',
   }
 
-DRIVE3_TO_DRIVE2_CAPABILITIES_TITLES_MAP = {
-  'capabilities.canComment': 'canComment',
-  'capabilities.canReadRevisions': 'canReadRevisions',
-  'capabilities.canCopy': 'copyable',
-  'capabilities.canEdit': 'editable',
-  'capabilities.canShare': 'shareable',
-  }
-
 DRIVE3_TO_DRIVE2_FILES_FIELDS_MAP = {
   'allowFileDiscovery': 'withLink',
   'createdTime': 'createdDate',

src/gam/gamlib/glclargs.py

@@ -577,6 +577,7 @@ class GamCLArgs():
   ARG_FILEDRIVELABELS = 'filedrivelabels'
   ARG_FILEINFO = 'fileinfo'
   ARG_FILELIST = 'filelist'
+  ARG_FILEPARENTTREE = 'fileparenttree'
   ARG_FILEPATH = 'filepath'
   ARG_FILEPATHS = 'filepaths'
   ARG_FILEREVISION = 'filerevision'
@@ -929,6 +930,7 @@ class GamCLArgs():
   OB_ORGUNIT_PATH = 'OrgUnitPath'
   OB_PARAMETER_VALUE = 'ParameterValue'
   OB_PASSWORD = 'Password'
+  OB_PERMISSION_ID_LIST = 'PermissionIDList'
   OB_PHOTO_FILENAME_PATTERN = 'FilenameNamePattern'
   OB_PRINTER_ID = 'PrinterID'
   OB_PRIVILEGE_LIST = 'PrivilegeList'
@@ -990,6 +992,7 @@ class GamCLArgs():
   OB_TRANSFER_ID = 'TransferID'
   OB_URI = 'URI'
   OB_URL = 'URL'
+  OB_URL_LIST = 'URLList'
   OB_USER_ENTITY = 'UserEntity'
   OB_USER_ITEM = 'UserItem'
   OB_USER_NAME = 'UserName'

src/gam/gamlib/glentity.py

@@ -84,7 +84,10 @@ class GamEntity():
   CHANNEL_PRODUCT = 'chpr'
   CHANNEL_SKU = 'chsk'
   CHAT_BOT = 'chbo'
+  CHAT_MANAGER_USER = 'chgu'
   CHAT_MEMBER = 'chme'
+  CHAT_MEMBER_GROUP = 'chmg'
+  CHAT_MEMBER_USER = 'chmu'
   CHAT_MESSAGE = 'chms'
   CHAT_MESSAGE_ID = 'chmi'
   CHAT_SPACE = 'chsp'
@@ -207,6 +210,7 @@ class GamEntity():
   FEATURE = 'feat'
   FIELD = 'fiel'
   FILE = 'file'
+  FILE_PARENT_TREE = 'fptr'
   FILTER = 'filt'
   FORM = 'form'
   FORM_RESPONSE = 'frmr'
@@ -413,9 +417,12 @@ class GamEntity():
     CHANNEL_PRODUCT: ['Channel Products', 'Channel Product'],
     CHANNEL_SKU: ['Channel SKUs', 'Channel SKU'],
     CHAT_BOT: ['Chat BOTs', 'Chat BOT'],
+    CHAT_MANAGER_USER: ['Chat User Managers', 'Chat User Manager'],
     CHAT_MESSAGE: ['Chat Messages', 'Chat Message'],
     CHAT_MESSAGE_ID: ['Chat Message IDs', 'Chat Message ID'],
     CHAT_MEMBER: ['Chat Members', 'Chat Member'],
+    CHAT_MEMBER_GROUP: ['Chat Group Members', 'Chat Group Member'],
+    CHAT_MEMBER_USER: ['Chat User Members', 'Chat User Member'],
     CHAT_SPACE: ['Chat Spaces', 'Chat Space'],
     CHAT_THREAD: ['Chat Threads', 'Chat Thread'],
     CHROME_APP: ['Chrome Applications', 'Chrome Application'],
@@ -536,6 +543,7 @@ class GamEntity():
     FEATURE: ['Features', 'Feature'],
     FIELD: ['Fields', 'Field'],
     FILE: ['Files', 'File'],
+    FILE_PARENT_TREE: ['File Parent Trees', 'File Parent Tree'],
     FILTER: ['Filters', 'Filter'],
     FORM: ['Forms', 'Form'],
     FORM_RESPONSE: ['Form Responses', 'Form Response'],

src/gam/gamlib/glgapi.py

@@ -21,6 +21,7 @@
 """
 # callGAPI throw reasons
 ABORTED = 'aborted'
+ABUSIVE_CONTENT_RESTRICTION = 'abusiveContentRestriction'
 ACCESS_NOT_CONFIGURED = 'accessNotConfigured'
 ALREADY_EXISTS = 'alreadyExists'
 AUTH_ERROR = 'authError'
@@ -155,6 +156,7 @@ TARGET_USER_ROLE_LIMITED_BY_LICENSE_RESTRICTION = 'targetUserRoleLimitedByLicens
 TEAMDRIVE_ALREADY_EXISTS = 'teamDriveAlreadyExists'
 TEAMDRIVE_DOMAIN_USERS_ONLY_RESTRICTION = 'teamDriveDomainUsersOnlyRestriction'
 TEAMDRIVE_TEAM_MEMBERS_ONLY_RESTRICTION = 'teamDriveTeamMembersOnlyRestriction'
+TEAMDRIVE_FILE_LIMIT_EXCEEDED = 'teamDriveFileLimitExceeded'
 TEAMDRIVE_HIERARCHY_TOO_DEEP = 'teamDriveHierarchyTooDeep'
 TEAMDRIVE_MEMBERSHIP_REQUIRED = 'teamDriveMembershipRequired'
 TEAMDRIVES_FOLDER_MOVE_IN_NOT_SUPPORTED = 'teamDrivesFolderMoveInNotSupported'
@@ -199,7 +201,8 @@ COURSE_ACCESS_THROW_REASONS = [NOT_FOUND, INSUFFICIENT_PERMISSIONS, PERMISSION_D
 DRIVE_USER_THROW_REASONS = [SERVICE_NOT_AVAILABLE, AUTH_ERROR, DOMAIN_POLICY]
 DRIVE_ACCESS_THROW_REASONS = DRIVE_USER_THROW_REASONS+[FILE_NOT_FOUND, FORBIDDEN, INTERNAL_ERROR, INSUFFICIENT_FILE_PERMISSIONS, UNKNOWN_ERROR, INVALID]
 DRIVE_COPY_THROW_REASONS = DRIVE_ACCESS_THROW_REASONS+[CANNOT_COPY_FILE, BAD_REQUEST, RESPONSE_PREPARATION_FAILURE, TEAMDRIVES_SHARING_RESTRICTION_NOT_ALLOWED,
-                                                       FIELD_NOT_WRITABLE, RATE_LIMIT_EXCEEDED, USER_RATE_LIMIT_EXCEEDED]
+                                                       FIELD_NOT_WRITABLE, RATE_LIMIT_EXCEEDED, USER_RATE_LIMIT_EXCEEDED,
+                                                       STORAGE_QUOTA_EXCEEDED, TEAMDRIVE_FILE_LIMIT_EXCEEDED, TEAMDRIVE_HIERARCHY_TOO_DEEP]
 DRIVE_GET_THROW_REASONS = DRIVE_USER_THROW_REASONS+[FILE_NOT_FOUND]
 DRIVE3_CREATE_ACL_THROW_REASONS = [BAD_REQUEST, INVALID, INVALID_SHARING_REQUEST, OWNERSHIP_CHANGE_ACROSS_DOMAIN_NOT_PERMITTED, CANNOT_SET_EXPIRATION,
                                    NOT_FOUND, TEAMDRIVE_DOMAIN_USERS_ONLY_RESTRICTION, TEAMDRIVE_TEAM_MEMBERS_ONLY_RESTRICTION,
@@ -213,7 +216,7 @@ DRIVE3_CREATE_ACL_THROW_REASONS = [BAD_REQUEST, INVALID, INVALID_SHARING_REQUEST
                                    FILE_ORGANIZER_NOT_YET_ENABLED_FOR_THIS_TEAMDRIVE,
                                    FILE_ORGANIZER_ON_FOLDERS_IN_SHARED_DRIVE_ONLY,
                                    FILE_ORGANIZER_ON_NON_TEAMDRIVE_NOT_SUPPORTED,
-                                   TEAMDRIVES_FOLDER_SHARING_NOT_SUPPORTED, INVALID_LINK_VISIBILITY]
+                                   TEAMDRIVES_FOLDER_SHARING_NOT_SUPPORTED, INVALID_LINK_VISIBILITY, ABUSIVE_CONTENT_RESTRICTION]
 DRIVE3_GET_ACL_REASONS = DRIVE_USER_THROW_REASONS+[FILE_NOT_FOUND, FORBIDDEN, INTERNAL_ERROR,
                                                    INSUFFICIENT_ADMINISTRATOR_PRIVILEGES, INSUFFICIENT_FILE_PERMISSIONS,
                                                    UNKNOWN_ERROR, INVALID]
@@ -244,7 +247,7 @@ DRIVE3_MODIFY_LABEL_THROW_REASONS = DRIVE_USER_THROW_REASONS+[FILE_NOT_FOUND, NO
                                                               LABEL_MUTATION_ILLEGAL_SELECTION, LABEL_MUTATION_UNKNOWN_FIELD]
 GMAIL_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST]
 GMAIL_LIST_THROW_REASONS = [FAILED_PRECONDITION, PERMISSION_DENIED, INVALID, INVALID_ARGUMENT]
-GMAIL_SMIME_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST, INVALID_ARGUMENT, FORBIDDEN, NOT_FOUND]
+GMAIL_SMIME_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST, INVALID_ARGUMENT, FORBIDDEN, NOT_FOUND, PERMISSION_DENIED]
 GROUP_GET_RETRY_REASONS = [INVALID, SYSTEM_ERROR, SERVICE_NOT_AVAILABLE]
 GROUP_CREATE_THROW_REASONS = [DUPLICATE, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, INVALID, INVALID_INPUT]
 GROUP_GET_THROW_REASONS = [GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR]
@@ -260,6 +263,7 @@ MEMBERS_THROW_REASONS = [GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_AP
 MEMBERS_RETRY_REASONS = [SYSTEM_ERROR, SERVICE_NOT_AVAILABLE]
 ORGUNIT_GET_THROW_REASONS = [INVALID_ORGUNIT, ORGUNIT_NOT_FOUND, BACKEND_ERROR, BAD_REQUEST, INVALID_CUSTOMER_ID, LOGIN_REQUIRED]
 PEOPLE_ACCESS_THROW_REASONS = [SERVICE_NOT_AVAILABLE, FORBIDDEN, PERMISSION_DENIED]
+RESELLER_THROW_REASONS = [BAD_REQUEST, RESOURCE_NOT_FOUND, FORBIDDEN, INVALID]
 SHEETS_ACCESS_THROW_REASONS = DRIVE_USER_THROW_REASONS+[NOT_FOUND, PERMISSION_DENIED, FORBIDDEN, INTERNAL_ERROR, INSUFFICIENT_FILE_PERMISSIONS,
                                                         BAD_REQUEST, INVALID, INVALID_ARGUMENT, FAILED_PRECONDITION]
 TASK_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST, PERMISSION_DENIED, INVALID, NOT_FOUND, ACCESS_NOT_CONFIGURED]
@@ -336,6 +340,8 @@ REASON_MESSAGE_MAP = {
 
 class aborted(Exception):
   pass
+class abusiveContentRestriction(Exception):
+  pass
 class accessNotConfigured(Exception):
   pass
 class alreadyExists(Exception):
@@ -598,6 +604,8 @@ class teamDriveDomainUsersOnlyRestriction(Exception):
   pass
 class teamDriveTeamMembersOnlyRestriction(Exception):
   pass
+class teamDriveFileLimitExceeded(Exception):
+  pass
 class teamDriveHierarchyTooDeep(Exception):
   pass
 class teamDriveMembershipRequired(Exception):
@@ -635,6 +643,7 @@ class userRateLimitExceeded(Exception):
 
 REASON_EXCEPTION_MAP = {
   ABORTED: aborted,
+  ABUSIVE_CONTENT_RESTRICTION: abusiveContentRestriction,
   ACCESS_NOT_CONFIGURED: accessNotConfigured,
   ALREADY_EXISTS: alreadyExists,
   AUTH_ERROR: authError,
@@ -766,6 +775,7 @@ REASON_EXCEPTION_MAP = {
   TEAMDRIVE_ALREADY_EXISTS: teamDriveAlreadyExists,
   TEAMDRIVE_DOMAIN_USERS_ONLY_RESTRICTION: teamDriveDomainUsersOnlyRestriction,
   TEAMDRIVE_TEAM_MEMBERS_ONLY_RESTRICTION: teamDriveTeamMembersOnlyRestriction,
+  TEAMDRIVE_FILE_LIMIT_EXCEEDED: teamDriveFileLimitExceeded,
   TEAMDRIVE_HIERARCHY_TOO_DEEP: teamDriveHierarchyTooDeep,
   TEAMDRIVE_MEMBERSHIP_REQUIRED: teamDriveMembershipRequired,
   TEAMDRIVES_FOLDER_MOVE_IN_NOT_SUPPORTED: teamDrivesFolderMoveInNotSupported,

src/gam/gamlib/glmsgs.py

@@ -338,6 +338,7 @@ NOT_A_MEMBER = 'Not a member'
 NOT_ACTIVE = 'Not Active'
 NOT_ALLOWED = 'Not Allowed'
 NOT_AN_ENTITY = 'Not a {0}'
+NOT_APPROPRIATE = 'Not Appropriate'
 NOT_COMPATIBLE = 'Not Compatible'
 NOT_COPYABLE = 'Not Copyable'
 NOT_COPYABLE_INTO_ITSELF = 'Not copyable into itself'
@@ -430,7 +431,7 @@ SELECTED = 'Selected'
 SERVICE_NOT_APPLICABLE = 'Service not applicable/Does not exist'
 SERVICE_NOT_APPLICABLE_THIS_ADDRESS = 'Service not applicable for this address: {0}'
 STARTING_THREAD = 'Starting thread'
-STATISTICS_COPY_FILE = 'Total: {0}, Copied: {1}, Shortcut created {2}, Shortcut exists {3}, Duplicate: {4}, Copy Failed: {5}, Not copyable: {6}, Permissions Failed: {7}, Protected Ranges Failed: {8}'
+STATISTICS_COPY_FILE = 'Total: {0}, Copied: {1}, Shortcut created {2}, Shortcut exists {3}, Duplicate: {4}, Copy Failed: {5}, Not copyable: {6}, In skipids: {7}, Permissions Failed: {8}, Protected Ranges Failed: {9}'
 STATISTICS_COPY_FOLDER = 'Total: {0}, Copied: {1}, Shortcut created {2}, Shortcut exists {3}, Duplicate: {4}, Merged: {5}, Copy Failed: {6}, Not writable: {7}, Permissions Failed: {8}'
 STATISTICS_MOVE_FILE = 'Total: {0}, Moved: {1}, Shortcut created {2}, Shortcut exists {3}, Duplicate: {4}, Move Failed: {5}, Not movable: {6}'
 STATISTICS_MOVE_FOLDER = 'Total: {0}, Moved: {1}, Shortcut created {2}, Shortcut exists {3}, Duplicate: {4}, Merged: {5}, Move Failed: {6}, Not writable: {7}'

src/gam/googleapiclient/__init__.py

@@ -0,0 +1,27 @@
+# Copyright 2014 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Set default logging handler to avoid "No handler found" warnings.
+import logging
+
+try:  # Python 2.7+
+    from logging import NullHandler
+except ImportError:
+
+    class NullHandler(logging.Handler):
+        def emit(self, record):
+            pass
+
+
+logging.getLogger(__name__).addHandler(NullHandler())

src/gam/googleapiclient/_auth.py

@@ -0,0 +1,167 @@
+# Copyright 2016 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Helpers for authentication using oauth2client or google-auth."""
+
+import httplib2
+
+try:
+    import google.auth
+    import google.auth.credentials
+
+    HAS_GOOGLE_AUTH = True
+except ImportError:  # pragma: NO COVER
+    HAS_GOOGLE_AUTH = False
+
+try:
+    import google_auth_httplib2
+except ImportError:  # pragma: NO COVER
+    google_auth_httplib2 = None
+
+try:
+    import oauth2client
+    import oauth2client.client
+
+    HAS_OAUTH2CLIENT = True
+except ImportError:  # pragma: NO COVER
+    HAS_OAUTH2CLIENT = False
+
+
+def credentials_from_file(filename, scopes=None, quota_project_id=None):
+    """Returns credentials loaded from a file."""
+    if HAS_GOOGLE_AUTH:
+        credentials, _ = google.auth.load_credentials_from_file(
+            filename, scopes=scopes, quota_project_id=quota_project_id
+        )
+        return credentials
+    else:
+        raise EnvironmentError(
+            "client_options.credentials_file is only supported in google-auth."
+        )
+
+
+def default_credentials(scopes=None, quota_project_id=None):
+    """Returns Application Default Credentials."""
+    if HAS_GOOGLE_AUTH:
+        credentials, _ = google.auth.default(
+            scopes=scopes, quota_project_id=quota_project_id
+        )
+        return credentials
+    elif HAS_OAUTH2CLIENT:
+        if scopes is not None or quota_project_id is not None:
+            raise EnvironmentError(
+                "client_options.scopes and client_options.quota_project_id are not supported in oauth2client."
+                "Please install google-auth."
+            )
+        return oauth2client.client.GoogleCredentials.get_application_default()
+    else:
+        raise EnvironmentError(
+            "No authentication library is available. Please install either "
+            "google-auth or oauth2client."
+        )
+
+
+def with_scopes(credentials, scopes):
+    """Scopes the credentials if necessary.
+
+    Args:
+        credentials (Union[
+            google.auth.credentials.Credentials,
+            oauth2client.client.Credentials]): The credentials to scope.
+        scopes (Sequence[str]): The list of scopes.
+
+    Returns:
+        Union[google.auth.credentials.Credentials,
+            oauth2client.client.Credentials]: The scoped credentials.
+    """
+    if HAS_GOOGLE_AUTH and isinstance(credentials, google.auth.credentials.Credentials):
+        return google.auth.credentials.with_scopes_if_required(credentials, scopes)
+    else:
+        try:
+            if credentials.create_scoped_required():
+                return credentials.create_scoped(scopes)
+            else:
+                return credentials
+        except AttributeError:
+            return credentials
+
+
+def authorized_http(credentials):
+    """Returns an http client that is authorized with the given credentials.
+
+    Args:
+        credentials (Union[
+            google.auth.credentials.Credentials,
+            oauth2client.client.Credentials]): The credentials to use.
+
+    Returns:
+        Union[httplib2.Http, google_auth_httplib2.AuthorizedHttp]: An
+            authorized http client.
+    """
+    from googleapiclient.http import build_http
+
+    if HAS_GOOGLE_AUTH and isinstance(credentials, google.auth.credentials.Credentials):
+        if google_auth_httplib2 is None:
+            raise ValueError(
+                "Credentials from google.auth specified, but "
+                "google-api-python-client is unable to use these credentials "
+                "unless google-auth-httplib2 is installed. Please install "
+                "google-auth-httplib2."
+            )
+        return google_auth_httplib2.AuthorizedHttp(credentials, http=build_http())
+    else:
+        return credentials.authorize(build_http())
+
+
+def refresh_credentials(credentials):
+    # Refresh must use a new http instance, as the one associated with the
+    # credentials could be a AuthorizedHttp or an oauth2client-decorated
+    # Http instance which would cause a weird recursive loop of refreshing
+    # and likely tear a hole in spacetime.
+    refresh_http = httplib2.Http()
+    if HAS_GOOGLE_AUTH and isinstance(credentials, google.auth.credentials.Credentials):
+        request = google_auth_httplib2.Request(refresh_http)
+        return credentials.refresh(request)
+    else:
+        return credentials.refresh(refresh_http)
+
+
+def apply_credentials(credentials, headers):
+    # oauth2client and google-auth have the same interface for this.
+    if not is_valid(credentials):
+        refresh_credentials(credentials)
+    return credentials.apply(headers)
+
+
+def is_valid(credentials):
+    if HAS_GOOGLE_AUTH and isinstance(credentials, google.auth.credentials.Credentials):
+        return credentials.valid
+    else:
+        return (
+            credentials.access_token is not None
+            and not credentials.access_token_expired
+        )
+
+
+def get_credentials_from_http(http):
+    if http is None:
+        return None
+    elif hasattr(http.request, "credentials"):
+        return http.request.credentials
+    elif hasattr(http, "credentials") and not isinstance(
+        http.credentials, httplib2.Credentials
+    ):
+        return http.credentials
+    else:
+        return None

src/gam/googleapiclient/_helpers.py

@@ -0,0 +1,207 @@
+# Copyright 2015 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Helper functions for commonly used utilities."""
+
+import functools
+import inspect
+import logging
+import urllib
+
+logger = logging.getLogger(__name__)
+
+POSITIONAL_WARNING = "WARNING"
+POSITIONAL_EXCEPTION = "EXCEPTION"
+POSITIONAL_IGNORE = "IGNORE"
+POSITIONAL_SET = frozenset(
+    [POSITIONAL_WARNING, POSITIONAL_EXCEPTION, POSITIONAL_IGNORE]
+)
+
+positional_parameters_enforcement = POSITIONAL_WARNING
+
+_SYM_LINK_MESSAGE = "File: {0}: Is a symbolic link."
+_IS_DIR_MESSAGE = "{0}: Is a directory"
+_MISSING_FILE_MESSAGE = "Cannot access {0}: No such file or directory"
+
+
+def positional(max_positional_args):
+    """A decorator to declare that only the first N arguments may be positional.
+
+    This decorator makes it easy to support Python 3 style keyword-only
+    parameters. For example, in Python 3 it is possible to write::
+
+        def fn(pos1, *, kwonly1=None, kwonly2=None):
+            ...
+
+    All named parameters after ``*`` must be a keyword::
+
+        fn(10, 'kw1', 'kw2')  # Raises exception.
+        fn(10, kwonly1='kw1')  # Ok.
+
+    Example
+    ^^^^^^^
+
+    To define a function like above, do::
+
+        @positional(1)
+        def fn(pos1, kwonly1=None, kwonly2=None):
+            ...
+
+    If no default value is provided to a keyword argument, it becomes a
+    required keyword argument::
+
+        @positional(0)
+        def fn(required_kw):
+            ...
+
+    This must be called with the keyword parameter::
+
+        fn()  # Raises exception.
+        fn(10)  # Raises exception.
+        fn(required_kw=10)  # Ok.
+
+    When defining instance or class methods always remember to account for
+    ``self`` and ``cls``::
+
+        class MyClass(object):
+
+            @positional(2)
+            def my_method(self, pos1, kwonly1=None):
+                ...
+
+            @classmethod
+            @positional(2)
+            def my_method(cls, pos1, kwonly1=None):
+                ...
+
+    The positional decorator behavior is controlled by
+    ``_helpers.positional_parameters_enforcement``, which may be set to
+    ``POSITIONAL_EXCEPTION``, ``POSITIONAL_WARNING`` or
+    ``POSITIONAL_IGNORE`` to raise an exception, log a warning, or do
+    nothing, respectively, if a declaration is violated.
+
+    Args:
+        max_positional_arguments: Maximum number of positional arguments. All
+                                  parameters after this index must be
+                                  keyword only.
+
+    Returns:
+        A decorator that prevents using arguments after max_positional_args
+        from being used as positional parameters.
+
+    Raises:
+        TypeError: if a keyword-only argument is provided as a positional
+                   parameter, but only if
+                   _helpers.positional_parameters_enforcement is set to
+                   POSITIONAL_EXCEPTION.
+    """
+
+    def positional_decorator(wrapped):
+        @functools.wraps(wrapped)
+        def positional_wrapper(*args, **kwargs):
+            if len(args) > max_positional_args:
+                plural_s = ""
+                if max_positional_args != 1:
+                    plural_s = "s"
+                message = (
+                    "{function}() takes at most {args_max} positional "
+                    "argument{plural} ({args_given} given)".format(
+                        function=wrapped.__name__,
+                        args_max=max_positional_args,
+                        args_given=len(args),
+                        plural=plural_s,
+                    )
+                )
+                if positional_parameters_enforcement == POSITIONAL_EXCEPTION:
+                    raise TypeError(message)
+                elif positional_parameters_enforcement == POSITIONAL_WARNING:
+                    logger.warning(message)
+            return wrapped(*args, **kwargs)
+
+        return positional_wrapper
+
+    if isinstance(max_positional_args, int):
+        return positional_decorator
+    else:
+        args, _, _, defaults, _, _, _ = inspect.getfullargspec(max_positional_args)
+        return positional(len(args) - len(defaults))(max_positional_args)
+
+
+def parse_unique_urlencoded(content):
+    """Parses unique key-value parameters from urlencoded content.
+
+    Args:
+        content: string, URL-encoded key-value pairs.
+
+    Returns:
+        dict, The key-value pairs from ``content``.
+
+    Raises:
+        ValueError: if one of the keys is repeated.
+    """
+    urlencoded_params = urllib.parse.parse_qs(content)
+    params = {}
+    for key, value in urlencoded_params.items():
+        if len(value) != 1:
+            msg = "URL-encoded content contains a repeated value:" "%s -> %s" % (
+                key,
+                ", ".join(value),
+            )
+            raise ValueError(msg)
+        params[key] = value[0]
+    return params
+
+
+def update_query_params(uri, params):
+    """Updates a URI with new query parameters.
+
+    If a given key from ``params`` is repeated in the ``uri``, then
+    the URI will be considered invalid and an error will occur.
+
+    If the URI is valid, then each value from ``params`` will
+    replace the corresponding value in the query parameters (if
+    it exists).
+
+    Args:
+        uri: string, A valid URI, with potential existing query parameters.
+        params: dict, A dictionary of query parameters.
+
+    Returns:
+        The same URI but with the new query parameters added.
+    """
+    parts = urllib.parse.urlparse(uri)
+    query_params = parse_unique_urlencoded(parts.query)
+    query_params.update(params)
+    new_query = urllib.parse.urlencode(query_params)
+    new_parts = parts._replace(query=new_query)
+    return urllib.parse.urlunparse(new_parts)
+
+
+def _add_query_parameter(url, name, value):
+    """Adds a query parameter to a url.
+
+    Replaces the current value if it already exists in the URL.
+
+    Args:
+        url: string, url to add the query parameter to.
+        name: string, query parameter name.
+        value: string, query parameter value.
+
+    Returns:
+        Updated query parameter. Does not update the url if value is None.
+    """
+    if value is None:
+        return url
+    else:
+        return update_query_params(url, {name: value})

src/gam/googleapiclient/channel.py

@@ -0,0 +1,315 @@
+# Copyright 2014 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Channel notifications support.
+
+Classes and functions to support channel subscriptions and notifications
+on those channels.
+
+Notes:
+  - This code is based on experimental APIs and is subject to change.
+  - Notification does not do deduplication of notification ids, that's up to
+    the receiver.
+  - Storing the Channel between calls is up to the caller.
+
+
+Example setting up a channel:
+
+  # Create a new channel that gets notifications via webhook.
+  channel = new_webhook_channel("https://example.com/my_web_hook")
+
+  # Store the channel, keyed by 'channel.id'. Store it before calling the
+  # watch method because notifications may start arriving before the watch
+  # method returns.
+  ...
+
+  resp = service.objects().watchAll(
+    bucket="some_bucket_id", body=channel.body()).execute()
+  channel.update(resp)
+
+  # Store the channel, keyed by 'channel.id'. Store it after being updated
+  # since the resource_id value will now be correct, and that's needed to
+  # stop a subscription.
+  ...
+
+
+An example Webhook implementation using webapp2. Note that webapp2 puts
+headers in a case insensitive dictionary, as headers aren't guaranteed to
+always be upper case.
+
+  id = self.request.headers[X_GOOG_CHANNEL_ID]
+
+  # Retrieve the channel by id.
+  channel = ...
+
+  # Parse notification from the headers, including validating the id.
+  n = notification_from_headers(channel, self.request.headers)
+
+  # Do app specific stuff with the notification here.
+  if n.resource_state == 'sync':
+    # Code to handle sync state.
+  elif n.resource_state == 'exists':
+    # Code to handle the exists state.
+  elif n.resource_state == 'not_exists':
+    # Code to handle the not exists state.
+
+
+Example of unsubscribing.
+
+  service.channels().stop(channel.body()).execute()
+"""
+from __future__ import absolute_import
+
+import datetime
+import uuid
+
+from googleapiclient import _helpers as util
+from googleapiclient import errors
+
+# The unix time epoch starts at midnight 1970.
+EPOCH = datetime.datetime(1970, 1, 1)
+
+# Map the names of the parameters in the JSON channel description to
+# the parameter names we use in the Channel class.
+CHANNEL_PARAMS = {
+    "address": "address",
+    "id": "id",
+    "expiration": "expiration",
+    "params": "params",
+    "resourceId": "resource_id",
+    "resourceUri": "resource_uri",
+    "type": "type",
+    "token": "token",
+}
+
+X_GOOG_CHANNEL_ID = "X-GOOG-CHANNEL-ID"
+X_GOOG_MESSAGE_NUMBER = "X-GOOG-MESSAGE-NUMBER"
+X_GOOG_RESOURCE_STATE = "X-GOOG-RESOURCE-STATE"
+X_GOOG_RESOURCE_URI = "X-GOOG-RESOURCE-URI"
+X_GOOG_RESOURCE_ID = "X-GOOG-RESOURCE-ID"
+
+
+def _upper_header_keys(headers):
+    new_headers = {}
+    for k, v in headers.items():
+        new_headers[k.upper()] = v
+    return new_headers
+
+
+class Notification(object):
+    """A Notification from a Channel.
+
+    Notifications are not usually constructed directly, but are returned
+    from functions like notification_from_headers().
+
+    Attributes:
+      message_number: int, The unique id number of this notification.
+      state: str, The state of the resource being monitored.
+      uri: str, The address of the resource being monitored.
+      resource_id: str, The unique identifier of the version of the resource at
+        this event.
+    """
+
+    @util.positional(5)
+    def __init__(self, message_number, state, resource_uri, resource_id):
+        """Notification constructor.
+
+        Args:
+          message_number: int, The unique id number of this notification.
+          state: str, The state of the resource being monitored. Can be one
+            of "exists", "not_exists", or "sync".
+          resource_uri: str, The address of the resource being monitored.
+          resource_id: str, The identifier of the watched resource.
+        """
+        self.message_number = message_number
+        self.state = state
+        self.resource_uri = resource_uri
+        self.resource_id = resource_id
+
+
+class Channel(object):
+    """A Channel for notifications.
+
+    Usually not constructed directly, instead it is returned from helper
+    functions like new_webhook_channel().
+
+    Attributes:
+      type: str, The type of delivery mechanism used by this channel. For
+        example, 'web_hook'.
+      id: str, A UUID for the channel.
+      token: str, An arbitrary string associated with the channel that
+        is delivered to the target address with each event delivered
+        over this channel.
+      address: str, The address of the receiving entity where events are
+        delivered. Specific to the channel type.
+      expiration: int, The time, in milliseconds from the epoch, when this
+        channel will expire.
+      params: dict, A dictionary of string to string, with additional parameters
+        controlling delivery channel behavior.
+      resource_id: str, An opaque id that identifies the resource that is
+        being watched. Stable across different API versions.
+      resource_uri: str, The canonicalized ID of the watched resource.
+    """
+
+    @util.positional(5)
+    def __init__(
+        self,
+        type,
+        id,
+        token,
+        address,
+        expiration=None,
+        params=None,
+        resource_id="",
+        resource_uri="",
+    ):
+        """Create a new Channel.
+
+        In user code, this Channel constructor will not typically be called
+        manually since there are functions for creating channels for each specific
+        type with a more customized set of arguments to pass.
+
+        Args:
+          type: str, The type of delivery mechanism used by this channel. For
+            example, 'web_hook'.
+          id: str, A UUID for the channel.
+          token: str, An arbitrary string associated with the channel that
+            is delivered to the target address with each event delivered
+            over this channel.
+          address: str,  The address of the receiving entity where events are
+            delivered. Specific to the channel type.
+          expiration: int, The time, in milliseconds from the epoch, when this
+            channel will expire.
+          params: dict, A dictionary of string to string, with additional parameters
+            controlling delivery channel behavior.
+          resource_id: str, An opaque id that identifies the resource that is
+            being watched. Stable across different API versions.
+          resource_uri: str, The canonicalized ID of the watched resource.
+        """
+        self.type = type
+        self.id = id
+        self.token = token
+        self.address = address
+        self.expiration = expiration
+        self.params = params
+        self.resource_id = resource_id
+        self.resource_uri = resource_uri
+
+    def body(self):
+        """Build a body from the Channel.
+
+        Constructs a dictionary that's appropriate for passing into watch()
+        methods as the value of body argument.
+
+        Returns:
+          A dictionary representation of the channel.
+        """
+        result = {
+            "id": self.id,
+            "token": self.token,
+            "type": self.type,
+            "address": self.address,
+        }
+        if self.params:
+            result["params"] = self.params
+        if self.resource_id:
+            result["resourceId"] = self.resource_id
+        if self.resource_uri:
+            result["resourceUri"] = self.resource_uri
+        if self.expiration:
+            result["expiration"] = self.expiration
+
+        return result
+
+    def update(self, resp):
+        """Update a channel with information from the response of watch().
+
+        When a request is sent to watch() a resource, the response returned
+        from the watch() request is a dictionary with updated channel information,
+        such as the resource_id, which is needed when stopping a subscription.
+
+        Args:
+          resp: dict, The response from a watch() method.
+        """
+        for json_name, param_name in CHANNEL_PARAMS.items():
+            value = resp.get(json_name)
+            if value is not None:
+                setattr(self, param_name, value)
+
+
+def notification_from_headers(channel, headers):
+    """Parse a notification from the webhook request headers, validate
+      the notification, and return a Notification object.
+
+    Args:
+      channel: Channel, The channel that the notification is associated with.
+      headers: dict, A dictionary like object that contains the request headers
+        from the webhook HTTP request.
+
+    Returns:
+      A Notification object.
+
+    Raises:
+      errors.InvalidNotificationError if the notification is invalid.
+      ValueError if the X-GOOG-MESSAGE-NUMBER can't be converted to an int.
+    """
+    headers = _upper_header_keys(headers)
+    channel_id = headers[X_GOOG_CHANNEL_ID]
+    if channel.id != channel_id:
+        raise errors.InvalidNotificationError(
+            "Channel id mismatch: %s != %s" % (channel.id, channel_id)
+        )
+    else:
+        message_number = int(headers[X_GOOG_MESSAGE_NUMBER])
+        state = headers[X_GOOG_RESOURCE_STATE]
+        resource_uri = headers[X_GOOG_RESOURCE_URI]
+        resource_id = headers[X_GOOG_RESOURCE_ID]
+        return Notification(message_number, state, resource_uri, resource_id)
+
+
+@util.positional(2)
+def new_webhook_channel(url, token=None, expiration=None, params=None):
+    """Create a new webhook Channel.
+
+    Args:
+      url: str, URL to post notifications to.
+      token: str, An arbitrary string associated with the channel that
+        is delivered to the target address with each notification delivered
+        over this channel.
+      expiration: datetime.datetime, A time in the future when the channel
+        should expire. Can also be None if the subscription should use the
+        default expiration. Note that different services may have different
+        limits on how long a subscription lasts. Check the response from the
+        watch() method to see the value the service has set for an expiration
+        time.
+      params: dict, Extra parameters to pass on channel creation. Currently
+        not used for webhook channels.
+    """
+    expiration_ms = 0
+    if expiration:
+        delta = expiration - EPOCH
+        expiration_ms = (
+            delta.microseconds / 1000 + (delta.seconds + delta.days * 24 * 3600) * 1000
+        )
+        if expiration_ms < 0:
+            expiration_ms = 0
+
+    return Channel(
+        "web_hook",
+        str(uuid.uuid4()),
+        token,
+        url,
+        expiration=expiration_ms,
+        params=params,
+    )

src/gam/googleapiclient/discovery_cache/__init__.py

@@ -0,0 +1,78 @@
+# Copyright 2014 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Caching utility for the discovery document."""
+
+from __future__ import absolute_import
+
+import logging
+import os
+
+LOGGER = logging.getLogger(__name__)
+
+DISCOVERY_DOC_MAX_AGE = 60 * 60 * 24  # 1 day
+DISCOVERY_DOC_DIR = os.path.join(
+    os.path.dirname(os.path.realpath(__file__)), "documents"
+)
+
+
+def autodetect():
+    """Detects an appropriate cache module and returns it.
+
+    Returns:
+      googleapiclient.discovery_cache.base.Cache, a cache object which
+      is auto detected, or None if no cache object is available.
+    """
+    if "GAE_ENV" in os.environ:
+        try:
+            from . import appengine_memcache
+
+            return appengine_memcache.cache
+        except Exception:
+            pass
+    try:
+        from . import file_cache
+
+        return file_cache.cache
+    except Exception:
+        LOGGER.info(
+            "file_cache is only supported with oauth2client<4.0.0", exc_info=False
+        )
+        return None
+
+
+def get_static_doc(serviceName, version):
+    """Retrieves the discovery document from the directory defined in
+    DISCOVERY_DOC_DIR corresponding to the serviceName and version provided.
+
+    Args:
+        serviceName: string, name of the service.
+        version: string, the version of the service.
+
+    Returns:
+        A string containing the contents of the JSON discovery document,
+        otherwise None if the JSON discovery document was not found.
+    """
+
+    content = None
+    doc_name = "{}.{}.json".format(serviceName, version)
+
+    try:
+        with open(os.path.join(DISCOVERY_DOC_DIR, doc_name), "r") as f:
+            content = f.read()
+    except FileNotFoundError:
+        # File does not exist. Nothing to do here.
+        pass
+
+    return content

src/gam/googleapiclient/discovery_cache/appengine_memcache.py

@@ -0,0 +1,55 @@
+# Copyright 2014 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""App Engine memcache based cache for the discovery document."""
+
+import logging
+
+# This is only an optional dependency because we only import this
+# module when google.appengine.api.memcache is available.
+from google.appengine.api import memcache
+
+from . import base
+from ..discovery_cache import DISCOVERY_DOC_MAX_AGE
+
+LOGGER = logging.getLogger(__name__)
+
+NAMESPACE = "google-api-client"
+
+
+class Cache(base.Cache):
+    """A cache with app engine memcache API."""
+
+    def __init__(self, max_age):
+        """Constructor.
+
+        Args:
+          max_age: Cache expiration in seconds.
+        """
+        self._max_age = max_age
+
+    def get(self, url):
+        try:
+            return memcache.get(url, namespace=NAMESPACE)
+        except Exception as e:
+            LOGGER.warning(e, exc_info=True)
+
+    def set(self, url, content):
+        try:
+            memcache.set(url, content, time=int(self._max_age), namespace=NAMESPACE)
+        except Exception as e:
+            LOGGER.warning(e, exc_info=True)
+
+
+cache = Cache(max_age=DISCOVERY_DOC_MAX_AGE)

src/gam/googleapiclient/discovery_cache/base.py

@@ -0,0 +1,46 @@
+# Copyright 2014 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""An abstract class for caching the discovery document."""
+
+import abc
+
+
+class Cache(object):
+    """A base abstract cache class."""
+
+    __metaclass__ = abc.ABCMeta
+
+    @abc.abstractmethod
+    def get(self, url):
+        """Gets the content from the memcache with a given key.
+
+        Args:
+          url: string, the key for the cache.
+
+        Returns:
+          object, the value in the cache for the given key, or None if the key is
+          not in the cache.
+        """
+        raise NotImplementedError()
+
+    @abc.abstractmethod
+    def set(self, url, content):
+        """Sets the given key and content in the cache.
+
+        Args:
+          url: string, the key for the cache.
+          content: string, the discovery document.
+        """
+        raise NotImplementedError()

src/gam/googleapiclient/discovery_cache/file_cache.py

@@ -0,0 +1,145 @@
+# Copyright 2014 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""File based cache for the discovery document.
+
+The cache is stored in a single file so that multiple processes can
+share the same cache. It locks the file whenever accessing to the
+file. When the cache content is corrupted, it will be initialized with
+an empty cache.
+"""
+
+from __future__ import division
+
+import datetime
+import json
+import logging
+import os
+import tempfile
+
+try:
+    from oauth2client.contrib.locked_file import LockedFile
+except ImportError:
+    # oauth2client < 2.0.0
+    try:
+        from oauth2client.locked_file import LockedFile
+    except ImportError:
+        # oauth2client > 4.0.0 or google-auth
+        raise ImportError(
+            "file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth"
+        )
+
+from . import base
+from ..discovery_cache import DISCOVERY_DOC_MAX_AGE
+
+LOGGER = logging.getLogger(__name__)
+
+FILENAME = "google-api-python-client-discovery-doc.cache"
+EPOCH = datetime.datetime(1970, 1, 1)
+
+
+def _to_timestamp(date):
+    try:
+        return (date - EPOCH).total_seconds()
+    except AttributeError:
+        # The following is the equivalent of total_seconds() in Python2.6.
+        # See also: https://docs.python.org/2/library/datetime.html
+        delta = date - EPOCH
+        return (
+            delta.microseconds + (delta.seconds + delta.days * 24 * 3600) * 10**6
+        ) / 10**6
+
+
+def _read_or_initialize_cache(f):
+    f.file_handle().seek(0)
+    try:
+        cache = json.load(f.file_handle())
+    except Exception:
+        # This means it opens the file for the first time, or the cache is
+        # corrupted, so initializing the file with an empty dict.
+        cache = {}
+        f.file_handle().truncate(0)
+        f.file_handle().seek(0)
+        json.dump(cache, f.file_handle())
+    return cache
+
+
+class Cache(base.Cache):
+    """A file based cache for the discovery documents."""
+
+    def __init__(self, max_age):
+        """Constructor.
+
+        Args:
+          max_age: Cache expiration in seconds.
+        """
+        self._max_age = max_age
+        self._file = os.path.join(tempfile.gettempdir(), FILENAME)
+        f = LockedFile(self._file, "a+", "r")
+        try:
+            f.open_and_lock()
+            if f.is_locked():
+                _read_or_initialize_cache(f)
+            # If we can not obtain the lock, other process or thread must
+            # have initialized the file.
+        except Exception as e:
+            LOGGER.warning(e, exc_info=True)
+        finally:
+            f.unlock_and_close()
+
+    def get(self, url):
+        f = LockedFile(self._file, "r+", "r")
+        try:
+            f.open_and_lock()
+            if f.is_locked():
+                cache = _read_or_initialize_cache(f)
+                if url in cache:
+                    content, t = cache.get(url, (None, 0))
+                    if _to_timestamp(datetime.datetime.now()) < t + self._max_age:
+                        return content
+                return None
+            else:
+                LOGGER.debug("Could not obtain a lock for the cache file.")
+                return None
+        except Exception as e:
+            LOGGER.warning(e, exc_info=True)
+        finally:
+            f.unlock_and_close()
+
+    def set(self, url, content):
+        f = LockedFile(self._file, "r+", "r")
+        try:
+            f.open_and_lock()
+            if f.is_locked():
+                cache = _read_or_initialize_cache(f)
+                cache[url] = (content, _to_timestamp(datetime.datetime.now()))
+                # Remove stale cache.
+                for k, (_, timestamp) in list(cache.items()):
+                    if (
+                        _to_timestamp(datetime.datetime.now())
+                        >= timestamp + self._max_age
+                    ):
+                        del cache[k]
+                f.file_handle().truncate(0)
+                f.file_handle().seek(0)
+                json.dump(cache, f.file_handle())
+            else:
+                LOGGER.debug("Could not obtain a lock for the cache file.")
+        except Exception as e:
+            LOGGER.warning(e, exc_info=True)
+        finally:
+            f.unlock_and_close()
+
+
+cache = Cache(max_age=DISCOVERY_DOC_MAX_AGE)

src/gam/googleapiclient/errors.py

@@ -0,0 +1,197 @@
+# Copyright 2014 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Errors for the library.
+
+All exceptions defined by the library
+should be defined in this file.
+"""
+from __future__ import absolute_import
+
+__author__ = "jcgregorio@google.com (Joe Gregorio)"
+
+import json
+
+from googleapiclient import _helpers as util
+
+
+class Error(Exception):
+    """Base error for this module."""
+
+    pass
+
+
+class HttpError(Error):
+    """HTTP data was invalid or unexpected."""
+
+    @util.positional(3)
+    def __init__(self, resp, content, uri=None):
+        self.resp = resp
+        if not isinstance(content, bytes):
+            raise TypeError("HTTP content should be bytes")
+        self.content = content
+        self.uri = uri
+        self.error_details = ""
+        self.reason = self._get_reason()
+
+    @property
+    def status_code(self):
+        """Return the HTTP status code from the response content."""
+        return self.resp.status
+
+    def _get_reason(self):
+        """Calculate the reason for the error from the response content."""
+        reason = self.resp.reason
+        try:
+            try:
+                data = json.loads(self.content.decode("utf-8"))
+            except json.JSONDecodeError:
+                # In case it is not json
+                data = self.content.decode("utf-8")
+            if isinstance(data, dict):
+                reason = data["error"]["message"]
+                error_detail_keyword = next(
+                    (
+                        kw
+                        for kw in ["detail", "details", "errors", "message"]
+                        if kw in data["error"]
+                    ),
+                    "",
+                )
+                if error_detail_keyword:
+                    self.error_details = data["error"][error_detail_keyword]
+            elif isinstance(data, list) and len(data) > 0:
+                first_error = data[0]
+                reason = first_error["error"]["message"]
+                if "details" in first_error["error"]:
+                    self.error_details = first_error["error"]["details"]
+            else:
+                self.error_details = data
+        except (ValueError, KeyError, TypeError):
+            pass
+        if reason is None:
+            reason = ""
+        return reason.strip()
+
+    def __repr__(self):
+        if self.error_details:
+            return '<HttpError %s when requesting %s returned "%s". Details: "%s">' % (
+                self.resp.status,
+                self.uri,
+                self.reason,
+                self.error_details,
+            )
+        elif self.uri:
+            return '<HttpError %s when requesting %s returned "%s">' % (
+                self.resp.status,
+                self.uri,
+                self.reason,
+            )
+        else:
+            return '<HttpError %s "%s">' % (self.resp.status, self.reason)
+
+    __str__ = __repr__
+
+
+class InvalidJsonError(Error):
+    """The JSON returned could not be parsed."""
+
+    pass
+
+
+class UnknownFileType(Error):
+    """File type unknown or unexpected."""
+
+    pass
+
+
+class UnknownLinkType(Error):
+    """Link type unknown or unexpected."""
+
+    pass
+
+
+class UnknownApiNameOrVersion(Error):
+    """No API with that name and version exists."""
+
+    pass
+
+
+class UnacceptableMimeTypeError(Error):
+    """That is an unacceptable mimetype for this operation."""
+
+    pass
+
+
+class MediaUploadSizeError(Error):
+    """Media is larger than the method can accept."""
+
+    pass
+
+
+class ResumableUploadError(HttpError):
+    """Error occurred during resumable upload."""
+
+    pass
+
+
+class InvalidChunkSizeError(Error):
+    """The given chunksize is not valid."""
+
+    pass
+
+
+class InvalidNotificationError(Error):
+    """The channel Notification is invalid."""
+
+    pass
+
+
+class BatchError(HttpError):
+    """Error occurred during batch operations."""
+
+    @util.positional(2)
+    def __init__(self, reason, resp=None, content=None):
+        self.resp = resp
+        self.content = content
+        self.reason = reason
+
+    def __repr__(self):
+        if getattr(self.resp, "status", None) is None:
+            return '<BatchError "%s">' % (self.reason)
+        else:
+            return '<BatchError %s "%s">' % (self.resp.status, self.reason)
+
+    __str__ = __repr__
+
+
+class UnexpectedMethodError(Error):
+    """Exception raised by RequestMockBuilder on unexpected calls."""
+
+    @util.positional(1)
+    def __init__(self, methodId=None):
+        """Constructor for an UnexpectedMethodError."""
+        super(UnexpectedMethodError, self).__init__(
+            "Received unexpected call %s" % methodId
+        )
+
+
+class UnexpectedBodyError(Error):
+    """Exception raised by RequestMockBuilder on unexpected bodies."""
+
+    def __init__(self, expected, provided):
+        """Constructor for an UnexpectedMethodError."""
+        super(UnexpectedBodyError, self).__init__(
+            "Expected: [%s] - Provided: [%s]" % (expected, provided)
+        )

src/gam/googleapiclient/mimeparse.py

@@ -0,0 +1,183 @@
+# Copyright 2014 Joe Gregorio
+#
+# Licensed under the MIT License
+
+"""MIME-Type Parser
+
+This module provides basic functions for handling mime-types. It can handle
+matching mime-types against a list of media-ranges. See section 14.1 of the
+HTTP specification [RFC 2616] for a complete explanation.
+
+   http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.1
+
+Contents:
+ - parse_mime_type():   Parses a mime-type into its component parts.
+ - parse_media_range(): Media-ranges are mime-types with wild-cards and a 'q'
+                          quality parameter.
+ - quality():           Determines the quality ('q') of a mime-type when
+                          compared against a list of media-ranges.
+ - quality_parsed():    Just like quality() except the second parameter must be
+                          pre-parsed.
+ - best_match():        Choose the mime-type with the highest quality ('q')
+                          from a list of candidates.
+"""
+from __future__ import absolute_import
+
+from functools import reduce
+
+__version__ = "0.1.3"
+__author__ = "Joe Gregorio"
+__email__ = "joe@bitworking.org"
+__license__ = "MIT License"
+__credits__ = ""
+
+
+def parse_mime_type(mime_type):
+    """Parses a mime-type into its component parts.
+
+    Carves up a mime-type and returns a tuple of the (type, subtype, params)
+    where 'params' is a dictionary of all the parameters for the media range.
+    For example, the media range 'application/xhtml;q=0.5' would get parsed
+    into:
+
+       ('application', 'xhtml', {'q', '0.5'})
+    """
+    parts = mime_type.split(";")
+    params = dict(
+        [tuple([s.strip() for s in param.split("=", 1)]) for param in parts[1:]]
+    )
+    full_type = parts[0].strip()
+    # Java URLConnection class sends an Accept header that includes a
+    # single '*'. Turn it into a legal wildcard.
+    if full_type == "*":
+        full_type = "*/*"
+    (type, subtype) = full_type.split("/")
+
+    return (type.strip(), subtype.strip(), params)
+
+
+def parse_media_range(range):
+    """Parse a media-range into its component parts.
+
+    Carves up a media range and returns a tuple of the (type, subtype,
+    params) where 'params' is a dictionary of all the parameters for the media
+    range.  For example, the media range 'application/*;q=0.5' would get parsed
+    into:
+
+       ('application', '*', {'q', '0.5'})
+
+    In addition this function also guarantees that there is a value for 'q'
+    in the params dictionary, filling it in with a proper default if
+    necessary.
+    """
+    (type, subtype, params) = parse_mime_type(range)
+    if (
+        "q" not in params
+        or not params["q"]
+        or not float(params["q"])
+        or float(params["q"]) > 1
+        or float(params["q"]) < 0
+    ):
+        params["q"] = "1"
+
+    return (type, subtype, params)
+
+
+def fitness_and_quality_parsed(mime_type, parsed_ranges):
+    """Find the best match for a mime-type amongst parsed media-ranges.
+
+    Find the best match for a given mime-type against a list of media_ranges
+    that have already been parsed by parse_media_range(). Returns a tuple of
+    the fitness value and the value of the 'q' quality parameter of the best
+    match, or (-1, 0) if no match was found. Just as for quality_parsed(),
+    'parsed_ranges' must be a list of parsed media ranges.
+    """
+    best_fitness = -1
+    best_fit_q = 0
+    (target_type, target_subtype, target_params) = parse_media_range(mime_type)
+    for (type, subtype, params) in parsed_ranges:
+        type_match = type == target_type or type == "*" or target_type == "*"
+        subtype_match = (
+            subtype == target_subtype or subtype == "*" or target_subtype == "*"
+        )
+        if type_match and subtype_match:
+            param_matches = reduce(
+                lambda x, y: x + y,
+                [
+                    1
+                    for (key, value) in target_params.items()
+                    if key != "q" and key in params and value == params[key]
+                ],
+                0,
+            )
+            fitness = (type == target_type) and 100 or 0
+            fitness += (subtype == target_subtype) and 10 or 0
+            fitness += param_matches
+            if fitness > best_fitness:
+                best_fitness = fitness
+                best_fit_q = params["q"]
+
+    return best_fitness, float(best_fit_q)
+
+
+def quality_parsed(mime_type, parsed_ranges):
+    """Find the best match for a mime-type amongst parsed media-ranges.
+
+    Find the best match for a given mime-type against a list of media_ranges
+    that have already been parsed by parse_media_range(). Returns the 'q'
+    quality parameter of the best match, 0 if no match was found. This function
+    bahaves the same as quality() except that 'parsed_ranges' must be a list of
+    parsed media ranges.
+    """
+
+    return fitness_and_quality_parsed(mime_type, parsed_ranges)[1]
+
+
+def quality(mime_type, ranges):
+    """Return the quality ('q') of a mime-type against a list of media-ranges.
+
+    Returns the quality 'q' of a mime-type when compared against the
+    media-ranges in ranges. For example:
+
+    >>> quality('text/html','text/*;q=0.3, text/html;q=0.7,
+                  text/html;level=1, text/html;level=2;q=0.4, */*;q=0.5')
+    0.7
+
+    """
+    parsed_ranges = [parse_media_range(r) for r in ranges.split(",")]
+
+    return quality_parsed(mime_type, parsed_ranges)
+
+
+def best_match(supported, header):
+    """Return mime-type with the highest quality ('q') from list of candidates.
+
+    Takes a list of supported mime-types and finds the best match for all the
+    media-ranges listed in header. The value of header must be a string that
+    conforms to the format of the HTTP Accept: header. The value of 'supported'
+    is a list of mime-types. The list of supported mime-types should be sorted
+    in order of increasing desirability, in case of a situation where there is
+    a tie.
+
+    >>> best_match(['application/xbel+xml', 'text/xml'],
+                   'text/*;q=0.5,*/*; q=0.1')
+    'text/xml'
+    """
+    split_header = _filter_blank(header.split(","))
+    parsed_header = [parse_media_range(r) for r in split_header]
+    weighted_matches = []
+    pos = 0
+    for mime_type in supported:
+        weighted_matches.append(
+            (fitness_and_quality_parsed(mime_type, parsed_header), pos, mime_type)
+        )
+        pos += 1
+    weighted_matches.sort()
+
+    return weighted_matches[-1][0][1] and weighted_matches[-1][2] or ""
+
+
+def _filter_blank(i):
+    for s in i:
+        if s.strip():
+            yield s

src/gam/googleapiclient/model.py

@@ -0,0 +1,409 @@
+# Copyright 2014 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Model objects for requests and responses.
+
+Each API may support one or more serializations, such
+as JSON, Atom, etc. The model classes are responsible
+for converting between the wire format and the Python
+object representation.
+"""
+from __future__ import absolute_import
+
+__author__ = "jcgregorio@google.com (Joe Gregorio)"
+
+import json
+import logging
+import platform
+import urllib
+
+from googleapiclient import version as googleapiclient_version
+from googleapiclient.errors import HttpError
+
+_LIBRARY_VERSION = googleapiclient_version.__version__
+_PY_VERSION = platform.python_version()
+
+LOGGER = logging.getLogger(__name__)
+
+dump_request_response = False
+
+
+def _abstract():
+    raise NotImplementedError("You need to override this function")
+
+
+class Model(object):
+    """Model base class.
+
+    All Model classes should implement this interface.
+    The Model serializes and de-serializes between a wire
+    format such as JSON and a Python object representation.
+    """
+
+    def request(self, headers, path_params, query_params, body_value):
+        """Updates outgoing requests with a serialized body.
+
+        Args:
+          headers: dict, request headers
+          path_params: dict, parameters that appear in the request path
+          query_params: dict, parameters that appear in the query
+          body_value: object, the request body as a Python object, which must be
+                      serializable.
+        Returns:
+          A tuple of (headers, path_params, query, body)
+
+          headers: dict, request headers
+          path_params: dict, parameters that appear in the request path
+          query: string, query part of the request URI
+          body: string, the body serialized in the desired wire format.
+        """
+        _abstract()
+
+    def response(self, resp, content):
+        """Convert the response wire format into a Python object.
+
+        Args:
+          resp: httplib2.Response, the HTTP response headers and status
+          content: string, the body of the HTTP response
+
+        Returns:
+          The body de-serialized as a Python object.
+
+        Raises:
+          googleapiclient.errors.HttpError if a non 2xx response is received.
+        """
+        _abstract()
+
+
+class BaseModel(Model):
+    """Base model class.
+
+    Subclasses should provide implementations for the "serialize" and
+    "deserialize" methods, as well as values for the following class attributes.
+
+    Attributes:
+      accept: The value to use for the HTTP Accept header.
+      content_type: The value to use for the HTTP Content-type header.
+      no_content_response: The value to return when deserializing a 204 "No
+          Content" response.
+      alt_param: The value to supply as the "alt" query parameter for requests.
+    """
+
+    accept = None
+    content_type = None
+    no_content_response = None
+    alt_param = None
+
+    def _log_request(self, headers, path_params, query, body):
+        """Logs debugging information about the request if requested."""
+        if dump_request_response:
+            LOGGER.info("--request-start--")
+            LOGGER.info("-headers-start-")
+            for h, v in headers.items():
+                LOGGER.info("%s: %s", h, v)
+            LOGGER.info("-headers-end-")
+            LOGGER.info("-path-parameters-start-")
+            for h, v in path_params.items():
+                LOGGER.info("%s: %s", h, v)
+            LOGGER.info("-path-parameters-end-")
+            LOGGER.info("body: %s", body)
+            LOGGER.info("query: %s", query)
+            LOGGER.info("--request-end--")
+
+    def request(self, headers, path_params, query_params, body_value):
+        """Updates outgoing requests with a serialized body.
+
+        Args:
+          headers: dict, request headers
+          path_params: dict, parameters that appear in the request path
+          query_params: dict, parameters that appear in the query
+          body_value: object, the request body as a Python object, which must be
+                      serializable by json.
+        Returns:
+          A tuple of (headers, path_params, query, body)
+
+          headers: dict, request headers
+          path_params: dict, parameters that appear in the request path
+          query: string, query part of the request URI
+          body: string, the body serialized as JSON
+        """
+        query = self._build_query(query_params)
+        headers["accept"] = self.accept
+        headers["accept-encoding"] = "gzip, deflate"
+        if "user-agent" in headers:
+            headers["user-agent"] += " "
+        else:
+            headers["user-agent"] = ""
+        headers["user-agent"] += "(gzip)"
+        if "x-goog-api-client" in headers:
+            headers["x-goog-api-client"] += " "
+        else:
+            headers["x-goog-api-client"] = ""
+        headers["x-goog-api-client"] += "gdcl/%s gl-python/%s" % (
+            _LIBRARY_VERSION,
+            _PY_VERSION,
+        )
+
+        if body_value is not None:
+            headers["content-type"] = self.content_type
+            body_value = self.serialize(body_value)
+        self._log_request(headers, path_params, query, body_value)
+        return (headers, path_params, query, body_value)
+
+    def _build_query(self, params):
+        """Builds a query string.
+
+        Args:
+          params: dict, the query parameters
+
+        Returns:
+          The query parameters properly encoded into an HTTP URI query string.
+        """
+        if self.alt_param is not None:
+            params.update({"alt": self.alt_param})
+        astuples = []
+        for key, value in params.items():
+            if type(value) == type([]):
+                for x in value:
+                    x = x.encode("utf-8")
+                    astuples.append((key, x))
+            else:
+                if isinstance(value, str) and callable(value.encode):
+                    value = value.encode("utf-8")
+                astuples.append((key, value))
+        return "?" + urllib.parse.urlencode(astuples)
+
+    def _log_response(self, resp, content):
+        """Logs debugging information about the response if requested."""
+        if dump_request_response:
+            LOGGER.info("--response-start--")
+            for h, v in resp.items():
+                LOGGER.info("%s: %s", h, v)
+            if content:
+                LOGGER.info(content)
+            LOGGER.info("--response-end--")
+
+    def response(self, resp, content):
+        """Convert the response wire format into a Python object.
+
+        Args:
+          resp: httplib2.Response, the HTTP response headers and status
+          content: string, the body of the HTTP response
+
+        Returns:
+          The body de-serialized as a Python object.
+
+        Raises:
+          googleapiclient.errors.HttpError if a non 2xx response is received.
+        """
+        self._log_response(resp, content)
+        # Error handling is TBD, for example, do we retry
+        # for some operation/error combinations?
+        if resp.status < 300:
+            if resp.status == 204:
+                # A 204: No Content response should be treated differently
+                # to all the other success states
+                return self.no_content_response
+            return self.deserialize(content)
+        else:
+            LOGGER.debug("Content from bad request was: %r" % content)
+            raise HttpError(resp, content)
+
+    def serialize(self, body_value):
+        """Perform the actual Python object serialization.
+
+        Args:
+          body_value: object, the request body as a Python object.
+
+        Returns:
+          string, the body in serialized form.
+        """
+        _abstract()
+
+    def deserialize(self, content):
+        """Perform the actual deserialization from response string to Python
+        object.
+
+        Args:
+          content: string, the body of the HTTP response
+
+        Returns:
+          The body de-serialized as a Python object.
+        """
+        _abstract()
+
+
+class JsonModel(BaseModel):
+    """Model class for JSON.
+
+    Serializes and de-serializes between JSON and the Python
+    object representation of HTTP request and response bodies.
+    """
+
+    accept = "application/json"
+    content_type = "application/json"
+    alt_param = "json"
+
+    def __init__(self, data_wrapper=False):
+        """Construct a JsonModel.
+
+        Args:
+          data_wrapper: boolean, wrap requests and responses in a data wrapper
+        """
+        self._data_wrapper = data_wrapper
+
+    def serialize(self, body_value):
+        if (
+            isinstance(body_value, dict)
+            and "data" not in body_value
+            and self._data_wrapper
+        ):
+            body_value = {"data": body_value}
+        return json.dumps(body_value)
+
+    def deserialize(self, content):
+        try:
+            content = content.decode("utf-8")
+        except AttributeError:
+            pass
+        try:
+            body = json.loads(content)
+        except json.decoder.JSONDecodeError:
+            body = content
+        else:
+            if self._data_wrapper and "data" in body:
+                body = body["data"]
+        return body
+
+    @property
+    def no_content_response(self):
+        return {}
+
+
+class RawModel(JsonModel):
+    """Model class for requests that don't return JSON.
+
+    Serializes and de-serializes between JSON and the Python
+    object representation of HTTP request, and returns the raw bytes
+    of the response body.
+    """
+
+    accept = "*/*"
+    content_type = "application/json"
+    alt_param = None
+
+    def deserialize(self, content):
+        return content
+
+    @property
+    def no_content_response(self):
+        return ""
+
+
+class MediaModel(JsonModel):
+    """Model class for requests that return Media.
+
+    Serializes and de-serializes between JSON and the Python
+    object representation of HTTP request, and returns the raw bytes
+    of the response body.
+    """
+
+    accept = "*/*"
+    content_type = "application/json"
+    alt_param = "media"
+
+    def deserialize(self, content):
+        return content
+
+    @property
+    def no_content_response(self):
+        return ""
+
+
+class ProtocolBufferModel(BaseModel):
+    """Model class for protocol buffers.
+
+    Serializes and de-serializes the binary protocol buffer sent in the HTTP
+    request and response bodies.
+    """
+
+    accept = "application/x-protobuf"
+    content_type = "application/x-protobuf"
+    alt_param = "proto"
+
+    def __init__(self, protocol_buffer):
+        """Constructs a ProtocolBufferModel.
+
+        The serialized protocol buffer returned in an HTTP response will be
+        de-serialized using the given protocol buffer class.
+
+        Args:
+          protocol_buffer: The protocol buffer class used to de-serialize a
+          response from the API.
+        """
+        self._protocol_buffer = protocol_buffer
+
+    def serialize(self, body_value):
+        return body_value.SerializeToString()
+
+    def deserialize(self, content):
+        return self._protocol_buffer.FromString(content)
+
+    @property
+    def no_content_response(self):
+        return self._protocol_buffer()
+
+
+def makepatch(original, modified):
+    """Create a patch object.
+
+    Some methods support PATCH, an efficient way to send updates to a resource.
+    This method allows the easy construction of patch bodies by looking at the
+    differences between a resource before and after it was modified.
+
+    Args:
+      original: object, the original deserialized resource
+      modified: object, the modified deserialized resource
+    Returns:
+      An object that contains only the changes from original to modified, in a
+      form suitable to pass to a PATCH method.
+
+    Example usage:
+      item = service.activities().get(postid=postid, userid=userid).execute()
+      original = copy.deepcopy(item)
+      item['object']['content'] = 'This is updated.'
+      service.activities.patch(postid=postid, userid=userid,
+        body=makepatch(original, item)).execute()
+    """
+    patch = {}
+    for key, original_value in original.items():
+        modified_value = modified.get(key, None)
+        if modified_value is None:
+            # Use None to signal that the element is deleted
+            patch[key] = None
+        elif original_value != modified_value:
+            if type(original_value) == type({}):
+                # Recursively descend objects
+                patch[key] = makepatch(original_value, modified_value)
+            else:
+                # In the case of simple types or arrays we just replace
+                patch[key] = modified_value
+        else:
+            # Don't add anything to patch if there's no change
+            pass
+    for key in modified:
+        if key not in original:
+            patch[key] = modified[key]
+
+    return patch

src/gam/googleapiclient/schema.py

@@ -0,0 +1,317 @@
+# Copyright 2014 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Schema processing for discovery based APIs
+
+Schemas holds an APIs discovery schemas. It can return those schema as
+deserialized JSON objects, or pretty print them as prototype objects that
+conform to the schema.
+
+For example, given the schema:
+
+ schema = \"\"\"{
+   "Foo": {
+    "type": "object",
+    "properties": {
+     "etag": {
+      "type": "string",
+      "description": "ETag of the collection."
+     },
+     "kind": {
+      "type": "string",
+      "description": "Type of the collection ('calendar#acl').",
+      "default": "calendar#acl"
+     },
+     "nextPageToken": {
+      "type": "string",
+      "description": "Token used to access the next
+         page of this result. Omitted if no further results are available."
+     }
+    }
+   }
+ }\"\"\"
+
+ s = Schemas(schema)
+ print s.prettyPrintByName('Foo')
+
+ Produces the following output:
+
+  {
+   "nextPageToken": "A String", # Token used to access the
+       # next page of this result. Omitted if no further results are available.
+   "kind": "A String", # Type of the collection ('calendar#acl').
+   "etag": "A String", # ETag of the collection.
+  },
+
+The constructor takes a discovery document in which to look up named schema.
+"""
+from __future__ import absolute_import
+
+# TODO(jcgregorio) support format, enum, minimum, maximum
+
+__author__ = "jcgregorio@google.com (Joe Gregorio)"
+
+
+from collections import OrderedDict
+
+from googleapiclient import _helpers as util
+
+
+class Schemas(object):
+    """Schemas for an API."""
+
+    def __init__(self, discovery):
+        """Constructor.
+
+        Args:
+          discovery: object, Deserialized discovery document from which we pull
+            out the named schema.
+        """
+        self.schemas = discovery.get("schemas", {})
+
+        # Cache of pretty printed schemas.
+        self.pretty = {}
+
+    @util.positional(2)
+    def _prettyPrintByName(self, name, seen=None, dent=0):
+        """Get pretty printed object prototype from the schema name.
+
+        Args:
+          name: string, Name of schema in the discovery document.
+          seen: list of string, Names of schema already seen. Used to handle
+            recursive definitions.
+
+        Returns:
+          string, A string that contains a prototype object with
+            comments that conforms to the given schema.
+        """
+        if seen is None:
+            seen = []
+
+        if name in seen:
+            # Do not fall into an infinite loop over recursive definitions.
+            return "# Object with schema name: %s" % name
+        seen.append(name)
+
+        if name not in self.pretty:
+            self.pretty[name] = _SchemaToStruct(
+                self.schemas[name], seen, dent=dent
+            ).to_str(self._prettyPrintByName)
+
+        seen.pop()
+
+        return self.pretty[name]
+
+    def prettyPrintByName(self, name):
+        """Get pretty printed object prototype from the schema name.
+
+        Args:
+          name: string, Name of schema in the discovery document.
+
+        Returns:
+          string, A string that contains a prototype object with
+            comments that conforms to the given schema.
+        """
+        # Return with trailing comma and newline removed.
+        return self._prettyPrintByName(name, seen=[], dent=0)[:-2]
+
+    @util.positional(2)
+    def _prettyPrintSchema(self, schema, seen=None, dent=0):
+        """Get pretty printed object prototype of schema.
+
+        Args:
+          schema: object, Parsed JSON schema.
+          seen: list of string, Names of schema already seen. Used to handle
+            recursive definitions.
+
+        Returns:
+          string, A string that contains a prototype object with
+            comments that conforms to the given schema.
+        """
+        if seen is None:
+            seen = []
+
+        return _SchemaToStruct(schema, seen, dent=dent).to_str(self._prettyPrintByName)
+
+    def prettyPrintSchema(self, schema):
+        """Get pretty printed object prototype of schema.
+
+        Args:
+          schema: object, Parsed JSON schema.
+
+        Returns:
+          string, A string that contains a prototype object with
+            comments that conforms to the given schema.
+        """
+        # Return with trailing comma and newline removed.
+        return self._prettyPrintSchema(schema, dent=0)[:-2]
+
+    def get(self, name, default=None):
+        """Get deserialized JSON schema from the schema name.
+
+        Args:
+          name: string, Schema name.
+          default: object, return value if name not found.
+        """
+        return self.schemas.get(name, default)
+
+
+class _SchemaToStruct(object):
+    """Convert schema to a prototype object."""
+
+    @util.positional(3)
+    def __init__(self, schema, seen, dent=0):
+        """Constructor.
+
+        Args:
+          schema: object, Parsed JSON schema.
+          seen: list, List of names of schema already seen while parsing. Used to
+            handle recursive definitions.
+          dent: int, Initial indentation depth.
+        """
+        # The result of this parsing kept as list of strings.
+        self.value = []
+
+        # The final value of the parsing.
+        self.string = None
+
+        # The parsed JSON schema.
+        self.schema = schema
+
+        # Indentation level.
+        self.dent = dent
+
+        # Method that when called returns a prototype object for the schema with
+        # the given name.
+        self.from_cache = None
+
+        # List of names of schema already seen while parsing.
+        self.seen = seen
+
+    def emit(self, text):
+        """Add text as a line to the output.
+
+        Args:
+          text: string, Text to output.
+        """
+        self.value.extend(["  " * self.dent, text, "\n"])
+
+    def emitBegin(self, text):
+        """Add text to the output, but with no line terminator.
+
+        Args:
+          text: string, Text to output.
+        """
+        self.value.extend(["  " * self.dent, text])
+
+    def emitEnd(self, text, comment):
+        """Add text and comment to the output with line terminator.
+
+        Args:
+          text: string, Text to output.
+          comment: string, Python comment.
+        """
+        if comment:
+            divider = "\n" + "  " * (self.dent + 2) + "# "
+            lines = comment.splitlines()
+            lines = [x.rstrip() for x in lines]
+            comment = divider.join(lines)
+            self.value.extend([text, " # ", comment, "\n"])
+        else:
+            self.value.extend([text, "\n"])
+
+    def indent(self):
+        """Increase indentation level."""
+        self.dent += 1
+
+    def undent(self):
+        """Decrease indentation level."""
+        self.dent -= 1
+
+    def _to_str_impl(self, schema):
+        """Prototype object based on the schema, in Python code with comments.
+
+        Args:
+          schema: object, Parsed JSON schema file.
+
+        Returns:
+          Prototype object based on the schema, in Python code with comments.
+        """
+        stype = schema.get("type")
+        if stype == "object":
+            self.emitEnd("{", schema.get("description", ""))
+            self.indent()
+            if "properties" in schema:
+                properties = schema.get("properties", {})
+                sorted_properties = OrderedDict(sorted(properties.items()))
+                for pname, pschema in sorted_properties.items():
+                    self.emitBegin('"%s": ' % pname)
+                    self._to_str_impl(pschema)
+            elif "additionalProperties" in schema:
+                self.emitBegin('"a_key": ')
+                self._to_str_impl(schema["additionalProperties"])
+            self.undent()
+            self.emit("},")
+        elif "$ref" in schema:
+            schemaName = schema["$ref"]
+            description = schema.get("description", "")
+            s = self.from_cache(schemaName, seen=self.seen)
+            parts = s.splitlines()
+            self.emitEnd(parts[0], description)
+            for line in parts[1:]:
+                self.emit(line.rstrip())
+        elif stype == "boolean":
+            value = schema.get("default", "True or False")
+            self.emitEnd("%s," % str(value), schema.get("description", ""))
+        elif stype == "string":
+            value = schema.get("default", "A String")
+            self.emitEnd('"%s",' % str(value), schema.get("description", ""))
+        elif stype == "integer":
+            value = schema.get("default", "42")
+            self.emitEnd("%s," % str(value), schema.get("description", ""))
+        elif stype == "number":
+            value = schema.get("default", "3.14")
+            self.emitEnd("%s," % str(value), schema.get("description", ""))
+        elif stype == "null":
+            self.emitEnd("None,", schema.get("description", ""))
+        elif stype == "any":
+            self.emitEnd('"",', schema.get("description", ""))
+        elif stype == "array":
+            self.emitEnd("[", schema.get("description"))
+            self.indent()
+            self.emitBegin("")
+            self._to_str_impl(schema["items"])
+            self.undent()
+            self.emit("],")
+        else:
+            self.emit("Unknown type! %s" % stype)
+            self.emitEnd("", "")
+
+        self.string = "".join(self.value)
+        return self.string
+
+    def to_str(self, from_cache):
+        """Prototype object based on the schema, in Python code with comments.
+
+        Args:
+          from_cache: callable(name, seen), Callable that retrieves an object
+             prototype for a schema with the given name. Seen is a list of schema
+             names already seen as we recursively descend the schema definition.
+
+        Returns:
+          Prototype object based on the schema, in Python code with comments.
+          The lines of the code will all be properly indented.
+        """
+        self.from_cache = from_cache
+        return self._to_str_impl(self.schema)

src/gam/googleapiclient/version.py

@@ -0,0 +1,15 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+__version__ = "2.114.0"