Updated gam print groups ... ciallfields|(cifields <CIGroupFieldNameList>) to account for an API shortcoming that failed to get all of the Cloud Identity fields.

2025-07-04 03:33:44 +00:00 · 2024-02-12 11:28:09 -08:00
parent 1c9f65f7ca
commit be3c6f10c7
7 changed files with 41 additions and 55 deletions
--- a/docs/GamUpdates.md
+++ b/docs/GamUpdates.md
@ -10,6 +10,11 @@ Add the `-s` option to the end of the above commands to suppress creating the `g

 See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation

+### 6.68.05
+
+Updated `gam print groups ... ciallfields|(cifields <CIGroupFieldNameList>)` to account for an
+API shortcoming that failed to get all of the Cloud Identity fields.
+
 ### 6.68.04

 Added option `skiprows <Integer>` to `gam csv|loop` that causes GAM to skip processing the first `<Integer>` filtered rows.
--- a/docs/How-to-Upgrade-from-Standard-GAM.md
+++ b/docs/How-to-Upgrade-from-Standard-GAM.md
@ -334,7 +334,7 @@ writes the credentials into the file oauth2.txt.
 admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt
 admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version
 WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found
-GAMADV-XTD3 6.68.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.68.05 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 MacOS Sonoma 14.2.1 x86_64
@ -1002,7 +1002,7 @@ writes the credentials into the file oauth2.txt.
 C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt
 C:\GAMADV-XTD3>gam version
 WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found
-GAMADV-XTD3 6.68.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.68.05 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 Windows-10-10.0.17134 AMD64
--- a/docs/Version-and-Help.md
+++ b/docs/Version-and-Help.md
@ -3,7 +3,7 @@
 Print the current version of Gam with details
 ```
 gam version
-GAMADV-XTD3 6.68.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.68.05 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 MacOS Sonoma 14.2.1 x86_64
@ -15,7 +15,7 @@ Time: 2023-06-02T21:10:00-07:00
 Print the current version of Gam with details and time offset information
 ```
 gam version timeoffset
-GAMADV-XTD3 6.68.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.68.05 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 MacOS Sonoma 14.2.1 x86_64
@ -27,7 +27,7 @@ Your system time differs from www.googleapis.com by less than 1 second
 Print the current version of Gam with extended details and SSL information
 ```
 gam version extended
-GAMADV-XTD3 6.68.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.68.05 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 MacOS Sonoma 14.2.1 x86_64
@ -64,7 +64,7 @@ MacOS High Sierra 10.13.6 x86_64
 Path: /Users/Admin/bin/gamadv-xtd3
 Version Check:
  Current: 5.35.08
-   Latest: 6.68.04
+   Latest: 6.68.05
 echo $?
 1
 ```
@ -72,7 +72,7 @@ echo $?
 Print the current version number without details
 ```
 gam version simple
-6.68.04
+6.68.05
 ```
 In Linux/MacOS you can do:
 ```
@ -82,7 +82,7 @@ echo $VER
 Print the current version of Gam and address of this Wiki
 ```
 gam help
-GAM 6.68.04 - https://github.com/taers232c/GAMADV-XTD3
+GAM 6.68.05 - https://github.com/taers232c/GAMADV-XTD3
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 MacOS Sonoma 14.2.1 x86_64
--- a/docs/gam.cfg.md
+++ b/docs/gam.cfg.md
@ -213,7 +213,7 @@ csv_output_header_force
        for inclusion in the CSV file written by a gam print command
        Default: ''
 csv_output_line_terminator
-p        Allowed values: cr, lf, crlf
+        Allowed values: cr, lf, crlf
        Designates character(s) used to terminate the lines of a CSV file.
        For Linux and Mac OS, this would typically be lf.
        For Windows, this would typically be crlf.
--- a/src/GamUpdate.txt
+++ b/src/GamUpdate.txt
@ -2,6 +2,11 @@

 Merged GAM-Team version

+6.68.05
+
+Updated `gam print groups ... ciallfields|(cifields <CIGroupFieldNameList>)` to account for an
+API shortcoming that failed to get all of the Cloud Identity fields.
+
 6.68.04

 Added option `skiprows <Integer>` to `gam csv|loop` that causes GAM to skip processing the first `<Integer>` filtered rows.
--- a/src/gam/init.py
+++ b/src/gam/init.py
@ -108,11 +108,6 @@ from filelock import FileLock

 from pathvalidate import sanitize_filename, sanitize_filepath

-# Do not import system library, includes discovery documents
-#import googleapiclient
-#import googleapiclient.discovery
-#import googleapiclient.errors
-#import googleapiclient.http
 import google.oauth2.credentials
 import google.oauth2.id_token
 import google.auth
@ -31804,7 +31799,6 @@ def doPrintGroups():
    if not showCIgroupKey:
      groupFieldsLists['ci'].append('groupKey(id)')
    cifields = ','.join(set(groupFieldsLists['ci']))
-    cifieldsnp = f'nextPageToken,groups({cifields})'
    ci = buildGAPIObject(API.CLOUDIDENTITY_GROUPS)
  getRolesSet = rolesSet.copy()
  if showOwnedBy:
@ -31843,24 +31837,6 @@ def doPrintGroups():
          badRequestWarning(Ent.GROUP, Ent.DOMAIN, kwargs['domain'])
        else:
          accessErrorExit(cd)
-      if getCloudIdentity:
-        printGettingAllAccountEntities(Ent.CLOUD_IDENTITY_GROUP)
-        try:
-          ciGroupList = callGAPIpages(ci.groups(), 'list', 'groups',
-                                      pageMessage=getPageMessage(showFirstLastItems=True), messageAttribute=['groupKey', 'id'],
-                                      throwReasons=GAPI.CIGROUP_LIST_THROW_REASONS, retryReasons=GAPI.CIGROUP_RETRY_REASONS,
-                                      parent=f'customers/{GC.Values[GC.CUSTOMER_ID]}', view='FULL',
-                                      fields=cifieldsnp, pageSize=500)
-        except (GAPI.resourceNotFound, GAPI.domainNotFound, GAPI.domainCannotUseApis,
-                GAPI.forbidden, GAPI.badRequest, GAPI.invalid, GAPI.invalidArgument,
-                GAPI.systemError, GAPI.permissionDenied, GAPI.serviceNotAvailable) as e:
-          accessErrorExitNonDirectory(API.CLOUDIDENTITY_GROUPS, str(e))
-        for ciGroup in ciGroupList:
-          key = ciGroup['groupKey']['id']
-          if not showCIgroupKey:
-            ciGroup.pop('groupKey')
-          ciGroups[key] = ciGroup.copy()
-        del ciGroupList
  else:
    svcargs = dict([('groupKey', None), ('fields', cdfields)]+GM.Globals[GM.EXTRA_ARGS_LIST])
    cdmethod = getattr(cd.groups(), 'get')
@ -31881,22 +31857,6 @@ def doPrintGroups():
        executeBatch(cdbatch)
        cdbatch = cd.new_batch_http_request(callback=_callbackProcessGroupBasic)
        cdbcount = 0
-      if getCloudIdentity:
-        _, name, groupEmail = convertGroupEmailToCloudID(ci, groupEmail, i, count)
-        printGettingEntityItemForWhom(Ent.CLOUD_IDENTITY_GROUP, groupEmail, i, count)
-        if name:
-          try:
-            ciGroup = callGAPI(ci.groups(), 'get',
-                               throwReasons=GAPI.CIGROUP_GET_THROW_REASONS, retryReasons=GAPI.CIGROUP_RETRY_REASONS,
-                               name=name, fields=cifields)
-            key = ciGroup['groupKey']['id']
-            if not showCIgroupKey:
-              ciGroup.pop('groupKey')
-            ciGroups[key] = ciGroup
-          except (GAPI.notFound, GAPI.domainNotFound, GAPI.domainCannotUseApis,
-                  GAPI.forbidden, GAPI.badRequest, GAPI.invalid,
-                  GAPI.systemError, GAPI.permissionDenied, GAPI.serviceNotAvailable) as e:
-            entityActionFailedWarning([Ent.GROUP, groupEmail, Ent.CLOUD_IDENTITY_GROUP, None], str(e), i, count)
    if cdbcount > 0:
      cdbatch.execute()
  required = 0
@ -31927,6 +31887,22 @@ def doPrintGroups():
    if not showDetails:
      _printGroupRow(groupEntity, None, None)
      continue
+    if getCloudIdentity:
+      _, name, groupEmail = convertGroupEmailToCloudID(ci, groupEmail, i, count)
+      printGettingEntityItemForWhom(Ent.CLOUD_IDENTITY_GROUP, groupEmail, i, count)
+      if name:
+        try:
+          ciGroup = callGAPI(ci.groups(), 'get',
+                             throwReasons=GAPI.CIGROUP_GET_THROW_REASONS, retryReasons=GAPI.CIGROUP_RETRY_REASONS,
+                             name=name, fields=cifields)
+          key = ciGroup['groupKey']['id']
+          if not showCIgroupKey:
+            ciGroup.pop('groupKey')
+          ciGroups[key] = ciGroup
+        except (GAPI.notFound, GAPI.domainNotFound, GAPI.domainCannotUseApis,
+                GAPI.forbidden, GAPI.badRequest, GAPI.invalid,
+                GAPI.systemError, GAPI.permissionDenied, GAPI.serviceNotAvailable) as e:
+          entityActionFailedWarning([Ent.GROUP, groupEmail, Ent.CLOUD_IDENTITY_GROUP, None], str(e), i, count)
    groupData[i] = {'entity': groupEntity, 'cloudIdentity': {}, 'settings': getSettings, 'members': [], 'required': required}
    if getRoles:
      printGettingEntityItemForWhom(getRoles, groupEmail, i, count)
@ -33075,7 +33051,7 @@ def doUpdateCIGroups():
          memberEmail = convertUIDtoEmailAddress(member, cd=cd, emailTypes='any', checkForCustomerId=True)
          try:
            memberName = callGAPI(ci.groups().memberships(), 'lookup',
-                                  throwReasons=GAPI.GROUP_GET_THROW_REASONS+[GAPI.NOT_FOUND, GAPI.PERMISSION_DENIED],
+                                  throwReasons=GAPI.CIGROUP_GET_THROW_REASONS,
                                  parent=parent, memberKey_id=member, fields='name').get('name')
            callGAPI(ci.groups().memberships(), 'delete',
                     throwReasons=GAPI.MEMBERS_THROW_REASONS+[GAPI.FAILED_PRECONDITION],
@ -33230,7 +33206,7 @@ def doUpdateCIGroups():
          memberEmail = convertUIDtoEmailAddress(member, cd=cd, emailTypes='any', checkForCustomerId=True)
          try:
            memberName = callGAPI(ci.groups().memberships(), 'lookup',
-                                  throwReasons=GAPI.GROUP_GET_THROW_REASONS+[GAPI.NOT_FOUND, GAPI.PERMISSION_DENIED],
+                                  throwReasons=GAPI.CIGROUP_GET_THROW_REASONS,
                                  parent=parent, memberKey_id=memberEmail, fields='name').get('name')
          except (GAPI.groupNotFound, GAPI.domainNotFound, GAPI.domainCannotUseApis, GAPI.invalid, GAPI.forbidden,
                  GAPI.notFound, GAPI.memberNotFound, GAPI.permissionDenied, GAPI.invalidMember) as e:
@ -48035,11 +48011,11 @@ def _showCalendar(calendar, j, jcount, FJQC, acls=None):
 def _processCalendarList(user, calId, j, jcount, cal, function, **kwargs):
  try:
    callGAPI(cal.calendarList(), function,
-             throwReasons=[GAPI.NOT_FOUND, GAPI.DUPLICATE, GAPI.UNKNOWN_ERROR,
+             throwReasons=[GAPI.NOT_FOUND, GAPI.DUPLICATE, GAPI.UNKNOWN_ERROR, GAPI.SERVICE_NOT_AVAILABLE,
                           GAPI.CANNOT_CHANGE_OWN_ACL, GAPI.CANNOT_CHANGE_OWN_PRIMARY_SUBSCRIPTION],
             **kwargs)
    entityActionPerformed([Ent.USER, user, Ent.CALENDAR, calId], j, jcount)
-  except (GAPI.notFound, GAPI.duplicate, GAPI.unknownError,
+  except (GAPI.notFound, GAPI.duplicate, GAPI.unknownError, GAPI.serviceNotAvailable,
          GAPI.cannotChangeOwnAcl, GAPI.cannotChangeOwnPrimarySubscription) as e:
    entityActionFailedWarning([Ent.USER, user, Ent.CALENDAR, calId], str(e), j, jcount)

--- a/src/gam/gamlib/glgapi.py
+++ b/src/gam/gamlib/glgapi.py
@ -184,7 +184,7 @@ ACTIVITY_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST]
 ALERT_THROW_REASONS = [SERVICE_NOT_AVAILABLE, AUTH_ERROR]
 CALENDAR_THROW_REASONS = [SERVICE_NOT_AVAILABLE, AUTH_ERROR, NOT_A_CALENDAR_USER]
 CIGROUP_CREATE_THROW_REASONS = [SERVICE_NOT_AVAILABLE, ALREADY_EXISTS, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, INVALID, INVALID_ARGUMENT, PERMISSION_DENIED, FAILED_PRECONDITION]
-CIGROUP_GET_THROW_REASONS = [SERVICE_NOT_AVAILABLE, NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR, PERMISSION_DENIED]
+CIGROUP_GET_THROW_REASONS = [SERVICE_NOT_AVAILABLE, NOT_FOUND, GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR, PERMISSION_DENIED]
 CIGROUP_LIST_THROW_REASONS = [SERVICE_NOT_AVAILABLE, RESOURCE_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, INVALID_ARGUMENT, SYSTEM_ERROR, PERMISSION_DENIED]
 CIGROUP_LIST_USERKEY_THROW_REASONS = CIGROUP_LIST_THROW_REASONS+[INVALID_ARGUMENT]
 CIGROUP_UPDATE_THROW_REASONS = [SERVICE_NOT_AVAILABLE, NOT_FOUND, GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS,
@ -250,9 +250,9 @@ DRIVE3_MODIFY_LABEL_THROW_REASONS = DRIVE_USER_THROW_REASONS+[FILE_NOT_FOUND, NO
 GMAIL_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST]
 GMAIL_LIST_THROW_REASONS = [FAILED_PRECONDITION, PERMISSION_DENIED, INVALID, INVALID_ARGUMENT]
 GMAIL_SMIME_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST, INVALID_ARGUMENT, FORBIDDEN, NOT_FOUND, PERMISSION_DENIED]
+GROUP_GET_THROW_REASONS = [GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR]
 GROUP_GET_RETRY_REASONS = [INVALID, SYSTEM_ERROR, SERVICE_NOT_AVAILABLE]
 GROUP_CREATE_THROW_REASONS = [DUPLICATE, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, INVALID, INVALID_INPUT]
-GROUP_GET_THROW_REASONS = [GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR]
 GROUP_UPDATE_THROW_REASONS = [GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, INVALID, INVALID_INPUT]
 GROUP_SETTINGS_THROW_REASONS = [NOT_FOUND, GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, SYSTEM_ERROR, PERMISSION_DENIED,
                                INVALID, INVALID_PARAMETER, INVALID_ATTRIBUTE_VALUE, INVALID_INPUT, SERVICE_LIMIT, SERVICE_NOT_AVAILABLE, AUTH_ERROR, REQUIRED]