Updated gam print groups ... ciallfields|(cifields <CIGroupFieldNameList>) to account for an API shortcoming that failed to get all of the Cloud Identity fields.

docs/GamUpdates.md

@@ -10,6 +10,11 @@ Add the `-s` option to the end of the above commands to suppress creating the `g
 
 See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation
 
+### 6.68.05
+
+Updated `gam print groups ... ciallfields|(cifields <CIGroupFieldNameList>)` to account for an
+API shortcoming that failed to get all of the Cloud Identity fields.
+
 ### 6.68.04
 
 Added option `skiprows <Integer>` to `gam csv|loop` that causes GAM to skip processing the first `<Integer>` filtered rows.

docs/How-to-Upgrade-from-Standard-GAM.md

@@ -334,7 +334,7 @@ writes the credentials into the file oauth2.txt.
 admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt
 admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version
 WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found
-GAMADV-XTD3 6.68.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.68.05 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 MacOS Sonoma 14.2.1 x86_64
@@ -1002,7 +1002,7 @@ writes the credentials into the file oauth2.txt.
 C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt
 C:\GAMADV-XTD3>gam version
 WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found
-GAMADV-XTD3 6.68.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.68.05 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 Windows-10-10.0.17134 AMD64

docs/Version-and-Help.md

@@ -3,7 +3,7 @@
 Print the current version of Gam with details
 ```
 gam version
-GAMADV-XTD3 6.68.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.68.05 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 MacOS Sonoma 14.2.1 x86_64
@@ -15,7 +15,7 @@ Time: 2023-06-02T21:10:00-07:00
 Print the current version of Gam with details and time offset information
 ```
 gam version timeoffset
-GAMADV-XTD3 6.68.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.68.05 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 MacOS Sonoma 14.2.1 x86_64
@@ -27,7 +27,7 @@ Your system time differs from www.googleapis.com by less than 1 second
 Print the current version of Gam with extended details and SSL information
 ```
 gam version extended
-GAMADV-XTD3 6.68.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.68.05 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 MacOS Sonoma 14.2.1 x86_64
@@ -64,7 +64,7 @@ MacOS High Sierra 10.13.6 x86_64
 Path: /Users/Admin/bin/gamadv-xtd3
 Version Check:
   Current: 5.35.08
-   Latest: 6.68.04
+   Latest: 6.68.05
 echo $?
 1
 ```
@@ -72,7 +72,7 @@ echo $?
 Print the current version number without details
 ```
 gam version simple
-6.68.04
+6.68.05
 ```
 In Linux/MacOS you can do:
 ```
@@ -82,7 +82,7 @@ echo $VER
 Print the current version of Gam and address of this Wiki
 ```
 gam help
-GAM 6.68.04 - https://github.com/taers232c/GAMADV-XTD3
+GAM 6.68.05 - https://github.com/taers232c/GAMADV-XTD3
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.2 64-bit final
 MacOS Sonoma 14.2.1 x86_64

docs/gam.cfg.md

@@ -213,7 +213,7 @@ csv_output_header_force
         for inclusion in the CSV file written by a gam print command
         Default: ''
 csv_output_line_terminator
-p        Allowed values: cr, lf, crlf
+        Allowed values: cr, lf, crlf
         Designates character(s) used to terminate the lines of a CSV file.
         For Linux and Mac OS, this would typically be lf.
         For Windows, this would typically be crlf.

src/GamUpdate.txt

@@ -2,6 +2,11 @@
 
 Merged GAM-Team version
 
+6.68.05
+
+Updated `gam print groups ... ciallfields|(cifields <CIGroupFieldNameList>)` to account for an
+API shortcoming that failed to get all of the Cloud Identity fields.
+
 6.68.04
 
 Added option `skiprows <Integer>` to `gam csv|loop` that causes GAM to skip processing the first `<Integer>` filtered rows.

src/gam/__init__.py

@@ -108,11 +108,6 @@ from filelock import FileLock
 
 from pathvalidate import sanitize_filename, sanitize_filepath
 
-# Do not import system library, includes discovery documents
-#import googleapiclient
-#import googleapiclient.discovery
-#import googleapiclient.errors
-#import googleapiclient.http
 import google.oauth2.credentials
 import google.oauth2.id_token
 import google.auth
@@ -31804,7 +31799,6 @@ def doPrintGroups():
     if not showCIgroupKey:
       groupFieldsLists['ci'].append('groupKey(id)')
     cifields = ','.join(set(groupFieldsLists['ci']))
-    cifieldsnp = f'nextPageToken,groups({cifields})'
     ci = buildGAPIObject(API.CLOUDIDENTITY_GROUPS)
   getRolesSet = rolesSet.copy()
   if showOwnedBy:
@@ -31843,24 +31837,6 @@ def doPrintGroups():
           badRequestWarning(Ent.GROUP, Ent.DOMAIN, kwargs['domain'])
         else:
           accessErrorExit(cd)
-      if getCloudIdentity:
-        printGettingAllAccountEntities(Ent.CLOUD_IDENTITY_GROUP)
-        try:
-          ciGroupList = callGAPIpages(ci.groups(), 'list', 'groups',
-                                      pageMessage=getPageMessage(showFirstLastItems=True), messageAttribute=['groupKey', 'id'],
-                                      throwReasons=GAPI.CIGROUP_LIST_THROW_REASONS, retryReasons=GAPI.CIGROUP_RETRY_REASONS,
-                                      parent=f'customers/{GC.Values[GC.CUSTOMER_ID]}', view='FULL',
-                                      fields=cifieldsnp, pageSize=500)
-        except (GAPI.resourceNotFound, GAPI.domainNotFound, GAPI.domainCannotUseApis,
-                GAPI.forbidden, GAPI.badRequest, GAPI.invalid, GAPI.invalidArgument,
-                GAPI.systemError, GAPI.permissionDenied, GAPI.serviceNotAvailable) as e:
-          accessErrorExitNonDirectory(API.CLOUDIDENTITY_GROUPS, str(e))
-        for ciGroup in ciGroupList:
-          key = ciGroup['groupKey']['id']
-          if not showCIgroupKey:
-            ciGroup.pop('groupKey')
-          ciGroups[key] = ciGroup.copy()
-        del ciGroupList
   else:
     svcargs = dict([('groupKey', None), ('fields', cdfields)]+GM.Globals[GM.EXTRA_ARGS_LIST])
     cdmethod = getattr(cd.groups(), 'get')
@@ -31881,22 +31857,6 @@ def doPrintGroups():
         executeBatch(cdbatch)
         cdbatch = cd.new_batch_http_request(callback=_callbackProcessGroupBasic)
         cdbcount = 0
-      if getCloudIdentity:
-        _, name, groupEmail = convertGroupEmailToCloudID(ci, groupEmail, i, count)
-        printGettingEntityItemForWhom(Ent.CLOUD_IDENTITY_GROUP, groupEmail, i, count)
-        if name:
-          try:
-            ciGroup = callGAPI(ci.groups(), 'get',
-                               throwReasons=GAPI.CIGROUP_GET_THROW_REASONS, retryReasons=GAPI.CIGROUP_RETRY_REASONS,
-                               name=name, fields=cifields)
-            key = ciGroup['groupKey']['id']
-            if not showCIgroupKey:
-              ciGroup.pop('groupKey')
-            ciGroups[key] = ciGroup
-          except (GAPI.notFound, GAPI.domainNotFound, GAPI.domainCannotUseApis,
-                  GAPI.forbidden, GAPI.badRequest, GAPI.invalid,
-                  GAPI.systemError, GAPI.permissionDenied, GAPI.serviceNotAvailable) as e:
-            entityActionFailedWarning([Ent.GROUP, groupEmail, Ent.CLOUD_IDENTITY_GROUP, None], str(e), i, count)
     if cdbcount > 0:
       cdbatch.execute()
   required = 0
@@ -31927,6 +31887,22 @@ def doPrintGroups():
     if not showDetails:
       _printGroupRow(groupEntity, None, None)
       continue
+    if getCloudIdentity:
+      _, name, groupEmail = convertGroupEmailToCloudID(ci, groupEmail, i, count)
+      printGettingEntityItemForWhom(Ent.CLOUD_IDENTITY_GROUP, groupEmail, i, count)
+      if name:
+        try:
+          ciGroup = callGAPI(ci.groups(), 'get',
+                             throwReasons=GAPI.CIGROUP_GET_THROW_REASONS, retryReasons=GAPI.CIGROUP_RETRY_REASONS,
+                             name=name, fields=cifields)
+          key = ciGroup['groupKey']['id']
+          if not showCIgroupKey:
+            ciGroup.pop('groupKey')
+          ciGroups[key] = ciGroup
+        except (GAPI.notFound, GAPI.domainNotFound, GAPI.domainCannotUseApis,
+                GAPI.forbidden, GAPI.badRequest, GAPI.invalid,
+                GAPI.systemError, GAPI.permissionDenied, GAPI.serviceNotAvailable) as e:
+          entityActionFailedWarning([Ent.GROUP, groupEmail, Ent.CLOUD_IDENTITY_GROUP, None], str(e), i, count)
     groupData[i] = {'entity': groupEntity, 'cloudIdentity': {}, 'settings': getSettings, 'members': [], 'required': required}
     if getRoles:
       printGettingEntityItemForWhom(getRoles, groupEmail, i, count)
@@ -33075,7 +33051,7 @@ def doUpdateCIGroups():
           memberEmail = convertUIDtoEmailAddress(member, cd=cd, emailTypes='any', checkForCustomerId=True)
           try:
             memberName = callGAPI(ci.groups().memberships(), 'lookup',
-                                  throwReasons=GAPI.GROUP_GET_THROW_REASONS+[GAPI.NOT_FOUND, GAPI.PERMISSION_DENIED],
+                                  throwReasons=GAPI.CIGROUP_GET_THROW_REASONS,
                                   parent=parent, memberKey_id=member, fields='name').get('name')
             callGAPI(ci.groups().memberships(), 'delete',
                      throwReasons=GAPI.MEMBERS_THROW_REASONS+[GAPI.FAILED_PRECONDITION],
@@ -33230,7 +33206,7 @@ def doUpdateCIGroups():
           memberEmail = convertUIDtoEmailAddress(member, cd=cd, emailTypes='any', checkForCustomerId=True)
           try:
             memberName = callGAPI(ci.groups().memberships(), 'lookup',
-                                  throwReasons=GAPI.GROUP_GET_THROW_REASONS+[GAPI.NOT_FOUND, GAPI.PERMISSION_DENIED],
+                                  throwReasons=GAPI.CIGROUP_GET_THROW_REASONS,
                                   parent=parent, memberKey_id=memberEmail, fields='name').get('name')
           except (GAPI.groupNotFound, GAPI.domainNotFound, GAPI.domainCannotUseApis, GAPI.invalid, GAPI.forbidden,
                   GAPI.notFound, GAPI.memberNotFound, GAPI.permissionDenied, GAPI.invalidMember) as e:
@@ -48035,11 +48011,11 @@ def _showCalendar(calendar, j, jcount, FJQC, acls=None):
 def _processCalendarList(user, calId, j, jcount, cal, function, **kwargs):
   try:
     callGAPI(cal.calendarList(), function,
-             throwReasons=[GAPI.NOT_FOUND, GAPI.DUPLICATE, GAPI.UNKNOWN_ERROR,
+             throwReasons=[GAPI.NOT_FOUND, GAPI.DUPLICATE, GAPI.UNKNOWN_ERROR, GAPI.SERVICE_NOT_AVAILABLE,
                            GAPI.CANNOT_CHANGE_OWN_ACL, GAPI.CANNOT_CHANGE_OWN_PRIMARY_SUBSCRIPTION],
              **kwargs)
     entityActionPerformed([Ent.USER, user, Ent.CALENDAR, calId], j, jcount)
-  except (GAPI.notFound, GAPI.duplicate, GAPI.unknownError,
+  except (GAPI.notFound, GAPI.duplicate, GAPI.unknownError, GAPI.serviceNotAvailable,
           GAPI.cannotChangeOwnAcl, GAPI.cannotChangeOwnPrimarySubscription) as e:
     entityActionFailedWarning([Ent.USER, user, Ent.CALENDAR, calId], str(e), j, jcount)
 

src/gam/gamlib/glgapi.py

@@ -184,7 +184,7 @@ ACTIVITY_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST]
 ALERT_THROW_REASONS = [SERVICE_NOT_AVAILABLE, AUTH_ERROR]
 CALENDAR_THROW_REASONS = [SERVICE_NOT_AVAILABLE, AUTH_ERROR, NOT_A_CALENDAR_USER]
 CIGROUP_CREATE_THROW_REASONS = [SERVICE_NOT_AVAILABLE, ALREADY_EXISTS, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, INVALID, INVALID_ARGUMENT, PERMISSION_DENIED, FAILED_PRECONDITION]
-CIGROUP_GET_THROW_REASONS = [SERVICE_NOT_AVAILABLE, NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR, PERMISSION_DENIED]
+CIGROUP_GET_THROW_REASONS = [SERVICE_NOT_AVAILABLE, NOT_FOUND, GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR, PERMISSION_DENIED]
 CIGROUP_LIST_THROW_REASONS = [SERVICE_NOT_AVAILABLE, RESOURCE_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, INVALID_ARGUMENT, SYSTEM_ERROR, PERMISSION_DENIED]
 CIGROUP_LIST_USERKEY_THROW_REASONS = CIGROUP_LIST_THROW_REASONS+[INVALID_ARGUMENT]
 CIGROUP_UPDATE_THROW_REASONS = [SERVICE_NOT_AVAILABLE, NOT_FOUND, GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS,
@@ -250,9 +250,9 @@ DRIVE3_MODIFY_LABEL_THROW_REASONS = DRIVE_USER_THROW_REASONS+[FILE_NOT_FOUND, NO
 GMAIL_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST]
 GMAIL_LIST_THROW_REASONS = [FAILED_PRECONDITION, PERMISSION_DENIED, INVALID, INVALID_ARGUMENT]
 GMAIL_SMIME_THROW_REASONS = [SERVICE_NOT_AVAILABLE, BAD_REQUEST, INVALID_ARGUMENT, FORBIDDEN, NOT_FOUND, PERMISSION_DENIED]
+GROUP_GET_THROW_REASONS = [GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR]
 GROUP_GET_RETRY_REASONS = [INVALID, SYSTEM_ERROR, SERVICE_NOT_AVAILABLE]
 GROUP_CREATE_THROW_REASONS = [DUPLICATE, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, INVALID, INVALID_INPUT]
-GROUP_GET_THROW_REASONS = [GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, BAD_REQUEST, INVALID, SYSTEM_ERROR]
 GROUP_UPDATE_THROW_REASONS = [GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, INVALID, INVALID_INPUT]
 GROUP_SETTINGS_THROW_REASONS = [NOT_FOUND, GROUP_NOT_FOUND, DOMAIN_NOT_FOUND, DOMAIN_CANNOT_USE_APIS, FORBIDDEN, SYSTEM_ERROR, PERMISSION_DENIED,
                                 INVALID, INVALID_PARAMETER, INVALID_ATTRIBUTE_VALUE, INVALID_INPUT, SERVICE_LIMIT, SERVICE_NOT_AVAILABLE, AUTH_ERROR, REQUIRED]