Updated gam create project

to simplify handling the situation when your workspace is configured to disable service account private key uploads.

docs/Authorization.md

@@ -31,6 +31,7 @@
   - [Update an existing Service Account key](#update-an-existing-service-account-key)
   - [Replace all existing Service Account keys](#replace-all-existing-service-account-keys)
   - [Delete Service Account keys](#delete-service-account-keys)
+  - [Upload a Service Account key to a service account with no keys](#upload-a-service-account-key-to-a-service-account-with-no-keys)
   - [Display Service Account keys](#display-service-account-keys)
 - [Manage Service Account access](#manage-service-account-access)
   - [Full Service Account access](#full-service-account-access)
@@ -209,8 +210,8 @@ perform these steps and then retry the create project command.
 
 ## Authorize Service Account Key Uploads
 
-If you try to create a project and get an error saying that Constraint `constraints/iam.disableServiceAccountKeyUpload violated for service account projects/gam-project-xxx`
+If you try to create a project and get an error saying that Constraint `constraints/iam.disableServiceAccountKeyUpload violated for service account projects/gam-project-xxx`,
-perform these steps and then retry the create project command.
+perform these steps and then you should be able to authorize and use your project.
 
 * Login as an existing super admin at console.cloud.google.com
 * In the upper left click the three lines to the left of Google Cloud and select IAM & Admin
@@ -242,9 +243,9 @@ perform these steps and then retry the create project command.
 * Click Done
 * Click Set Policy
 
-Do the following to upload the service account key:
+Wait a couple of minutes for the policy updates to complete and then do the following to upload the service account key:
 ```
-gam update sakey
+gam upload sakey
 ```
 
 ## Authorize GAM to create projects
@@ -856,10 +857,26 @@ delete a service account key for a distributed copy of an `oauth2service.json` f
 that user's service account access.
 
 You can disable your current Service Account key if you specify the `doit` argument. This is your
-acknowledgement that you will have to manually create a new Service Account key in the Developer's Console.
+acknowledgement that you will have to manually create a new Service Account key in the Developer's Console
+or upload a new key with the `gam upload sakey` command.
 ```
 gam delete sakeys <ServiceAccountKeyList>+ [doit]
 ```
+## Upload a Service Account key to a service account with no keys
+There are two cases where you will use this command:
+* Your workspace is configured to disable service account private key uploads.
+* All of your service account keys have been deleted, eith mamually or with the `gam delete sakeys` command.
+
+The `oauth2service.json` file is updated with the new private key. If you had previously distributed
+any `oauth2service.json` file to other users, you must redistribute the updated file with the new key.
+```
+gam upload sakey
+        (algorithm KEY_ALG_RSA_1024|KEY_ALG_RSA_2048)|
+        ((localkeysize 1024|2048|4096 [validityhours <Number>])|
+        (yubikey yubikey_pin yubikey_slot AUTHENTICATION 
+         yubikey_serialnumber <Number>
+         [localkeysize 1024|2048|4096])
+```
 ## Display Service Account keys
 There are system keys and user keys; user keys are what Gam uses; GCP uses system keys.
 

docs/GamUpdates.md

@@ -10,6 +10,12 @@ Add the `-s` option to the end of the above commands to suppress creating the `g
 
 See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation
 
+### 6.75.00
+
+Updated `gam create project` to simplify handling the situation when your workspace is configured to disable service account private key uploads.
+
+Added command `gam upload sakey` to aid in this process.
+
 ### 6.74.02
 
 Fixed bug in `gam <UserTypeEntity> print shareddrives ... formatjson` that caused a trap.

docs/How-to-Upgrade-from-Standard-GAM.md

@@ -335,7 +335,7 @@ writes the credentials into the file oauth2.txt.
 admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt
 admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version
 WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found
-GAMADV-XTD3 6.74.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.75.00 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.3 64-bit final
 MacOS Sonoma 14.4.1 x86_64
@@ -1009,7 +1009,7 @@ writes the credentials into the file oauth2.txt.
 C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt
 C:\GAMADV-XTD3>gam version
 WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found
-GAMADV-XTD3 6.74.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.75.00 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.3 64-bit final
 Windows-10-10.0.17134 AMD64

docs/Version-and-Help.md

@@ -3,7 +3,7 @@
 Print the current version of Gam with details
 ```
 gam version
-GAMADV-XTD3 6.74.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.75.00 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.3 64-bit final
 MacOS Sonoma 14.4.1 x86_64
@@ -15,7 +15,7 @@ Time: 2023-06-02T21:10:00-07:00
 Print the current version of Gam with details and time offset information
 ```
 gam version timeoffset
-GAMADV-XTD3 6.74.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.75.00 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.3 64-bit final
 MacOS Sonoma 14.4.1 x86_64
@@ -27,7 +27,7 @@ Your system time differs from www.googleapis.com by less than 1 second
 Print the current version of Gam with extended details and SSL information
 ```
 gam version extended
-GAMADV-XTD3 6.74.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
+GAMADV-XTD3 6.75.00 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.3 64-bit final
 MacOS Sonoma 14.4.1 x86_64
@@ -64,7 +64,7 @@ MacOS High Sierra 10.13.6 x86_64
 Path: /Users/Admin/bin/gamadv-xtd3
 Version Check:
   Current: 5.35.08
-   Latest: 6.74.02
+   Latest: 6.75.00
 echo $?
 1
 ```
@@ -72,7 +72,7 @@ echo $?
 Print the current version number without details
 ```
 gam version simple
-6.74.02
+6.75.00
 ```
 In Linux/MacOS you can do:
 ```
@@ -82,7 +82,7 @@ echo $VER
 Print the current version of Gam and address of this Wiki
 ```
 gam help
-GAM 6.74.02 - https://github.com/taers232c/GAMADV-XTD3
+GAM 6.75.00 - https://github.com/taers232c/GAMADV-XTD3
 Ross Scroggs <ross.scroggs@gmail.com>
 Python 3.12.3 64-bit final
 MacOS Sonoma 14.4.1 x86_64

src/GamCommands.txt

@@ -1406,6 +1406,13 @@ gam rotate sakey|sakeys retain_none
          yubikey_serialnumber <Number>
          [localkeysize 1024|2048|4096])
 
+gam upload sakey
+        (algorithm KEY_ALG_RSA_1024|KEY_ALG_RSA_2048)|
+        ((localkeysize 1024|2048|4096 [validityhours <Number>])|
+        (yubikey yubikey_pin yubikey_slot AUTHENTICATION|SIGNATURE
+         yubikey_serialnumber <Number>
+         [localkeysize 1024|2048|4096])
+
 gam delete sakeys <ServiceAccountKeyList>+ [doit]
 gam show sakeys [all|system|user]
 

src/GamUpdate.txt

@@ -2,6 +2,12 @@
 
 Merged GAM-Team version
 
+6.75.00
+
+Updated `gam create project` to simplify handling the situation when your workspace is configured to disable service account private key uploads.
+
+Added command `gam upload sakey` to aid in this process.
+
 6.74.02
 
 Fixed bug in `gam <UserTypeEntity> print shareddrives ... formatjson` that caused a trap.

src/gam/__init__.py

@@ -11251,8 +11251,6 @@ def _createClientSecretsOauth2service(httpObj, login_hint, appInfo, projectInfo,
     return
   if appInfo:
     setGAMProjectConsentScreen(httpObj, projectInfo['projectId'], appInfo)
-  if not _createOauth2serviceJSON(httpObj, projectInfo, svcAcctInfo):
-    return
   console_url = f'https://console.cloud.google.com/apis/credentials/oauthclient?project={projectInfo["projectId"]}&authuser={login_hint}'
   csHttpObj = getHttpObj()
   while True:
@@ -11284,6 +11282,8 @@ def _createClientSecretsOauth2service(httpObj, login_hint, appInfo, projectInfo,
   sys.stdout.write(Msg.GO_BACK_TO_YOUR_BROWSER_AND_CLICK_OK_TO_CLOSE_THE_OAUTH_CLIENT_POPUP)
   sys.stdout.write(Msg.TRUST_GAM_CLIENT_ID.format(GAM, client_id))
   readStdin('')
+  if not _createOauth2serviceJSON(httpObj, projectInfo, svcAcctInfo):
+    return
   sys.stdout.write(Msg.YOUR_GAM_PROJECT_IS_CREATED_AND_READY_TO_USE)
 
 def _getProjects(crm, pfilter, returnNF=False):
@@ -12335,8 +12335,9 @@ def doProcessSvcAcctKeys(mode=None, iam=None, projectId=None, clientEmail=None,
   local_key_size = 2048
   validityHours = 0
   body = {}
-  if iam is None:
+  if iam is None or mode == 'upload':
-    _, iam = buildGAPIServiceObject(API.IAM, None)
+    if iam is None:
+      _, iam = buildGAPIServiceObject(API.IAM, None)
     _getSvcAcctData()
     currentPrivateKeyId, projectId, clientEmail, clientId = _getSvcAcctKeyProjectClientFields()
     # dict() ensures we have a real copy, not pointer
@@ -12413,6 +12414,7 @@ def doProcessSvcAcctKeys(mode=None, iam=None, projectId=None, clientEmail=None,
         result = callGAPI(iam.projects().serviceAccounts().keys(), 'upload',
                           throwReasons=[GAPI.NOT_FOUND, GAPI.BAD_REQUEST, GAPI.PERMISSION_DENIED, GAPI.FAILED_PRECONDITION],
                           name=name, body={'publicKeyData': publicKeyData})
+        newPrivateKeyId = result['name'].rsplit('/', 1)[-1]
         break
       except GAPI.notFound as e:
         if retry == maxRetries:
@@ -12424,10 +12426,19 @@ def doProcessSvcAcctKeys(mode=None, iam=None, projectId=None, clientEmail=None,
           entityActionFailedWarning([Ent.PROJECT, projectId, Ent.SVCACCT, clientEmail], Msg.UPDATE_PROJECT_TO_VIEW_MANAGE_SAKEYS)
           return False
         waitForCompletion(retry)
-      except (GAPI.badRequest, GAPI.failedPrecondition) as e:
+      except GAPI.badRequest as e:
         entityActionFailedWarning([Ent.PROJECT, projectId, Ent.SVCACCT, clientEmail], str(e))
         return False
-    newPrivateKeyId = result['name'].rsplit('/', 1)[-1]
+      except GAPI.failedPrecondition as e:
+        entityActionFailedWarning([Ent.PROJECT, projectId, Ent.SVCACCT, clientEmail], str(e))
+        if 'iam.disableServiceAccountKeyUpload' not in str(e):
+          return False
+        if retry == maxRetries or mode != 'upload':
+          sys.stdout.write(Msg.ENABLE_SERVICE_ACCOUNT_PRIVATE_KEY_UPLOAD.format(projectId))
+          new_data['private_key'] = ''
+          newPrivateKeyId = ''
+          break
+        waitForCompletion(retry)
     new_data['private_key_id'] = newPrivateKeyId
     oauth2service_data = _formatOAuth2ServiceData(new_data)
   else:
@@ -12438,6 +12449,7 @@ def doProcessSvcAcctKeys(mode=None, iam=None, projectId=None, clientEmail=None,
         result = callGAPI(iam.projects().serviceAccounts().keys(), 'create',
                           throwReasons=[GAPI.BAD_REQUEST, GAPI.PERMISSION_DENIED],
                           name=name, body=body)
+        newPrivateKeyId = result['name'].rsplit('/', 1)[-1]
         break
       except GAPI.permissionDenied:
         if retry == maxRetries:
@@ -12447,9 +12459,9 @@ def doProcessSvcAcctKeys(mode=None, iam=None, projectId=None, clientEmail=None,
       except GAPI.badRequest as e:
         entityActionFailedWarning([Ent.PROJECT, projectId, Ent.SVCACCT, clientEmail], str(e))
         return False
-    newPrivateKeyId = result['name'].rsplit('/', 1)[-1]
     oauth2service_data = base64.b64decode(result['privateKeyData']).decode(UTF8)
-  entityActionPerformed([Ent.PROJECT, projectId, Ent.SVCACCT, clientEmail, Ent.SVCACCT_KEY, newPrivateKeyId])
+  if newPrivateKeyId != '':
+    entityActionPerformed([Ent.PROJECT, projectId, Ent.SVCACCT, clientEmail, Ent.SVCACCT_KEY, newPrivateKeyId])
   if GM.Globals[GM.SVCACCT_SCOPES_DEFINED]:
     try:
       GM.Globals[GM.OAUTH2SERVICE_JSON_DATA] = json.loads(oauth2service_data)
@@ -12461,35 +12473,36 @@ def doProcessSvcAcctKeys(mode=None, iam=None, projectId=None, clientEmail=None,
   Act.Set(Act.UPDATE)
   entityActionPerformed([Ent.OAUTH2SERVICE_JSON_FILE, GC.Values[GC.OAUTH2SERVICE_JSON],
                          Ent.SVCACCT_KEY, newPrivateKeyId])
-  if mode != 'retainexisting':
+  if mode in {'retainexisting', 'upload'}:
-    Act.Set(Act.REVOKE)
+    return newPrivateKeyId != ''
-    count = len(keys) if mode == 'retainnone' else 1
+  Act.Set(Act.REVOKE)
-    entityPerformActionNumItems([Ent.PROJECT, projectId, Ent.SVCACCT, clientEmail], count, Ent.SVCACCT_KEY)
+  count = len(keys) if mode == 'retainnone' else 1
-    Ind.Increment()
+  entityPerformActionNumItems([Ent.PROJECT, projectId, Ent.SVCACCT, clientEmail], count, Ent.SVCACCT_KEY)
-    i = 0
+  Ind.Increment()
-    for key in keys:
+  i = 0
-      keyName = key['name'].rsplit('/', 1)[-1]
+  for key in keys:
-      if mode == 'retainnone' or keyName == currentPrivateKeyId and keyName != newPrivateKeyId:
+    keyName = key['name'].rsplit('/', 1)[-1]
-        i += 1
+    if mode == 'retainnone' or keyName == currentPrivateKeyId and keyName != newPrivateKeyId:
-        maxRetries = 5
+      i += 1
-        for retry in range(1, maxRetries+1):
+      maxRetries = 5
-          try:
+      for retry in range(1, maxRetries+1):
-            callGAPI(iam.projects().serviceAccounts().keys(), 'delete',
+        try:
-                     throwReasons=[GAPI.BAD_REQUEST, GAPI.PERMISSION_DENIED],
+          callGAPI(iam.projects().serviceAccounts().keys(), 'delete',
-                     name=key['name'])
+                   throwReasons=[GAPI.BAD_REQUEST, GAPI.PERMISSION_DENIED],
-            entityActionPerformed([Ent.SVCACCT_KEY, keyName], i, count)
+                   name=key['name'])
-            break
+          entityActionPerformed([Ent.SVCACCT_KEY, keyName], i, count)
-          except GAPI.permissionDenied:
-            if retry == maxRetries:
-              entityActionFailedWarning([Ent.SVCACCT_KEY, keyName], Msg.UPDATE_PROJECT_TO_VIEW_MANAGE_SAKEYS)
-              break
-            waitForCompletion(retry)
-          except GAPI.badRequest as e:
-            entityActionFailedWarning([Ent.SVCACCT_KEY, keyName], str(e), i, count)
-            break
-        if mode != 'retainnone':
           break
-    Ind.Decrement()
+        except GAPI.permissionDenied:
+          if retry == maxRetries:
+            entityActionFailedWarning([Ent.SVCACCT_KEY, keyName], Msg.UPDATE_PROJECT_TO_VIEW_MANAGE_SAKEYS)
+            break
+          waitForCompletion(retry)
+        except GAPI.badRequest as e:
+          entityActionFailedWarning([Ent.SVCACCT_KEY, keyName], str(e), i, count)
+          break
+      if mode != 'retainnone':
+        break
+  Ind.Decrement()
   return True
 
 # gam create sakey|sakeys
@@ -12524,6 +12537,20 @@ def doUpdateSvcAcctKeys():
 def doReplaceSvcAcctKeys():
   doProcessSvcAcctKeys(mode='retainnone')
 
+# gam upload sakey|sakeys
+#	(algorithm KEY_ALG_RSA_1024|KEY_ALG_RSA_2048)|
+#	((localkeysize 1024|2048|4096 [validityhours <Number>])|
+#	(yubikey yubikey_pin yubikey_slot AUTHENTICATION
+#	 yubikey_serialnumber <String>
+#	 [localkeysize 1024|2048|4096])
+def doUploadSvcAcctKeys():
+  _, httpObj, _, _, _, _ = _getLoginHintProjectInfo(True)
+  iam = getAPIService(API.IAM, httpObj)
+  if doProcessSvcAcctKeys(mode='upload', iam=iam):
+    sa_email = GM.Globals[GM.OAUTH2SERVICE_JSON_DATA]['client_email']
+    _grantRotateRights(iam, GM.Globals[GM.OAUTH2SERVICE_JSON_DATA]['project_id'], sa_email, sa_email)
+    sys.stdout.write(Msg.YOUR_GAM_PROJECT_IS_CREATED_AND_READY_TO_USE)
+
 # gam delete sakeys <ServiceAccountKeyList>
 def doDeleteSvcAcctKeys():
   _, iam = buildGAPIServiceObject(API.IAM, None)
@@ -73215,6 +73242,11 @@ MAIN_COMMANDS_WITH_OBJECTS = {
       Cmd.ARG_USERS:		doSuspendUnsuspendUsers,
      }
     ),
+  'upload':
+    (Act.USE,
+     {Cmd.ARG_SAKEY:		doUploadSvcAcctKeys,
+     }
+    ),
   'use':
     (Act.USE,
      {Cmd.ARG_PROJECT:		doUseProject,

src/gam/gamlib/glmsgs.py

@@ -91,7 +91,21 @@ Please go to:
 8. Press enter here on the terminal once trust is complete.
 '''
 
-YOUR_GAM_PROJECT_IS_CREATED_AND_READY_TO_USE = 'That\'s it! Your GAM Project is created and ready to use.\n'
+ENABLE_SERVICE_ACCOUNT_PRIVATE_KEY_UPLOAD = '''
+Your workspace is configured to disable service account private key uploads.
+
+Please go to:
+
+  https://github.com/taers232c/GAMADV-XTD3/wiki/Authorization#authorize-service-account-key-uploads
+
+Follow the steps to allow a service account private key upload for the project ({0}) just created.
+Once those steps are completed, you can continue with your project authentication.
+'''
+
+YOUR_GAM_PROJECT_IS_CREATED_AND_READY_TO_USE = '''
+That\'s it! Your GAM Project is created and ready to use.
+Proceed to the authentication steps.
+'''
 
 # check|update service messages in order of appearance
 SYSTEM_TIME_STATUS = 'System time status'