deepblue/GoogleDriveManagement
1
0
Fork 0
mirror of https://github.com/GAM-team/GAM.git synced 2026-06-18 21:21:37 +00:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity

sa email address cleanup

Browse Source
This commit is contained in:
Ross Scroggs
2024-06-26 07:07:11 -07:00
parent f8dafa294d
commit ab65890455
13 changed files with 104 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
docs/Bulk-Processing.md
View File

@@ -8,6 +8,7 @@
- [CSV files](#csv-files)
- [CSV files with redirection and select](#csv-files-with-redirection-and-select)
- [Automatic batch processing](#automatic-batch-processing)
- [Process Google Sheet commands and save results](#process-google-sheet-commands-and-save-results)
## Introduction
Batch and CSV file processing can improve performance by executing Gam commands in parallel.
@@ -156,3 +157,18 @@ If you want to select a `gam.cfg` section for the command, you must select and s
```
gam select <Section> save config auto_batch_min 1 redirect csv ./filelistperms.csv multiprocess group sales@domain.com print filelist fields id,name,mimetype,basicpermissions
```
## Process Google Sheet commands and save results
You want to process data from a Google Sheet tab and save the results to another tab in the same sheet.
Make a Google sheet with two tabs: Commands, Results; get the File ID and the two tab IDs.
Put your command data in the Commands tab.
Run your command, write the results to Results.txt
```
gam redirect stdout ./Results.txt multiprocess redirect stderr stdout csv gsheet user@domain.com <FileID> id:<CommandsTabID> gam ... Command
```
Upload Results.txt to the Results tab of the sheet.
```
gam user user@domain.com update drivefile <FileID> localfile Results.txt retainname gsheet id:<ResultsTabID>
```

11
docs/GamUpdates.md
View File

@@ -10,6 +10,17 @@ Add the `-s` option to the end of the above commands to suppress creating the `g
See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation
### 6.77.03
Thanks to jay, added the following Colab License SKUs:
```
1010500001 - Colab Pro
1010500002 - Colab Pro+
```
Thanks to Jay, updated `gam print|show admins` to properly display addresses
of service accounts with admin role assignments.
### 6.77.02
Cleaned up problems with some of the new Chat API asadmin commands.

4
docs/How-to-Upgrade-from-Standard-GAM.md
View File

@@ -335,7 +335,7 @@ writes the credentials into the file oauth2.txt.
admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt
admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version
WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found
GAMADV-XTD3 6.77.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.77.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.4 64-bit final
MacOS Sonoma 14.5 x86_64
@@ -1009,7 +1009,7 @@ writes the credentials into the file oauth2.txt.
C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt
C:\GAMADV-XTD3>gam version
WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found
GAMADV-XTD3 6.77.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.77.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.4 64-bit final
Windows-10-10.0.17134 AMD64

6
docs/Licenses.md
View File

@@ -24,6 +24,7 @@
| Cloud Identity Free | 101001 |
| Cloud Identity Premium | 101005 |
| Cloud Search | 101035 |
| Colab | 101050 |
| Education Endpoint Management | 101049 |
| Gemini | 101047 |
| Google Chrome Device Management | Google-Chrome-Device-Management |
@@ -49,6 +50,8 @@
| Cloud Identity Free | 1010010001 | cloudidentity |
| Cloud Identity Premium | 1010050001 | cloudidentitypremium |
| Cloud Search | 1010350001 | cloudsearch |
| Colab Pro | 1010500001 | colabpro |
| Colab Pro+ | 1010500002 | colabpro+ | colabproplus |
| Endpoint Education Upgrade | 1010490001 | eeu |
| G Suite Basic | Google-Apps-For-Business | gsuitebasic |
| G Suite Business | Google-Apps-Unlimited | gsuitebusiness |
@@ -123,6 +126,7 @@
101043 |
101047 |
101049 |
101050 |
Google-Apps |
Google-Chrome-Device-Management |
Google-Drive-storage |
@@ -151,6 +155,8 @@
cloudidentity | identity | 1010010001 | Cloud Identity |
cloudidentitypremium | identitypremium | 1010050001 | Cloud Identity Premium |
cloudsearch | 1010350001 | Cloud Search |
colabpro | 1010500001 | Colab Pro |
colabpro+ | colabproplus | 1010500002 | Colab Pro+ |
eeu | 1010490001 | SKU Endpoint Education Upgrade |
geminibiz | 1010470003 | Gemini Business |
geminiedu | 1010470004 | Gemini Education |

2
docs/Reseller.md
View File

@@ -70,6 +70,8 @@ Thanks to Duncan Isaksen-Loxton for a script to help manage multiple domains.
cloudidentity | identity | 1010010001 | Cloud Identity |
cloudidentitypremium | identitypremium | 1010050001 | Cloud Identity Premium |
cloudsearch | 1010350001 | Cloud Search |
colabpro | 1010500001 | Colab Pro |
colabpro+ | colabproplus | 1010500002 | Colab Pro+ |
eeu | 1010490001 | SKU Endpoint Education Upgrade |
geminibiz | 1010470003 | Gemini Business |
geminiedu | 1010470004 | Gemini Education |

2
docs/Users-Chat.md
View File

@@ -360,6 +360,8 @@ By default, Gam displays the information as an indented list of keys and values.
### Display information about all chat spaces, asadmin
For query and orderby information, see: https://developers.google.com/workspace/chat/api/reference/rest/v1/spaces/search
Only spaces of `<ChatSpaceType>` `space` are displayed; spaces of `<ChatSpaceType>` `groupchat` and `directmessage` are not displayed.
```
gam <UserItem> show chatspaces asadmin
[query <String>] [querytime<String> <Time>]

1
docs/Users-Drive-Files-Manage.md
View File

@@ -519,6 +519,7 @@ You can update a specific sheet within a Google spreadsheet or add a new sheet t
* `addsheet <String>` - Specify a sheet name to be added to the Google Sheets file
* `charset <Charset>` - Specify the character set of the local file; if not specified, the value of `charset` from `gam.cfg` will be used
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
If you want the Google spreadsheet to retain its name, specify: `retainname localfile LocalFile.csv`.
By default, the user, file name, updated file name and id values are displayed on stdout.

12
docs/Version-and-Help.md
View File

@@ -3,7 +3,7 @@
Print the current version of Gam with details
```
gam version
GAMADV-XTD3 6.77.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.77.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.4 64-bit final
MacOS Sonoma 14.5 x86_64
@@ -15,7 +15,7 @@ Time: 2023-06-02T21:10:00-07:00
Print the current version of Gam with details and time offset information
```
gam version timeoffset
GAMADV-XTD3 6.77.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.77.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.4 64-bit final
MacOS Sonoma 14.5 x86_64
@@ -27,7 +27,7 @@ Your system time differs from www.googleapis.com by less than 1 second
Print the current version of Gam with extended details and SSL information
```
gam version extended
GAMADV-XTD3 6.77.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.77.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.4 64-bit final
MacOS Sonoma 14.5 x86_64
@@ -64,7 +64,7 @@ MacOS High Sierra 10.13.6 x86_64
Path: /Users/Admin/bin/gamadv-xtd3
Version Check:
Current: 5.35.08
Latest: 6.77.02
Latest: 6.77.03
echo $?
1
```
@@ -72,7 +72,7 @@ echo $?
Print the current version number without details
```
gam version simple
6.77.02
6.77.03
```
In Linux/MacOS you can do:
```
@@ -82,7 +82,7 @@ echo $VER
Print the current version of Gam and address of this Wiki
```
gam help
GAM 6.77.02 - https://github.com/taers232c/GAMADV-XTD3
GAM 6.77.03 - https://github.com/taers232c/GAMADV-XTD3
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.4 64-bit final
MacOS Sonoma 14.5 x86_64

3
src/GamCommands.txt
View File

@@ -244,6 +244,7 @@ If an item contains spaces, it should be surrounded by ".
101043 |
101047 |
101049 |
101050 |
Google-Apps |
Google-Chrome-Device-Management |
Google-Drive-storage |
@@ -270,6 +271,8 @@ If an item contains spaces, it should be surrounded by ".
cloudidentity | identity | 1010010001 | Cloud Identity |
cloudidentitypremium | identitypremium | 1010050001 | Cloud Identity Premium |
cloudsearch | 1010350001 | Cloud Search |
colabpro | 1010500001 | Colab Pro |
colabpro+ | colabproplus | 1010500002 | Colab Pro+ |
eeu | 1010490001 | SKU Endpoint Education Upgrade |
geminibiz | 1010470003 | Gemini Business |
geminiedu | 1010470004 | Gemini Education |

11
src/GamUpdate.txt
View File

@@ -2,6 +2,17 @@
Merged GAM-Team version
6.77.03
Thanks to jay, added the following Colab License SKUs:
```
1010500001 - Colab Pro
1010500002 - Colab Pro+
```
Thanks to Jay, updated `gam print|show admins` to properly display addresses
of service accounts with admin role assignments.
6.77.02
Cleaned up problems with some of the new Chat API asadmin commands.

90
src/gam/__init__.py
View File

@@ -5485,8 +5485,7 @@ def buildGAPIObject(api, credentials=None):
API_Scopes = set(API.VAULT_SCOPES) if api == API.VAULT else set()
GM.Globals[GM.CURRENT_CLIENT_API] = api
GM.Globals[GM.CURRENT_CLIENT_API_SCOPES] = API_Scopes.intersection(GM.Globals[GM.CREDENTIALS_SCOPES])
scopeless_apis = {API.OAUTH2, API.CHROMEVERSIONHISTORY, API.SERVICEACCOUNTLOOKUP}
if api not in scopeless_apis and not GM.Globals[GM.CURRENT_CLIENT_API_SCOPES]:
if api not in API.SCOPELESS_APIS and not GM.Globals[GM.CURRENT_CLIENT_API_SCOPES]:
systemErrorExit(NO_SCOPES_FOR_API_RC, Msg.NO_SCOPES_FOR_API.format(API.getAPIName(api)))
if not GC.Values[GC.DOMAIN]:
GC.Values[GC.DOMAIN] = GM.Globals[GM.DECODED_ID_TOKEN].get('hd', 'UNKNOWN').lower()
@@ -5604,7 +5603,6 @@ def getSitesObject(entityType=Ent.DOMAIN, entityName=None, i=0, count=0):
sitesObject.debug = True
return (userEmail, sitesObject)
def getUserEmailFromID(uid, cd):
try:
result = callGAPI(cd.users(), 'get',
@@ -5624,6 +5622,26 @@ def getGroupEmailFromID(uid, cd):
except (GAPI.groupNotFound, GAPI.domainNotFound, GAPI.domainCannotUseApis, GAPI.forbidden, GAPI.badRequest):
return None
def getServiceAccountEmailFromID(account_id, sal=None):
if sal is None:
sal = buildGAPIObject('serviceaccountlookup')
try:
certs = callGAPI(sal.serviceaccounts(), 'lookup',
throwReasons = [GAPI.BAD_REQUEST, GAPI.RESOURCE_NOT_FOUND, GAPI.INVALID_ARGUMENT],
account=account_id)
except (GAPI.badRequest, GAPI.resourceNotFound, GAPI.invalidArgument):
return None
sa_cn_rx = r'CN=(.+)\.(.+).iam\.gservice.*'
sa_emails = []
for _, raw_cert in certs.items():
cert = x509.load_pem_x509_certificate(raw_cert.encode(), default_backend())
mg = re.match(sa_cn_rx, cert.issuer.rfc4514_string())
if mg:
sa_email = f'{mg.group(1)}@{mg.group(2)}.iam.gserviceaccount.com'
if sa_email not in sa_emails:
sa_emails.append(sa_email)
return GC.Values[GC.CSV_OUTPUT_FIELD_DELIMITER].join(sa_emails)
# Convert UID to email address and type
def convertUIDtoEmailAddressWithType(emailAddressOrUID, cd=None, sal=None, emailTypes=None,
checkForCustomerId=False, ciGroupsAPI=False, aliasAllowed=True):
@@ -5677,35 +5695,11 @@ def convertUIDtoEmailAddressWithType(emailAddressOrUID, cd=None, sal=None, email
except (GAPI.badRequest, GAPI.resourceNotFound, GAPI.forbidden):
pass
if 'serviceaccount' in emailTypes:
if sal is None:
sal = buildGAPIObject(API.SERVICEACCOUNTLOOKUP)
uid = getServiceAccountEmailFromID(normalizedEmailAddressOrUID, sal)
if uid:
return (uid, 'serviceaccount')
return (normalizedEmailAddressOrUID, 'unknown')
def getServiceAccountEmailFromID(account_id, sal=None):
if sal is None:
sal = buildGAPIObject('serviceaccountlookup')
throwReasons = [GAPI.BAD_REQUEST,
GAPI.RESOURCE_NOT_FOUND,
GAPI.INVALID_ARGUMENT]
try:
certs = callGAPI(sal.serviceaccounts(),
'lookup',
account=account_id,
throwReasons=throwReasons)
except (GAPI.badRequest, GAPI.resourceNotFound, GAPI.invalidArgument):
return
sa_cn_rx = r'CN=.*\.gserviceaccount\.com$'
sa_emails = []
for kid, raw_cert in certs.items():
cert = x509.load_pem_x509_certificate(raw_cert.encode(), default_backend())
subject = cert.issuer.rfc4514_string()
if re.match(sa_cn_rx, subject):
sa_emails.append(subject[3:])
return ' or '.join(sa_emails)
# Convert UID to email address
def convertUIDtoEmailAddress(emailAddressOrUID, cd=None, emailTypes=None,
checkForCustomerId=False, ciGroupsAPI=False, aliasAllowed=True):
@@ -16287,10 +16281,8 @@ def doInfoAdminRole():
fields = ','.join(set(fieldsList))
try:
role = callGAPI(cd.roles(), 'get',
throwReasons=[GAPI.BAD_REQUEST,
GAPI.CUSTOMER_NOT_FOUND,
GAPI.FORBIDDEN]+[GAPI.NOT_FOUND,
GAPI.FAILED_PRECONDITION],
throwReasons=[GAPI.NOT_FOUND, GAPI.FORBIDDEN, GAPI.FAILED_PRECONDITION,
GAPI.BAD_REQUEST, GAPI.CUSTOMER_NOT_FOUND],
customer=GC.Values[GC.CUSTOMER_ID], roleId=roleId, fields=fields)
role.setdefault('isSuperAdminRole', False)
role.setdefault('isSystemRole', False)
@@ -16434,8 +16426,15 @@ def doDeleteAdmin():
except (GAPI.badRequest, GAPI.customerNotFound):
accessErrorExit(cd)
ASSIGNEE_EMAILTYPE_TOFIELD_MAP = {
'user': 'assignedToUser',
'group': 'assignedToGroup',
'serviceaccount': 'assignedToServiceAccount',
}
PRINT_ADMIN_FIELDS = ['roleAssignmentId', 'roleId', 'assignedTo', 'scopeType', 'orgUnitId']
PRINT_ADMIN_TITLES = ['roleAssignmentId', 'roleId', 'role', 'assignedTo', 'assignedToUser', 'assignedToGroup', 'scopeType', 'orgUnitId', 'orgUnit']
PRINT_ADMIN_TITLES = ['roleAssignmentId', 'roleId', 'role',
'assignedTo', 'assignedToUser', 'assignedToGroup', 'assignedToServiceAccount',
'scopeType', 'orgUnitId', 'orgUnit']
# gam print admins [todrive <ToDriveAttribute>*]
# [user|group <EmailAddress>|<UniqueID>] [role <RoleItem>] [condition]
@@ -16449,10 +16448,8 @@ def doPrintShowAdmins():
if roleId not in rolePrivileges:
try:
rolePrivileges[roleId] = callGAPI(cd.roles(), 'get',
throwReasons=[GAPI.BAD_REQUEST,
GAPI.CUSTOMER_NOT_FOUND,
GAPI.FORBIDDEN]+[GAPI.NOT_FOUND,
GAPI.FAILED_PRECONDITION],
throwReasons=[GAPI.NOT_FOUND, GAPI.FORBIDDEN, GAPI.FAILED_PRECONDITION,
GAPI.BAD_REQUEST, GAPI.CUSTOMER_NOT_FOUND],
customer=GC.Values[GC.CUSTOMER_ID],
roleId=roleId,
fields='rolePrivileges')
@@ -16468,26 +16465,13 @@ def doPrintShowAdmins():
assignedTo = admin['assignedTo']
if assignedTo not in assignedToIdEmailMap:
assigneeType = admin.get('assigneeType')
if assigneeType == 'user':
assignedToField = 'assignedToUser'
elif assigneeType == 'group':
assignedToField = 'assignedToGroup'
elif assigneeType == 'serviceaccount':
assignedToField = 'assignedToServiceAccount'
else:
assignedToField = None
emailTypes = ['user', 'group', 'serviceaccount']
assignedToField = ASSIGNEE_EMAILTYPE_TOFIELD_MAP.get(assigneeType, None)
assigneeEmail, assigneeType = convertUIDtoEmailAddressWithType(f'uid:{assignedTo}',
cd,
sal,
emailTypes=emailTypes)
if not assignedToField and assigneeType in ['user', 'group', 'serviceaccount']:
if assigneeType == 'user':
assignedToField = 'assignedToUser'
elif assigneeType == 'group':
assignedToField = 'assignedToGroup'
elif assigneeType == 'serviceaccount':
assignedToField = 'assignedToServiceAccount'
emailTypes=list(ASSIGNEE_EMAILTYPE_TOFIELD_MAP.keys()))
if not assignedToField and assigneeType in ASSIGNEE_EMAILTYPE_TOFIELD_MAP:
assignedToField = ASSIGNEE_EMAILTYPE_TOFIELD_MAP[assigneeType]
assignedToIdEmailMap[assignedTo] = {'assignedToField': assignedToField, 'assigneeEmail': assigneeEmail}
assignedToField = assignedToIdEmailMap[assignedTo]['assignedToField']
if assignedToField:

6
src/gam/gamlib/glapi.py
View File

@@ -121,6 +121,12 @@ JWT_APIS = {
ORGPOLICY: [CLOUD_PLATFORM_SCOPE],
}
#
SCOPELESS_APIS = {
CHROMEVERSIONHISTORY,
OAUTH2,
SERVICEACCOUNTLOOKUP,
}
#
APIS_NEEDING_ACCESS_TOKEN = {
CBCM: ['https://www.googleapis.com/auth/admin.directory.device.chromebrowsers']
}

2
src/gam/gamlib/glcfg.py
View File

@@ -292,7 +292,7 @@ TODRIVE_USER = 'todrive_user'
TRUNCATE_CLIENT_ID = 'truncate_client_id'
# Update CrOS org unit with orgUnitId
UPDATE_CROS_OU_WITH_ID = 'update_cros_ou_with_id'
# Use chat asadmin where possible
# Use admin access for chat where possible
USE_CHAT_ADMIN_ACCESS = 'use_chat_admin_access'
# Use course owner for course access
USE_COURSE_OWNER_ACCESS = 'use_course_owner_access'