Added options showvalidcolumn and addcsvdata to `gam print use

This commit is contained in:
Ross Scroggs
2024-02-23 14:41:45 -08:00
parent b03a43777d
commit 883979f5f5
8 changed files with 141 additions and 32 deletions

View File

@@ -89,15 +89,6 @@ If you run a Google Workspace Education SKU, verify that the super admin you'll
* Choose "All users are 18 or older"
* Click "SAVE"
Verify whether the super admin you'll be using is in an OU where reauthentication is required.
* Access the admin console and go to Security -> Overview
* Scroll down and open Google Cloud session control section
* Select the OU containing the super admin
* If Require reauthentication is selected and Exempt Trusted apps is not checked, you'll have to do `gam oauth create` at whatever frequency is specified
* If that sounds unappealing, check Exempt Trusted apps
* Click "OVERRIDE"
* Follow the steps below to mark GAM as a trusted app
Based on your domain policies, you may have to mark GAM as a trusted app. These steps are performed after a project is created.
* Access the admin console and go to Security -> Access and data control -> API controls
* Check Trust internal, domain-owned apps
@@ -114,6 +105,15 @@ Based on your domain policies, you may have to mark GAM as a trusted app. These
* Click Next/Continue
* Click Finish
Verify whether the super admin you'll be using is in an OU where reauthentication is required.
* Access the admin console and go to Security -> Overview
* Scroll down and open Google Cloud session control section
* Select the OU containing the super admin
* If Require reauthentication is selected and Exempt Trusted apps is not checked, you'll have to do `gam oauth create` at whatever frequency is specified
* If that sounds unappealing, check Exempt Trusted apps
* Click "OVERRIDE"
* Follow the steps below to mark GAM as a trusted app
## Headless computers and Cloud Shells
With many thanks to Jay, `gam oauth create` now uses a new client access authentication flow
as required by Google for headless computers/cloud shells; this is required as of February 28, 2022.

View File

@@ -10,6 +10,18 @@ Add the `-s` option to the end of the above commands to suppress creating the `g
See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation
### 6.70.04
Added option `showvalidcolumn` to `gam print users` that can be used to identify whether
users are defined in the domain. Typically, you would read CSV file of email addresses
to verify as domain members.
* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Users#verify-domain-membership
Added option `addcsvdata <FieldName> <String>` to `gam print users` that adds
additional columns of data to the CSV file output. Typically, you would read CSV file of email addresses
to generate a CSV file of results and copy data from the input CSV to the outout CSV.
### 6.70.03
Renamed license product DuetAI to Gemini

View File

@@ -334,7 +334,7 @@ writes the credentials into the file oauth2.txt.
admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt
admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version
WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found
GAMADV-XTD3 6.70.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.70.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.2 64-bit final
MacOS Sonoma 14.2.1 x86_64
@@ -1002,7 +1002,7 @@ writes the credentials into the file oauth2.txt.
C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt
C:\GAMADV-XTD3>gam version
WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found
GAMADV-XTD3 6.70.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.70.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.2 64-bit final
Windows-10-10.0.17134 AMD64

View File

@@ -39,6 +39,7 @@
- [Print domain counts for users specified by `<UserTypeEntity>`](#print-domain-counts-for-users-specified-by-usertypeentity)
- [Print user list](#print-user-list)
- [Display user counts](#display-user-counts)
- [Verify domain membership]($verify-domain-membership)
## API documentation
* https://developers.google.com/admin-sdk/directory/reference/rest/v1/users
@@ -981,14 +982,16 @@ gam print users [todrive <ToDriveAttribute>*]
[limittoou <OrgUnitItem>] [deleted_only|only_deleted])
[orderby <UserOrderByFieldName> [ascending|descending]]
[groups|groupsincolumns]
[license|licenses|licence|licences]
[license|licenses|licence|licences|licensebyuser|licensesbyuser|licencebyuser|licencesbyuser]
[onelicenseperrow|onelicenceperrow]
[(products|product <ProductIDList>)|(skus|sku <SKUIDList>)]
[schemas|custom|customschemas all|<SchemaNameList>]
[emailpart|emailparts|username]
[userview] [allfields|basic|full|(<UserFieldName>*|fields <UserFieldNameList>)]
[delimiter <Character>] [sortheaders [<Boolean>]] [scalarsfirst [<Boolean>]]
[formatjson [quotechar <Character>]] [quoteplusphonenumbers]
[issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
[showvalidcolumn] (addcsvdata <FieldName> <String>)*
```
By default, users in all domains in the account are selected; these options allow selection of subsets of users:
@@ -1008,11 +1011,13 @@ gam print users [todrive <ToDriveAttribute>*] select <UserTypeEntity>
[onelicenseperrow|onelicenceperrow]
[(products|product <ProductIDList>)|(skus|sku <SKUIDList>)]
[schemas|custom|customschemas all|<SchemaNameList>]
[emailpart|emailparts|username][schemas|custom all|<SchemaNameList>]
[userview] [allfields|basic|full|(<UserFieldName>*|fields <UserFieldNameList>)]
[emailpart|emailparts|username]
[userview] [basic|full|allfields|(<UserFieldName>*|fields <UserFieldNameList>)]
[delimiter <Character>] [sortheaders [<Boolean>]] [scalarsfirst [<Boolean>]]
[formatjson [quotechar <Character>]] [quoteplusphonenumbers]
[issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
[showvalidcolumn] (addcsvdata <FieldName> <String>)*
gam <UserTypeEntity> print users [todrive <ToDriveAttribute>*]
[orderby <UserOrderByFieldName> [ascending|descending]]
[groups|groupsincolumns]
@@ -1021,10 +1026,11 @@ gam <UserTypeEntity> print users [todrive <ToDriveAttribute>*]
[(products|product <ProductIDList>)|(skus|sku <SKUIDList>)]
[schemas|custom|customschemas all|<SchemaNameList>]
[emailpart|emailparts|username]
[userview] [allfields|basic|full|(<UserFieldName>*|fields <UserFieldNameList>)]
[userview] [basic|full|allfields|(<UserFieldName>*|fields <UserFieldNameList>)]
[delimiter <Character>] [sortheaders [<Boolean>]] [scalarsfirst [<Boolean>]]
[formatjson [quotechar <Character>]] [quoteplusphonenumbers]
[issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
[showvalidcolumn] (addcsvdata <FieldName> <String>)*
```
By default, Gam gets no group membership information for each user. The `groups` and `groupsincolumns`
@@ -1079,6 +1085,17 @@ In the output, primaryEmail is the always the first column; these options contro
* `<UserFieldName>*|fields <UserFieldNameList>` - The columns appear in the order that the fields are specified.
* `scalarsfirst [true]` - When columns are sorted by name, scalar fields appear before repeating fields.
By default, if `<UserTypeEntity>` includes an email address the is not a user member of the domain,
an error message is generated.
```
User: testuserxxx@domain.com, Does not exist
```
Using option `showvalidcolumn`, a new column `Found` indicates domain membership; no errors are generated
Add additional columns of data from the command line to the output
* `addcsvdata <FieldName> <String>`
By default, Gam displays the information as columns of fields; the following option causes the output to be in JSON format:
* `formatjson` - Display the fields in JSON format.
@@ -1249,3 +1266,34 @@ count=$(gam print users query "orgUnitPath='/Students/Middle School'" showitemco
Windows PowerShell
count = & gam print users query "orgUnitPath='/Students/Middle School'" showitemcountonly
```
## Verify domain membership
You have a CSV file of email addresses and want to verify of the addresses are valid users in your domain.
```
# Users.csv
$ more Users.csv
primaryEmail,name
testuser1@domain.com,Test User 1
testuserxxx@domain.com,Test User XXX
testuser2@domain.com,Test User 2
# Without showvalidcolumn, non-domain users generate an error
$ gam redirect csv - multiprocess csv Users.csv gam user "~primaryEmail" print users fields primaryemail,id addcsvdata name "~name"
2024-02-23T11:29:00.407-08:00,0/3,Using 3 processes...
2024-02-23T11:29:00.410-08:00,0,Processing item 3/3
User: testuserxxx@domain.com, Does not exist
2024-02-23T11:29:06.511-08:00,0/3,Processing complete
primaryEmail,id,name
testuser1@domain.com,118080758787650801331,Test User 1
testuser2@domain.com,107344800159717682514,Test User 2
# Using showvalidcolumn, a new column `Valid` indicates domain membership; no errors are generated
$ gam redirect csv - multiprocess csv Users.csv gam user "~primaryEmail" print users fields primaryemail,id addcsvdata name "~name" showvalidcolumn
2024-02-23T11:29:22.287-08:00,0/3,Using 3 processes...
2024-02-23T11:29:22.292-08:00,0,Processing item 3/3
2024-02-23T11:29:23.366-08:00,0/3,Processing complete
primaryEmail,id,Valid,name
testuser1@domain.com,118080758787650801331,True,Test User 1
testuserxxx@domain.com,,False,Test User XXX
testuser2@domain.com,107344800159717682514,True,Test User 2
```

View File

@@ -3,7 +3,7 @@
Print the current version of Gam with details
```
gam version
GAMADV-XTD3 6.70.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.70.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.2 64-bit final
MacOS Sonoma 14.2.1 x86_64
@@ -15,7 +15,7 @@ Time: 2023-06-02T21:10:00-07:00
Print the current version of Gam with details and time offset information
```
gam version timeoffset
GAMADV-XTD3 6.70.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.70.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.2 64-bit final
MacOS Sonoma 14.2.1 x86_64
@@ -27,7 +27,7 @@ Your system time differs from www.googleapis.com by less than 1 second
Print the current version of Gam with extended details and SSL information
```
gam version extended
GAMADV-XTD3 6.70.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.70.04 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.2 64-bit final
MacOS Sonoma 14.2.1 x86_64
@@ -64,7 +64,7 @@ MacOS High Sierra 10.13.6 x86_64
Path: /Users/Admin/bin/gamadv-xtd3
Version Check:
Current: 5.35.08
Latest: 6.70.03
Latest: 6.70.04
echo $?
1
```
@@ -72,7 +72,7 @@ echo $?
Print the current version number without details
```
gam version simple
6.70.03
6.70.04
```
In Linux/MacOS you can do:
```
@@ -82,7 +82,7 @@ echo $VER
Print the current version of Gam and address of this Wiki
```
gam help
GAM 6.70.03 - https://github.com/taers232c/GAMADV-XTD3
GAM 6.70.04 - https://github.com/taers232c/GAMADV-XTD3
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.2 64-bit final
MacOS Sonoma 14.2.1 x86_64

View File

@@ -5413,10 +5413,11 @@ gam print users [todrive <ToDriveAttribute>*]
[(products|product <ProductIDList>)|(skus|sku <SKUIDList>)]
[schemas|custom|customschemas all|<SchemaNameList>]
[emailpart|emailparts|username]
[userview] [basic|full|allfields | <UserFieldName>* | fields <UserFieldNameList>]
[userview] [basic|full|allfields|(<UserFieldName>*|fields <UserFieldNameList>)]
[delimiter <Character>] [sortheaders [<Boolean>]] [scalarsfirst [<Boolean>]]
[formatjson [quotechar <Character>]] [quoteplusphonenumbers]
[issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
[showvalidcolumn] (addcsvdata <FieldName> <String>)*
[showitemcountonly]
Print fields for specified users.
@@ -5429,10 +5430,11 @@ gam print users [todrive <ToDriveAttribute>*] select <UserTypeEntity>
[(products|product <ProductIDList>)|(skus|sku <SKUIDList>)]
[schemas|custom|customschemas all|<SchemaNameList>]
[emailpart|emailparts|username]
[userview] [basic|full|allfields | <UserFieldName>* | fields <UserFieldNameList>]
[userview] [basic|full|allfields|(<UserFieldName>*|fields <UserFieldNameList>)]
[delimiter <Character>] [sortheaders [<Boolean>]] [scalarsfirst [<Boolean>]]
[formatjson [quotechar <Character>]] [quoteplusphonenumbers]
[issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
[showvalidcolumn] (addcsvdata <FieldName> <String>)*
[showitemcountonly]
gam <UserTypeEntity> print users [todrive <ToDriveAttribute>*]
@@ -5443,10 +5445,11 @@ gam <UserTypeEntity> print users [todrive <ToDriveAttribute>*]
[(products|product <ProductIDList>)|(skus|sku <SKUIDList>)]
[schemas|custom|customschemas all|<SchemaNameList>]
[emailpart|emailparts|username]
[userview] [basic|full|allfields | <UserFieldName>* | fields <UserFieldNameList>]
[userview] [basic|full|allfields|(<UserFieldName>*|fields <UserFieldNameList>)]
[delimiter <Character>] [sortheaders [<Boolean>]] [scalarsfirst [<Boolean>]]
[formatjson [quotechar <Character>]] [quoteplusphonenumbers]
[issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
[showvalidcolumn] (addcsvdata <FieldName> <String>)*
[showitemcountonly]
The first column will always be primaryEmail; the remaining field names will be sorted if allfields, basic, full or sortheaders is specified;

View File

@@ -2,6 +2,18 @@
Merged GAM-Team version
6.70.04
Added option `showvalidcolumn` to `gam print users` that can be used to identify whether
users are defined in the domain. Typically, you would read CSV file of email addresses
to verify as domain members.
* See: https://github.com/taers232c/GAMADV-XTD3/wiki/Users#verify-domain-membership
Added option `addcsvdata <FieldName> <String>` to `gam print users` that adds
additional columns of data to the CSV file output. Typically, you would read CSV file of email addresses
to generate a CSV file of results and copy data from the input CSV to the outout CSV.
6.70.03
Renamed license product DuetAI to Gemini

View File

@@ -42672,6 +42672,7 @@ USERS_INDEXED_TITLES = ['addresses', 'aliases', 'nonEditableAliases', 'emails',
# [delimiter <Character>] [sortheaders] [formatjson [quotechar <Character>]] [quoteplusphonenumbers]
# [issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
# [showitemcountonly]
# [showvalidcolumn] (addcsvdata <FieldName> <String>)*
#
# gam <UserTypeEntity> print users [todrive <ToDriveAttribute>*]
# [groups|groupsincolumns]
@@ -42683,6 +42684,7 @@ USERS_INDEXED_TITLES = ['addresses', 'aliases', 'nonEditableAliases', 'emails',
# [delimiter <Character>] [sortheaders] [formatjson [quotechar <Character>]] [quoteplusphonenumbers]
# [issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
# [showitemcountonly]
# [showvalidcolumn] (addcsvdata <FieldName> <String>)*
#
# gam print users [todrive <ToDriveAttribute>*]
# ([domain <DomainName>] [(query <QueryUser>)|(queries <QueryUserList>)]
@@ -42690,6 +42692,7 @@ USERS_INDEXED_TITLES = ['addresses', 'aliases', 'nonEditableAliases', 'emails',
# [formatjson [quotechar <Character>]] [countonly]
# [issuspended <Boolean>] [aliasmatchpattern <RegularExpression>]
# [showitemcountonly]
# [showvalidcolumn] (addcsvdata <FieldName> <String>)*
#
# gam <UserTypeEntity> print users [todrive <ToDriveAttribute>*]
# [formatjson [quotechar <Character>]] [countonly]
@@ -42697,16 +42700,23 @@ USERS_INDEXED_TITLES = ['addresses', 'aliases', 'nonEditableAliases', 'emails',
# [showitemcountonly]
def doPrintUsers(entityList=None):
def _writeUserEntity(userEntity):
if addCSVData:
userEntity.update(addCSVData)
row = flattenJSON(userEntity, skipObjects=USER_SKIP_OBJECTS, timeObjects=USER_TIME_OBJECTS)
if not FJQC.formatJSON:
csvPF.WriteRowTitles(row)
elif csvPF.CheckRowTitles(row):
csvPF.WriteRowNoFilter({'primaryEmail': userEntity['primaryEmail'],
'JSON': json.dumps(cleanJSON(userEntity, skipObjects=USER_SKIP_OBJECTS, timeObjects=USER_TIME_OBJECTS),
ensure_ascii=False, sort_keys=True)})
row = {'primaryEmail': userEntity['primaryEmail']}
if showValidColumn:
row[showValidColumn] = userEntity[showValidColumn]
row['JSON'] = json.dumps(cleanJSON(userEntity, skipObjects=USER_SKIP_OBJECTS, timeObjects=USER_TIME_OBJECTS),
ensure_ascii=False, sort_keys=True)
csvPF.WriteRowNoFilter(row)
def _printUser(userEntity, i, count):
if isSuspended is None or isSuspended == userEntity.get('suspended', isSuspended):
if showValidColumn:
userEntity[showValidColumn] = True
userEmail = userEntity['primaryEmail']
if printOptions['emailParts']:
if userEmail.find('@') != -1:
@@ -42785,7 +42795,10 @@ def doPrintUsers(entityList=None):
else:
http_status, reason, message = checkGAPIError(exception)
if reason in GAPI.USER_GET_THROW_REASONS:
entityUnknownWarning(Ent.USER, ri[RI_ITEM], int(ri[RI_J]), int(ri[RI_JCOUNT]))
if not showValidColumn:
entityUnknownWarning(Ent.USER, ri[RI_ITEM], int(ri[RI_J]), int(ri[RI_JCOUNT]))
else:
_writeUserEntity({'primaryEmail': ri[RI_ITEM], showValidColumn: False})
elif (reason == GAPI.INVALID_INPUT) and customFieldMask:
entityActionFailedWarning([Ent.USER, ri[RI_ITEM]], invalidUserSchema(customFieldMask), int(ri[RI_J]), int(ri[RI_JCOUNT]))
elif reason not in GAPI.DEFAULT_RETRY_REASONS:
@@ -42799,7 +42812,10 @@ def doPrintUsers(entityList=None):
userKey=ri[RI_ITEM], projection=projection, customFieldMask=customFieldMask, viewType=viewType, fields=fields)
_printUser(user, int(ri[RI_J]), int(ri[RI_JCOUNT]))
except (GAPI.userNotFound, GAPI.resourceNotFound):
entityUnknownWarning(Ent.USER, ri[RI_ITEM], int(ri[RI_J]), int(ri[RI_JCOUNT]))
if not showValidColumn:
entityUnknownWarning(Ent.USER, ri[RI_ITEM], int(ri[RI_J]), int(ri[RI_JCOUNT]))
else:
_writeUserEntity({'primaryEmail': ri[RI_ITEM], showValidColumn: False})
except (GAPI.domainNotFound, GAPI.domainCannotUseApis, GAPI.forbidden,
GAPI.badRequest, GAPI.backendError, GAPI.systemError, GAPI.rateLimitExceeded) as e:
entityActionFailedWarning([Ent.USER, ri[RI_ITEM]], str(e), int(ri[RI_J]), int(ri[RI_JCOUNT]))
@@ -42836,7 +42852,9 @@ def doPrintUsers(entityList=None):
aliasMatchPattern = isSuspended = orgUnitPath = orgUnitPathLower = orderBy = sortOrder = None
viewType = 'admin_view'
delimiter = GC.Values[GC.CSV_OUTPUT_FIELD_DELIMITER]
showValidColumn = ''
showItemCountOnly = False
addCSVData = {}
while Cmd.ArgumentsRemaining():
myarg = getArgument()
if myarg == 'todrive':
@@ -42913,6 +42931,11 @@ def doPrintUsers(entityList=None):
quotePlusPhoneNumbers = True
elif myarg == 'showitemcountonly':
showItemCountOnly = True
elif myarg == 'showvalidcolumn':
showValidColumn = 'Valid'
elif myarg == 'addcsvdata':
k = getString(Cmd.OB_STRING)
addCSVData[k] = getString(Cmd.OB_STRING, minLen=0)
else:
FJQC.GetFormatJSONQuoteChar(myarg, False)
_, _, entityList = getEntityArgument(entityList)
@@ -42926,8 +42949,16 @@ def doPrintUsers(entityList=None):
else:
if FJQC.formatJSON:
printOptions['sortHeaders'] = False
csvPF.SetJSONTitles(['primaryEmail', 'JSON'])
titles = ['primaryEmail']
if showValidColumn:
titles.append(showValidColumn)
titles.append('JSON')
csvPF.SetJSONTitles(titles)
else:
if showValidColumn:
csvPF.AddTitles([showValidColumn])
if addCSVData:
csvPF.AddTitles(sorted(addCSVData.keys()))
if printOptions['getGroupFeed']:
if not printOptions['groupsInColumns']:
csvPF.AddTitles(['GroupsCount', 'Groups'])
@@ -43013,7 +43044,7 @@ def doPrintUsers(entityList=None):
# If no individual fields were specified (allfields, basic, full) or individual fields other than primaryEmail were specified, look up each user
if isSuspended is not None:
fieldsList.append('suspended')
if projectionSet or len(set(fieldsList)) > 1:
if projectionSet or len(set(fieldsList)) > 1 or showValidColumn:
jcount = len(entityList)
fields = getFieldsFromFieldsList(fieldsList)
if GC.Values[GC.BATCH_SIZE] > 1 and jcount > 1:
@@ -43045,7 +43076,10 @@ def doPrintUsers(entityList=None):
userKey=userEmail, projection=projection, customFieldMask=customFieldMask, viewType=viewType, fields=fields)
_printUser(user, j, jcount)
except (GAPI.userNotFound, GAPI.resourceNotFound):
entityUnknownWarning(Ent.USER, userEmail, j, jcount)
if not showValidColumn:
entityUnknownWarning(Ent.USER, userEmail, j, jcount)
else:
_writeUserEntity({'primaryEmail': userEmail, showValidColumn: False})
except (GAPI.domainNotFound, GAPI.domainCannotUseApis, GAPI.forbidden,
GAPI.badRequest, GAPI.backendError, GAPI.systemError, GAPI.rateLimitExceeded) as e:
entityActionFailedWarning([Ent.USER, userEmail], str(e), j, jcount)