deepblue/GoogleDriveManagement
1
0
Fork 0
mirror of https://github.com/GAM-team/GAM.git synced 2025-07-05 04:03:35 +00:00
Code Issues Actions 5 Packages Projects Releases Wiki Activity

Multiple changes

Browse Source 
Added support for Duet AI license.

Added `api_call_tries_limit` variable to `gam.cfg` that limits the number of tries
for Google API calls that return an error that indicates a retry should be performed. The default value is 10 and the range of allowable values is 3-10.

Code cleanup for retry loops

Initial (not announced, in preview mode) code for Chat API support of group members and role management

Allow spaces/xxx and space/xxx when specifying chat spaces
This commit is contained in:
Ross Scroggs
2023-09-20 11:26:16 -07:00
parent 9999abe462
commit 38c78228aa
11 changed files with 191 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/CSV-Output-Filtering.md
View File

@ -197,11 +197,11 @@ gam config csv_output_row_drop_filter <RowValueFilterJSONList> ...
You optionally specify whether all or any value filters must match for the row to be excluded from the output.
* `csv_output_row_filter_drop_mode allmatch` - If all value filters match, the row is excluded from the output
* `csv_output_row_filter_drop_mode anymatch` - If any value filter matches, the row is excluded from the output; this is the default
* `csv_output_row_drop_filter_mode allmatch` - If all value filters match, the row is excluded from the output
* `csv_output_row_drop_filter_mode anymatch` - If any value filter matches, the row is excluded from the output; this is the default
```
gam config csv_output_row_filter_drop_mode allmatch csv_output_row_drop_filter <RowValueFilterList> ...
gam config csv_output_row_filter_drop_mode allmatch csv_output_row_drop_filter <RowValueFilterJSONList> ...
gam config csv_output_row_drop_filter_mode allmatch csv_output_row_drop_filter <RowValueFilterList> ...
gam config csv_output_row_drop_filter_mode allmatch csv_output_row_drop_filter <RowValueFilterJSONList> ...
```
### Matches

10
docs/GamUpdates.md
View File

@ -10,6 +10,16 @@ Add the `-s` option to the end of the above commands to suppress creating the `g
See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation.
### 6.63.17
Added support for Duet AI license.
* ProductID - 101047
* SKUID - 101047001 | duetai
Added `api_call_tries_limit` variable to `gam.cfg` that limits the number of tries
for Google API calls that return an error that indicates a retry should be performed.
The default value is 10 and the range of allowable values is 3-10.
### 6.63.16
Arguments `noinherit`, `blockinheritance` and `blockinheritance true` have been removed from the following

8
docs/How-to-Upgrade-from-Standard-GAM.md
View File

@ -106,6 +106,7 @@ Section: DEFAULT
admin_email = ''
api_calls_rate_check = false
api_calls_rate_limit = 100
api_calls_tries_limit = 10
auto_batch_min = 0
bail_on_internal_error_tries = 2
batch_size = 50
@ -333,7 +334,7 @@ writes the credentials into the file oauth2.txt.
admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt
admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version
WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found
GAMADV-XTD3 6.63.16 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.63.17 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.10.8 64-bit final
MacOS High Sierra 10.13.6 x86_64
@ -534,6 +535,7 @@ Section: DEFAULT
admin_email = ''
api_calls_rate_check = false
api_calls_rate_limit = 100
api_calls_tries_limit = 10
auto_batch_min = 0
bail_on_internal_error_tries = 2
batch_size = 50
@ -735,6 +737,7 @@ Section: DEFAULT
admin_email = ''
api_calls_rate_check = false
api_calls_rate_limit = 100
api_calls_tries_limit = 10
auto_batch_min = 0
bail_on_internal_error_tries = 2
batch_size = 50
@ -981,7 +984,7 @@ writes the credentials into the file oauth2.txt.
C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt
C:\GAMADV-XTD3>gam version
WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found
GAMADV-XTD3 6.63.16 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.63.17 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.11.5 64-bit final
Windows-10-10.0.17134 AMD64
@ -1182,6 +1185,7 @@ Section: DEFAULT
admin_email = ''
api_calls_rate_check = false
api_calls_rate_limit = 100
api_calls_tries_limit = 10
auto_batch_min = 0
bail_on_internal_error_tries = 2
batch_size = 50

4
docs/Licenses.md
View File

@ -24,6 +24,7 @@
| Cloud Identity Free | 101001 |
| Cloud Identity Premium | 101005 |
| Cloud Search | 101035 |
| Duet AI | 101047 |
| Google Chrome Device Management | Google-Chrome-Device-Management |
| Google Drive Storage | Google-Drive-storage |
| Google Meet Global Dialing | 101036 |
@ -44,6 +45,7 @@
| Cloud Identity Free | 1010010001 | cloudidentity |
| Cloud Identity Premium | 1010050001 | cloudidentitypremium |
| Cloud Search | 1010350001 | cloudsearch |
| Duet AI | 1010470001 | duetai |
| G Suite Basic | Google-Apps-For-Business | gsuitebasic |
| G Suite Business | Google-Apps-Unlimited | gsuitebusiness |
| G Suite Legacy | Google-Apps | standard |
@ -108,6 +110,7 @@
101038 |
101039 |
101040 |
101047 |
Google-Apps |
Google-Chrome-Device-Management |
Google-Drive-storage |
@ -134,6 +137,7 @@
cloudidentity | identity | 1010010001 |
cloudidentitypremium | identitypremium | 1010050001 |
cloudsearch | 1010350001 |
duetai | 101047001 |
gsuitebasic | gafb | gafw | basic | Google-Apps-For-Business |
gsuitebusiness | gau | gsb | unlimited | Google-Apps-Unlimited |
gsuitebusinessarchived | gsbau | businessarchived | 1010340002 |

12
docs/Version-and-Help.md
View File

@ -3,7 +3,7 @@
Print the current version of Gam with details
```
gam version
GAMADV-XTD3 6.63.16 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.63.17 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.11.5 64-bit final
MacOS Monterey 12.6.6 x86_64
@ -15,7 +15,7 @@ Time: 2023-06-02T21:10:00-07:00
Print the current version of Gam with details and time offset information
```
gam version timeoffset
GAMADV-XTD3 6.63.16 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.63.17 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.11.5 64-bit final
MacOS Monterey 12.6.6 x86_64
@ -27,7 +27,7 @@ Your system time differs from www.googleapis.com by less than 1 second
Print the current version of Gam with extended details and SSL information
```
gam version extended
GAMADV-XTD3 6.63.16 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.63.17 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.11.5 64-bit final
MacOS Monterey 12.6.6 x86_64
@ -64,7 +64,7 @@ MacOS High Sierra 10.13.6 x86_64
Path: /Users/Admin/bin/gamadv-xtd3
Version Check:
Current: 5.35.08
Latest: 6.63.16
Latest: 6.63.17
echo $?
1
```
@ -72,7 +72,7 @@ echo $?
Print the current version number without details
```
gam version simple
6.63.16
6.63.17
```
In Linux/MacOS you can do:
```
@ -82,7 +82,7 @@ echo $VER
Print the current version of Gam and address of this Wiki
```
gam help
GAM 6.63.16 - https://github.com/taers232c/GAMADV-XTD3
GAM 6.63.17 - https://github.com/taers232c/GAMADV-XTD3
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.11.5 64-bit final
MacOS Monterey 12.6.6 x86_64

7
docs/gam.cfg.md
View File

@ -61,6 +61,11 @@ api_calls_rate_limit
Limit on number of Google API calls per 60 seconds
Default: 1000
Range: 100 - Unlimited
api_calls_tries_limit
Limit the number of tries for Google API calls that return an error
that indicates a retry should be performed
Default: 10
Range: 3-10
auto_batch_min
Automatically generate gam batch command if number of users
specified in gam users xxx command exceeds this number
@ -582,6 +587,7 @@ Section: DEFAULT
admin_email = ''
api_calls_rate_check = false
api_calls_rate_limit = 100
api_calls_tries_limit = 10
auto_batch_min = 0
bail_on_internal_error_tries = 2
batch_size = 50
@ -773,6 +779,7 @@ activity_max_results = 100
admin_email = ''
api_calls_rate_check = false
api_calls_rate_limit = 1000
api_calls_tries_limit = 10
auto_batch_min = 0
bail_on_internal_error_tries = 2
batch_size = 50

2
src/GamCommands.txt
View File

@ -240,6 +240,7 @@ If an item contains spaces, it should be surrounded by ".
101038 |
101039 |
101040 |
101047 |
Google-Apps |
Google-Chrome-Device-Management |
Google-Drive-storage |
@ -264,6 +265,7 @@ If an item contains spaces, it should be surrounded by ".
cloudidentity | identity | 1010010001 |
cloudidentitypremium | identitypremium | 1010050001 |
cloudsearch | 1010350001 |
duetai | 101047001 |
gsuitebasic | gafb | gafw | basic | Google-Apps-For-Business |
gsuitebusiness | gau | gsb | unlimited | Google-Apps-Unlimited |
gsuitebusinessarchived | gsbau | businessarchived | 1010340002 |

10
src/GamUpdate.txt
View File

@ -2,6 +2,16 @@
Merged GAM-Team version
6.63.17
Added support for Duet AI license.
* ProductID - 101047
* SKUID - 101047001 | duetai
Added `api_call_tries_limit` variable to `gam.cfg` that limits the number of tries
for Google API calls that return an error that indicates a retry should be performed.
The default value is 10 and the range of allowable values is 3-10.
6.63.16
Arguments `noinherit`, `blockinheritance` and `blockinheritance true` have been removed from the following

218
src/gam/__init__.py
View File

@ -3054,15 +3054,15 @@ def getGSheetData():
f = TemporaryFile(mode='w+', encoding=UTF8)
if GC.Values[GC.DEBUG_LEVEL] > 0:
sys.stderr.write(f'Debug: spreadsheetUrl: {spreadsheetUrl}\n')
retries = 3
for n in range(1, retries+1):
triesLimit = 3
for n in range(1, triesLimit+1):
_, content = drive._http.request(uri=spreadsheetUrl, method='GET')
# Check for HTML error message instead of data
if content[0:15] != b'<!DOCTYPE html>':
break
tg = HTML_TITLE_PATTERN.match(content[0:600].decode('utf-8'))
errMsg = tg.group(1) if tg else 'Unknown error'
getGDocSheetDataRetryWarning([Ent.USER, user, Ent.SPREADSHEET, result['name'], sheetEntity['sheetType'], sheetEntity['sheetValue']], errMsg, n, retries)
getGDocSheetDataRetryWarning([Ent.USER, user, Ent.SPREADSHEET, result['name'], sheetEntity['sheetType'], sheetEntity['sheetValue']], errMsg, n, triesLimit)
time.sleep(20)
else:
getGDocSheetDataFailedExit([Ent.USER, user, Ent.SPREADSHEET, result['name'], sheetEntity['sheetType'], sheetEntity['sheetValue']], errMsg)
@ -4553,16 +4553,16 @@ def getClientCredentials(forceRefresh=False, forceWrite=False, filename=None, ap
if not credentials:
invalidOauth2TxtExit('')
if credentials.expired or forceRefresh:
retries = 3
for n in range(1, retries+1):
triesLimit = 3
for n in range(1, triesLimit+1):
try:
credentials.refresh(transportCreateRequest())
if writeCreds or forceWrite:
writeClientCredentials(credentials, filename or GC.Values[GC.OAUTH2_TXT])
break
except (httplib2.HttpLib2Error, google.auth.exceptions.TransportError, RuntimeError) as e:
if n != retries:
waitOnFailure(n, retries, NETWORK_ERROR_RC, str(e))
if n != triesLimit:
waitOnFailure(n, triesLimit, NETWORK_ERROR_RC, str(e))
continue
handleServerError(e)
except google.auth.exceptions.RefreshError as e:
@ -4576,10 +4576,10 @@ def getClientCredentials(forceRefresh=False, forceWrite=False, filename=None, ap
handleOAuthTokenError(e, False)
return credentials
def waitOnFailure(n, retries, error_code, error_message):
def waitOnFailure(n, triesLimit, error_code, error_message):
delta = min(2 ** n, 60)+float(random.randint(1, 1000))/1000
if n > 3:
writeStderr(f'Temporary error: {error_code} - {error_message}, Backing off: {int(delta)} seconds, Retry: {n}/{retries}\n')
writeStderr(f'Temporary error: {error_code} - {error_message}, Backing off: {int(delta)} seconds, Retry: {n}/{triesLimit}\n')
flushStderr()
time.sleep(delta)
if GC.Values[GC.SHOW_API_CALLS_RETRY_DATA]:
@ -4614,8 +4614,8 @@ def getService(api, httpObj):
clearServiceCache(service)
return service
if not hasLocalJSON:
retries = 3
for n in range(1, retries+1):
triesLimit = 3
for n in range(1, triesLimit+1):
try:
service = googleapiclient.discovery.build(api, version, http=httpObj, cache_discovery=False,
discoveryServiceUrl=DISCOVERY_URIS[v2discovery], static_discovery=False)
@ -4627,20 +4627,20 @@ def getService(api, httpObj):
except googleapiclient.errors.UnknownApiNameOrVersion as e:
systemErrorExit(GOOGLE_API_ERROR_RC, Msg.UNKNOWN_API_OR_VERSION.format(str(e), __author__))
except (googleapiclient.errors.InvalidJsonError, KeyError, ValueError) as e:
if n != retries:
waitOnFailure(n, retries, INVALID_JSON_RC, str(e))
if n != triesLimit:
waitOnFailure(n, triesLimit, INVALID_JSON_RC, str(e))
continue
systemErrorExit(INVALID_JSON_RC, str(e))
except (http_client.ResponseNotReady, OSError, googleapiclient.errors.HttpError) as e:
errMsg = f'Connection error: {str(e) or repr(e)}'
if n != retries:
waitOnFailure(n, retries, SOCKET_ERROR_RC, errMsg)
if n != triesLimit:
waitOnFailure(n, triesLimit, SOCKET_ERROR_RC, errMsg)
continue
systemErrorExit(SOCKET_ERROR_RC, errMsg)
except (httplib2.HttpLib2Error, google.auth.exceptions.TransportError, RuntimeError) as e:
if n != retries:
if n != triesLimit:
httpObj.connections = {}
waitOnFailure(n, retries, NETWORK_ERROR_RC, str(e))
waitOnFailure(n, triesLimit, NETWORK_ERROR_RC, str(e))
continue
handleServerError(e)
disc_file, discovery = readDiscoveryFile(f'{api}-{version}')
@ -4897,27 +4897,28 @@ def checkGDataError(e, service):
def callGData(service, function,
bailOnInternalServerError=False, softErrors=False,
throwErrors=None, retryErrors=None,
throwErrors=None, retryErrors=None, triesLimit=0,
**kwargs):
if throwErrors is None:
throwErrors = []
if retryErrors is None:
retryErrors = []
if triesLimit == 0:
triesLimit = GC.Values[GC.API_CALLS_TRIES_LIMIT]
allRetryErrors = GDATA.NON_TERMINATING_ERRORS+retryErrors
method = getattr(service, function)
retries = 10
if GC.Values[GC.API_CALLS_RATE_CHECK]:
checkAPICallsRate()
for n in range(1, retries+1):
for n in range(1, triesLimit+1):
try:
return method(**kwargs)
except (gdata.service.RequestError, gdata.apps.service.AppsForYourDomainException) as e:
error_code, error_message = checkGDataError(e, service)
if (n != retries) and (error_code in allRetryErrors):
if (n != triesLimit) and (error_code in allRetryErrors):
if (error_code == GDATA.INTERNAL_SERVER_ERROR and
bailOnInternalServerError and n == GC.Values[GC.BAIL_ON_INTERNAL_ERROR_TRIES]):
raise GDATA.ERROR_CODE_EXCEPTION_MAP[error_code](error_message)
waitOnFailure(n, retries, error_code, error_message)
waitOnFailure(n, triesLimit, error_code, error_message)
continue
if error_code in throwErrors:
if error_code in GDATA.ERROR_CODE_EXCEPTION_MAP:
@ -4930,8 +4931,8 @@ def callGData(service, function,
APIAccessDeniedExit()
systemErrorExit(GOOGLE_API_ERROR_RC, f'{error_code} - {error_message}')
except (httplib2.HttpLib2Error, google.auth.exceptions.TransportError, RuntimeError) as e:
if n != retries:
waitOnFailure(n, retries, NETWORK_ERROR_RC, str(e))
if n != triesLimit:
waitOnFailure(n, triesLimit, NETWORK_ERROR_RC, str(e))
continue
handleServerError(e)
except google.auth.exceptions.RefreshError as e:
@ -4941,8 +4942,8 @@ def callGData(service, function,
raise GDATA.ERROR_CODE_EXCEPTION_MAP[GDATA.SERVICE_NOT_APPLICABLE](str(e))
except (http_client.ResponseNotReady, OSError) as e:
errMsg = f'Connection error: {str(e) or repr(e)}'
if n != retries:
waitOnFailure(n, retries, SOCKET_ERROR_RC, errMsg)
if n != triesLimit:
waitOnFailure(n, triesLimit, SOCKET_ERROR_RC, errMsg)
continue
if softErrors:
writeStderr(f'\n{ERROR_PREFIX}{errMsg} - Giving up.\n')
@ -5163,18 +5164,20 @@ def checkGAPIError(e, softErrors=False, retryOnHttpError=False, mapNotFound=True
def callGAPI(service, function,
bailOnInternalError=False, bailOnTransientError=False, bailOnInvalidError=False,
softErrors=False, mapNotFound=True,
throwReasons=None, retryReasons=None, retries=10,
throwReasons=None, retryReasons=None, triesLimit=0,
**kwargs):
if throwReasons is None:
throwReasons = []
if retryReasons is None:
retryReasons = []
if triesLimit == 0:
triesLimit = GC.Values[GC.API_CALLS_TRIES_LIMIT]
allRetryReasons = GAPI.DEFAULT_RETRY_REASONS+retryReasons
method = getattr(service, function)
svcparms = dict(list(kwargs.items())+GM.Globals[GM.EXTRA_ARGS_LIST])
if GC.Values[GC.API_CALLS_RATE_CHECK]:
checkAPICallsRate()
for n in range(1, retries+1):
for n in range(1, triesLimit+1):
try:
return method(**svcparms).execute()
except googleapiclient.errors.HttpError as e:
@ -5190,7 +5193,7 @@ def callGAPI(service, function,
continue
if http_status == 0:
return None
if (n != retries) and ((reason in allRetryReasons) or
if (n != triesLimit) and ((reason in allRetryReasons) or
(GC.Values[GC.RETRY_API_SERVICE_NOT_AVAILABLE] and (reason == GAPI.SERVICE_NOT_AVAILABLE))):
if (reason in [GAPI.INTERNAL_ERROR, GAPI.BACKEND_ERROR] and
bailOnInternalError and n == GC.Values[GC.BAIL_ON_INTERNAL_ERROR_TRIES]):
@ -5198,7 +5201,7 @@ def callGAPI(service, function,
if (reason in [GAPI.INVALID] and
bailOnInvalidError and n == GC.Values[GC.BAIL_ON_INTERNAL_ERROR_TRIES]):
raise GAPI.REASON_EXCEPTION_MAP[reason](message)
waitOnFailure(n, retries, reason, message)
waitOnFailure(n, triesLimit, reason, message)
if reason == GAPI.TRANSIENT_ERROR and bailOnTransientError:
raise GAPI.REASON_EXCEPTION_MAP[reason](message)
continue
@ -5213,9 +5216,9 @@ def callGAPI(service, function,
APIAccessDeniedExit()
systemErrorExit(HTTP_ERROR_RC, formatHTTPError(http_status, reason, message))
except (httplib2.HttpLib2Error, google.auth.exceptions.TransportError, RuntimeError) as e:
if n != retries:
if n != triesLimit:
service._http.connections = {}
waitOnFailure(n, retries, NETWORK_ERROR_RC, str(e))
waitOnFailure(n, triesLimit, NETWORK_ERROR_RC, str(e))
continue
handleServerError(e)
except google.auth.exceptions.RefreshError as e:
@ -5225,8 +5228,8 @@ def callGAPI(service, function,
raise GAPI.REASON_EXCEPTION_MAP[GAPI.SERVICE_NOT_AVAILABLE](str(e))
except (http_client.ResponseNotReady, OSError) as e:
errMsg = f'Connection error: {str(e) or repr(e)}'
if n != retries:
waitOnFailure(n, retries, SOCKET_ERROR_RC, errMsg)
if n != triesLimit:
waitOnFailure(n, triesLimit, SOCKET_ERROR_RC, errMsg)
continue
if softErrors:
writeStderr(f'\n{ERROR_PREFIX}{errMsg} - Giving up.\n')
@ -5443,22 +5446,22 @@ def buildGAPIServiceObject(api, user, i=0, count=0, displayError=True):
service = getService(api, httpObj)
credentials = getSvcAcctCredentials(api, userEmail)
request = transportCreateRequest(httpObj)
retries = 3
for n in range(1, retries+1):
triesLimit = 3
for n in range(1, triesLimit+1):
try:
credentials.refresh(request)
service._http = transportAuthorizedHttp(credentials, http=httpObj)
return (userEmail, service)
except (httplib2.HttpLib2Error, google.auth.exceptions.TransportError, RuntimeError) as e:
if n != retries:
if n != triesLimit:
httpObj.connections = {}
waitOnFailure(n, retries, NETWORK_ERROR_RC, str(e))
waitOnFailure(n, triesLimit, NETWORK_ERROR_RC, str(e))
continue
handleServerError(e)
except google.auth.exceptions.RefreshError as e:
if isinstance(e.args, tuple):
e = e.args[0]
if n < retries:
if n < triesLimit:
if isinstance(e, str):
eContent = e
else:
@ -8875,16 +8878,16 @@ def _getServerTLSUsed(location):
_, netloc, _, _, _, _ = urlparse(url)
conn = 'https:'+netloc
httpObj = getHttpObj()
retries = 5
for n in range(1, retries+1):
triesLimit = 5
for n in range(1, triesLimit+1):
try:
httpObj.request(url, headers={'user-agent': GAM_USER_AGENT})
cipher_name, tls_ver, _ = httpObj.connections[conn].sock.cipher()
return tls_ver, cipher_name
except (httplib2.HttpLib2Error, RuntimeError) as e:
if n != retries:
if n != triesLimit:
httpObj.connections = {}
waitOnFailure(n, retries, NETWORK_ERROR_RC, str(e))
waitOnFailure(n, triesLimit, NETWORK_ERROR_RC, str(e))
continue
handleServerError(e)
@ -24552,8 +24555,12 @@ def getChatSpace(myarg):
chatSpace = getString(Cmd.OB_CHAT_SPACE)
if chatSpace.startswith('spaces/'):
return chatSpace
return 'spaces/'+chatSpace
return Cmd.Previous() # /spaces/xxx
if not chatSpace.startswith('space/'):
return 'spaces/'+chatSpace
_, chatSpace = chatSpace.split('/', 1)
else: # myarg.startswith('spaces/') or myarg.startswith('space/')
_, chatSpace = Cmd.Previous().split('/', 1)
return 'spaces/'+chatSpace
def _cleanChatSpace(space):
space.pop('type', None)
@ -24709,7 +24716,7 @@ def updateChatSpace(users):
body = {}
while Cmd.ArgumentsRemaining():
myarg = getArgument()
if myarg == 'space' or myarg.startswith('spaces/'):
if myarg == 'space' or myarg.startswith('spaces/') or myarg.startswith('space/'):
name = getChatSpace(myarg)
elif getChatSpaceParameters(myarg, body, CHAT_UPDATE_SPACE_TYPE_MAP):
pass
@ -24741,7 +24748,7 @@ def deleteChatSpace(users):
name = None
while Cmd.ArgumentsRemaining():
myarg = getArgument()
if myarg == 'space' or myarg.startswith('spaces/'):
if myarg == 'space' or myarg.startswith('spaces/') or myarg.startswith('space/'):
name = getChatSpace(myarg)
else:
unknownArgumentExit()
@ -24768,7 +24775,7 @@ def infoChatSpace(users, name=None):
function = 'get' if name is None else 'findDirectMessage'
while Cmd.ArgumentsRemaining():
myarg = getArgument()
if function == 'get' and (myarg == 'space' or myarg.startswith('spaces/')):
if function == 'get' and (myarg == 'space' or myarg.startswith('spaces/') or myarg.startswith('space/')):
name = getChatSpace(myarg)
else:
FJQC.GetFormatJSON(myarg)
@ -24917,7 +24924,7 @@ CHAT_MEMBER_TYPE_MAP = {
# gam <UserTypeEntity> create chatmember <ChatSpace>
# [type human|bot]
# (user <UserItem>)* (members <UserTypeEntity>)*
# (user <UserItem>)* (members <UserTypeEntity>)* (group <GroupItem>)*
# [formatjson|returnidonly]
def createChatMember(users):
def addMembers(members, field, entityType, i, count):
@ -24953,16 +24960,19 @@ def createChatMember(users):
parent = None
mtype = CHAT_MEMBER_TYPE_MAP['human']
userList = []
groupList = []
returnIdOnly = False
while Cmd.ArgumentsRemaining():
myarg = getArgument()
if myarg == 'space' or myarg.startswith('spaces/'):
if myarg == 'space' or myarg.startswith('spaces/') or myarg.startswith('space/'):
parent = getChatSpace(myarg)
elif myarg == 'user':
userList.append(getEmailAddress(returnUIDprefix='uid:'))
elif myarg in {'member', 'members'}:
_, members = getEntityToModify(defaultEntityType=Cmd.ENTITY_USERS)
userList.extend(members)
elif myarg == 'group':
groupList.append(getEmailAddress(returnUIDprefix='uid:'))
elif myarg == 'type':
mtype = getChoice(CHAT_MEMBER_TYPE_MAP, mapChoice=True)
elif myarg == 'returnidonly':
@ -24971,54 +24981,79 @@ def createChatMember(users):
FJQC.GetFormatJSON(myarg)
if not parent:
missingArgumentExit('space')
if not userList:
missingArgumentExit('user|members')
if not userList and not groupList:
missingArgumentExit('user|members|group')
userMembers = []
for user in userList:
name = normalizeEmailAddressOrUID(user)
userMembers.append({'member': {'name': f'users/{name}', 'type': mtype}})
groupMembers = []
for group in groupList:
name = normalizeEmailAddressOrUID(group)
groupMembers.append({'groupMember': {'name': f'groups/{name}'}})
i, count, users = getEntityArgument(users)
for user in users:
i += 1
user, chat, kvList = buildChatServiceObject(API.CHAT_MEMBERSHIPS, user, i, count, [Ent.CHAT_SPACE, parent, Ent.CHAT_MEMBER, ''])
if not chat:
continue
addMembers(userMembers, 'member', Ent.USER, i, count)
if userMembers:
addMembers(userMembers, 'member', Ent.USER, i, count)
if groupMembers:
addMembers(groupMembers, 'groupMember', Ent.GROUP, i, count)
CHAT_MEMBER_ROLE_CHOICES_MAP = {
'member': 'ROLE_MEMBER',
'manager': 'ROLE_MANAGER'
}
# gam <UserTypeEntity> delete chatmember <ChatSpace>
# ((user <UserItem>)|(members <UserTypeEntity>))+
# ((user <UserItem>)|(members <UserTypeEntity>)|(group <GroupItem>))+
# gam <UserTypeEntity> remove chatmember members <ChatMemberList>
def deleteChatMember(users):
# gam <UserTypeEntity> update chatmember <ChatSpace>
# ((user <UserItem>)|(members <UserTypeEntity>))+ role member|manager
# gam <UserTypeEntity> modify chatmember members <ChatMemberList> role member|manager
def deleteUpdateChatMember(users):
cd = buildGAPIObject(API.DIRECTORY)
action = Act.Get()
deleteMode = action in {Act.DELETE, Act.REMOVE}
parent = None
body = {}
memberNames = []
userList = []
userGroupList = []
while Cmd.ArgumentsRemaining():
myarg = getArgument()
if action == Act.REMOVE:
if action in {Act.UPDATE, Act.MODIFY} and myarg == 'role':
body['role'] = getChoice(CHAT_MEMBER_ROLE_CHOICES_MAP, mapChoice=True)
continue
if action in {Act.REMOVE, Act.MODIFY}:
if myarg in {'member', 'members'}:
memberNames.extend(getString(Cmd.OB_CHAT_MEMBER).replace(',', ' ').split())
else:
unknownArgumentExit()
else: # Act.DELETE
if myarg == 'space' or myarg.startswith('spaces/'):
else: # {Act.DELETE, Act.UPDATE}
if myarg == 'space' or myarg.startswith('spaces/') or myarg.startswith('space/'):
parent = getChatSpace(myarg)
elif myarg == 'user':
userList.append(getEmailAddress(returnUIDprefix='uid:'))
userGroupList.append(getEmailAddress(returnUIDprefix='uid:'))
elif myarg in {'member', 'members'}:
_, members = getEntityToModify(defaultEntityType=Cmd.ENTITY_USERS)
userList.extend(members)
userGroupList.extend(members)
elif deleteMode and myarg == 'group':
userGroupList.append(getEmailAddress(returnUIDprefix='uid:'))
else:
unknownArgumentExit()
if action == Act.REMOVE:
if not deleteMode and 'role' not in body:
missingArgumentExit('role')
if action in {Act.REMOVE, Act.MODIFY}:
if not memberNames:
missingArgumentExit('members')
else: # Act.DELETE
else: # {Act.DELETE, Act.UPDATE}
if not parent:
missingArgumentExit('space')
if not userList:
missingArgumentExit('user|members')
for user in userList:
if not userGroupList:
missingArgumentExit('user|members|group')
for user in userGroupList:
name = normalizeEmailAddressOrUID(user)
memberNames.append(f'{parent}/members/{name}')
i, count, users = getEntityArgument(users)
@ -25036,11 +25071,21 @@ def deleteChatMember(users):
j += 1
kvList[-1] = name
try:
callGAPI(chat.spaces().members(), 'delete',
bailOnInternalError=True,
throwReasons=[GAPI.NOT_FOUND, GAPI.INVALID_ARGUMENT, GAPI.PERMISSION_DENIED, GAPI.INTERNAL_ERROR],
name=name)
entityActionPerformed(kvList, j, jcount)
if deleteMode:
callGAPI(chat.spaces().members(), 'delete',
bailOnInternalError=True,
throwReasons=[GAPI.NOT_FOUND, GAPI.INVALID_ARGUMENT, GAPI.PERMISSION_DENIED, GAPI.INTERNAL_ERROR],
name=name)
entityActionPerformed(kvList, j, jcount)
else:
member = callGAPI(chat.spaces().members(), 'patch',
bailOnInternalError=True,
throwReasons=[GAPI.NOT_FOUND, GAPI.INVALID_ARGUMENT, GAPI.PERMISSION_DENIED, GAPI.INTERNAL_ERROR],
name=name, updateMask='role', body=body)
_getChatMemberEmail(cd, member)
Ind.Increment()
_showChatMember(member, None, j, jcount)
Ind.Decrement()
except GAPI.notFound as e:
entityActionFailedWarning(kvList, str(e), j, jcount)
except (GAPI.invalidArgument, GAPI.permissionDenied, GAPI.internalError) as e:
@ -25119,7 +25164,7 @@ def printShowChatMembers(users):
myarg = getArgument()
if csvPF and myarg == 'todrive':
csvPF.GetTodriveParameters()
elif myarg == 'space' or myarg.startswith('spaces/'):
elif myarg == 'space' or myarg.startswith('spaces/') or myarg.startswith('space/'):
parent = getChatSpace(myarg)
elif myarg == 'showinvited':
kwargs['showInvited'] = getBoolean()
@ -25197,7 +25242,7 @@ def createChatMessage(users):
returnIdOnly = False
while Cmd.ArgumentsRemaining():
myarg = getArgument()
if myarg == 'space' or myarg.startswith('spaces/'):
if myarg == 'space' or myarg.startswith('spaces/') or myarg.startswith('space/'):
parent = getChatSpace(myarg)
elif myarg == 'thread':
body.setdefault('thread', {})
@ -25399,7 +25444,7 @@ def printShowChatMessages(users):
myarg = getArgument()
if csvPF and myarg == 'todrive':
csvPF.GetTodriveParameters()
elif myarg == 'space' or myarg.startswith('spaces/'):
elif myarg == 'space' or myarg.startswith('spaces/') or myarg.startswith('space/'):
parent = getChatSpace(myarg)
elif myarg == 'showdeleted':
showDeleted = getBoolean()
@ -45224,7 +45269,7 @@ def _batchAddItemsToCourse(croom, courseId, i, count, addParticipants, role):
throwReasons=[GAPI.NOT_FOUND, GAPI.FORBIDDEN, GAPI.BACKEND_ERROR,
GAPI.ALREADY_EXISTS, GAPI.FAILED_PRECONDITION,
GAPI.QUOTA_EXCEEDED, GAPI.SERVICE_NOT_AVAILABLE],
retryReasons=[GAPI.NOT_FOUND, GAPI.SERVICE_NOT_AVAILABLE], retries=10 if reason != GAPI.NOT_FOUND else 3,
retryReasons=[GAPI.NOT_FOUND, GAPI.SERVICE_NOT_AVAILABLE], triesLimit=0 if reason != GAPI.NOT_FOUND else 3,
courseId=addCourseIdScope(ri[RI_ENTITY]),
body={attribute: ri[RI_ITEM] if ri[RI_ROLE] != Ent.COURSE_ALIAS else addCourseAliasScope(ri[RI_ITEM])},
fields='')
@ -45301,7 +45346,7 @@ def _batchRemoveItemsFromCourse(croom, courseId, i, count, removeParticipants, r
callGAPI(service, 'delete',
throwReasons=[GAPI.NOT_FOUND, GAPI.FORBIDDEN, GAPI.PERMISSION_DENIED,
GAPI.QUOTA_EXCEEDED, GAPI.SERVICE_NOT_AVAILABLE],
retryReasons=[GAPI.NOT_FOUND, GAPI.SERVICE_NOT_AVAILABLE], retries=10 if reason != GAPI.NOT_FOUND else 3,
retryReasons=[GAPI.NOT_FOUND, GAPI.SERVICE_NOT_AVAILABLE], triesLimit=0 if reason != GAPI.NOT_FOUND else 3,
courseId=addCourseIdScope(ri[RI_ENTITY]),
body={attribute: ri[RI_ITEM] if ri[RI_ROLE] != Ent.COURSE_ALIAS else addCourseAliasScope(ri[RI_ITEM])},
fields='')
@ -57044,14 +57089,14 @@ def transferDrive(users):
if removeSourceParents:
op = 'Remove Source Parents'
callGAPI(sourceDrive.files(), 'update',
throwReasons=GAPI.DRIVE_ACCESS_THROW_REASONS, retryReasons=[GAPI.BAD_REQUEST, GAPI.FILE_NOT_FOUND], retries=3,
throwReasons=GAPI.DRIVE_ACCESS_THROW_REASONS, retryReasons=[GAPI.BAD_REQUEST, GAPI.FILE_NOT_FOUND], triesLimit=3,
fileId=childFileId, removeParents=','.join(removeSourceParents), fields='')
actionUser = targetUser
if addTargetParent or removeTargetParents:
op = 'Add/Remove Target Parents'
callGAPI(targetDrive.files(), 'update',
throwReasons=GAPI.DRIVE_ACCESS_THROW_REASONS+[GAPI.INSUFFICIENT_PARENT_PERMISSIONS],
retryReasons=[GAPI.BAD_REQUEST, GAPI.FILE_NOT_FOUND], retries=3,
retryReasons=[GAPI.BAD_REQUEST, GAPI.FILE_NOT_FOUND], triesLimit=3,
fileId=childFileId,
addParents=addTargetParent, removeParents=','.join(removeTargetParents), fields='')
entityModifierNewValueItemValueListActionPerformed([Ent.USER, sourceUser, childFileType, childFileName], Act.MODIFIER_TO, None, [Ent.USER, targetUser], j, jcount)
@ -57168,7 +57213,7 @@ def transferDrive(users):
try:
callGAPI(sourceDrive.files(), 'update',
throwReasons=GAPI.DRIVE_ACCESS_THROW_REASONS+[GAPI.BAD_REQUEST],
retryReasons=[GAPI.FILE_NOT_FOUND], retries=3,
retryReasons=[GAPI.FILE_NOT_FOUND], triesLimit=3,
fileId=childFileId,
removeParents=','.join(existingParentIds), body={}, fields='')
except (GAPI.fileNotFound, GAPI.forbidden, GAPI.internalError, GAPI.insufficientFilePermissions, GAPI.unknownError,
@ -57184,7 +57229,7 @@ def transferDrive(users):
# try:
# callGAPI(targetDrive.files(), 'update',
# throwReasons=GAPI.DRIVE_ACCESS_THROW_REASONS+[GAPI.BAD_REQUEST, GAPI.CANNOT_ADD_PARENT, GAPI.INSUFFICIENT_PARENT_PERMISSIONS],
# retryReasons=[GAPI.FILE_NOT_FOUND], retries=3,
# retryReasons=[GAPI.FILE_NOT_FOUND], triesLimit=3,
# fileId=childFileId,
# addParents=mappedParentId, body={}, fields='')
# except (GAPI.fileNotFound, GAPI.forbidden, GAPI.internalError, GAPI.insufficientFilePermissions, GAPI.insufficientParentPermissions, GAPI.unknownError,
@ -59370,6 +59415,8 @@ def printShowDriveFileACLs(users, useDomainAdminAccess=False):
useDomainAdminAccess = True
elif myarg == 'pmselect':
pmselect = True
elif myarg == 'pmfilter': # Ignore, this is the default behavior
pass
elif PM.ProcessArgument(myarg):
pass
elif myarg == 'includepermissionsforview':
@ -66987,13 +67034,14 @@ def _processForwardingAddress(user, i, count, emailAddress, j, jcount, gmail, fu
userDefined = True
try:
result = callGAPI(gmail.users().settings().forwardingAddresses(), function,
throwReasons=GAPI.GMAIL_THROW_REASONS+[GAPI.NOT_FOUND, GAPI.ALREADY_EXISTS, GAPI.DUPLICATE, GAPI.INVALID_ARGUMENT],
throwReasons=GAPI.GMAIL_THROW_REASONS+[GAPI.NOT_FOUND, GAPI.ALREADY_EXISTS, GAPI.DUPLICATE,
GAPI.INVALID_ARGUMENT, GAPI.FAILED_PRECONDITION],
userId='me', **kwargs)
if function == 'get':
_showForwardingAddress(j, count, result)
else:
entityActionPerformed([Ent.USER, user, Ent.FORWARDING_ADDRESS, emailAddress], j, jcount)
except (GAPI.notFound, GAPI.alreadyExists, GAPI.duplicate, GAPI.invalidArgument) as e:
except (GAPI.notFound, GAPI.alreadyExists, GAPI.duplicate, GAPI.invalidArgument, GAPI.failedPrecondition) as e:
entityActionFailedWarning([Ent.USER, user, Ent.FORWARDING_ADDRESS, emailAddress], str(e), j, jcount)
except (GAPI.serviceNotAvailable, GAPI.badRequest):
entityServiceNotApplicableWarning(Ent.USER, user, i, count)
@ -70719,7 +70767,7 @@ USER_COMMANDS_WITH_OBJECTS = {
Cmd.ARG_BACKUPCODE: deleteBackupCodes,
Cmd.ARG_CALENDAR: deleteCalendars,
Cmd.ARG_CALENDARACL: deleteCalendarACLs,
Cmd.ARG_CHATMEMBER: deleteChatMember,
Cmd.ARG_CHATMEMBER: deleteUpdateChatMember,
Cmd.ARG_CHATMESSAGE: deleteChatMessage,
Cmd.ARG_CHATSPACE: deleteChatSpace,
Cmd.ARG_CLASSROOMINVITATION: deleteClassroomInvitations,
@ -70837,6 +70885,7 @@ USER_COMMANDS_WITH_OBJECTS = {
'modify':
(Act.MODIFY,
{Cmd.ARG_CALENDAR: modifyCalendars,
Cmd.ARG_CHATMEMBER: deleteUpdateChatMember,
Cmd.ARG_MESSAGE: processMessages,
Cmd.ARG_THREAD: processThreads,
}
@ -70939,7 +70988,7 @@ USER_COMMANDS_WITH_OBJECTS = {
'remove':
(Act.REMOVE,
{Cmd.ARG_CALENDAR: removeCalendars,
Cmd.ARG_CHATMEMBER: deleteChatMember,
Cmd.ARG_CHATMEMBER: deleteUpdateChatMember,
}
),
'replacedomain':
@ -71083,6 +71132,7 @@ USER_COMMANDS_WITH_OBJECTS = {
Cmd.ARG_CALATTENDEES: updateCalendarAttendees,
Cmd.ARG_CALENDAR: updateCalendars,
Cmd.ARG_CALENDARACL: updateCalendarACLs,
Cmd.ARG_CHATMEMBER: deleteUpdateChatMember,
Cmd.ARG_CHATMESSAGE: updateChatMessage,
Cmd.ARG_CHATSPACE: updateChatSpace,
Cmd.ARG_LOOKERSTUDIOPERMISSION: processLookerStudioPermissions,

4
src/gam/gamlib/glcfg.py
View File

@ -50,6 +50,8 @@ ADMIN_EMAIL = 'admin_email'
API_CALLS_RATE_CHECK = 'api_calls_rate_check'
# API calls per 100 seconds limit
API_CALLS_RATE_LIMIT = 'api_calls_rate_limit'
# API calls tries limit
API_CALLS_TRIES_LIMIT = 'api_calls_tries_limit'
# Automatically generate gam batch command if number of users specified in gam users xxx command exceeds this number
# Default: 0, do not automatically generate gam batch commands
AUTO_BATCH_MIN = 'auto_batch_min'
@ -297,6 +299,7 @@ Defaults = {
ADMIN_EMAIL: '',
API_CALLS_RATE_CHECK: FALSE,
API_CALLS_RATE_LIMIT: '100',
API_CALLS_TRIES_LIMIT: '10',
AUTO_BATCH_MIN: '0',
BAIL_ON_INTERNAL_ERROR_TRIES: '2',
BATCH_SIZE: '50',
@ -448,6 +451,7 @@ VAR_INFO = {
ADMIN_EMAIL: {VAR_TYPE: TYPE_STRING, VAR_ENVVAR: 'GA_ADMIN_EMAIL', VAR_LIMITS: (0, None)},
API_CALLS_RATE_CHECK: {VAR_TYPE: TYPE_BOOLEAN},
API_CALLS_RATE_LIMIT: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (50, None)},
API_CALLS_TRIES_LIMIT: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (3, 10)},
AUTO_BATCH_MIN: {VAR_TYPE: TYPE_INTEGER, VAR_ENVVAR: 'GAM_AUTOBATCH', VAR_LIMITS: (0, 100)},
BAIL_ON_INTERNAL_ERROR_TRIES: {VAR_TYPE: TYPE_INTEGER, VAR_LIMITS: (1, 10)},
BATCH_SIZE: {VAR_TYPE: TYPE_INTEGER, VAR_ENVVAR: 'GAM_BATCH_SIZE', VAR_LIMITS: (1, 1000)},

3
src/gam/gamlib/glskus.py
View File

@ -33,6 +33,7 @@ _PRODUCTS = {
'101038': 'AppSheet',
'101039': 'Assured Controls',
'101040': 'Beyond Corp Enterprise',
'101047': 'Duet AI',
'Google-Apps': 'Google Workspace',
'Google-Chrome-Device-Management': 'Google Chrome Device Management',
'Google-Drive-storage': 'Google Drive Storage',
@ -81,6 +82,8 @@ _SKUS = {
'product': '101039', 'aliases': ['assuredcontrols'], 'displayName': 'Assured Controls'},
'1010400001': {
'product': '101040', 'aliases': ['beyondcorp', 'beyondcorpenterprise', 'bce'], 'displayName': 'Beyond Corp Enterprise'},
'1010470001': {
'product': '101047', 'aliases': ['duetai'], 'displayName': 'Duet AI for Enterprise'},
'Google-Apps': {
'product': 'Google-Apps', 'aliases': ['standard', 'free'], 'displayName': 'G Suite Legacy'},
'Google-Apps-For-Business': {