deepblue/GoogleDriveManagement
1
0
Fork 0
mirror of https://github.com/GAM-team/GAM.git synced 2025-07-05 20:23:34 +00:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Fixed bug in gam batch|tbatch

Browse Source
This commit is contained in:
Ross Scroggs
2024-05-02 15:14:11 -07:00
parent 42d33786a1
commit ab5aa02bf8
6 changed files with 35 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
docs/Bulk-Processing.md
View File

@ -42,6 +42,7 @@ Batch files can contain the following types of lines:
* GAM prints \<String\> and waits for the user to press any key * GAM prints \<String\> and waits for the user to press any key
* GAM continues * GAM continues
* sleep \<Integer\> - Batch processing will suspend for \<Integer\> seconds before the next command line is processed * sleep \<Integer\> - Batch processing will suspend for \<Integer\> seconds before the next command line is processed
* To be effective, this should immediately follow commit-batch
* print \<String\> - Print \<String\> on stderr * print \<String\> - Print \<String\> on stderr
* set \<KeywordString\> \<ValueString\> * set \<KeywordString\> \<ValueString\>
* Subsequent lines will have %\<KeywordString\>% replaced with \<ValueString\> * Subsequent lines will have %\<KeywordString\>% replaced with \<ValueString\>
@ -129,25 +130,29 @@ gam csv gsheet <user> <fileID> "<sheetName>" gam update user "~primaryEmail" add
## CSV files with redirection and select ## CSV files with redirection and select
You should use the `multiprocess` option on any redirected files: `csv`, `stdout`, `stderr`. You should use the `multiprocess` option on any redirected files: `csv`, `stdout`, `stderr`.
``` ```
gam redirect csv ./filelistperms.csv multiprocess csv Users.csv gam user ~primaryEmail print filelist fields id,title,permissions,owners.emailaddress gam redirect csv ./filelistperms.csv multiprocess csv Users.csv gam user ~primaryEmail print filelist fields id,name,mimetype,basicpermissions
gam redirect csv - multiprocess todrive csv Users.csv gam user ~primaryEmail print filelist fields id,name,mimetype,basicpermissions
``` ```
If you want to select a `gam.cfg` section for the command, you can select the section at the outer `gam` and save it If you want to select a `gam.cfg` section for the command, you can select the section at the outer `gam` and save it
or select the section at the inner `gam`. or select the section at the inner `gam`.
``` ```
gam select <Section> save redirect csv ./filelistperms.csv multiprocess csv Users.csv gam user ~primaryEmail print filelist fields id,title,permissions,owners.emailaddress gam select <Section> save redirect csv ./filelistperms.csv multiprocess csv Users.csv gam user ~primaryEmail print filelist fields id,name,mimetype,basicpermissions
gam redirect csv ./filelistperms.csv multiprocess csv Users.csv gam select <Section> user ~primaryEmail print filelist fields id,title,permissions,owners.emailaddress gam redirect csv ./filelistperms.csv multiprocess csv Users.csv gam select <Section> user ~primaryEmail print filelist fields id,name,mimetype,basicpermissions
gam select <Section> save redirect csv - multiprocess todrive csv Users.csv gam user ~primaryEmail print filelist fields id,name,mimetype,basicpermissions
gam redirect csv - multiprocess todrive csv Users.csv gam select <Section> user ~primaryEmail print filelist fields id,name,mimetype,basicpermissions
``` ```
## Automatic batch processing ## Automatic batch processing
You can enable automatic batch (parallel) processing when issuing commands of the form `gam <UserTypeEntity> ...`. You can enable automatic batch (parallel) processing when issuing commands of the form `gam <UserTypeEntity> ...`.
In the following example, if the number of users in group sales@domain.com exceeds 1, then the `print filelist` command will be processed in parallel. In the following example, if the number of users in group sales@domain.com exceeds 1, then the `print filelist` command will be processed in parallel.
``` ```
gam config auto_batch_min 1 redirect csv ./filelistperms.csv multiprocess group sales@domain.com print filelist fields id,title,permissions,owners.emailaddress gam config auto_batch_min 1 redirect csv ./filelistperms.csv multiprocess group sales@domain.com print filelist fields id,name,mimetype,basicpermissions
gam config auto_batch_min 1 redirect csv - multiprocess todrive group sales@domain.com print filelist fields id,name,mimetype,basicpermissions
``` ```
With automatic batch processing, you should use the `multiprocess` option on any redirected files: `csv`, `stdout`, `stderr`. With automatic batch processing, you should use the `multiprocess` option on any redirected files: `csv`, `stdout`, `stderr`.
If you want to select a `gam.cfg` section for the command, you must select and save it for it to be processed correctly. If you want to select a `gam.cfg` section for the command, you must select and save it for it to be processed correctly.
``` ```
gam select <Section> save config auto_batch_min 1 redirect csv ./filelistperms.csv multiprocess group sales@domain.com print filelist fields id,title,permissions,owners.emailaddress gam select <Section> save config auto_batch_min 1 redirect csv ./filelistperms.csv multiprocess group sales@domain.com print filelist fields id,name,mimetype,basicpermissions
``` ```

7
docs/GamUpdates.md
View File

@ -10,6 +10,13 @@ Add the `-s` option to the end of the above commands to suppress creating the `g
See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation
### 6.75.03
Fixed bug in `gam batch|tbatch` where the line `sleep <Integer>` in the batch file caused the error:
```
ERROR: Invalid argument: Expected <gam|commit-batch|print>
```
### 6.75.02 ### 6.75.02
Updated `gam report <ActivityApplictionName>` to retry/handle the following error: Updated `gam report <ActivityApplictionName>` to retry/handle the following error:

4
docs/How-to-Upgrade-from-Standard-GAM.md
View File

@ -335,7 +335,7 @@ writes the credentials into the file oauth2.txt.
admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt
admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version
WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found
GAMADV-XTD3 6.75.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource GAMADV-XTD3 6.75.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.3 64-bit final Python 3.12.3 64-bit final
MacOS Sonoma 14.4.1 x86_64 MacOS Sonoma 14.4.1 x86_64
@ -1009,7 +1009,7 @@ writes the credentials into the file oauth2.txt.
C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt
C:\GAMADV-XTD3>gam version C:\GAMADV-XTD3>gam version
WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found
GAMADV-XTD3 6.75.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource GAMADV-XTD3 6.75.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.3 64-bit final Python 3.12.3 64-bit final
Windows-10-10.0.17134 AMD64 Windows-10-10.0.17134 AMD64

12
docs/Version-and-Help.md
View File

@ -3,7 +3,7 @@
Print the current version of Gam with details Print the current version of Gam with details
``` ```
gam version gam version
GAMADV-XTD3 6.75.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource GAMADV-XTD3 6.75.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.3 64-bit final Python 3.12.3 64-bit final
MacOS Sonoma 14.4.1 x86_64 MacOS Sonoma 14.4.1 x86_64
@ -15,7 +15,7 @@ Time: 2023-06-02T21:10:00-07:00
Print the current version of Gam with details and time offset information Print the current version of Gam with details and time offset information
``` ```
gam version timeoffset gam version timeoffset
GAMADV-XTD3 6.75.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource GAMADV-XTD3 6.75.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.3 64-bit final Python 3.12.3 64-bit final
MacOS Sonoma 14.4.1 x86_64 MacOS Sonoma 14.4.1 x86_64
@ -27,7 +27,7 @@ Your system time differs from www.googleapis.com by less than 1 second
Print the current version of Gam with extended details and SSL information Print the current version of Gam with extended details and SSL information
``` ```
gam version extended gam version extended
GAMADV-XTD3 6.75.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource GAMADV-XTD3 6.75.03 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.3 64-bit final Python 3.12.3 64-bit final
MacOS Sonoma 14.4.1 x86_64 MacOS Sonoma 14.4.1 x86_64
@ -64,7 +64,7 @@ MacOS High Sierra 10.13.6 x86_64
Path: /Users/Admin/bin/gamadv-xtd3 Path: /Users/Admin/bin/gamadv-xtd3
Version Check: Version Check:
Current: 5.35.08 Current: 5.35.08
Latest: 6.75.02 Latest: 6.75.03
echo $? echo $?
1 1
``` ```
@ -72,7 +72,7 @@ echo $?
Print the current version number without details Print the current version number without details
``` ```
gam version simple gam version simple
6.75.02 6.75.03
``` ```
In Linux/MacOS you can do: In Linux/MacOS you can do:
``` ```
@ -82,7 +82,7 @@ echo $VER
Print the current version of Gam and address of this Wiki Print the current version of Gam and address of this Wiki
``` ```
gam help gam help
GAM 6.75.02 - https://github.com/taers232c/GAMADV-XTD3 GAM 6.75.03 - https://github.com/taers232c/GAMADV-XTD3
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.3 64-bit final Python 3.12.3 64-bit final
MacOS Sonoma 14.4.1 x86_64 MacOS Sonoma 14.4.1 x86_64

7
src/GamUpdate.txt
View File

@ -2,6 +2,13 @@
Merged GAM-Team version Merged GAM-Team version
6.75.03
Fixed bug in `gam batch|tbatch` where the line `sleep <Integer>` in the batch file caused the error:
```
ERROR: Invalid argument: Expected <gam|commit-batch|print>
```
6.75.02 6.75.02
Updated `gam report <ActivityApplictionName>` to retry/handle the following error: Updated `gam report <ActivityApplictionName>` to retry/handle the following error:

6
src/gam/__init__.py
View File

@ -9895,8 +9895,8 @@ def threadBatchWorker(showCmds=False, numItems=0):
batchWriteStderr(f'{currentISOformatTimeStamp()},{pid}/{numItems},Error,{str(e)},{logCmd}\n') batchWriteStderr(f'{currentISOformatTimeStamp()},{pid}/{numItems},Error,{str(e)},{logCmd}\n')
GM.Globals[GM.TBATCH_QUEUE].task_done() GM.Globals[GM.TBATCH_QUEUE].task_done()
BATCH_COMMANDS = [Cmd.GAM_CMD, Cmd.COMMIT_BATCH_CMD, Cmd.PRINT_CMD] BATCH_COMMANDS = [Cmd.GAM_CMD, Cmd.COMMIT_BATCH_CMD, Cmd.PRINT_CMD, Cmd.SLEEP_CMD]
TBATCH_COMMANDS = [Cmd.GAM_CMD, Cmd.COMMIT_BATCH_CMD, Cmd.EXECUTE_CMD, Cmd.PRINT_CMD] TBATCH_COMMANDS = [Cmd.GAM_CMD, Cmd.COMMIT_BATCH_CMD, Cmd.EXECUTE_CMD, Cmd.PRINT_CMD, Cmd.SLEEP_CMD]
def ThreadBatchGAMCommands(items, showCmds): def ThreadBatchGAMCommands(items, showCmds):
if not items: if not items:
@ -9934,7 +9934,7 @@ def ThreadBatchGAMCommands(items, showCmds):
batchWriteStderr(f'{currentISOformatTimeStamp()},0/{numItems},{Cmd.QuotedArgumentList(item[1:])}\n') batchWriteStderr(f'{currentISOformatTimeStamp()},0/{numItems},{Cmd.QuotedArgumentList(item[1:])}\n')
continue continue
if item[0] == Cmd.SLEEP_CMD: if item[0] == Cmd.SLEEP_CMD:
batchWriteStderr(f'{currentISOformatTimeStamp()},0/{numItems},Sleepiing {item[1]} seconds\n') batchWriteStderr(f'{currentISOformatTimeStamp()},0/{numItems},Sleeping {item[1]} seconds\n')
time.sleep(int(item[1])) time.sleep(int(item[1]))
continue continue
pid += 1 pid += 1