deepblue/GoogleDriveManagement
1
0
Fork 0
mirror of https://github.com/GAM-team/GAM.git synced 2026-07-03 20:31:35 +00:00
Code Issues Actions 19 Packages Projects Releases Wiki Activity

dditional updates on MacOS when a gam csv command is interrupted with a contol-C.

Browse Source
This commit is contained in:
Ross Scroggs
2023-11-18 14:22:38 -08:00
parent f0cddbe7c2
commit 7cfb16c1f5
5 changed files with 73 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/GamUpdates.md
View File

@@ -10,6 +10,10 @@ Add the `-s` option to the end of the above commands to suppress creating the `g
See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation
### 6.65.12
Additional updates on MacOS when a `gam csv` command is interrupted with a contol-C.
### 6.65.11 ### 6.65.11
Updated multiprocessing to handle the following error that occurs on MacOS when a `gam csv` command Updated multiprocessing to handle the following error that occurs on MacOS when a `gam csv` command

4
docs/How-to-Upgrade-from-Standard-GAM.md
View File

@@ -334,7 +334,7 @@ writes the credentials into the file oauth2.txt.
admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt
admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version
WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found
GAMADV-XTD3 6.65.11 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource GAMADV-XTD3 6.65.12 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.10.8 64-bit final Python 3.10.8 64-bit final
MacOS High Sierra 10.13.6 x86_64 MacOS High Sierra 10.13.6 x86_64
@@ -1002,7 +1002,7 @@ writes the credentials into the file oauth2.txt.
C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt
C:\GAMADV-XTD3>gam version C:\GAMADV-XTD3>gam version
WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found
GAMADV-XTD3 6.65.11 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource GAMADV-XTD3 6.65.12 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.0 64-bit final Python 3.12.0 64-bit final
Windows-10-10.0.17134 AMD64 Windows-10-10.0.17134 AMD64

12
docs/Version-and-Help.md
View File

@@ -4,7 +4,7 @@
Print the current version of Gam with details Print the current version of Gam with details
``` ```
gam version gam version
GAMADV-XTD3 6.65.11 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource GAMADV-XTD3 6.65.12 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.0 64-bit final Python 3.12.0 64-bit final
MacOS Monterey 12.7 x86_64 MacOS Monterey 12.7 x86_64
@@ -16,7 +16,7 @@ Time: 2023-06-02T21:10:00-07:00
Print the current version of Gam with details and time offset information Print the current version of Gam with details and time offset information
``` ```
gam version timeoffset gam version timeoffset
GAMADV-XTD3 6.65.11 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource GAMADV-XTD3 6.65.12 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.0 64-bit final Python 3.12.0 64-bit final
MacOS Monterey 12.7 x86_64 MacOS Monterey 12.7 x86_64
@@ -28,7 +28,7 @@ Your system time differs from www.googleapis.com by less than 1 second
Print the current version of Gam with extended details and SSL information Print the current version of Gam with extended details and SSL information
``` ```
gam version extended gam version extended
GAMADV-XTD3 6.65.11 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource GAMADV-XTD3 6.65.12 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.0 64-bit final Python 3.12.0 64-bit final
MacOS Monterey 12.7 x86_64 MacOS Monterey 12.7 x86_64
@@ -65,7 +65,7 @@ MacOS High Sierra 10.13.6 x86_64
Path: /Users/Admin/bin/gamadv-xtd3 Path: /Users/Admin/bin/gamadv-xtd3
Version Check: Version Check:
Current: 5.35.08 Current: 5.35.08
Latest: 6.65.11 Latest: 6.65.12
echo $? echo $?
1 1
``` ```
@@ -73,7 +73,7 @@ echo $?
Print the current version number without details Print the current version number without details
``` ```
gam version simple gam version simple
6.65.11 6.65.12
``` ```
In Linux/MacOS you can do: In Linux/MacOS you can do:
``` ```
@@ -83,7 +83,7 @@ echo $VER
Print the current version of Gam and address of this Wiki Print the current version of Gam and address of this Wiki
``` ```
gam help gam help
GAM 6.65.11 - https://github.com/taers232c/GAMADV-XTD3 GAM 6.65.12 - https://github.com/taers232c/GAMADV-XTD3
Ross Scroggs <ross.scroggs@gmail.com> Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.0 64-bit final Python 3.12.0 64-bit final
MacOS Monterey 12.7 x86_64 MacOS Monterey 12.7 x86_64

4
src/GamUpdate.txt
View File

@@ -2,6 +2,10 @@
Merged GAM-Team version Merged GAM-Team version
6.65.12
Additional updates on MacOS when a `gam csv` command is interrupted with a contol-C.
6.65.11 6.65.11
Updated multiprocessing to handle the following error that occurs on MacOS when a `gam csv` command Updated multiprocessing to handle the following error that occurs on MacOS when a `gam csv` command

44
src/gam/__init__.py
View File

@@ -9234,9 +9234,10 @@ def CSVFileQueueHandler(mpQueue, mpQueueStdout, mpQueueStderr, csvPF, datetimeNo
GM.Globals[GM.DATETIME_NOW] = datetimeNow GM.Globals[GM.DATETIME_NOW] = datetimeNow
GC.Values[GC.TIMEZONE] = tzinfo GC.Values[GC.TIMEZONE] = tzinfo
GC.Values[GC.OUTPUT_TIMEFORMAT] = output_timeformat GC.Values[GC.OUTPUT_TIMEFORMAT] = output_timeformat
if sys.platform.startswith('win'): # if sys.platform.startswith('win'):
signal.signal(signal.SIGINT, signal.SIG_IGN) # signal.signal(signal.SIGINT, signal.SIG_IGN)
if multiprocessing.get_start_method() == 'spawn': if multiprocessing.get_start_method() == 'spawn':
signal.signal(signal.SIGINT, signal.SIG_IGN)
Cmd = glclargs.GamCLArgs() Cmd = glclargs.GamCLArgs()
else: else:
csvPF.SetColumnDelimiter(GC.Values[GC.CSV_OUTPUT_COLUMN_DELIMITER]) csvPF.SetColumnDelimiter(GC.Values[GC.CSV_OUTPUT_COLUMN_DELIMITER])
@@ -9340,9 +9341,10 @@ def StdQueueHandler(mpQueue, stdtype, gmGlobals, gcValues):
except IOError as e: except IOError as e:
systemErrorExit(FILE_ERROR_RC, fdErrorMessage(fd, GM.Globals[stdtype][GM.REDIRECT_NAME], e)) systemErrorExit(FILE_ERROR_RC, fdErrorMessage(fd, GM.Globals[stdtype][GM.REDIRECT_NAME], e))
if sys.platform.startswith('win'): # if sys.platform.startswith('win'):
signal.signal(signal.SIGINT, signal.SIG_IGN) # signal.signal(signal.SIGINT, signal.SIG_IGN)
if multiprocessing.get_start_method() == 'spawn': if multiprocessing.get_start_method() == 'spawn':
signal.signal(signal.SIGINT, signal.SIG_IGN)
GM.Globals = gmGlobals.copy() GM.Globals = gmGlobals.copy()
GC.Values = gcValues.copy() GC.Values = gcValues.copy()
pid0DataItem = [KEYBOARD_INTERRUPT_RC, None] pid0DataItem = [KEYBOARD_INTERRUPT_RC, None]
@@ -9433,7 +9435,8 @@ def ProcessGAMCommandMulti(pid, numItems, logCmd, mpQueueCSVFile, mpQueueStdout,
with mplock: with mplock:
initializeLogging() initializeLogging()
if sys.platform.startswith('win'): # if sys.platform.startswith('win'):
if multiprocessing.get_start_method() == 'spawn':
signal.signal(signal.SIGINT, signal.SIG_IGN) signal.signal(signal.SIGINT, signal.SIG_IGN)
GM.Globals[GM.API_CALLS_RETRY_DATA] = {} GM.Globals[GM.API_CALLS_RETRY_DATA] = {}
GM.Globals[GM.CMDLOG_LOGGER] = None GM.Globals[GM.CMDLOG_LOGGER] = None
@@ -9539,6 +9542,18 @@ def MultiprocessGAMCommands(items, showCmds):
if GM.Globals[GM.MULTIPROCESS_EXIT_CONDITION] is not None and checkChildProcessRC(result[1]): if GM.Globals[GM.MULTIPROCESS_EXIT_CONDITION] is not None and checkChildProcessRC(result[1]):
GM.Globals[GM.MULTIPROCESS_EXIT_PROCESSING] = True GM.Globals[GM.MULTIPROCESS_EXIT_PROCESSING] = True
def signal_handler(sig, frame):
controlC['trapped'] = True
def handleControlC(source):
batchWriteStderr(f'Control-C (Multiprocess-{source})\n')
setSysExitRC(KEYBOARD_INTERRUPT_RC)
batchWriteStderr(Msg.BATCH_CSV_TERMINATE_N_PROCESSES.format(currentISOformatTimeStamp(),
numItems, poolProcessResults[0],
PROCESS_PLURAL_SINGULAR[poolProcessResults[0] == 1]))
pool.terminate()
controlC['trapped'] = False
if not items: if not items:
return return
GM.Globals[GM.NUM_BATCH_ITEMS] = numItems = len(items) GM.Globals[GM.NUM_BATCH_ITEMS] = numItems = len(items)
@@ -9589,7 +9604,9 @@ def MultiprocessGAMCommands(items, showCmds):
mpQueueCSVFile, mpQueueHandlerCSVFile = initializeCSVFileQueueHandler(mpManager, mpQueueStdout, mpQueueStderr) mpQueueCSVFile, mpQueueHandlerCSVFile = initializeCSVFileQueueHandler(mpManager, mpQueueStdout, mpQueueStderr)
else: else:
mpQueueCSVFile = None mpQueueCSVFile = None
signal.signal(signal.SIGINT, origSigintHandler) # signal.signal(signal.SIGINT, origSigintHandler)
controlC = {'trapped': False}
signal.signal(signal.SIGINT, signal_handler)
batchWriteStderr(Msg.USING_N_PROCESSES.format(currentISOformatTimeStamp(), batchWriteStderr(Msg.USING_N_PROCESSES.format(currentISOformatTimeStamp(),
numItems, numPoolProcesses, numItems, numPoolProcesses,
PROCESS_PLURAL_SINGULAR[numPoolProcesses == 1])) PROCESS_PLURAL_SINGULAR[numPoolProcesses == 1]))
@@ -9599,6 +9616,8 @@ def MultiprocessGAMCommands(items, showCmds):
for item in items: for item in items:
if GM.Globals[GM.MULTIPROCESS_EXIT_PROCESSING]: if GM.Globals[GM.MULTIPROCESS_EXIT_PROCESSING]:
break break
if controlC['trapped']:
break
if item[0] == Cmd.COMMIT_BATCH_CMD: if item[0] == Cmd.COMMIT_BATCH_CMD:
batchWriteStderr(Msg.COMMIT_BATCH_WAIT_N_PROCESSES.format(currentISOformatTimeStamp(), batchWriteStderr(Msg.COMMIT_BATCH_WAIT_N_PROCESSES.format(currentISOformatTimeStamp(),
numItems, poolProcessResults[0], numItems, poolProcessResults[0],
@@ -9660,6 +9679,7 @@ def MultiprocessGAMCommands(items, showCmds):
break break
time.sleep(1) time.sleep(1)
processWaitStart = time.time() processWaitStart = time.time()
if not controlC['trapped']:
if GC.Values[GC.PROCESS_WAIT_LIMIT] > 0: if GC.Values[GC.PROCESS_WAIT_LIMIT] > 0:
waitRemaining = GC.Values[GC.PROCESS_WAIT_LIMIT] waitRemaining = GC.Values[GC.PROCESS_WAIT_LIMIT]
else: else:
@@ -9687,14 +9707,11 @@ def MultiprocessGAMCommands(items, showCmds):
pool.terminate() pool.terminate()
break break
waitRemaining = GC.Values[GC.PROCESS_WAIT_LIMIT] - delta waitRemaining = GC.Values[GC.PROCESS_WAIT_LIMIT] - delta
except KeyboardInterrupt:
setSysExitRC(KEYBOARD_INTERRUPT_RC)
batchWriteStderr(Msg.BATCH_CSV_TERMINATE_N_PROCESSES.format(currentISOformatTimeStamp(),
numItems, poolProcessResults[0],
PROCESS_PLURAL_SINGULAR[poolProcessResults[0] == 1]))
pool.terminate()
else:
pool.close() pool.close()
else:
handleControlC('SIG')
except KeyboardInterrupt:
handleControlC('KBI')
pool.join() pool.join()
batchWriteStderr(Msg.BATCH_CSV_PROCESSING_COMPLETE.format(currentISOformatTimeStamp(), numItems)) batchWriteStderr(Msg.BATCH_CSV_PROCESSING_COMPLETE.format(currentISOformatTimeStamp(), numItems))
if mpQueueCSVFile: if mpQueueCSVFile:
@@ -71801,6 +71818,7 @@ def ProcessGAMCommand(args, processGamCfg=True, inLoop=False, closeSTD=True):
CROS_COMMANDS_WITH_OBJECTS[CL_command][CMD_FUNCTION][CL_objectName](entityList) CROS_COMMANDS_WITH_OBJECTS[CL_command][CMD_FUNCTION][CL_objectName](entityList)
sys.exit(GM.Globals[GM.SYSEXITRC]) sys.exit(GM.Globals[GM.SYSEXITRC])
except KeyboardInterrupt: except KeyboardInterrupt:
batchWriteStderr('Control-C\n')
setSysExitRC(KEYBOARD_INTERRUPT_RC) setSysExitRC(KEYBOARD_INTERRUPT_RC)
showAPICallsRetryData() showAPICallsRetryData()
adjustRedirectedSTDFilesIfNotMultiprocessing() adjustRedirectedSTDFilesIfNotMultiprocessing()