Compare commits

..

466 Commits
v4.80 ... v4.97

Author SHA1 Message Date
Ross Scroggs
08e790ebbb Set errors='backslashreplace' on whatever charset user has chosen (#1067)
User can set GAM_CHARSET; default is UTF8
2020-01-02 12:05:58 -05:00
Jay Lee
75eec07207 switch to backslashreplace for so generated txt can be read back in to GAM 2020-01-02 10:38:08 -05:00
Jay Lee
22d841144c Merge branch 'master' of https://github.com/jay0lee/GAM 2020-01-01 20:19:42 -05:00
Jay Lee
8a9a08a2d2 Use new sakey code during project create 2020-01-01 20:19:35 -05:00
Ross Scroggs
497251186d Cleanup (#1066)
* Cleanup

gam,py:
In show sakays, only mark USER-MANAGED keys as current: true/false

gapi/__init__.py:
pylint cleanup: import order, indentation

gapi/errors.py:
pylint cleanup: import order, unused import, indentation

Recognize 502 Bad Gateway/Gateway Timeout: treat as retryable errors

This last change does not directly handle the refresh problem in Issue #1063 but in my version this seems to be the right solution to the 502 gateways errors

* In show sakeys, indicate whcij key was used to authenticate

* Show all sakeys by default

* Include unused import

* Remove unused import, fix unit tests
2020-01-01 15:04:45 -05:00
Jay Lee
2c0a005d3e modify gapi error tests 2020-01-01 13:06:24 -05:00
Ross Scroggs
bea7981bdb When rotating an sakey, only delete the existing key, no others (#1064)
* When rotating an sakey, only delete the existing key, no others

* Make sakeys retention explicit

* Fix bug

* Rotate, local key generation default

* Allow explicit specification of retain_none in rotate sakeys

It is still the default value.
2020-01-01 08:22:57 -05:00
Ross Scroggs
0be104fd02 pylint cleanup, Code fixes, Improved sakeys commands (#1062)
* pylint cleanup, Code fixes, Improved sakeys commands

pylint cleanup:
Import order: 77
Indentation: Numerous
Unused variable: 1121/1143
Unneeded else: 1367, 2760, 3194, 5029, 5825, 10105, 10116, 10127
Type checking: 7329, 7829
Static method: 13574

Code fixes: 3803, 3806, 5962, 5969, 6041, 6067

Improved sakeys commands: 7841/7949

Use sakeys instead of keys for claity
14172/3, 14273/4, 14383/4

* Allow for future fields

Co-authored-by: Jay Lee <jay0lee@gmail.com>
2019-12-30 13:20:03 -05:00
Jay Lee
ea1cdd553c fix few parameters after restructuring of fileutils 2019-12-30 10:07:44 -05:00
ejochman
6a14964f8f Move file utilities and add tests (#1059)
* Move file utilities and add tests

Separates common file operations into their own component with
appropriate unit tests.

* Remove use of unittest.mock.mock_open

Its behavior is slightly different across Python 3.5, 3.6, and 3.7
making it difficult to test across Python versions.
2019-12-28 10:47:04 -05:00
Jay Lee
cb9f5eab14 attempt at more universal stdin/stdout unicode and test with labels 2019-12-23 16:23:11 -05:00
Jay Lee
4ad972f7fe allow generating SA keys locally 2019-12-21 15:06:48 -05:00
Jay Lee
3925d6a467 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-12-20 11:47:11 -05:00
Jay Lee
6fcc59169c gam rotate keys - rekey service accounts 2019-12-20 11:46:53 -05:00
ejochman
02eebf186c Move the page-related call() methods and add tests (#1053)
* Move callGAPIItems and add unit tests

* Move the page-related call() methods and add tests

Continues work on jay0lee/GAM#147

* Account for rand addition that rounds to 1.0

In this case, sleep time could be equal to 61
2019-12-19 19:43:03 -05:00
Jay Lee
5b7d0fd3b8 travis cleanup 2019-12-19 19:40:46 -05:00
Jay Lee
aa4e72844b Python 3.8.1 2019-12-19 14:44:51 -05:00
Jay Lee
f9617a101b GAM 4.97 2019-12-19 13:48:05 -05:00
Jay Lee
c641596d42 disable Github actions 2019-12-19 13:45:58 -05:00
Jay Lee
d389b8ad14 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-12-19 13:44:17 -05:00
Jay Lee
6248958c94 cleanup 2019-12-19 13:44:07 -05:00
Jay Lee
e0e4c46329 Two fixes for project create
- Google changed URL for consent / client_id creation
- Python 3.5 httplib2 returns content as bytes breaking short URLs
2019-12-19 10:32:32 -05:00
Jay Lee
fdca4c2822 Remove workaround and require passlib 1.7.2 2019-12-16 11:05:53 -05:00
Jay Lee
cbc6eeabda fix retry test
Wait may be exactly 61 seconds if rand = 1 so use AssertLessEqual.
2019-12-15 16:25:36 -05:00
Ross Scroggs
dea2958d9d Update code to allow recovery phone to be cleared (#1052) 2019-12-09 13:50:32 -05:00
ejochman
a0c410be0e Begin breaking apart gam.py into logical pieces (#1047)
* Begin breaking apart gam.py into logical pieces

Start with one of the deepest parts of the stack, Google API request execution calls and associated errors. Critical information printing functions and application control logic are also broken out into their own components.

This change also adds unit tests for migrated content and makes code more PEP8 compliant.

This commit starts work on  jay0lee/GAM#147

* Add unit tests to Travis config

* Swap assert_called_once() with assertEqual() and Mock.call_count

Makes tests compatible with Python 3.5. assert_called_once() is only available in Python 3.6+
2019-12-07 09:50:56 -05:00
Ross Scroggs
4cc1a97a0a Fix label deletion (#1044)
Updated `gam delete labels` to process labels in reverse hierarchial order to avoid deleting a parent label before all of its child labels are deleted.
2019-12-02 10:04:05 -05:00
Jay Lee
ae6ac851c7 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-11-08 20:10:55 -05:00
Jay Lee
0f39b991df Test with PyPy 2019-11-08 20:10:17 -05:00
Ross Scroggs
6e9068c952 Fix race condition/fix rename labels (#1043)
* Fix race condition

From filelock.py in UnixFileLock
        # Do not remove the lockfile:^M
        #^M
        #   https://github.com/benediktschmitt/py-filelock/issues/31^M
        #   https://stackoverflow.com/questions/17708885/flock-removing-locked-file-without-race-condition^M

* Encode (){} in label: query
2019-11-05 18:08:47 -05:00
Ross Scroggs
891e5967db Fix variable name (#1038) 2019-10-22 16:36:43 -04:00
Jay Lee
542f21d58d Update linux-arm64-before-install.sh 2019-10-21 20:47:44 -04:00
Jay Lee
1af3f9f196 Update linux-arm64-before-install.sh 2019-10-21 20:18:01 -04:00
Jay Lee
f0ca2e2601 fix arm install, 4.96 2019-10-21 15:55:49 -04:00
Ross Scroggs
84f0296917 Fix multiprocessing imports, check for 32-bit GAM on 64-bit Windows (#1035)
* Fix multiprocessing imports

* Cleanup multiprocessing imports

* Check for 32-bit GAM on 64-bit Windows
2019-10-21 15:54:13 -04:00
Jay Lee
96ad8c15c6 Xenial ARM, pre-install ruby 2019-10-21 15:07:28 -04:00
Jay Lee
d77d45b5bc fix arm cache 2019-10-21 14:30:56 -04:00
Jay Lee
dc788a68f8 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-10-21 12:37:22 -04:00
Jay Lee
d3af49972c install ruby on arm64 early 2019-10-21 12:37:06 -04:00
Jay Lee
c7a732a61e Update .travis.yml 2019-10-21 11:33:18 -04:00
Jay Lee
d28a412204 always show cpuinfo 2019-10-18 20:04:44 -04:00
Jay Lee
2722d97a7d show cpuinfo 2019-10-18 20:02:12 -04:00
Jay Lee
79c62d86cc latest google-auth-oauthlib while keeping code_verifier enabled 2019-10-17 13:35:42 -04:00
Jay Lee
9f1dcc4c9f fix current path so we can find requirements.txt on arm 2019-10-17 12:36:07 -04:00
Jay Lee
4230f49bd9 arm fixes 2019-10-17 11:50:08 -04:00
Jay Lee
9fe9171798 fix path 2019-10-17 10:43:23 -04:00
Jay Lee
dac3b79f4d more logging 2019-10-17 10:04:28 -04:00
Jay Lee
083c2f4e9b hack fix for passlib bug relying on deprecated time.clock on win32 2019-10-17 09:01:22 -04:00
Jay Lee
17f88eb4e7 travis win fixes 2019-10-17 06:04:31 -04:00
Jay Lee
2798e89925 pyver was issue, add * 2019-10-16 17:08:28 -04:00
Jay Lee
79db8f2df3 find the guilty one 2019-10-16 17:04:05 -04:00
Jay Lee
089aadd729 one to many ] 2019-10-16 16:59:13 -04:00
Jay Lee
201e37d185 another shot 2019-10-16 16:56:56 -04:00
Jay Lee
9d6b569ddb reverse another part 2019-10-16 16:46:06 -04:00
Jay Lee
c7b026fd1d correct cache check 2019-10-16 16:40:44 -04:00
Jay Lee
66b95abb96 test Python 3.7, fix linux cache check 2019-10-16 16:33:43 -04:00
Jay Lee
e2ef8a293d more travis 2019-10-16 16:23:41 -04:00
Jay Lee
b25f6e041c fix Python paths for pip install 2019-10-16 15:34:48 -04:00
Jay Lee
4607580e6f fix windir, fork for MacOS 2019-10-16 14:04:19 -04:00
Jay Lee
f2d48c0e8f list files on Win 2019-10-16 12:11:04 -04:00
Jay Lee
c9d12e21d8 call freeze_support() unconditionally since it's a NOOP on non-relevant platforms 2019-10-16 11:58:32 -04:00
Jay Lee
2f4764a3f2 manual pip install, no ssl copy 2019-10-16 11:31:36 -04:00
Jay Lee
8b67039fa6 use pyinstaller dev 2019-10-14 20:52:54 -04:00
Jay Lee
8cf8d51c79 fixes for Python 3.8.0 2019-10-14 20:03:46 -04:00
Jay Lee
90226c6981 Python 3.8.0 2019-10-14 19:12:04 -04:00
Jay Lee
91a248bdbe another attempt to allow arm64 to fail 2019-10-14 17:13:55 -04:00
Jay Lee
0d3bfacc84 more travis fixes 2019-10-14 16:14:25 -04:00
Jay Lee
ac3c156a0b python3 not python 2019-10-14 14:45:00 -04:00
Jay Lee
f24d403705 set pip/python vars right after they are installed 2019-10-14 14:41:54 -04:00
Jay Lee
da3b16c0d4 watch cache 2019-10-14 11:42:33 -04:00
Jay Lee
2b2cb03784 Update linux-x86_64-before-install.sh 2019-10-13 19:00:35 -04:00
Jay Lee
19d0ff3d46 Update .travis.yml 2019-10-13 18:58:27 -04:00
Ross Scroggs
04b6b0ad76 Allow simplifed versions of GAM_CSV_ROW_FILTER (#1029)
export GAM_CSV_ROW_FILTER="'suspended:boolean:true', 'suspensionReason:regex:ADMIN'"

export GAM_CSV_ROW_FILTER='{"suspended": "boolean:true", "suspensionReason": "regex:ADMIN"}'
2019-10-13 17:56:49 -04:00
Jay Lee
6a5fb33306 further linux cleanup, re-enable e2e 2019-10-13 14:06:29 -04:00
Jay Lee
d0262ea6ae Update linux-x86_64-before-install.sh 2019-10-13 12:48:17 -04:00
Jay Lee
512c2ee000 Update linux-x86_64-before-install.sh 2019-10-13 10:53:11 -04:00
Jay Lee
e708e885f6 Update linux-x86_64-before-install.sh 2019-10-13 10:25:52 -04:00
Jay Lee
cc3b7b8124 Update linux-arm64-before-install.sh 2019-10-11 20:52:43 -04:00
Jay Lee
39c9deb456 more fi 2019-10-11 19:55:38 -04:00
Jay Lee
7991790f94 reformat linux x86 before install, attempt to fix ruby on arm64 2019-10-11 19:51:36 -04:00
Jay Lee
5f418d3f1a build VM 2019-10-11 17:52:12 -04:00
Jay Lee
5047bf5466 Only one ARM version, re-disable filter_secrets 2019-10-11 17:04:07 -04:00
Jay Lee
dca7c26b9d ARM64 Xenial+Bionic, test re-enable filter_secrets 2019-10-11 16:51:59 -04:00
Jay Lee
c6173e2957 whereibelong case 2019-10-11 14:05:19 -04:00
Jay Lee
b646023c41 /home/jayhlee instead of ~ 2019-10-11 13:19:15 -04:00
Jay Lee
45e3b01b15 cleanup arm build 2019-10-11 13:10:04 -04:00
Jay Lee
97515ab758 add zlib dev packages for arm 2019-10-11 12:28:49 -04:00
Jay Lee
d85328e729 dist arm 2019-10-11 11:53:36 -04:00
Jay Lee
0aa7082391 disable arm secrets 2019-10-11 10:58:05 -04:00
Jay Lee
946431b83f arm 2019-10-11 07:58:22 -04:00
Jay Lee
a101e3e7a6 pip 2019-10-10 20:53:57 -04:00
Jay Lee
f73120dbaa try test instead of build 2019-10-10 20:48:36 -04:00
Jay Lee
a55879c93a attempt to fix arm hangs 2019-10-10 20:42:20 -04:00
Jay Lee
fe02af151d arm64 build files 2019-10-10 20:35:33 -04:00
Jay Lee
f1a963fa9c Update .travis.yml 2019-10-10 20:01:04 -04:00
Jay Lee
8c6dee4213 Arm64 compile 2019-10-10 19:59:09 -04:00
Jay Lee
9208b4c4dd fix vsort check 2019-10-10 11:24:27 -04:00
Jay Lee
0e04b34852 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-10-10 11:16:36 -04:00
Jay Lee
796e35e8a4 Perform test install on MacOS/Linux 2019-10-10 11:16:06 -04:00
Jay Lee
72f0ae906f Fix gam-install.sh on Cloud Shell and MacOS 10.12 2019-10-10 11:12:57 -04:00
Ross Scroggs
464482d197 Document check serviceaccount scopes argument (#1023) 2019-10-05 11:23:17 -04:00
Jay Lee
48ce39a645 Custom scopes for check serviceaccount 2019-10-03 21:37:03 -04:00
Jay Lee
f993240d2b quotes 2019-10-02 10:08:39 -04:00
Jay Lee
83fd9babef run github install action only on Linux/MacOS 2019-10-02 10:05:23 -04:00
Jay Lee
66fb0cf8fc fix MacOS doesn't support case insensitive, allow specifying version 2019-10-02 09:46:01 -04:00
Jay Lee
459ac84d29 fix glibc/MacOS issue where equal version failed check 2019-10-01 20:15:46 -04:00
Jay Lee
736b833d52 newer glibc 2019-10-01 15:56:48 -04:00
Jay Lee
dc37020d73 gam-install.sh for MacOS versions 2019-10-01 15:46:01 -04:00
Jay Lee
93916d4ed1 remove timeout that caused sporadic failure 2019-10-01 12:44:22 -04:00
Jay Lee
1d1e48acb7 One lock for r/w, cleanup .lock file. Fixes #1011.
Keep lock in place thru read and possible write of oauth2.txt. This
allows only a single process to refresh credentials, others won't see the token
until post-refresh and we avoid multiple refreshes in parallel.

filelock can cleanup after itself on Windows but has to avoid a deadlock on *nix.
Try to cleanup the .lock file for it.
2019-10-01 10:59:46 -04:00
Jay Lee
1884e1a111 file locking for oauth2.txt 2019-09-30 16:18:24 -04:00
Jay Lee
2d0396da21 GAM 4.95, Ross' changes in #1018, re-enable older Python tests 2019-09-28 12:15:31 -04:00
Jay Lee
cce47ba723 another attempt to fix bionic 2019-09-28 07:00:34 -04:00
Jay Lee
2831680d14 stop building MacOS 10.1[01] 2019-09-28 06:42:44 -04:00
Jay Lee
5e3374acbc unset LD_LIBRARY_PATH to fix deploy 2019-09-28 06:34:24 -04:00
Jay Lee
13d7de9501 travis 2019-09-27 22:43:14 -04:00
Jay Lee
d78abc92f2 update ruby gems 2019-09-27 21:32:16 -04:00
Jay Lee
07672eb874 use newer ruby on github 2019-09-27 21:12:15 -04:00
Jay Lee
e7225ce487 travis 2019-09-27 19:56:13 -04:00
Jay Lee
cb24d3bf78 travis 2019-09-27 19:52:37 -04:00
Jay Lee
8360019080 parallel tests 2019-09-27 18:59:52 -04:00
Jay Lee
f3b34bea26 travis 2019-09-27 16:50:33 -04:00
Jay Lee
edf09a2d7b travis 2019-09-27 16:14:47 -04:00
Jay Lee
1de445c5b8 travis 2019-09-27 15:28:31 -04:00
Jay Lee
8d7f307173 more travis 2019-09-27 14:59:46 -04:00
Jay Lee
0a23a6d084 more travis 2019-09-27 14:37:59 -04:00
Jay Lee
87fa70be2c travis 2019-09-27 13:40:41 -04:00
Jay Lee
b57c62fe1b mroe travis 2019-09-27 12:34:43 -04:00
Jay Lee
a8c1051e0f more travis work 2019-09-27 11:43:27 -04:00
Jay Lee
d7ba12e729 flush travis cache, osx libraries 2019-09-27 10:17:23 -04:00
Jay Lee
1d3c47f3fd more diag 2019-09-18 18:52:26 -04:00
Jay Lee
4ae81bae99 brew upgrade on MacOS 2019-09-18 18:19:19 -04:00
Jay Lee
2fc301d061 full path to win python 2019-09-18 18:14:58 -04:00
Jay Lee
fc6e6d1ab6 verbose logging on MacOS 2019-09-18 17:50:30 -04:00
Jay Lee
9cb4ee9d6f figure out what's up with path 2019-09-18 17:44:37 -04:00
Jay Lee
0e1da6982b call python 2019-09-18 16:40:21 -04:00
Jay Lee
28573b47a8 more build fixes 2019-09-18 16:35:28 -04:00
Jay Lee
851bd1ef14 not gam 2019-09-18 15:47:25 -04:00
Jay Lee
0c7d64563d missing $ for bash var 2019-09-18 15:35:29 -04:00
Jay Lee
b984f62bbf a_atleast_b tool 2019-09-18 15:16:16 -04:00
Jay Lee
a89e0936c2 OpenSSL 1.1.1d on Windows 2019-09-18 14:59:33 -04:00
Jay Lee
677146d905 OpenSSL 1.1.1d 2019-09-18 14:11:15 -04:00
Ross Scroggs
00b7ead8bb Standard Got messages to always show total_items (#1015) 2019-09-18 08:40:57 -04:00
Ross Scroggs
dce5016261 Add recoveryEmail/Phone to users field list (#1012) 2019-09-13 10:33:03 -04:00
Ross Scroggs
2bc759778c Keep pylint happy (#1008) 2019-09-12 18:49:46 -04:00
Jay Lee
3aa6869a4b fix MacOS vars 2019-09-06 11:21:39 -04:00
Jay Lee
209fdfd5b9 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-09-06 11:15:27 -04:00
Jay Lee
eec0df14b5 MacOS codenames 2019-09-06 11:14:03 -04:00
Ross Scroggs
7e2810d33d Fix code to avoid trap when description is long (#1004)
* Fix code to avoid trap when description is long

* Code cleanup
2019-09-06 11:12:42 -04:00
Jay Lee
78404c8cd3 cleanup MacOS 2019-09-06 10:56:22 -04:00
Jay Lee
f05ceecf8e no x86 on GitHub Actions :-/ 2019-09-06 10:45:41 -04:00
Jay Lee
bcef526213 More recognizable OS info in gam version 2019-09-06 10:44:06 -04:00
Jay Lee
00d5767246 show Python MacOS versions 2019-09-06 07:17:47 -04:00
Jay Lee
6655301bfe Merge branch 'master' of https://github.com/jay0lee/GAM 2019-09-05 19:52:08 -04:00
Jay Lee
5beff97f95 test building on multiple MacOS versions 2019-09-05 19:51:46 -04:00
Jay Lee
cfa6b49bab Update pythonpackage.yml 2019-09-05 14:42:00 -04:00
Jay Lee
a659d5fada Update pythonpackage.yml 2019-09-05 13:54:05 -04:00
Jay Lee
c1521bfa3f Update pythonpackage.yml 2019-09-05 13:47:27 -04:00
Jay Lee
096e6c911a Update pythonpackage.yml 2019-09-05 13:02:54 -04:00
Jay Lee
26f7cd38e5 add runs-on 2019-09-05 12:56:12 -04:00
Jay Lee
802541c09f Test run of GitHub Actions 2019-09-05 12:52:42 -04:00
Jay Lee
cfd36c2836 GAM 4.94, pull in Ross changes in #1003 2019-08-30 11:45:52 -04:00
Jay Lee
7689ac7bed disable ssl verify on time sync so we know when clock is WAY off 2019-08-30 11:15:56 -04:00
Jay Lee
7a5ba99b36 Even better SA check 2019-08-30 11:02:42 -04:00
Ross Scroggs
8f4a40bc9a Add timeoffset option to gam version (#1002)
* Add timeoffset option to gam version

* Update timeOffset checking
2019-08-29 14:32:45 -04:00
Jay Lee
e3ab846d70 update user passwords 2019-08-27 14:28:04 -04:00
Jay Lee
29db574bc5 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-08-27 10:19:48 -04:00
Jay Lee
4851d5b62f upgrade MUSL 2019-08-27 10:19:33 -04:00
Jay Lee
caef16bdee add email scope to SA, check serviceaccount verifies proper DwD and scopes for token 2019-08-27 10:18:36 -04:00
Ross Scroggs
1a0f9ab66a handle empty recoveryPhone (#999)
* handle empty recoveryPhone

* Test empty recoveryemail/recoveryphone

* Handle autoUpdateExpiration for CrOS
2019-08-21 15:03:05 -04:00
Ross Scroggs
021c3bfb13 Handle more errors with short URL oauth create (#998) 2019-08-18 17:34:55 -04:00
Jay Lee
b3dfa41df6 GAM user agent on short URL 2019-08-15 18:21:54 -04:00
Jay Lee
5f94263db2 Use _createHttpObj() for short URL 2019-08-15 18:15:56 -04:00
Jay Lee
68d8e46b4c force pip upgrades 2019-08-15 14:50:23 -04:00
Jay Lee
ed221b0d7b just use static recovery phone 2019-08-15 13:31:26 -04:00
Jay Lee
1243563cd4 try to make MacOS tr happy 2019-08-15 12:38:47 -04:00
Jay Lee
1170457a39 GAM 4.93, remove *MAX_RESULTS config options 2019-08-15 12:35:10 -04:00
Jay Lee
435ed9f568 use area code 212 always 2019-08-15 12:18:36 -04:00
Jay Lee
81884e48d0 Support recovery email/phone for users 2019-08-15 12:00:36 -04:00
Jay Lee
a7be6d233b confirm API call supports maxResults before checking value 2019-08-15 10:08:03 -04:00
Jay Lee
584ddba1a5 Dynamic maxResults from discovery or exception 2019-08-14 16:11:14 -04:00
Jay Lee
2bc6c8bca0 200 or bust for URL shorten 2019-08-13 08:35:06 -04:00
Jay Lee
fc1e81a01d oauthbrowser.txt, GAM 4.92 2019-08-12 14:02:59 -04:00
Jay Lee
eebfaaf373 finish pyinstaller install 2019-08-12 12:48:01 -04:00
Jay Lee
652223d9bc Pyinstaller windows 64 bit bootloader compile 2019-08-12 12:29:51 -04:00
Jay Lee
e75664fd2e roll Python 3.5 back to Xenial 2019-08-12 12:08:09 -04:00
Jay Lee
556278b216 Use bionic for Python source tests 2019-08-12 11:57:24 -04:00
Jay Lee
f9bfaa98bb fix cros/mobile print 2019-08-12 11:47:44 -04:00
Jay Lee
b6bd2da6ce Short OAuth URLs, make console flow default to reduce issues 2019-08-12 11:00:26 -04:00
Jay Lee
7c36a6b601 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-08-12 10:41:22 -04:00
Ross Scroggs
413924b11a Generalize expression to find group settings values (#994) 2019-08-12 10:41:17 -04:00
Jay Lee
251883dae5 add cros/mobile print tests 2019-08-10 15:46:52 -04:00
Jay Lee
7e4d0da8fb GAM 4.91 2019-08-10 15:39:14 -04:00
Ross Scroggs
3fc2aeed4d Fix group settings (#990)
* Fix group settings

Clean up md5MatchesFile

* Simplify doGetCustomerInfo

Avoid trap when doPrintDomains returned a domain creationTime with no fraction.
Traceback (most recent call last):
  File "gam.py", line 14761, in <module>
  File "gam.py", line 14159, in ProcessGAMCommand
  File "gam.py", line 2053, in doGetDomainInfo
  File "gam.py", line 2088, in doGetCustomerInfo
  File "_strptime.py", line 577, in _strptime_datetime
  File "_strptime.py", line 359, in _strptime
ValueError: time data '2019-04-23 16:14:56' does not match format '%Y-%m-%d %H:%
M:%S.%f'

lansa.co.uk,True,2019-04-23 16:14:13.041000,secondary,
lansa.com.au,True,2019-04-23 16:14:56,secondary
2019-08-10 15:37:44 -04:00
Ross Scroggs
7f4f785f0b Lower default limit for 500 to 200 for devices (#987)
* Lower default limit for 500 to 200 for devices

* Lower limit from 200 to 100 to handle mobile devices
2019-08-05 18:16:46 -04:00
Jay Lee
9f5920989d remove bionic for now, re-enable e2e tests 2019-08-01 11:32:23 -04:00
Jay Lee
62bceb30c5 GAM 4.90, update mobile adjustments 2019-08-01 11:15:10 -04:00
Jay Lee
8c736e52ac Empty body no longer required 2019-08-01 10:58:01 -04:00
Jay Lee
2669079a31 don't cleanup 2019-07-27 10:01:33 -04:00
Jay Lee
cbfd93e440 Timeout optimized Python build so we can cache partial testing... 2019-07-27 09:36:01 -04:00
Jay Lee
ffd1c297e5 force configure 2019-07-26 15:51:35 -04:00
Jay Lee
1a11cb04f9 clean and optimize python build 2019-07-26 15:38:23 -04:00
Jay Lee
438fd5b59a turn off unsafe to speed up Python for moment 2019-07-26 15:23:58 -04:00
Jay Lee
2fef5e2cfa minimal tests so Bionic Python 3.7 can build and cache 2019-07-26 14:22:35 -04:00
Jay Lee
db8ad38fd3 try curl instead of wget 2019-07-26 13:28:24 -04:00
Jay Lee
cbb2722291 figure out what's up with Python wget 2019-07-26 13:07:27 -04:00
Jay Lee
be1c7f2167 Bionic Beaver build 2019-07-26 12:37:00 -04:00
Jay Lee
61f4a137b0 32-bit bootloader, fix path 2019-07-26 12:34:40 -04:00
Jay Lee
dac9e91428 fix python path 2019-07-26 12:14:45 -04:00
Jay Lee
83c64f1f71 few more fixes 2019-07-26 11:53:12 -04:00
Jay Lee
443f4e707b Use MacOS Python binary, fix Win32 Pyinstaller build 2019-07-26 11:33:00 -04:00
Jay Lee
70bf2a05f3 pyinstaller version 2019-07-26 11:09:12 -04:00
Jay Lee
33a4747677 rebuild w32 PyInstaller bootloader to avoid AV issues 2019-07-26 10:43:44 -04:00
Ross Scroggs
a4cce17767 Fix typos (#981) 2019-07-26 09:27:19 -04:00
Ross Scroggs
67cd03d3f1 I suggest match_users instead of if_users (#980)
* I suggest match_users instead of if_users

* Allow if_users or match_users
2019-07-26 09:06:45 -04:00
Jay Lee
1f88c18f94 Update gam.py 2019-07-24 15:07:12 -04:00
Jay Lee
d2fc706b17 fix doit mobile update 2019-07-24 12:28:22 -04:00
Jay Lee
b5e5786813 Require doit argument to update >1 devices 2019-07-24 12:21:30 -04:00
Jay Lee
38e741c788 Update mobile devices by query 2019-07-24 11:16:54 -04:00
Jay Lee
e9a0b85682 retry notFound when changing group settings after create to handle sporadic latency issues that cause failure 2019-07-23 10:09:55 -04:00
Jay Lee
5ab3602c2a gam download storagebucket initial work 2019-07-20 10:08:48 -04:00
Jay Lee
dd4bf7b144 10 second timeout on update check (default seems to be 2 min) 2019-07-20 10:07:34 -04:00
Ross Scroggs
922326c5ce Code cleanup (#975) 2019-07-17 08:41:38 -04:00
Jay Lee
c69be414ca Update gam.py
Fix one more case.
2019-07-16 17:45:03 -04:00
Ross Scroggs
10bc47402c Cleanup (#974)
* Cleanup

parent is not valid with use project

* Cleanup

* Cleanup

* Code fix
2019-07-16 17:42:42 -04:00
Ross Scroggs
193e42cf22 Add new forms of create/use project (#973) 2019-07-16 12:50:36 -04:00
Jay Lee
e0f58e5264 Allow setting project parent 2019-07-15 13:24:54 -04:00
Jay Lee
f14e48320c noupx, strip osx/linux 2019-07-14 17:33:50 -04:00
Jay Lee
474fcd33a6 disable PyInstaller debug 2019-07-13 15:12:50 -04:00
Jay Lee
a2a9ffc895 force pip upgrades 2019-07-13 14:40:01 -04:00
Jay Lee
b02416b32c Actually use GAM_CA_FILE env var 2019-07-12 15:10:20 -04:00
Jay Lee
fa52d9e89e Merge branch 'master' of https://github.com/jay0lee/GAM 2019-07-11 09:55:41 -04:00
Jay Lee
6383aa594a Batch Drive Deletes 2019-07-11 09:55:20 -04:00
Ross Scroggs
54eb59c27b Convert team to shared in vault export; prepare for the future (#971)
* Convert team to shared in vault export

Restore _getValidCourseStates(croom), it is used by _getCourseStates in print courses/course-participants

* Update GamCommands.txt

* Fix typo

* Update gam.py
2019-07-10 15:35:06 -04:00
Ross Scroggs
3877f8309b Update GamCommands.txt (#969) 2019-07-10 12:17:53 -04:00
Jay Lee
d8b0681831 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-07-10 12:17:13 -04:00
Jay Lee
cd8303dbea GAM 4.89 2019-07-10 12:17:00 -04:00
Ross Scroggs
ab51d6e931 Update GamCommands.txt (#968) 2019-07-10 11:33:49 -04:00
Jay Lee
b7e402dca2 confidential vault, generalize enums from discovery, query RCs 2019-07-10 09:33:21 -04:00
Jay Lee
842040a8b3 run e2e tests again 2019-07-09 21:39:27 -04:00
Jay Lee
1205da5d34 Update linux-x86_64-before-install.sh 2019-07-09 20:36:33 -04:00
Jay Lee
f40824fedd disable e2e so Python 3.7.4 compile finishes 2019-07-09 13:18:29 -04:00
Jay Lee
67fa0cbc61 } 2019-07-09 12:23:29 -04:00
Jay Lee
e029c77f76 Python 3.7.4, accept newer Python/OpenSSL versions 2019-07-09 11:09:09 -04:00
Jay Lee
2b23ae4e67 remove dnspython requirement, minor fixes 2019-07-02 12:21:51 -04:00
Jay Lee
c8ecc23c9c Remove dnspython in favor of simple Google DNS JSON API 2019-07-02 11:13:31 -04:00
Ross Scroggs
94f8959879 Handle group members with no status (#962)
* Handle group members with no status

* Omit Advanced  form

* Update gam.py
2019-07-01 12:02:08 -04:00
Jay Lee
2cdb8eb44d fix message header argument, make sure we remove all headers in cases of duplicate header or non-matching case 2019-06-27 14:39:03 -04:00
Ross Scroggs
ebc1d1ecb3 Clean up sendOrDropEmail (#961)
* Clean up sendOrDropEmail

* Date allowed in all commands, only sets kwargs for import/insert

* Two updates

Allow headers in draft/import/insert/send email
Quote arguments in todrive decscription

* Make requested changes

* Don't user sendser as an alias for from as they can be different things in SMTP

* On import message, default to not checking for spam
2019-06-27 09:55:00 -04:00
Jay Lee
d9e99334d2 new options and improvements to message send/drop/draft 2019-06-25 13:24:36 -04:00
Jay Lee
a06776bbbd Update var.py 2019-06-24 10:12:32 -04:00
Jay Lee
aecd725a71 Update .travis.yml 2019-06-21 21:11:59 -04:00
Jay Lee
ad8e9364e1 Update .travis.yml 2019-06-21 20:49:33 -04:00
Ross Scroggs
b812fef1c3 Update draft/import/insert/sendemail (#959)
* Update draft/import/insert/sendemail

If possible, use same option names as already exist in my Gam for same commands.

Include charset with file.

* Code cleanup
2019-06-21 20:22:44 -04:00
Ross Scroggs
56371214b0 Document gam <UserTypeEntity> sendemail (#958) 2019-06-20 18:32:28 -04:00
Jay Lee
3669a86f41 use gam.py, send to admin 2019-06-20 16:40:27 -04:00
Jay Lee
4095bf63ef Email send/import/insert/draft 2019-06-20 16:29:46 -04:00
Jay Lee
d75321ca8a put on the breaks :-) 2019-06-20 11:52:07 -04:00
Jay Lee
c931e1cdd7 retry on version extended also 2019-06-20 11:43:45 -04:00
Jay Lee
ea8cda72c7 full email on send 2019-06-19 16:46:14 -04:00
Jay Lee
f0bcd7888a don't use runCmdforUsers() 2019-06-19 15:53:48 -04:00
Jay Lee
6a08a66221 fi 2019-06-19 15:45:58 -04:00
Jay Lee
5e58edf598 command to send (spartan) test messages 2019-06-19 15:41:01 -04:00
Jay Lee
1e79772ec1 catch/retry DNS errors if we need to refresh token on startup 2019-06-19 15:09:58 -04:00
Jay Lee
3461ce053f cleanup print jobs 2019-06-19 12:49:50 -04:00
Jay Lee
9c84ce30e8 retry on ServerNotFound DNS issues 2019-06-19 12:15:23 -04:00
Jay Lee
93ca63e133 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-06-18 14:54:49 -04:00
Jay Lee
e367c86fce fix issue with over-escaping aue model 2019-06-18 14:54:33 -04:00
Ross Scroggs
34dc12994a Three updates (#956)
Add notregex to GAM_CSV_ROW_FILTER to allow selection rows that don't have a particular value

Standarize formatting timestamps

Display mobile.patchSecurityLevel as a date/time
2019-06-18 14:03:19 -04:00
Jay Lee
df4de5ce4b move CrOS AUE dates to dynamic file 2019-06-18 11:20:24 -04:00
Jay Lee
ea630480b2 create cros-aue-dates.json 2019-06-18 11:14:43 -04:00
Jay Lee
db1159cd0d remove custom code verifier patch and force google-auth-oauthlib==0.4.0.
Next release may break code_verifier auto-gen based on
https://github.com/googleapis/google-auth-library-python-oauthlib/pull/48 so we lock at 0.4.0 for now.
2019-06-18 10:23:50 -04:00
Ross Scroggs
ccf1dc0585 Update calendar ACL commands (#953)
* Update calendar ACL commands

Your change was not backwards compatible;  sys.argv[4] was previously ignored

* Code cleanup
2019-06-17 19:52:08 -04:00
Jay Lee
1cb96cf057 Print Calendars ACLs, delete by ACL id 2019-06-12 14:25:10 -04:00
Jay Lee
2bc8a114c1 GAM 4.87 2019-06-12 13:46:14 -04:00
Ross Scroggs
7f183b9edc Prevent traps when filtering CSV rows (#951) 2019-06-12 13:44:01 -04:00
Jay Lee
f86be17834 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-06-12 13:42:48 -04:00
Jay Lee
6854e3729a prefer id_token_jwt if present in oauth2.txt. Fixes #952 2019-06-12 13:41:16 -04:00
Ross Scroggs
5f48b1e16f httplib2 throws RuntimeError when OpenSSL is below 1.1 (#948) 2019-06-07 19:15:31 -04:00
Jay Lee
aea92be8d3 Update .travis.yml 2019-06-07 17:15:16 -04:00
Jay Lee
b4c5d6c626 merge changes 2019-06-07 16:30:51 -04:00
Jay Lee
0e2845082b include headers on version request 2019-06-07 16:29:28 -04:00
Ross Scroggs
705a40d035 Handle unknown server in gam version extended location (#947)
Line 847 drops a training space
2019-06-07 16:11:34 -04:00
Jay Lee
f85a072708 downgrade patchelf to compile on precise 2019-06-07 15:34:01 -04:00
Jay Lee
e990387660 fix precise patchelf compile 2019-06-07 15:21:11 -04:00
Jay Lee
feb76f504c build musl and patchelf 2019-06-07 14:21:52 -04:00
Jay Lee
6a33173a49 Use precise for legacy build 2019-06-07 13:58:16 -04:00
Jay Lee
5e8e9765fa use 2019-06-07 12:09:20 -04:00
Jay Lee
2a6cdb9b14 fix rule 2019-06-07 11:33:52 -04:00
Jay Lee
642c0fa216 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-06-07 11:28:36 -04:00
Jay Lee
9d5e79725c generalize TLS test 2019-06-07 11:28:16 -04:00
Ross Scroggs
db301d2635 Fix case where event has no parameters (#946) 2019-06-07 10:08:49 -04:00
Jay Lee
d7283d17e2 AUE updates 2019-06-07 09:44:54 -04:00
Ross Scroggs
bd7b58ad43 Pylint cleanup (#944)
* Pylint cleanup

* Handle Boolean values in activity cleanup
2019-06-07 05:28:10 -04:00
Jay Lee
27c0e5f8a6 use httplib2 TLS min/max feature, standardize http obj creation 2019-06-06 16:40:23 -04:00
Jay Lee
7a439a3e07 GAM 4.86 2019-06-06 13:23:50 -04:00
Jay Lee
fff6e6717f Merge branch 'master' of https://github.com/jay0lee/GAM 2019-06-06 12:31:21 -04:00
Jay Lee
2671764e99 cleanup report token and other activity reports 2019-06-06 12:30:40 -04:00
Ross Scroggs
4f12e2affb Various cleanup (#943)
* Various cleanup

146, 152 - pylint
1752, 1762 - Python 3
1742, 7695/7703, 7773/7778 - Standardize getting project id from client_secrets.json

* Fix bug in update group;  specifying delivery_option causes failure
2019-06-06 12:29:59 -04:00
Jay Lee
e1faab524b OpenSSL 1.1.1c 2019-05-30 12:57:58 -04:00
Jay Lee
6c5585d059 standardize char choice strings 2019-05-30 10:12:11 -04:00
Jay Lee
dc678dd510 include punctuation in random passwords 2019-05-29 19:19:25 -04:00
Jay Lee
7b70c5a745 switch back to choice() over sample()
sample() decreases randomness because a char will only be chosen from
the pool once for the string. Consider:

random.sample(string.digits, 10)

you'll always get the numbers 0-9 in some random order whereas:

random.choice(string.digits) for _ in range(10)

will give you greater randomness as there are 10 choices for every char.
2019-05-29 16:58:08 -04:00
Jay Lee
2e8190ce29 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-05-29 16:03:16 -04:00
Jay Lee
14b09b91df Temporarily implement OAuth 2.0 PKCE
Add OAuth 2.0 PKCE support while we wait for upstream
google-auth-oauthlib to implement. See
https://tools.ietf.org/html/rfc7636 for more details.
2019-05-29 16:00:10 -04:00
Ross Scroggs
b77d7eaadc Mr. Consistency strikes again (#941) 2019-05-29 14:22:55 -04:00
Jay Lee
c68e0bfbc4 Remove SA DwD enablement steps on project create
The step to enable domain-wide delegation for a service account is
unnecessary. Even when unchecked, the service account gets access to the
scopes that admin has granted in admin console (gam user <email> check
serviceaccount).
2019-05-28 14:02:46 -04:00
Jay Lee
8a26e99dfc don't store scopes 2019-05-28 10:49:10 -04:00
Ross Scroggs
9e2dd11617 Handle missing credentials, e.g., two gam oauth deletes in a row (#938)
* Handle missing credentials, e.g., two gam oauth deletes in a row

* Add scopes back to oauth2.txt

If scopes are in oauth2.txt, an advanced gam user can use it unchanged. My code does preemptive error checking to detect API scope mismatches early on.

* Suppress token details unless requested

* Bring on the details

* Update scopes used to make oauth2.txt
2019-05-27 10:25:38 -04:00
Ross Scroggs
19f01007f4 Clean up random string generation, cleanup (#937)
* Clean up random string generation

You don't want printable.

string.printable: String of characters which are considered printable. This is a combination of digits, letters, punctuation, and whitespace.
string.whitespace: A string containing all characters that are considered whitespace. On most systems this includes the characters space, tab, linefeed, return, formfeed, and vertical tab.

* Cleanup
2019-05-25 17:40:20 -04:00
Jay Lee
2c4bbbbbfb cleanup random string generation 2019-05-23 21:26:41 -04:00
Jay Lee
71536c50a2 set auth prompts equivalent to what we had with oauth2client 2019-05-23 11:10:24 -04:00
Jay Lee
1d118a9ca3 use oldest domain creation as customer creation date 2019-05-23 10:19:37 -04:00
Jay Lee
94f6c45291 expand info for gam oauth info 2019-05-22 13:00:06 -04:00
Jay Lee
fd4a64f6a7 gam oauth refresh to force refresh 2019-05-22 12:42:44 -04:00
Jay Lee
d8ba15ab98 iss seems to be with and without https:// 2019-05-22 12:22:57 -04:00
Jay Lee
a0a2e1359e more cleanup for google-auth 2019-05-22 12:12:31 -04:00
Jay Lee
34b32da1e6 change deps 2019-05-22 10:33:42 -04:00
Jay Lee
f9af688bea replace deprecaed oauth2client with google-auth
Early work, much remains to clean things up and patch all the remaining
holes...
2019-05-22 10:17:00 -04:00
Jay Lee
8829bc3a65 filter activity reports by orgUnitID 2019-05-20 10:44:51 -04:00
Jay Lee
59d8e5a853 change order to allow more time for processes to finish 2019-05-18 10:40:03 -04:00
Ross Scroggs
fddd77e87c Update events (#932)
* Update printevents

The column header should be calendarId not primaryEmail as the user could specify a resource calendar for instance.

The order should be calendarId,id: major,minor as in other print commands.

* Event cleanup

* More event cleanup
2019-05-18 10:10:53 -04:00
ejochman
eed3fb1ed9 Remove unused imports (#931)
Also corrects identified common misspellings
2019-05-18 10:08:02 -04:00
Jay Lee
fbaf9272a8 Update windows-x86-before-install.sh 2019-05-18 08:37:52 -04:00
Jay Lee
97df7c75b1 Update windows-x86_64-before-install.sh 2019-05-18 08:36:06 -04:00
Jay Lee
358892869b fix paths 2019-05-18 08:05:31 -04:00
Jay Lee
7cca371c22 Update windows-x86_64-before-install.sh 2019-05-18 07:23:16 -04:00
Jay Lee
8b1c8b36ce Update windows-x86-before-install.sh 2019-05-18 07:21:47 -04:00
Jay Lee
86154adc92 full paths 2019-05-17 22:17:31 -04:00
Jay Lee
bfc0b57f62 fix cp 2019-05-17 21:54:16 -04:00
Jay Lee
d67110e771 check for OpenSSL version 2019-05-17 21:40:01 -04:00
Jay Lee
2d7c382c2f figure out OpenSSL silent install 2019-05-17 21:38:34 -04:00
Jay Lee
4536cd13ef MainInstaller sub-folder 2019-05-17 17:27:14 -04:00
Jay Lee
627db97b30 use 7-z for extract 2019-05-17 16:45:40 -04:00
Jay Lee
bbdce9536b [ not { 2019-05-17 16:15:27 -04:00
Jay Lee
5f0644d924 extract msi for openssl 2019-05-17 16:01:08 -04:00
Jay Lee
0b5dffc5a5 more fi 2019-05-17 15:22:54 -04:00
Jay Lee
33c85e9ed6 allow PRs to succeed, Win SSL 1.1.1b 2019-05-17 15:18:41 -04:00
Jay Lee
c04164e47a fix osx build 2019-05-17 14:17:21 -04:00
Jay Lee
06e7545f33 fix before-install 2019-05-17 13:26:12 -04:00
Jay Lee
0399d96fd4 cache ssl and python, optimize python 2019-05-17 13:18:12 -04:00
Jay Lee
347688ac8c Merge branch 'master' of https://github.com/jay0lee/GAM 2019-05-17 12:07:54 -04:00
Jay Lee
37ce64acb7 print and move events 2019-05-17 12:07:24 -04:00
Ross Scroggs
4b8bc4e7ea Fix Issue #917 (#930) 2019-05-17 09:28:28 -04:00
Ross Scroggs
d9ba83217f Update parse-aue.py (#929) 2019-05-16 14:26:19 -04:00
Jay Lee
1d658ca1ac GAM 4.85 2019-05-16 12:39:49 -04:00
Jay Lee
65f94ff465 lowercase on AUE guess, many more exceptions 2019-05-16 11:55:37 -04:00
Ross Scroggs
1b010fbd07 Optimize AUE guessing (#928)
Only look up a model once, cache the result
2019-05-16 11:44:05 -04:00
Ross Scroggs
a1bce42387 Update AUE exceptions (#927) 2019-05-15 20:29:37 -04:00
Ross Scroggs
4808f18ca0 Cleanup guessAUE (#926) 2019-05-15 18:05:32 -04:00
Jay Lee
b41baf19b4 another model 2019-05-15 16:42:36 -04:00
Jay Lee
9d78fa8825 GAM 4.84 2019-05-15 16:23:07 -04:00
Jay Lee
2a6a424ce0 AUE is 1st of month, fix few models 2019-05-15 16:20:48 -04:00
Jay Lee
c05c040241 Guess CrOS AUE date 2019-05-15 12:30:22 -04:00
Ross Scroggs
3d3f2b535b Fix bug, encoded deviceId not included in output (#924) 2019-05-14 10:42:10 -04:00
Ross Scroggs
49bf1f675a Code cleanup (#923)
keys(), we don't need no stinkin' keys()
2019-05-12 17:08:13 -04:00
Ross Scroggs
50f3ac493c Update GamCommands.txt (#922) 2019-05-12 12:39:31 -04:00
Jay Lee
690302f2b3 Full support for archived user 2019-05-11 11:24:06 -04:00
Jay Lee
af7b387f94 disable resource sa tests again 2019-05-10 15:02:48 -04:00
Jay Lee
1c38c47e4a v4.83, re-enable resource tests 2019-05-10 11:05:55 -04:00
Jay Lee
74525efc37 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-05-10 10:20:54 -04:00
Jay Lee
985db74216 use external script to set row filters in travis 2019-05-10 10:20:39 -04:00
Ross Scroggs
6d4a34d971 Code cleanup (#921) 2019-05-10 09:57:15 -04:00
Jay Lee
43b17cce0e escape colon 2019-05-09 12:24:16 -04:00
Jay Lee
0fc11de8bf fix travis 2019-05-09 12:07:53 -04:00
Jay Lee
39f5b1dbc2 more GCP fixes, testing 2019-05-09 12:04:56 -04:00
Jay Lee
f3596856ba escape $ 2019-05-09 11:30:53 -04:00
Jay Lee
fc7e8b4deb fix .travis.yml 2019-05-09 11:26:07 -04:00
Jay Lee
4178b4a61e fix printer reg py3.5, less aggressive bulk user on travis 2019-05-09 11:16:21 -04:00
Jay Lee
05d5fdde53 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-05-09 10:59:32 -04:00
Jay Lee
3be1e97863 gam print commands in travis 2019-05-09 10:59:02 -04:00
Ross Scroggs
0cffb8202d Update encoding so Python REs with \ can be processed (#918)
* Update encoding so Python REs with \ can be processed

Fix bug in printShowDelegates

* uid:ResourceId can't be downshifted

* Appease pylint, fix print formatting

* Fix row filter count matching

* Fix indentation
2019-05-09 10:34:12 -04:00
Jay Lee
657ce1cbbf Update .travis.yml 2019-05-08 07:18:09 -04:00
Jay Lee
f8ee80be19 Update .travis.yml 2019-05-08 06:58:59 -04:00
Jay Lee
70de8b8719 print archived state for users, fix delegate count 2019-05-07 17:57:11 -04:00
Jay Lee
f972f0b8e4 really fix osx gampath 2019-05-07 17:12:26 -04:00
Jay Lee
8c025758c9 fix osx gampath 2019-05-07 15:55:28 -04:00
Jay Lee
2713bd9bfc show delegate count 2019-05-07 15:23:29 -04:00
Jay Lee
cdc067c4b7 = not - 2019-05-07 15:07:24 -04:00
Jay Lee
a18ab16d9d more cleanup 2019-05-07 15:00:09 -04:00
Jay Lee
7bdde3ec68 limit delegates to 25, fix building create 2019-05-07 14:50:40 -04:00
Jay Lee
2c89f988a1 travis cleanup 2019-05-07 14:40:41 -04:00
Jay Lee
ead7c7403a actually add enc oauth2service file 2019-05-07 14:23:55 -04:00
Jay Lee
5ce67e5f5c Service account testing 2019-05-07 14:15:37 -04:00
Jay Lee
b34b2d8e2a Allow identifying resources by id 2019-05-07 13:59:00 -04:00
Jay Lee
64d1dec3d8 quiet wget 2019-05-06 16:41:05 -04:00
Jay Lee
7f14bbcc22 CSV commands for travis 2019-05-06 16:30:15 -04:00
Jay Lee
9396cdef4a test more GAM commands 2019-05-06 16:12:47 -04:00
Jay Lee
b49f759ef9 try python nightly again 2019-05-06 15:51:13 -04:00
Jay Lee
3087e0cbdb fix 3.8-dev 2019-05-06 14:45:55 -04:00
Jay Lee
9992eaad2b cleanup lastupdatecheck.txt to keep out of archives 2019-05-06 14:37:44 -04:00
Jay Lee
33f5e74e45 Python 3.5 fix 2019-05-06 14:30:22 -04:00
Jay Lee
9a068f02da Run GAM test commands against live domain 2019-05-06 14:13:43 -04:00
Jay Lee
b71a4f1d0b Merge branch 'master' of https://github.com/jay0lee/GAM 2019-05-06 11:51:01 -04:00
Jay Lee
42950550cc deploy on build VMTYPE 2019-05-06 11:50:10 -04:00
Ross Scroggs
86842bbb02 Add other OAUTH2 token errors (#915) 2019-05-06 11:40:09 -04:00
Jay Lee
4ca0277e63 split out build vs test some more 2019-05-06 11:37:41 -04:00
Jay Lee
671ac52201 make sure GC_TLS_MIN_VERSION is None if Python doesn't support tls min version 2019-05-06 11:12:00 -04:00
Ross Scroggs
78c71babda Python 3.5/6 don't have the version attributes (#914)
* Python 3.5/6 don't have the version attributes

* Code cleanup

* Exit on unsupported request

* Change message

* Fix error response handling

ERROR: Authentication Token Error - ('invalid_grant: Invalid email or User ID', '{\n  "error": "invalid_grant",\n  "error_description": "Invalid email or User ID"\n}')
2019-05-06 10:46:39 -04:00
Jay Lee
19196a2ee4 set platform/gamos for test 2019-05-06 10:37:49 -04:00
Jay Lee
843c5f0687 fix deploy 2019-05-06 10:25:30 -04:00
Jay Lee
260160ade1 VMTYPE env variable for build vs test 2019-05-06 10:23:45 -04:00
Jay Lee
06525ff690 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-05-06 10:01:42 -04:00
Jay Lee
ac2ccc2d03 Test on Python 3.5, 3.6, 3.8-dev and nightly 2019-05-06 10:01:36 -04:00
Jay Lee
41eba0c873 Update windows-x86_64-before-install.sh 2019-05-05 12:56:36 -04:00
Jay Lee
f6672496d1 Update windows-x86-before-install.sh 2019-05-05 12:55:00 -04:00
ejochman
004b51f3cd Gate usage on minimum supported Python version (#912)
3.7 is recommended, but 3.5 should be currently supported
2019-05-03 17:59:51 -04:00
Ross Scroggs
1046716304 Fix for Python 3 (#913)
* Fix for Python 3

* Another fix for Python 3
2019-05-03 17:58:31 -04:00
Ross Scroggs
187b7a8c39 Multiple updates (#910)
* Use integer division to get minutes

* Update to latest  Drive API

* Clean up language output formatting; encode control characters in mobile deviceId
2019-05-02 12:44:30 -04:00
Jay Lee
27177e3ef4 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-04-30 08:28:49 -04:00
Jay Lee
367eaae47f Google Voice SKUs
As defined at
https://developers.google.com/admin-sdk/licensing/v1/how-tos/products
2019-04-30 08:28:23 -04:00
Ross Scroggs
dcee433547 Update GamCommands.txt (#908) 2019-04-29 20:09:06 -04:00
Jay Lee
ea74e24024 GAM 4.82 2019-04-29 15:37:19 -04:00
Jay Lee
6353ddf58a get/set Gmail language 2019-04-29 15:36:49 -04:00
Jay Lee
911d83cfd8 deprecate whatsnew.txt, handle multiple GAM glibc versions 2019-04-29 11:38:00 -04:00
Jay Lee
49fc1c4f7e remove noverifyssl.txt in favor of GAM_CA_FILE
completely disabling SSL hostname verification is very dangerous and
unnecessary. Instead, allow admin to set GAM_CA_FILE to point to their
own file with certificate authorities. This file would presumably
include their own certificate authority when doing "man in the middle"
SSL/TLS inspection.
2019-04-29 10:21:56 -04:00
Ross Scroggs
097eb07fcc Use regular expressions in GAM_CSV_HEADER_FILTER (#907)
* Use regular expressions in GAM_CSV_HEADER_FILTER

* Handle no matching column titles
2019-04-29 09:39:31 -04:00
Ross Scroggs
00f992259b Python 3 cleanup (#906) 2019-04-28 13:31:34 -04:00
Jay Lee
ce655173f8 Update .travis.yml 2019-04-27 09:15:07 -04:00
Jay Lee
ec1c146362 add glibc to archive name 2019-04-25 14:02:52 -04:00
Ross Scroggs
4098a9b70f Update GAM_CSV_ROW_FILTER (#901)
* Update GAM_CSV_ROW_FILTER

* Improve GAM_CSV_ROW_FILTER

* Improve again GAM_CSV_ROW_FILTER

* Improve error messages
2019-04-25 13:11:11 -04:00
Jay Lee
b617d5cab0 TLS min/max config options with sane defaults 2019-04-25 13:04:26 -04:00
Jay Lee
4b29fda924 GAM 4.81 2019-04-24 15:57:59 -04:00
Jay Lee
37d07c9bd8 shell wrapper for staticx no longer needed 2019-04-24 15:56:38 -04:00
Ross Scroggs
0cf964073d Code cleanup (#900)
* Code cleanup

* Add missing _

* Add missing character

One character was missing from the prefix, I assumed :, did you want a space?

* Put missing variable back

* More cleanup repairs
2019-04-24 13:40:35 -04:00
Jay Lee
298e161658 v4.81 2019-04-23 21:29:17 -04:00
Jay Lee
997b2e84da staticx from git 2019-04-23 20:39:26 -04:00
Jay Lee
c135866f0d deps 2019-04-23 20:34:18 -04:00
Jay Lee
509f125e3a Precise for precise 2019-04-23 20:15:02 -04:00
Jay Lee
90c51a2d51 dist 2019-04-23 20:00:39 -04:00
Jay Lee
2b58539588 staticx only for oldest 2019-04-23 19:39:38 -04:00
Jay Lee
8b3bf26edf No bionic, append glibc version 2019-04-23 19:20:10 -04:00
Jay Lee
d4190496fb Try bionic, fix pyver grep 2019-04-23 18:52:37 -04:00
Jay Lee
a8ceb40953 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-04-23 18:42:19 -04:00
Jay Lee
e47b6a32de more tests to confirm sane build. 2019-04-23 18:42:13 -04:00
Jay Lee
77ed31d975 Update osx-x86_64-install.sh 2019-04-23 18:06:22 -04:00
Jay Lee
2d4a24554f Update osx-x86_64-before-install.sh 2019-04-23 17:49:43 -04:00
Jay Lee
4fcade37c2 mypath 2019-04-23 17:26:14 -04:00
Jay Lee
71978841b1 macos 2019-04-23 17:01:38 -04:00
Jay Lee
6aab88abce extended 2019-04-23 16:38:00 -04:00
Jay Lee
7bc5ad35cb build patchelf 2019-04-23 16:37:07 -04:00
Jay Lee
751020a589 staticx fix 2019-04-23 16:17:07 -04:00
Jay Lee
fdfd6e80fb record path 2019-04-23 16:03:45 -04:00
Jay Lee
ee3b6e471b record path 2019-04-23 16:01:57 -04:00
Jay Lee
b818d6a1c1 disable python optimizations causing failure 2019-04-23 15:49:49 -04:00
Jay Lee
221cd9826d : 2019-04-23 15:11:59 -04:00
Jay Lee
5afda1dc2f all 3 2019-04-23 15:10:40 -04:00
Jay Lee
108ca38f10 more fixes 2019-04-23 14:43:07 -04:00
Jay Lee
f104c7acdc build-deps 2019-04-23 14:08:48 -04:00
Jay Lee
a62a7d4da4 more build stuff 2019-04-23 13:38:27 -04:00
Jay Lee
2f0fa38f3d more fixes 2019-04-23 13:28:33 -04:00
Jay Lee
2243a4e8eb separate precise compiler 2019-04-23 13:25:14 -04:00
Jay Lee
4e15cb6618 Custom OpenSSL/Python compile 2019-04-23 13:22:32 -04:00
Ross Scroggs
2e103a2d69 Set Python 3.7 (#899) 2019-04-23 13:06:54 -04:00
Jay Lee
22c7609e0d more openssl copy stuff 2019-04-23 11:38:34 -04:00
Jay Lee
04504cdb2e Copy newer OpenSSL DLLs 2019-04-23 11:27:47 -04:00
Jay Lee
294211bf84 Merge branch 'master' of https://github.com/jay0lee/GAM 2019-04-23 11:12:06 -04:00
Jay Lee
0f7e0a4c87 Windows OpenSSL 1.1.1 2019-04-23 11:12:00 -04:00
Ross Scroggs
351e9ab929 Not needed in Python 3 (#898) 2019-04-23 11:05:29 -04:00
Jay Lee
6d16eae210 upgrade OpenSSL OSX, show OpenSSL info 2019-04-23 10:39:16 -04:00
Jay Lee
f0d0345fcd 'gam version extended' to show OpenSSL version, TLS used 2019-04-23 10:31:10 -04:00
Jay Lee
1c8cb1a617 dynamic cacerts.txt, r for text, rb for data 2019-04-22 15:59:02 -04:00
Jay Lee
bc065d4b31 split lines again 2019-04-22 14:23:27 -04:00
Ross Scroggs
71c9d90b39 pylint cleanup, Python 3 cleanup/fixes (#897)
* pylint cleanup, Python 3 cleanup/fixes

* More python 3 fixes
2019-04-22 14:19:31 -04:00
Jay Lee
dd7d4923be readlink works on older OSes than realpath 2019-04-21 21:38:39 -04:00
Jay Lee
f348405d46 Trusty 2019-04-21 21:35:44 -04:00
42 changed files with 6736 additions and 4912 deletions

View File

@@ -1,35 +1,195 @@
if: tag IS blank
env:
global:
- BUILD_PYTHON_VERSION=3.8.1
- BUILD_OPENSSL_VERSION=1.1.1d
- PATCHELF_VERSION=0.9
- PYINSTALLER_VERSION=3.5
- secure: "FSKvLaiqhKz21SVgAQZI3bSX34Ffyev4l+R2G//QXNDu6UVQcuFsykzw+eZEG7fkhotXr8BMDL7xIkookiL8eLwUtcd/Z95HCjPBBHcmCSQleyvuuJBxdrQ9xldmiGLzMCYiumSH9OH4uJhQ39Yjnjsa8TK+PlTci6a/BTzlYyBSyDYDf7Iv/uhfQPDHL3pNwrQPHf4fL6/jcvo+uaPcv83AVZkNzZjjyoi9Aa+uh9xlbyHg11jp44463qqxoxTdYik3pYuXRBPjknjOGcnFHqn+QOVSdRQoiwbmT8xVuYuCzTv9THhuJ//i5u7s4y3Xyl7u17B3tdm86UlMpQHy/w9EsYaSBPOU4oPNomRtOnTSugh0v9ZBwptP5XfbslII/iA+LQdzTHhchn0W0CRyDqjOMSestWlrsq5NZJtBJTYHbebllOhEI7xbj9tY+re1zFWSPMOPgHJP23ovsdk3hD9OT93AzRHInCx5IxL6QvEgRhAancRuGkf2rGP0g/vX9fQ0Il3rNMSQxHB5CyHUBtUJ9nhU79YkMDZicD0jFMEwjWJO3itAp3ynoLXRgktgQCYUfgc9SpdWKD5SXLCYnSo22JD3D1P6h2EertRHaoKRLb+CRXQC/lM8uh/W+BjA2Xe6Vut2I/72ndjM+10T7E2xk1CFyCH37a5p8cH26Fs="
- secure: "J9380tGLOZWa7dSH1y5Il8T5JQpN6ad81gI6VR1HIU0svpRdjgikyDA7ca2MKYDUYYY9yVSkTV6gCl6iIU/9+SKaYugpP+tkvdGYkC2moJdcTgYM/WOnIK9ExQ3BPhN1neGxJjPTwKo1ft27mtZ2I5vuCiBwIcnKWLnKPyW3PD+mWpfqiLuEzkHoAh6G3jC4qbcCrZDeX/knE+PzqESUEi+8k1G8gYcSDWujba9ypSsqZ8T/MXagGla6l7y2Rz+/KZTJmFHwKAA10V+xPLVqxoiqi4ar66yUqy0BamwRXPcseI+ns3Q+4lUpMqVQ5GlRy7LF1xC8myjmcAexXk0F9hg+CMzewKI8UgmQH/ZJvQZEh8s6mW26+CqA4d3zMQkWaR0WtEtpiuH7AGHCflIqvEQ6UiG7ia3B8iZfW2wl0j/kqx4OuHkS3r0pWKVVIIvCj9Ow2BHP7SpiV1AcUGsVxzwbgTh67fitna3Z3c6Uj8ccQlNr7ZIt1az6Wf3w5njijkLOiBpQSLKunTTCTSge/JzBTKUcie3RE9vzirl58gUxAt36nDtPWnory+RttMZrOkBVbTeSxp+IUe8pNwLFPHABsafXsjkfzBOtFmm+0ZXWt2Rlog5NvlemJfQUWDlsL4g+BSakzN+4sIPKzSauWDHyaEeULY7Uprkil6c5zwo="
- secure: "szcjWHPr0Bf1KCkyTrV5Fu3ADhWk+pg8YWucjXHdybmhaQIKG7iBNg8LJ5d0OBTwAg31wK4ZgyLVSa2gKrAZ3UeDjykJFsR711xDSQOod51Wrgqu4FbXDewE817DUk3Cwe1l5DCu3/fjEw4vbm8B/qb7iMTRKCq6hJd97FwT5oauP0QHNPer9JjrW4F0Hk9ttkgEU2dXWvBMsTJsDOGNI3ddABE2HskxV4T4thelDYGKBDHhUOAsRwSjXgWy77Tvz98psPIvd+6+WPYNRdRWcPDyAR3Z1O/fNjUymrQI6eMaHoSFrmhDS5lbhjINRfdUmECyfCfIFeLWWiw4g4bq7l+4HBORbei55tAIjhEsxJQoqHi0Q5dD5TFh8IiWqowkFbpvNonMSIpKtB0cyT5jU1G/jRA7MPcIvSrdzHaDkoDNHJgAeZfgjOhzTGYYD19lGIljz5BQBcNFZY2dJbja+Jr4He2CMAOBOdERa4Zn1VyNfOmd8Bn5hu0C9D2ybnSCxjXXq5TRiktR8X7WycVZYfqMZXAwP9FEHVitJ4MZEGUc7S92K5gX4wmjcJjLS+Xo/0nsduQm8PuiMjbcPM7/oGx8Xm1KuSfHdKWMBoaesPaDvRX+YcuiNstXf1DkCWl72TsFABzddlNUMl/s2YSKkCSHAJ5ILqrB28Gx89kzVlg="
- secure: "CPsDgSoZIHLyjpUbYEsx1GbB+UZcPXCEsz9qT6XRM+qMFKLlSnHxoJ7gMrJtqfjTh+1gLB3UTjQjFr+jAenqLWzaJh7Zdtpg9BOG6aXC8CAi1h0U5ZMNSA/+5lQxOXuhN8HmXI1r8xPNRFXBdZFea+072/2HJTwmgYlVTTQ8FrbXQJCzs2cFqxnVeFmuG3N49AJuoy3B+P2DMpqPzABbQt7Jf5H9Foq+16iXxhRkYA8/8H2nF7ZE8IdJuRpqhBUoPF/8Mt2QBLIlvNIdIzFEy2O7ZhwL9Dt08AG9v5c91QPzjsLok2e+5hFGaeMpGQJCE1V3uHyrOAeyZs1QYr7qBWbjQCVh0Phxz9yOA9RPfnQdjyJTq4QEj6vVYvCi5K/CGp/DnnLnGyYJKZOt7nBx02fTVI136UXqt29eF5NRkCpnUqah2s0OXkijsw5Y9LsbwiUxELKtCthESOwN8e/ZNvn/gjPfZWIaB0gupRNugL8Xo9Cx6vFmaW8wzm1IutJvo1mWvkWMvuYYjvd4aDyP6s7PFnSX7DoD/pfxoQuzjwHO2OD93nCOx0ofnNojLNooeJPKLikiwS4qKc8exWd+TlnccKSkXO8Y8S+XRgeE652YNlf+9DH79lNLeK0N0W2tRExX5JaEyJQCuvK0WZi02kxvjmUHhwLAOv9ueC2UCRs="
- secure: "JMv9mkSQBJXvUznXcyvngFaOfd2fHYEQQTH8BnS03pqAL0HkWSheG/R2HFJlJv1VJ9MDMf+wVzwMvU/kzkOT68nIgyFWnLBkT6fw+Mw7t/dG96/nOh7DPuaTVzKS0xRbMhDaqZOA9GW13TVnpVdIw79vbhQM69N9Gj80j8oM1cHgMi+fkJSDU3EN/vMJOGKSB3DTpyqAG/nYyZ38tLib8ic12za/YL3HIu8QRHa3fr37+cyrVKgGebGg++yK34p8lC1W4apiO30drmHVQhKWrWnmdcssdGmVM7NystMGGAzUwsJcmRFJuREv1LXiijDzjIRVduceeRYgPi1KHB5ZdL9vji0gM1eHYZZefhcJb6WgPDbCtbjdlCId4v+1bNfsh+dMmhM+vBZDtDEt3UC3MBeYTmklQT2w9zCXHuPBQigj7W4zLm6GxnAXife0SQMfmr736QBvSLUJWtd2tRgS+dRG/LvWxrP5Urvfgs8iRtEVZLDpRR+bSjvLs1UEKLN13KMKYuVwieHbxJn2kY7d5wmZVBYdaokl0yrTkZZ6J9xIphxM8GXJU1BnMZY9zn+Xq3jm576QYnNYUwCtUjOis00Ct4UVuuFqIqZ7bJL7GT4AHTENAk+9GtXVCo3/jnv161rxgoSdnUG6VmrpnyA9jSpcIvMVcE129oghBJ2+PkU="
- secure: "otePbb9W6CAuwzayL0dDAvCryzF2s5HHAIytvMW/xnzOuRMR03lumj9bhg56YCB/nIuTncAjToYmsMh2Acrp1Xcx8/DzqKse1K9nkfJgsB+EYeXmy8pVJnCxhHeLCzUPJjUfRoPBm+5GHAkeoviUyxenwbOFm3/Uhtay7i2ZU8K1FsECqx4FZXVh8rrGHqFGzHNimqDDvFUQM6f9a+BevEio8+/aDooaFfvVqPixDYXpi2+99DB/0hZKwcUpj6pOUUszATkAl0kJdOSWI6E7bQa63i6SfhcBNHfvbDqhpcAGs1TIXHbXwWGfySABKgT9mkq64CY3bc/QZ6HeWe6P5tJmt4mGBxDMOWMYj4qtCskQCCCiY9+sCx61Sj8W5w8exjQvvvFA088cKtbCKAx8FIil4CuYPtIjDQFkiNI2bC6BmkUen7qe4z7dSF2AZPb6CuSsBjoXL8Ezle3e1pRrkR+SYbxMSaZVcQoG8hinTqaXhlS5gXi97ZzRrJWn3tUB6JWEBeEkIfTJmrJolbLYIMcADNhKenrW8k6nV32JkvMsCgMFCIkJsIZeuQ9ZYgSZ3CXPODux/D2/4/gNa+353Fs3DTlDQxRTgoaYi9UDIRrKZAkoELFclxuGSKMCCheevQ8FMCmXfxUocwLSgjlO7g4rTJa9Kggn7VltucXiVWY="
- secure: "WKdB+WpZG/7SKpPpgM6DAwzVg0QQuxFMJEZE324pagJ74xv+GlObTLm6kU4SAZf6TuRSChTXDAD4paJXkxpq3LlXjp9aKxPASbG5PgZMnGL6bZ40c/UUqrwVy2HLnXPKSNuDBX1RidGXmH3u4YvufjguNnlOLvHYfMHIRGjiEL4ZvC38GB/3YDmID1zI6w8NzTCpFvxeNdg1qhhOBUKyt+icRUwBu8DfgLidzTrO0j5jo3UeqO/w3E9t3nnRgSiO25mBYwWySm1QBK1VS3F5iG9jo1J6GKMOFbWi3lOBoqnWotpfwID+p1vMNX35phHwmMrtoBYfMTV4A2CvQpGyQhn58qyMoKnhLz+NIOxlHN83qcu65bTMG+7ji0pgUl3jhp3ZPyWUjRYQpoVAg5f3UAnUzJgBrOUC0N60ukJVacT/kvkO10CLfz+0+eefW53r+GCkTi2m8iDezZn1olinpLs+Mt0M4VfQ52RVtq3WB6uxN7RjgtrK5XGi2zVAsvfHp+vqFQ6uu+iioiWNji88cTlMuKheo+FWwozNQzd8+4Vxe9w01mLek57Q0dpXiKWL3vMS4Qc4T9E6UzutllDyg/c61LW6Afcqw3jL4HN0UNG3zFefno47oKrmB13HcxakLEC8sdNfE1kWGvB7jqD0snXsz/8rz7rf0IqnK0kaCAc="
- secure: "Lah0Q4bWyEJXTBLZHCNkuEuU5wlDQzLF6aQwW47RyVtyRDYW8uQRQFkbb/3oj4QAAgXl0sOcDFnRaaWmlOStPIIYKKVY8Mk6ufyDCz8inWoPNJoSbdqbAi761HacGUsEfSNDrPNB7fst47UZaX7WNAO3q1QjCCGSEJQVpJE2MDAjcSJhAgLPt3JRFSOSwvqxcDLZI+wF9AM81OWNp6QisAdI4LPUJK3L4M4dLh9+apFRb1OMld++scWah1SB5Qj6tPzMvX96mp1XfWB1fJXuf8Yvks02H6ZcAXubK2HXe3iJZKVqGB67kShNN52P+wg0ZZO4OP4kZ2PXahnB7hhxIfEfWsNVy/w4ww1N6K907BfT6RDmsgNP1ZP8kpq6H4pTnWgAy9WDxjatzbNFAHtMzakGjYJNFxZJVco2FL0ipNQ7htoVAA4sUb+VbRQv4O/oZTLMNnnco19+TFJzuZuS4Rjxj/zX63gXkj/W7wou3hw+NpI/PL2hUXIc5imXSLfVQwri8Nl+6IOjV2gWR/vR1VhQqbLsTF6TZQKNI4lvbRTs1nNeqNMW1lHizI3r9Kernj2noAsxJK7wFTZ64OUkQvSNphfmVow29JYQKXbpxbmRrdnmfmtnMsxUngpx9WsTPhrprt5hIAqiBspHemrs+H3LiIml6IY/3l9bcPlc0gA="
- secure: "sNc7kC0CuH/TCZh2WwEN51GcA1fSpcliCHpFB+WX7ieQiRu3xKn2avby/T7vbvX0viXRER59arFGQF4i/dyr2g4tlZLVRYjPeiApfduKZ7Lb+vZGro3cWesfHG6Abk2VcgZZli1IDrgrHH5qAWnA9xNnKvKBL9NDM73Zj92BmlDFVEzadTii8brWvced/YP3jNXEmM5ZIufgpe2yidBB2bLWYJXb3Cf1MvzMG4tqNAtZTrI32q50mokz/uTqp3MRJ+cR8sOI+2+2xSbT0zZGLSRZf96/7FKtE0QIDxdWAe4XdlHq1CluRVk30Ju5BEn0QzoYLryCIuw3JjDl1Yksw4IA5imljZJlOmWa2l6fX0HNxMw+z0R/1d2HARA8BY7/uQKv4guV3Cf3jpsWoKSsM1WxqOqsuEFOoRQ2eQNJEaSuC6+j/vzNoj61pOuG0R9OC2PFcFCZ9fomIrZMse+7M3WIj4+mp7e+JDK8DgVdUlqkBVCx1Ospseb5pm6lDx8F5NbgqZgGXgyoWVpqZnyYOoOutezMoD6MI2wXzJaepV/L3+LD5f6q3DAa/sRAEEsBFGyMHXiPYbziEiy8Hz09Sz3inT5rzS8OLOinwAI2sHiIYHTl340XfWdYz6AlNhYLCGwwmtkntbjOj5UVW06IgBBx44ujpZSUjv7SOrACPGU="
- secure: "t9/mC8eSsxXIB2vjcZGtGISxrSY3Yd+XS4+/i1YG1mxSov3R6UdhVl2blLgrvFfVCSo61YMAzJbXKtZlXPzhLDXxpsSlWiatjNrKKo4D2unnlHsyEMJ4wfz8cJ8pKyynP7Kc1ZuaqTSMcEZgk85tlew8Zy/VH6g3Mc/7DvP4SxEDRsP+DdCplZc0vbxHDaV3iC93bwNRfy1UQypwJJ2WQviRema3Umneg8hVl2V35zbaBoy45ubCkUoCbRCDaUyoHA10GrE1OUOLsioar/dj6K2W3EhAtehHLWrUZhhl07rayrrRDQYDTdebifB/MWlvR5FjM/Dr5M4k2ciss4ol3IR5LypRLD+/YBtzeIuqkDbUkaBowY+oUj6OWlzEbzAUrUNa5mnyR2jhr8ivZUeEEWLxljsu8gWq65mzgiZt/u+kVCnMLciUv0//0nrsNsEMI9pau2ZxbcpItFVKdZYXFdmHWG+qPCgMcbgsUM3xqaIc7fNwbk2Aa6erIGqD2VkwWzD/xksweg4lsgQ0N1tXMfoWWKh4Xj/OFom+S+3w9uSx/jQma7nXm+PKQL8dIo7rqza3fz9biq5T6mhwUrqCpFIJv7mrMwaTT1UfOchjiLL3CGeO7Amv7Avmh3fhMPbypF0sws79d7ewZbyv+oSzn+pxlCdzBU1GYx6veApbzhg="
- secure: "ljVx3TgpBJ/ylMKDVmXabi9UNi5YvrRM5UBTrRk7XPu1YFYS4FI1GcGlyvYhToc4fKt4jLhX9qU+s/rZY+odO0x/HpmJglMBCrY+QcWOzuyaP1U5dCET+evuqFdEAZIzLQc4VDjL1aQLZh+OG7bjoBClVAan6a+pmW0yxBC6rNtCWTESG4rY3wOeTpoI0Q1gM7gg5Zkj4Z+yLvYeJdoKHijM7C3/R/VVTqUFqArk+Js7Qb2qTqm03SHP0ahRQA8XSfbPebSkJyX9oLbidanBEaQE6sqnp9Qh+8VGcnn7VkSu6oq2+ZXz4xlSMrH2Iv2JXl68Td51LsLo9BxaMCL68ssgTFfXPSrrcLwholNEt1pXk5nhBl1l6MZ1UwUJyBm+AXZp/4sCK9/P0rGa2d1rOcpOz7nobH7BDktqEJkrR6VzkTMx1aOwtF+JSt7SJQ1RrRdm9uKfOZZsnw17+VgVAHo/ttY0C3cRl10oaF1C/IdliDfa5gJdZ2VSZtJxyewqKwGiZrqCRv2fQyIuGsqfHXsyHVL6q1KfVcHjaXBvh0o6xZ6duieFT4FNHg7clv1qPQV+cLh4L11nugiihRTeYQtKyUnP5YIL+jlcGNM6KqKhF9RN5c+zOqWNmEcz6O8nljY5mFWdIxL6dFfE3+4wpw7snFP0PrWIWl5SmrU5ipY="
- secure: "L0ivSUmbOHNmKWVR2zeYcVR+Xr6780dYA829MyksXfg9OrieTS+qVqSODKexFW89dp4Wf8xFdT9f16xSqjA/xSuX12uiISMiTcFKP39ZnQZ//NDkx1T4pZaeNiysqT+2Ys9OaKTWxn8luFtAYp+DaluancQtEw6+M8H3jQyQZimGHl0QB6BK2A5VA2vko+7ebIdwu5Df5E7lc/LG42H+DdSFkSj7Meu5XMyPHSZRAdOE5doO++tqFKzgs2WbDRHRUP4R4LqtDlGn8+5qnQtQKUN4V9UGrKM95R34BhgUlgOqOFAjFDug71/1/rRyv9XnzKkAdTIbxV7nmxSL+APhQDqpwPr01EP9gtZBOycswV/igJYotgqkhzwNAYrmwOA5Ta8S18Ck0feDEqT20w8yKS4QwV2Ihg4q7CqDFu7i+9iSRUCd/RVrGtG0U5Zo1b3+1JcNxXH/ErUeyHPfrk4vktxfHmU+omqwkfvTRy5upn0Ycr57YHOJc/Iyur7vE07HcnBfAwV0d5KO6HJ7M1n6hmMxeyKmf+qiyQQnphySvHHAfa/9Sec7omwwKv4bn33DwFGc8GWvL6cZXWhmGhFvd+LslpZl0vZ+7Bz0tXlAg4t8V48y/ZtyoSpny0HP4UstdA43PttvZ6q2y8LUNk4LBP5btKmp27Fszuj/rldtj5g="
- secure: "Dox8JthAJqWT9eh3Jt4Morbf4pGN9OjduJXe/lYMsmFvqNg7b94P2QdumWBPmVjDq4YjDVirMejBA/TNwORgKfg7pI7MOw+qqoHpT9xyPecXi3ecyBay13e16p0GNRlc5pUu5JcU8sgCpttvM0EAw6bOuQIhnnkIFesbOvwoxGYjMzjmWMNuikR3CjKbo0LDtD5NJXT1OMSqrRuh8NM/BoKyn/kCdSaq9wI2GUMbkg09/kFJkQOvtMXPkM7dIhr/9UC0ouMIyqe/MHa8O6Y4xESdqiTql9uz1+eZfHIRrgFlHfxDvkMv87Cx5OuL+O+qeT/a+RYLCRJspMoq94IYHGg2iyEfBO2YAkogl53wiEf2KF2JdiNGT0xId7bxCJj3efTuCAXV1oqaHpJli1Mhvs7zPEtf78B4tkWEgjhGr5pBLIlbhNjS5wtTHJX4BUzoiP+wODj4h7rjPAah42nWF8XOMlboVi56sOCLjiHBOvYObqyhSfiQxoi2XHphsrZqw6H03tr4Kqd9HVmuoSvRiv+NOu24Ubr6MrrQM2/G72TrTx0/aBlt8Dx5nx2oWZ9ZMiDUR3XlvDLUi45SpY5qESXz08nRlcdS9EvUpK7C+77bNvX+A3dIhsxnxuNaf2naf+QnYYbvh7q4Qbrj4v6EMYS90Uky1JHdoc2wMua8J+w="
- secure: "Is2Lv/rxKKrXnxFns9KQYseD02tjY9qbgSteVtJavG1cLJDvkTwb6X+Thvgo4cxk5fQPiXScrQaYzHjVVuNleD+dyD1HC/8CU2Xq+tjBhPjdcccHFSbk06DpcETmLGRyMORXG9JkYlgXHLLXtu/9icppWEHgra+zvchVL2YDhofYne8FMNBb/lq0AAC2wgzAS33tW1+57HaYnzl0hf6+Q6lwqoH2/aTfGMRFDgyJ0HK+5IVfLnQJ+OuGFSrj8/0FWSggR5+EXDIddDgovFgaCghMjHYp21bzn0eIAJtuNFFultwk/UC1lT9joXTKEzgLTAh+w13yz1T3x9rNuv6FDKCotBIS/ZDtPmgvyZ8xvB4SzyULnTRSVn7YvspKR6PAO/qxGNudUD5H8tRKer4qKnKjHzSUcVBlRHb4yE6FqqY1Z9RLEcomWO43nJwb6saNHR9BYedyi0gA+EbA+P259QFClW1dWEQ2LQhDa+0VRssOqZ0BQblPFyz+e5Vc9kfAMbOuoss8fjkiYv+twXv7nT27xrVT5okfKDSiy5opZD6d36N5FibZPYiMrVx00YZdkFB+5EqQuJ7lqKUMkZJTeApLzj+h+/4aAOWd3paj5ghv7m+9ReohsNKFHyjaSy97RhMAZjzqgMMdD8rjUSKDhvNKvvQECWHlaXwL129GB0s="
- secure: "l7vWxfcu1RgXbStq76Mzz2I5Iu4e31729OyLYqulXZzft3wO+idvgQKy/JSwajiKgOxlpBuI0wrncgIUYshcRvE4yB0y9+QIMDTegJzTADtRSUyVNCIZfTgvtOvzrlW0iCdVsxLBtYcJWJVPdjF2q59ED1jahd1AuJJqX5e61gfr0eZ9+cNiHbX1u3VpmGchFNWQF4KvebE4WKs5xWEds+AtbTODdQq3H6kKQK3fTVJnbz6WMO+bEWgWI7orfSE20lku/3Q3eMLhNOcPwH8WnUoTTvDWol5Cq5NfPhkKF5aV7kIbNXkxswM7yBPAPumBiXdM1BHpfd4+0YQ/fqRtnqxw85HEYpT8dRewHimCl4IccgGCR+G0tK8RNleKL28GWrz86gRVpXMEhOU3ILwb8as3SdQWLwhXXy5uKGJmsdrCNAH4/8eu3XczO5VN2wOo7narYuBgGcl2eLW9TOLyNVKFKxbQnDLBiOybLNoOV42DCRYt7v3Eknkv1Z0dmX6n23q1z9if9kkfAFgRQWsTbNZyWeIwWuX8b2a0Zq0znS3JflSKxzqS7RHADfBOJEVM86AiGkw0XkR4o31yDrY+zOrkJ3GxfV/HpqG7LzCd9tFeexanneDCE1FJhCkDjpnc0Mdyx+1gVf+u2MkgrU0BbCf2EESBAlC/FTRvxTmk/6s="
- secure: "bvtQ348bxKwTtB8X0zMxeTsM0jEJozbS6/rzH/88Fk90a+KO8SdXen0Kj9/LahV4duMn2xTTRmxMCVj424FbcVTgBkJpIO7btqTcNASORkmK+9wGTK6Bgb9R5sHLbVrbrMYJzsMQR0MWxE8ibPLlyom+ssUIqr7HAjnRyYSDiKChGhgBfdE/0G9OE/DuQaU/ZkTuEYfc4527QNLJ8Bt1aLDdCHJLxzCojNaTHB215Tz7dmpnJjWa45BXxkMwLTpxIJuY7wA7K+2UBJfLvZv5QiPYyeUlosV7FwhCN9e90+dc2x8HPJbwe/Ysv5dm8/FfNTgTe6IkKw1z9kev9/W1tYCtqMzn9GnlIH2000ZUomhHKbc4UUl8sskF2jGr14rSz7pfaTyvXNvK7nKLKM7mZAO/cAaQvZoGEx0s6B7a7YX38NmJWcM0UUD603n4G9uVZNoxjyr8HWC6pho8MY8/u5aFKMPd+C8DaDWeSzNJDZRyi62v+t2iyyZuQUtVXtnDpv8GQW4tYFIJCkkrTm5VNNgD2x3WRsEr3LsBa3BonOmi3a1VrdIpyidDzBEWFUR59a/C0lU5J2UX5W1vu/Pnr9/8p898sDvvDbluz47OR9xoVOjtmGdt/ENJa4EOY1q/eRIN6zrt/cwwjIKgTcYOVk7DFDMXQuYlDeEFuHSJ5Xs="
- secure: "VWY3aJGMFB+E5ObJpUUXNQrnkEpA1pbfey8Z0pe9X/n5ubmDBIAfCPP7hqT/lAF/1hj5+HA6oWEa2FzeJZOL9q8cuymHzkehiX9n3f6GktirMtEoG3Hk/CiW/ZxKf9KyBNRVsqsOiXvfzMcMSU6ZrSV+4ZHIVl0zO5cehSaaiTTqVU5K474N5TCc8SAvpSRpxjZVRXy8rWF7WMaG4Xm6aa/IELccjJL8s6sahk3wT++5pzewxv4XBWf4bNA7+MNLpnzKmm2S5T/uohsw00F/j82g9xFT3qYm6vgaR4ql0aNvxd9gZ/6vdVpqBIUZmcFi9yTSFSsSSH2cNGFk3IU9Ecdh1Psi3B5pTpYzy7BbLWyKo70wBkOoY8qCwJ/wtj+2TcfR72qR8ryXgkbIoXxe04VkPqRNEezk3U+UxUvLLDEWthxjuCRCydT8FaCfYm9N+l5Un8TNmY4W9mOSu5IXKJ7oSQDrJG65Z2uZKQrWm0ToOo1nNLFclFuM+K8JWNOKysfh451LHdbmA0rkoaLUat2ttDBtQpbw3h+Hwn3SW6BfELg6WDJXHyAPNRMPSc2Grk2o7mFfTMJhHQG4U+k2bL+AHJU0nPWzuEJyMraquqKOCcHz56A+6j1PN3wpLbIC18nNPN9Q3TDPiBmwzwEg+zLu0DKTu3Oe4IRKRNTuocI="
- secure: "BSZK3v2RYsNKv/8lBZ/dKagYF+znGXIeDY271A2nCS8DXx+y0octwI5RT08+UqdD1AvhPOPjdnSXG+M2NOSLlfBoXF/0t9S39+qy3EhGgxYAcWcVDXnrOrHEOALg1cQkCA2D/CG+eeG/36SmAn6FINTLaAOilxWGGCeL0KOB5mqPPnPogaYhowIwDkj0Hfsfrw8011XzgbufNr15GIYJ+nP7f41NcgIbnNkqXY6Q1jiWs1DuKXNDr5WGPqztHtVf7RtcpVmShzptAcAMdRqCGKF7atCZhXPbd69j3qf7F533/tCSCIrszR/Wp4BKlJfVCRCk1oiEcfRvprJqeFJ+0WJCVzyD7hXfnkBg2Tb5PmzvxOFM+hsqRu4mY3UdjqXukzAe5O0O6Uo8sqzqQUIjUooRnNQ4GdPd+7wMbyBZn4PJaW6YTi/7zg7mAegqot6uyEGGpWb6iFYrf6DX75GUfTciNktv+ez0AeqYFwNBLgcOsmaq+3V+aR+dIxsxIzC/i5GCTHvMYOIp6Q2tPCuIug0tX6uxm++vMOoMLK+vG1fVQ0Cd8m6yQIWEXUu0VBs1MTlJD+vX6LsK8CGo0Dt3ZS3JmC4TGGo9CBSEkeDvnKvQeX6x3NBIFWvgt+Hjxh4S9U+vW+AaUXpV9k+NEhesC/8Ys0UGbGYsTDzHdx9TMBY="
- secure: "sskZYJ/+xmHh5GNvw3QRf10+sBGOjlub29jOTxjXIRYUyZfkhjsF8CR9NP59CBFuqgN716Mj1uEVEcx0aaepr/zWweQSnqa9lZgFD9nDYALVNh1b4oyqfhYG1sw57ZHdBsJOBF4mKBnahly/QQj11Ya25GyIS2IewwCWVNtCOJYietSssG9l0+qc+RPG5Ub/lEKA3VRrtbUuApcIYszt33DumgZ4wzkzICl1SuOy8V15vHVlkBqASJheEDa4Hia+eAMo6e7OCE5KWjl7K2MxTvtszwFi7lgMCyPCOy6DkaULACfBnQeloh42B2Qhd/ta04vuRKg6y4fKSrnegpeOAa9aGxF1ufvG9IvPsRTsu4+w63b9xsQUN9MyDgxcJe0YeUHPgPYL6mqAjCIKvL62LlIyrQ8IAteoh3MC+4Xb8crXaLGINTcLwYvLwxsHMuC/58hdQ23I4DqnyZGAS3L1IhpX4QvYN6xc7H+ptaiQttweoH5VpMAduSrmSnNmvLOc4g2PvbRES6D95moQ7qk7iX6vpu+PevL5HtQCbB8SFyFLTYWsqZaG+4NrjpIi2hLzUT35meHwrvxG/MsfeqOVlnBfa93VnX75vxOkRFNY56sOtrTJaiVZ+rQUdszAZz3KGPbyrSlz3KxV+ZKxZH6/0oFteAJttXFjWQdICnKGDSo="
- secure: "sdzU5bPs1w7Nzf3F5Gtk/iq2Kfq3zfLQNGcehLZp1gJXzNf7F6HZLOn6GaXQ/5RVlhqR5nOmzMpVQs88rZv+6PE6YqGMugTxHIQeNmXtGnuEDJSBvGT9Ok8ENxcwKwL93g9SCd9P8mTspxklOsW2Bk3No2+Zlc/aQguBcX44TwYF42KuBU4O7oS6pUg8NjnuQp2zTyhp0ouzyAudatPyu1BLci/3lbx+MehutQw1y4Om6g8tfwf950UONZQdqq9MBluu7yYb1oHkdC1J4qgwdCZkJslWIwQHCH5UE4AW9iVG0qVrLpzBGtV27Kfy/Vf8r2gMYzbiur2h2+zzWSDm8/bk8YLn7u1FBpjGJdJ0pX0ZrZr+hMV2vH3e54NvA5WyRU7tw8mZ1PoDYd/FM5KXIYbscSbSRCqTsbPgyVIHuoOWXeeSe+/Ef1ifZv3HHf6ARfUIWupKfAipxChc7QUMI/HEQ9QPsqgBaooZD9chGsWAgv+8tFxdteqkx4Yh+AuZp1rVykB/9vAamUBebQxy0oeGm65j6X1rksfjAPkfeDYB7L4Ruy7tUwPtvYrAHPoWrf9O0g/qDRsw0vdqp42CjszpIxhuPhVRDx0i0wquqw2LnIU3ejRv2R8d1SecVKBBcVsWLFvc9iR4rNIi5JnxITRtiwL1Xv3Vcgx7WwxExtE="
- secure: "f9n1KmU5NuV4jGkrhLNiPD+3Cy7t4D7Rq8YsPlmyB2A6u6IHMuOVP95IwH6Zt0cmMDBrZGthj3/0iu5kzxzqD0m135PT17wSEqsDfDMyKRZJQuYSO/ESVxnSca3afRH7Ds7/ipQVd/ljZgwEnW3JMaOQiIdbbIquNOOTeY0/wsXkreDamXZaKKqUoeadkAV4AkKhM0xcMg+Vni1i71TYPBWrZPLVAu3ZrSvU/cE5mtBUIkbr9EgsEE+WR23QCgtwxKzNxrXetBcPXDsJb98/ABgpoItm5Ko/Zk6pkib44f+iQtn7Y6j3lieELCH5Sn0uy3RrMxkl7xicB7zPYME94NEPHCmshyVsH3RxWfcBG4kauRNBCLYLl5HYF2t1lWZ6In5qlx8xN3Tf0KrbM3vzAEOnnfZt83h3q5OuWl+jzTOcv9xHmeW0lnwEEfS0nxUV3KDqLFBczcUxKBmsA2aWnJZ2HV+kls6OaWZ987m6V2pIGR2uviGT7I4ngjCSOJzbwYbvJbJqYAI97FWP/5pqv97xHLCSSJh6TBO4NMQ7Ib/W1XT6NZyPOjNGSLpd79UA3cTzU2+UYr/7RxZpAKONSAMTJh7CX451XQwgovpFZ2quXs2BzqcVpy8AlUc45ygnFOALOANAkcRP5QFhmff0jpXstjPW83/GksbtaiLhIiQ="
cache:
directories:
- $HOME/.cache/pip
- $HOME/python
- $HOME/ssl
matrix:
fast_finish: true
allow_failures:
- arch: arm64
include:
- os: linux
name: "Linux 64-bit"
name: "Linux 64-bit Bionic"
dist: bionic
language: bash
env:
- GAMOS=linux
- PLATFORM=x86_64
- VMTYPE=build
- os: linux
name: "Linux 64-bit Xenial"
dist: xenial
language: python
python:
- 3.7.3
language: bash
env:
- GAMOS=linux
- PLATFORM=x86_64
- VMTYPE=build
- os: linux
dist: bionic
arch: arm64
name: "Linux ARM64 Bionic"
language: bash
filter_secrets: false
addons:
apt:
packages:
- ruby
env:
- GAMOS=linux
- PLATFORM=arm64
- VMTYPE=build
- os: linux
dist: xenial
arch: arm64
name: "Linux ARM64 Xenial"
language: bash
filter_secrets: false
addons:
apt:
packages:
- ruby
env:
- GAMOS=linux
- PLATFORM=arm64
- VMTYPE=build
- os: linux
name: "Linux 64-bit Trusty"
dist: trusty
language: bash
env:
- GAMOS=linux
- PLATFORM=x86_64
- VMTYPE=build
- os: linux
name: "Linux 64-bit Precise"
dist: precise
language: bash
env:
- GAMOS=linux
- PLATFORM=x86_64
- VMTYPE=build
- os: linux
name: "Python 3.5 Source Testing"
dist: xenial
language: python
python:
- "3.5"
env:
- GAMOS=linux
- PLATFORM=x86_64
- VMTYPE=test
- os: linux
name: "Python 3.6 Source Testing"
dist: bionic
language: python
python:
- "3.6"
env:
- GAMOS=linux
- PLATFORM=x86_64
- VMTYPE=test
- os: linux
name: "Python 3.7 Source Testing"
dist: bionic
language: python
python:
- "3.7"
env:
- GAMOS=linux
- PLATFORM=x86_64
- VMTYPE=test
- os: linux
name: "Python nightly Source Testing"
dist: bionic
language: python
python:
- "nightly"
env:
- GAMOS=linux
- PLATFORM=x86_64
- VMTYPE=test
- os: linux
name: "Python PyPi Source Testing"
dist: xenial
language: python
python: pypy3.5
env:
- GAMOS=linux
- PLATFORM=x86_64
- VMTYPE=test
- os: osx
name: "MacOS 64-bit"
name: "MacOS 10.12"
language: generic
osx_image: xcode9.2
env:
- GAMOS=macos
- PLATFORM=x86_64
- VMTYPE=build
- os: osx
name: "MacOS 10.13"
language: generic
osx_image: xcode10.1
env:
- GAMOS=macos
- PLATFORM=x86_64
- VMTYPE=build
- os: osx
name: "MacOS 10.14"
language: generic
osx_image: xcode11.3
env:
- GAMOS=macos
- PLATFORM=x86_64
- VMTYPE=build
- os: windows
name: "Windows 64-bit"
language: shell
filter_secrets: false
env:
- GAMOS=windows
PLATFORM=x86_64
- PLATFORM=x86_64
- VMTYPE=build
- os: windows
name: "Windows 32-bit"
language: shell
filter_secrets: false
env:
- GAMOS=windows
- PLATFORM=x86
- VMTYPE=build
before_cache:
- echo "this is before cache"
before_install:
- source src/travis/$TRAVIS_OS_NAME-$PLATFORM-before-install.sh
@@ -38,10 +198,107 @@ install:
- source src/travis/$TRAVIS_OS_NAME-$PLATFORM-install.sh
script:
- gam/gam version
# Discover and run all Python unit tests
- $python -m unittest discover -s ./ -p "*_test.py"
- $gam version extended
- $gam version | grep travis # travis should be part of the path (not /tmp or such)
# determine which Python version GAM is built with and ensure it's at least build version from above.
- if [ "$VMTYPE" == "build" ]; then vline=$($gam version | grep "Python "); python_line=($vline); this_python=${python_line[1]}; $python tools/a_atleast_b.py $this_python $BUILD_PYTHON_VERSION; fi
# determine which OpenSSL version GAM is built with and ensure it's at least build version from above.
- if [ "$VMTYPE" == "build" ]; then vline=$($gam version extended | grep "OpenSSL "); openssl_line=($vline); this_openssl=${openssl_line[1]}; $python tools/a_atleast_b.py $this_openssl $BUILD_OPENSSL_VERSION; fi
- if [ "$VMTYPE" == "build" ]; then $gam version extended | grep TLSv1\.[23]; fi # Builds should default TLS 1.2 or 1.3 to Google
- if [ "$VMTYPE" == "build" ]; then GAM_TLS_MIN_VERSION=TLSv1_2 $gam version extended location tls-v1-0.badssl.com:1010; [[ $? == 3 ]]; fi # expect fail since server doesn't support our TLS version
- export jid="$(cut -d'.' -f2 <<<"$TRAVIS_JOB_NUMBER")"
- if [ "$TRAVIS_EVENT_TYPE" != "pull_request" ]; then export e2e=true; fi
- if [ "$e2e" = true ]; then export gam_user=gam-travis-$jid@pdl.jaylee.us; fi
- if [ "$e2e" = true ]; then openssl aes-256-cbc -K $encrypted_ab10ec38326e_key -iv $encrypted_ab10ec38326e_iv -in travis/oauth2service.json.enc -out $gampath/oauth2service.json -d; fi
- if [ "$e2e" = true ]; then cat travis/cfg_template.json | python travis/svars-write.py &> /dev/null; fi
- if [ "$e2e" = true ]; then $gam info domain; fi
- if [ "$e2e" = true ]; then $gam oauth info; fi
- if [ "$e2e" = true ]; then $gam oauth refresh; fi
- if [ "$e2e" = true ]; then $gam info user; fi
- if [ "$e2e" = true ]; then export tstamp=$(date +%s%3N);
export newbase=travis-test-$jid-$tstamp;
export newuser=$newbase@pdl.jaylee.us;
export newgroup=$newbase-group@pdl.jaylee.us;
export newalias=$newbase-alias@pdl.jaylee.us;
export newbuilding=$newbase-building;
export newresource=$newbase-resource;
export GAM_THREADS=5; fi
- if [ "$e2e" = true ]; then echo email > sample.csv;
for i in {01..20};
do echo $newbase-bulkuser-$i >> sample.csv;
done; fi
- if [ "$e2e" = true ]; then $gam create user $newuser firstname Travis lastname $jid password random recoveryphone 12125121110 recoveryemail jay0lee@gmail.com travis.jid $jid; fi
- if [ "$e2e" = true ]; then $gam user $gam_user sendemail recipient $newuser subject "test message $newbase" message "Travis test message"; fi
- if [ "$e2e" = true ]; then $gam create group $newgroup name "Travis $jid group" description "This is a description" isarchived true; fi
- if [ "$e2e" = true ]; then $gam user $newuser add license gsuitebusiness; fi
- if [ "$e2e" = true ]; then $gam update group $newgroup add owner $gam_user; fi
- if [ "$e2e" = true ]; then $gam update group $newgroup add member $newuser; fi
- if [ "$e2e" = true ]; then $gam csv sample.csv gam create user ~~email~~ firstname "Travis Bulk" lastname ~~email~~ travis.jid $jid; fi
- if [ "$e2e" = true ]; then $gam csv sample.csv gam update user ~~email~~ recoveryphone 12125121110 recoveryemail jay0lee@gmail.com password random; fi
- if [ "$e2e" = true ]; then $gam csv sample.csv gam update user ~~email~~ recoveryphone "" recoveryemail ""; fi
- if [ "$e2e" = true ]; then $gam csv sample.csv gam user ~email add license gsuitebusiness; fi
- if [ "$e2e" = true ]; then $gam csv sample.csv gam user $gam_user sendemail recipient ~~email~~@pdl.jaylee.us subject "test message $newbase" message "Travis test message"; fi
- if [ "$e2e" = true ]; then $gam csv sample.csv gam update group $newgroup add member ~email; fi
- if [ "$e2e" = true ]; then $gam info group $newgroup; fi
- if [ "$e2e" = true ]; then $gam user $gam_user check serviceaccount; fi
- if [ "$e2e" = true ]; then $gam user $newuser imap on; fi
- if [ "$e2e" = true ]; then $gam user $newuser show imap; fi
- if [ "$e2e" = true ]; then $gam csv sample.csv gam user $newuser delegate to ~email; fi
- if [ "$e2e" = true ]; then $gam user $newuser show delegates; fi
- if [ "$e2e" = true ]; then $gam user $newuser label "✔ unicode checkmark ✔"; fi
- if [ "$e2e" = true ]; then $gam user $newuser show labels; fi
- if [ "$e2e" = true ]; then $gam user $newuser show labels > labels.txt; fi
- if [ "$e2e" = true ]; then $gam user $gam_user importemail subject "Travis import $newbase" message "This is a test import" labels IMPORTANT,UNREAD,INBOX,STARRED; fi
- if [ "$e2e" = true ]; then $gam user $gam_user insertemail subject "Travis insert $newbase" file gam.py labels INBOX,UNREAD; fi # yep body is gam code
- if [ "$e2e" = true ]; then $gam user $gam_user sendemail subject "Travis send $gam_user $newbase" file gam.py recipient admin@pdl.jaylee.us; fi
- if [ "$e2e" = true ]; then $gam user $gam_user draftemail subject "Travis draft $newbase" message "Draft message test"; fi
- if [ "$e2e" = true ]; then $gam users "$gam_user $newbase-bulkuser-01 $newbase-bulkuser-02 $newbase-bulkuser-03" delete messages query in:anywhere maxtodelete 99999 doit; fi
- if [ "$e2e" = true ]; then $gam users "$newbase-bulkuser-04 $newbase-bulkuser-05 $newbase-bulkuser-06" trash messages query in:anywhere maxtotrash 99999 doit; fi
- if [ "$e2e" = true ]; then $gam users "$newbase-bulkuser-07 $newbase-bulkuser-08 $newbase-bulkuser-09" modify messages query in:anywhere maxtomodify 99999 addlabel IMPORTANT addlabel STARRED doit; fi
- if [ "$e2e" = true ]; then $gam user $newuser delete label --ALL_LABELS--; fi
- if [ "$e2e" = true ]; then $gam create feature name Whiteboard-$newbase; fi
- if [ "$e2e" = true ]; then $gam create feature name VC-$newbase; fi
- if [ "$e2e" = true ]; then $gam create building "My Building - $newbase" id $newbuilding floors 1,2,3,4,5,6,7,8,9,10,11,12,14,15 description "No 13th floor here..."; fi
- if [ "$e2e" = true ]; then $gam create resource $newresource "Resource Calendar $tstamp" capacity 25 features Whiteboard-$newbase,VC-$newbase building $newbuilding floor 15 type Room; fi
- if [ "$e2e" = true ]; then $gam info resource $newresource; fi
- if [ "$e2e" = true ]; then $gam user $newuser show filelist; fi
- if [ "$e2e" = true ]; then $gam calendar $gam_user printacl | $gam csv - gam calendar $gam_user delete id ~id; fi # clear ACLs
- if [ "$e2e" = true ]; then $gam calendar $gam_user update read domain; fi
- if [ "$e2e" = true ]; then $gam calendar $gam_user update freebusy default; fi
- if [ "$e2e" = true ]; then $gam calendar $gam_user add editor $newuser; fi
- if [ "$e2e" = true ]; then $gam calendar $gam_user showacl; fi
- if [ "$e2e" = true ]; then $gam calendar $gam_user printacl | $gam csv - gam calendar $gam_user delete id ~id; fi
- if [ "$e2e" = true ]; then $gam printer register; fi
- if [ "$e2e" = true ]; then source travis/set_printer_csv_filter.sh; fi
- if [ "$e2e" = true ]; then $gam print printers > printers.csv; fi
- if [ "$e2e" = true ]; then unset GAM_CSV_ROW_FILTER; fi
- if [ "$e2e" = true ]; then $gam csv printers.csv gam printer ~id add USER $newgroup; fi
- if [ "$e2e" = true ]; then $gam csv printers.csv gam printjob ~id submit https://www.github.com/jay0lee/GAM; fi
- if [ "$e2e" = true ]; then $gam csv printers.csv gam info printer ~id; fi
- if [ "$e2e" = true ]; then $gam print printjobs; fi
- if [ "$e2e" = true ]; then $gam csv printers.csv gam printjob ~id fetch; fi
- if [ "$e2e" = true ]; then $gam print printjobs | $gam csv - gam printjob ~id delete; fi
- if [ "$e2e" = true ]; then $gam csv printers.csv gam delete printer ~id; fi
- if [ "$e2e" = true ]; then $gam csv sample.csv gam user ~email add calendar id:$newresource; fi
- if [ "$e2e" = true ]; then $gam delete resource $newresource; fi
- if [ "$e2e" = true ]; then $gam delete feature Whiteboard-$newbase; fi
- if [ "$e2e" = true ]; then $gam delete feature VC-$newbase; fi
- if [ "$e2e" = true ]; then $gam delete building $newbuilding; fi
- if [ "$e2e" = true ]; then $gam delete group $newgroup; fi
- if [ "$e2e" = true ]; then $gam create alias $newalias user $newuser; fi
- if [ "$e2e" = true ]; then $gam whatis $newuser; fi
- if [ "$e2e" = true ]; then $gam user $gam_user show tokens; fi
- if [ "$e2e" = true ]; then $gam delete user $newuser; fi
- if [ "$e2e" = true ]; then $gam print users query "travis.jid=$jid" | $gam csv - gam delete user ~primaryEmail; fi
- if [ "$e2e" = true ]; then $gam print mobile; fi
- if [ "$e2e" = true ]; then $gam print cros allfields nolists; fi
- if [ "$TRAVIS_OS_NAME" != "windows" ]; then bash <(curl -s -S -L https://git.io/install-gam) -l; fi
before_deploy:
- export TRAVIS_TAG="preview"
- unset LD_LIBRARY_PATH
deploy:
provider: releases
@@ -55,3 +312,4 @@ deploy:
all_branches: true
on:
repo: jay0lee/GAM
condition: $VMTYPE = build

View File

@@ -74,6 +74,13 @@ If an item contains spaces, it should be surrounded by ".
Google-Drive-storage|
Google-Vault|
101001|101005|101031
<ProductID> ::=
Google-Apps|
Google-Chrome-Device-Management|
Google-Coordinate|
Google-Drive-storage|
Google-Vault|
101001|101005|101006|101031|101033|101034
<SKUID> ::=
cloudidentity|identity|1010010001|
cloudidentitypremium|identitypremium|1010050001|
@@ -81,11 +88,13 @@ If an item contains spaces, it should be surrounded by ".
gafb|gafw|basic|gsuitebasic|Google-Apps-For-Business|
gafg|gsuitegovernment|gsuitegov|Google-Apps-For-Government|
gams|postini|gsuitegams|gsuitepostini|gsuitemessagesecurity|Google-Apps-For-Postini|
gal|lite|gsuitelite|Google-Apps-Lite|
gau|unlimited|gsuitebusiness|Google-Apps-Unlimited|
gae|enterprise|gsuiteenterprise|1010020020|
gal|gsl|lite|gsuitelite|Google-Apps-Lite|
gau|gsb|unlimited|gsuitebusiness|Google-Apps-Unlimited|
gae|gse|enterprise|gsuiteenterprise|1010020020|
gsefe|e4e|gsuiteenterpriseeducation|1010310002|
gsefes|e4es|gsuiteenterpriseeducationstudent|1010310003|
gsbau|businessarchived|gsuitebusinessarchived|
gseau|enterprisearchived|gsuiteenterprisearchived|
chrome|cdm|googlechromedevicemanagement|Google-Chrome-Device-Management|
coordinate|googlecoordinate|Google-Coordinate|
d4e|driveenterprise|drive4enterprise|
@@ -118,7 +127,7 @@ If an item contains spaces, it should be surrounded by ".
<MilliSeconds> ::= <Digit><Digit><Digit>
<Date> ::=
<Year>-<Month>-<Day> |
(+|-)<Number>(d|w)
(+|-)<Number>(d|w|y)
<Time> ::=
<Year>-<Month>-<Day>(<Space>|T)<Hour>:<Minute>:<Second>[.<MilliSeconds>](Z|(+|-(<Hour>:<Minute>))) |
(+|-)<Number>(m|h|d|w)
@@ -131,9 +140,11 @@ If an item contains spaces, it should be surrounded by ".
<AccessToken> ::= <String>
<ACLScope> ::= [user:]<EmailAddress>|group:<EmailAddress>|domain[:<DomainName>]|default
<APIScopeURL> ::= <String>
<ASPID> ::= <String>
<BuildingID> ::= <String>|id:<String>
<CalendarACLRole> ::= editor|freebusy|freebusyreader|owner|reader|writer
<CalendarACLRuleID> ::= user:<EmailAddress>|group:<EmailAddress>|domain:<DomainName>|default
<CalendarColorIndex> ::= <Number in range 1-24>
<CalendarItem> ::= <EmailAddress>|<String>
<ChatRoom> ::= <String>
@@ -211,6 +222,7 @@ If an item contains spaces, it should be surrounded by ".
<SchemaName> ::= <String>
<Section> ::= <String>
<SerialNumber> ::= <String>
<ServiceAccountKey> ::= <String>
<S/MIMEID> ::= <String>
<SMTPHostName> ::= <String>
<StudentItem> ::= <EmailAddress>|<UniqueID>|<String>
@@ -247,16 +259,20 @@ If an item contains spaces, it should be surrounded by ".
annotatedassetid|assedid|asset|
annotatedlocation|location|
annotateduser|user|
autoupdateexpiration|
bootmode|
cpustatusreports|
devicefiles|
deviceid|
diskvolumereports|
dockmacaddress|
ethernetmacaddress|
ethernetmacaddress0|
firmwareversion|
lastenrollmenttime|
lastsync|
macaddress|
manufacturedate|
meid|
model|
notes|
@@ -289,6 +305,7 @@ If an item contains spaces, it should be surrounded by ".
cancomment|
canreadrevisions|
copyable|
copyrequireswriterpermission|
createddate|createdtime|
description|
editable|
@@ -511,6 +528,8 @@ If an item contains spaces, it should be surrounded by ".
phones|phone|
posixaccounts|posix|
primaryemail|username|
recoveryemail|
recoveryphone|
relations|relation|
ssh|sshkeys|sshpublickeys|
suspended|
@@ -535,6 +554,7 @@ Items, separated by spaces, with spaces, commas or single quotes in the items th
"'it em' 'it,em' \"it'em\""
<ACLList> ::= "<ACLScope>(,<ACLScope>)*"
<APIScopeURLList> ::= "<APIScopeURL>(,<APIScopeURL>)*"
<ASPIDList> ::= "<ASPID>(,<ASPID>)*"
<CalendarList> ::= "<CalendarItem>(,<CalendarItem>)*"
<ChatRoomList> ::= "<ChatRoom>(,<ChatRoom>)*"
@@ -573,6 +593,7 @@ Items, separated by spaces, with spaces, commas or single quotes in the items th
<SKUIDList> ="<SKUID>(,<SKUID>)*"
<SchemaNameList> ::= "<SchemaName>(,<SchemaName>)*"
<SerialNumberList> ::= "<SerialNumber>(,<SerialNumber>)*"
<ServiceAccountKeyList> ::= "<ServiceAccountKey>(,<ServiceAccountKey>)*"
<TeamDriveIDList> ::= "<TeamDriveID>(,<TeamDriveID>)*"
<UserFieldNameList> ::= "<UserFieldName>(,<UserFieldName>)*"
<UserList> ::= "<UserItem>(,<UserItem>)*"
@@ -654,24 +675,18 @@ Specify a collection of Users by directly specifying them or by specifiying item
<DriveFileAddAttributes> ::=
(localfile <FileName>)|
(convert)|(ocr)|(ocrlanguage <Language>)|(restricted|restrict)|(starred|star)|(trashed|trash)|(viewed|view)|
(convert)|(ocr)|(ocrlanguage <Language>)|
(restricted|restrict)|(starred|star)|(trashed|trash)|(viewed|view)|
copyrequireswriterpermission|
(lastviewedbyme <Time>)|(modifieddate|modifiedtime <Time>)|(description <String>)|(mimetype <MimeType>)|
(parentid <DriveFolderID>)|(parentname <DriveFolderName>)|(anyownerparentname <DriveFolderName>)|writerscantshare
<DriveFileUpdateAttributes> ::=
(localfile <FileName>)|
(convert)|(ocr)|(ocrlanguage <Language>)|(restricted|restrict <Boolean>)|(starred|star <Boolean>)|(trashed|trash <Boolean>)|(viewed|view <Boolean>)|
(convert)|(ocr)|(ocrlanguage <Language>)|
(restricted|restrict <Boolean>)|(starred|star <Boolean>)|(trashed|trash <Boolean>)|(viewed|view <Boolean>)|
(copyrequireswriterpermission <Boolean>)|
(lastviewedbyme <Time>)|(modifieddate <Time>)|(description <String>)|(mimetype <MimeType>)|
(parentid <DriveFolderID>)|(parentname <DriveFolderName>)|(anyownerparentname <DriveFolderName>)|writerscantshare
<EventAttributes> ::=
(anyonecanaddself)|(guestscantinviteothers)|(guestscantseeothers)|(notifyattendees)|(available)|(visibility default|public|prvate)|(tentative)|
(attendee <EmailAddress>)|(optionalattendee <EmailAddress>)|
(description <String>)|(summary <String>)|(location <String>)|(id <String>)|
(source <String> <URL>)|(privateproperty <PropertyKey> <PropertyValue>)|(sharedproperty <PropertyKey> <PropertyValue>)|
(recurrence <RRULE, EXRULE, RDATE and EXDATE line>)|
(start allday <Date>)|(start <Time>)|(end allday <Date>)|(end <Time>)|(timezone <Timezone>)|
(noreminders|(reminder <Number> email|popup|sms))|
(colorindex|colorid <EventColorIndex>)
<GroupSettingsAttribute> ::=
(allowexternalmembers <Boolean>)|
(allowwebposting <Boolean>)|
@@ -776,6 +791,8 @@ Specify a collection of Users by directly specifying them or by specifiying item
(note clear|([text_html|text_plain] <String>|(file <FileName> [charset <Charset>])))|
(org|ou|orgunitpath <OrgUnitPath>)
(password random|<Password>)|
(recoveryemail <EmailAddress>)|
(recoveryphone <string>)|
(suspended <Boolean>)|
(<SchemaName>.<FieldName> [multivalued|multivalue|value|multinonempty [type home|other|work|(custom <String>)]] <String>)
<UserMultiAttributes> ::=
@@ -799,7 +816,7 @@ Specify a collection of Users by directly specifying them or by specifiying item
<UserBasicAttribute>|
<UserMultiAttribute>
gam version [check] [simple]
gam version [check|checkrc|simple|extended] [timeoffset] [location <HostName>]
gam help
gam batch <FileName>|- [charset <Charset>]
@@ -813,17 +830,25 @@ Example: gam csv Users.csv gam update user "~primaryEmail" address type work uns
Each user (~primaryEmail, e.g. foo@bar.com) would have their work address updated
gam create project [<EmailAddress>] [<ProjectID>]
gam create project [admin <EmailAddress>] [project <ProjectID>] [parent <String>]
gam use project [<EmailAddress>] [<ProjectID>]
gam use project [admin <EmailAddress>] [project <ProjectID>]
gam update project [<EmailAddress>] [gam|<ProjectID>|(filter <String>)]
gam delete project [<EmailAddress>] [gam|<ProjectID>|(filter <String>)]
gam show projects [<EmailAddress>] [all|gam|<ProjectID>|(filter <String>)]
gam print projects [<EmailAddress>] [all|gam|<ProjectID>|(filter <String>)] [todrive]
gam rotate sakey|sakeys [retain_none|retain_existing|replace_current]
[(algorithm KEY_ALG_RSA_1024|KEY_ALG_RSA_2048)|(localkeysize 1024|2048|4096)]
gam delete sakey|sakeys <ServiceAccountKeyList>+ [doit]
gam show sakey|sakeys [all|system|user]
gam oauth|oauth2 create|request [<EmailAddress>]
gam oauth|oauth2 delete|revoke
gam oauth|oauth2 info|verify [<AccessToken>]
gam oauth|oauth2 info|verify [accesstoken <AccessToken>] [idtoken <IDToken>] [showsecret]
gam oauth|oauth2 refresh
gam <UserTypeEntity> check serviceaccount
gam <UserTypeEntity> check serviceaccount [scope|scopes <APIScopeURLList>]
gam whatis <EmailItem>
@@ -871,7 +896,7 @@ gam info resoldsubscriptions <CustomerID> [customer_auth_token <String>]
<ReportsAppList> ::= "<ReportsApp>(,<ReportsApp>)*"
gam report users|user [todrive] [date <Date>] [fulldatarequired all|<ReportsAppList>]
[(user all|<UserItem>)|(orgunit|org|ou <OrgUnitPath>)] [filter|filters <String>] [fields|parameters <String>]
[(user <UserItem>)|(orgunit|org|ou <OrgUnitPath>)] [filter|filters <String>] [fields|parameters <String>]
gam report customers|customer|domain [todrive] [date <Date>] [fulldatarequired all|<ReportsAppList>]
[fields|parameters <String>]
gam report admin|calendar|calendars|drive|docs|doc|groups|group|logins|login|mobile|tokens|token [todrive]
@@ -939,11 +964,57 @@ gam print aliases|nicknames [todrive] [shownoneditable] [nogroups] [nousers] [(q
gam calendar <CalendarItem> add <CalendarACLRole> ([user] <EmailAddress>)|(group <EmailAddress>)|(domain [<DomainName>])|default [sendnotifications <Boolean>]
gam calendar <CalendarItem> update <CalendarACLRole> ([user] <EmailAddress>)|(group <EmailAddress>)|(domain [<DomainName>])|default [sendnotifications <Boolean>]
gam calendar <CalendarItem> del|delete <CalendarACLRole> <EmailAddress>|(domain [<DomainName>])|default
gam calendar <CalendarItem> del|delete id <CalendarACLRuleID>
gam calendar <CalendarItem> showacl
gam calendar <CalendarItem> printacl [todrive]
gam calendar <CalendarItem> addevent <EventAttributes>+
gam calendar <CalendarItem> deleteevent (id|eventid <EventID>)* (query|eventquery <QueryCalendar>)* [doit] [notifyattendees]
<EventNotificationAttribute> ::=
notifyattendees|(sendnotifications <Boolean>)|(sendupdates all|enternalonly|none)
The following attributes are equivalent:
notifyattendees - sendupdates all
sendnotifications false - sendupdates none
sendnotifications true - sendupdates all
<EventAttributes> ::=
anyonecanaddself|
(attendee <EmailAddress>)|
available|
(colorindex|colorid <EventColorIndex>)
(description <String>)|
(end (allday <Date>)|<Time>)|
guestscantinviteothers|
guestscantseeothers|
(id <String>)|
(location <String>)|
(noreminders| (reminder <Number> email|popup|sms))|
(optionalattendee <EmailAddress>)|
(privateproperty <PropertyKey> <PropertyValue>)|
(recurrence <RRULE, EXRULE, RDATE and EXDATE line>)|
(sharedproperty <PropertyKey> <PropertyValue>)|
(source <String> <URL>)|
(start (allday <Date>)|<Time>)|
(summary <String>)|
tentative|
(timezone <Timezone>)|
(visibility default|public|prvate)
<EventSelectProperty:> ::=
(after <Time>)|
(before <Time>)|
includeeleted|
includehidden|
(query <QueryCalendar>)|
(updatedmin <Time>)
<EventDisplayProperty> ::=
(timezone <TimeZone>)
gam calendar <CalendarItem> addevent <EventAttributes>+ [<EventNotificationAttribute>]
gam calendar <CalendarItem> deleteevent id|eventid <EventID> [doit] [<EventNotificationAttribute>]
gam calendar <CalendarItem> moveevent id|eventid <EventID> [doit] [<EventNotificationAttribute>]
gam calendar <CalendarItem> wipe
gam calendar <CalendarItem> printevents <EventSelectProperty>* <EventDisplayProperty>* [todrive]
<CalendarSettings> ::=
summary <String>|
@@ -954,11 +1025,11 @@ gam calendar <CalendarItem> wipe
gam calendar <CalendarItem> modify <CalendarSettings>+
gam update cros <CrOSEntity> (<CrOSAttributes>+)|(action deprovision_same_model_replace|deprovision_different_model_replace|deprovision_retiring_device|disable|reenable [acknowledge_device_touch_requirement])
gam info cros <CrOSEntity> [nolists] [listlimit <Number>] [start <Date>] [end <Date>]
gam info cros <CrOSEntity> [guessaue] [nolists] [listlimit <Number>] [start <Date>] [end <Date>]
[basic|full|allfields] <CrOSFieldName>* [fields <CrOSFieldNameList>] [downloadfile latest|<Time>] [targetfolder <FilePath>]
gam print cros [todrive] [(query <QueryCrOS>)|(queries <QueryCrOSList>)] [limittoou <OrgUnitItem>]
[orderby <CrOSOrderByFieldName> [ascending|descending]] [nolists|<CrOSListFieldName>*] [listlimit <Number>] [start <Date>] [end <Date>]
[orderby <CrOSOrderByFieldName> [ascending|descending]] [guessaue] [nolists|<CrOSListFieldName>*] [listlimit <Number>] [start <Date>] [end <Date>]
[basic|full|allfields] <CrOSFieldName>* [fields <CrOSFieldNameList>] [sortheaders]
gam <CrOSTypeEntity> print
@@ -1001,7 +1072,7 @@ The listlimit <Number> argument limits the number of recent users, time ranges a
The start <Date> and end <Date> arguments filter the time ranges.
Delimiter defaults to comma.
gam update mobile <MobileID> action <MobileAction>
gam update mobile <MobileID>|query:<QueryMobile> action <MobileAction> [doit] [if_users|match_users <UserTypeEntity>]
gam delete mobile <MobileID>
gam info mobile <MobileID>
gam print mobile [todrive] [(query <QueryMobile>)|(queries <QueryMobileList>)] [basic|full] [orderby <MobileOrderByFieldName> [ascending|descending]]
@@ -1026,6 +1097,7 @@ gam info member <UserItem> <GroupItem>
gam print group-members|groups-members [todrive]
([domain <DomainName>] ([member <UserItem>]|[query <QueryGroup>]))|[group|group_ns|group_susp <GroupItem>] [notsuspended|suspended]
[roles <GroupRoleList>] [membernames] [fields <MembersFieldNameList>]
[includederivedmembership]
gam print licenses [todrive] [(products|product <ProductIDList>)|(skus|sku <SKUIDList>)|allskus|gsuite] [countsonly]
gam show license|licenses|licence|licences [(products|product <ProductIDList>)|(skus|sku <SKUIDList>)|allskus|gsuite]
@@ -1045,7 +1117,7 @@ gam create resource <ResourceID> <Name> <ResourceAttributes>*
gam update resource <ResourceID> <ResourceAttributes>*
gam delete resource <ResourceID>
gam info resource <ResourceID>
gam print resources [todrive] [allfields] <ResourceFieldName>*
gam print resources [todrive] [allfields] <ResourceFieldName>* [query <String>]
gam create schema|schemas <SchemaName> <SchemaFieldDefinition>+
gam update schema <SchemaName> <SchemaFieldDefinition>* (deletefield <FieldName>)*
@@ -1126,9 +1198,11 @@ gam print printjobs [todrive] [printer|printerid <PrinterID>]
gam create vaultexport|export matter <MatterItem> [name <name>] corpus <drive|mail|groups|hangouts_chat>
(accounts <EmailAddressList>) | (orgunit|ou <OrgUnitPath>) | (teamdrives <TeamDriveList>) | (rooms <ChatRoomList>) | everyone
[scope <all_data|held_data|unprocessed_data>]
[terms <terms>] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>] [timezone <TimeZone>] [format mbox|pst]
[excludedrafts <Boolean>] [driveversiondate <Date>|<Time>] [includeteamdrives] [includerooms]
[includeaccessinfo <Boolean>]
[terms <terms>] [start|starttime <Date>|<Time>] [end|endtime <Date>|<Time>] [timezone <TimeZone>]
[excludedrafts <Boolean>] [format mbox|pst] [showconfidentialmodecontent <Boolean>]
[includerooms]
[driveversiondate <Date>|<Time>] [includeshareddrives|includeteamdrives] [includeaccessinfo <Boolean>]
[region any|europe|us]
gam delete export <MatterItem> <ExportItem>
gam info export <MatterItem> <ExportItem>
gam print exports [todrive] [matters <MatterItemList>]
@@ -1239,6 +1313,21 @@ gam <UserTypeEntity> untrash messages query <QueryGmail> [doit] [max_to_untrash|
gam <UserTypeEntity> show gmailprofile [todrive]
gam <UserTypeEntity> draftemail [recipient|to <EmailAddress>] [from <EmailAddress>]
[subject <String>] [(message <String>)|(file <FileName> [charset <Charset>])]
gam <UserTypeEntity> importemail [recipient|to <EmailAddress>] [from <EmailAddress>]
[subject <String>] [(message <String>)|(file <FileName> [charset <Charset>])]
[labels <LabelNameList>] (header <String> <String>)*
[deleted] [date <Time>]
[nevercheckspam] [processforcalendar]
gam <UserTypeEntity> insertemail [recipient|to <EmailAddress>] [from <EmailAddress>]
[subject <String>] [(message <String>)|(file <FileName> [charset <Charset>])]
[labels <LabelNameList>] (header <String> <String>)*
[deleted] [date <Time>]
gam <UserTypeEntity> sendemail [recipient|to <EmailAddress>] [from <EmailAddress>]
[subject <String>] [(message <String>)|(file <FileName> [charset <Charset>])]
(header <String> <String>)*
gam <UserTypeEntity> create|add delegate|delegates <EmailAddress>
gam <UserTypeEntity> delegate|delegates to <EmailAddress>
gam <UserTypeEntity> delete|del delegate|delegates <EmailAddress>
@@ -1269,6 +1358,9 @@ gam <UserTypeEntity> show imap|imap4
gam <UserTypeEntity> pop|pop3 <Boolean> [for allmail|newmail|mailfromnowon|fromnowown] [action keep|leaveininbox|archive|delete|trash|markread]
gam <UserTypeEntity> show pop|pop3
gam <UserTypeEntity> language <Language>
gam <UserTypeEntity> show language
gam <UserTypeEntity> [create|add] sendas <EmailAddress> <Name> [signature|sig <String>|(file <FileName> [charset <Charset>]) (replace <Tag> <String>)*] [html] [replyto <EmailAddress>] [default] [treatasalias <Boolean>]
[smtpmsa.host <SMTPHostName> smtpmsa.port 25|465|587 smtpmsa.username <UserName> smtpmsa.password <Password> [smtpmsa.securitymode none|ssl|starttls]]
gam <UserTypeEntity> update sendas <EmailAddress> [name <Name>] [signature|sig <String>|(file <FileName> [charset <Charset>]) (replace <Tag> <String>)*] [html] [replyto <EmailAddress>] [default] [treatasalias <Boolean>]

File diff suppressed because it is too large Load Diff

66
src/controlflow.py Normal file
View File

@@ -0,0 +1,66 @@
"""Methods related to the central control flow of an application."""
import random
import sys
import time
import display # TODO: Change to relative import when gam is setup as a package
from var import MESSAGE_HEADER_NOT_FOUND_IN_CSV_HEADERS
from var import MESSAGE_INVALID_JSON
def system_error_exit(return_code, message):
"""Raises a system exit with the given return code and message.
Args:
return_code: Int, the return code to yield when the system exits.
message: An error message to print before the system exits.
"""
if message:
display.print_error(message)
sys.exit(return_code)
def csv_field_error_exit(field_name, field_names):
"""Raises a system exit when a CSV field is malformed.
Args:
field_name: The CSV field name for which a header does not exist in the
existing CSV headers.
field_names: The known list of CSV headers.
"""
system_error_exit(
2,
MESSAGE_HEADER_NOT_FOUND_IN_CSV_HEADERS.format(field_name,
','.join(field_names)))
def invalid_json_exit(file_name):
"""Raises a sysyem exit when invalid JSON content is encountered."""
system_error_exit(17, MESSAGE_INVALID_JSON.format(file_name))
def wait_on_failure(current_attempt_num,
total_num_retries,
error_message,
error_print_threshold=3):
"""Executes an exponential backoff-style system sleep.
Args:
current_attempt_num: Int, the current number of retries.
total_num_retries: Int, the total number of times the current action will be
retried.
error_message: String, a message to be displayed that will give more context
around why the action is being retried.
error_print_threshold: Int, the number of attempts which will have their
error messages suppressed. Any current_attempt_num greater than
error_print_threshold will print the prescribed error.
"""
wait_on_fail = min(2**current_attempt_num,
60) + float(random.randint(1, 1000)) / 1000
if current_attempt_num > error_print_threshold:
sys.stderr.write(
'Temporary error: {0}, Backing off: {1} seconds, Retry: {2}/{3}\n'
.format(error_message, int(wait_on_fail), current_attempt_num,
total_num_retries))
sys.stderr.flush()
time.sleep(wait_on_fail)

108
src/controlflow_test.py Normal file
View File

@@ -0,0 +1,108 @@
"""Tests for controlflow."""
import unittest
from unittest.mock import patch
import controlflow
class ControlFlowTest(unittest.TestCase):
def test_system_error_exit_raises_systemexit_error(self):
with self.assertRaises(SystemExit):
controlflow.system_error_exit(1, 'exit message')
def test_system_error_exit_raises_systemexit_with_return_code(self):
with self.assertRaises(SystemExit) as context_manager:
controlflow.system_error_exit(100, 'exit message')
self.assertEqual(context_manager.exception.code, 100)
@patch.object(controlflow.display, 'print_error')
def test_system_error_exit_prints_error_before_exiting(self, mock_print_err):
with self.assertRaises(SystemExit):
controlflow.system_error_exit(100, 'exit message')
self.assertIn('exit message', mock_print_err.call_args[0][0])
def test_csv_field_error_exit_raises_systemexit_error(self):
with self.assertRaises(SystemExit):
controlflow.csv_field_error_exit('aField',
['unusedField1', 'unusedField2'])
def test_csv_field_error_exit_exits_code_2(self):
with self.assertRaises(SystemExit) as context_manager:
controlflow.csv_field_error_exit('aField',
['unusedField1', 'unusedField2'])
self.assertEqual(context_manager.exception.code, 2)
@patch.object(controlflow.display, 'print_error')
def test_csv_field_error_exit_prints_error_details(self, mock_print_err):
with self.assertRaises(SystemExit):
controlflow.csv_field_error_exit('aField',
['unusedField1', 'unusedField2'])
printed_message = mock_print_err.call_args[0][0]
self.assertIn('aField', printed_message)
self.assertIn('unusedField1', printed_message)
self.assertIn('unusedField2', printed_message)
def test_invalid_json_exit_raises_systemexit_error(self):
with self.assertRaises(SystemExit):
controlflow.invalid_json_exit('filename')
def test_invalid_json_exit_exit_exits_code_17(self):
with self.assertRaises(SystemExit) as context_manager:
controlflow.invalid_json_exit('filename')
self.assertEqual(context_manager.exception.code, 17)
@patch.object(controlflow.display, 'print_error')
def test_invalid_json_exit_prints_error_details(self, mock_print_err):
with self.assertRaises(SystemExit):
controlflow.invalid_json_exit('filename')
printed_message = mock_print_err.call_args[0][0]
self.assertIn('filename', printed_message)
@patch.object(controlflow.time, 'sleep')
def test_wait_on_failure_waits_exponentially(self, mock_sleep):
controlflow.wait_on_failure(1, 5, 'Backoff attempt #1')
controlflow.wait_on_failure(2, 5, 'Backoff attempt #2')
controlflow.wait_on_failure(3, 5, 'Backoff attempt #3')
sleep_calls = mock_sleep.call_args_list
self.assertGreaterEqual(sleep_calls[0][0][0], 2**1)
self.assertGreaterEqual(sleep_calls[1][0][0], 2**2)
self.assertGreaterEqual(sleep_calls[2][0][0], 2**3)
@patch.object(controlflow.time, 'sleep')
def test_wait_on_failure_does_not_exceed_60_secs_wait(self, mock_sleep):
total_attempts = 20
for attempt in range(1, total_attempts + 1):
controlflow.wait_on_failure(
attempt,
total_attempts,
'Attempt #%s' % attempt,
# Suppress messages while we make a lot of attempts.
error_print_threshold=total_attempts + 1)
# Wait time may be between 60 and 61 secs, due to rand addition.
self.assertLessEqual(mock_sleep.call_args[0][0], 61)
# Prevent the system from actually sleeping and thus slowing down the test.
@patch.object(controlflow.time, 'sleep')
@patch.object(controlflow.sys.stderr, 'write')
def test_wait_on_failure_prints_errors(self, mock_stderr_write,
unused_mock_sleep):
message = 'An error message to display'
controlflow.wait_on_failure(1, 5, message, error_print_threshold=0)
self.assertIn(message, mock_stderr_write.call_args[0][0])
@patch.object(controlflow.time, 'sleep')
@patch.object(controlflow.sys.stderr, 'write')
def test_wait_on_failure_only_prints_after_threshold(self, mock_stderr_write,
unused_mock_sleep):
total_attempts = 5
threshold = 3
for attempt in range(1, total_attempts + 1):
controlflow.wait_on_failure(
attempt,
total_attempts,
'Attempt #%s' % attempt,
error_print_threshold=threshold)
self.assertEqual(total_attempts - threshold, mock_stderr_write.call_count)

256
src/cros-aue-dates.json Normal file
View File

@@ -0,0 +1,256 @@
{
"acer ac700": "2016-08-01T00:00:00.000Z",
"acer c7 chromebook": "2017-10-01T00:00:00.000Z",
"acer c7 chromebook (c710)": "2017-10-01T00:00:00.000Z",
"acer c720 chromebook": "2019-06-01T00:00:00.000Z",
"acer c740 chromebook": "2019-06-01T00:00:00.000Z",
"acer chromebase": "2020-08-01T00:00:00.000Z",
"acer chromebase 24": "2021-06-01T00:00:00.000Z",
"acer chromebook 11 (c720, c720p)": "2019-06-01T00:00:00.000Z",
"acer chromebook 11 (c732, c732t, c732l, c732lt)": "2023-11-01T00:00:00.000Z",
"acer chromebook 11 (c740)": "2020-06-01T00:00:00.000Z",
"acer chromebook 11 (c771, c771t)": "2022-11-01T00:00:00.000Z",
"acer chromebook 11 (cb3-111, c730, c730e)": "2019-08-01T00:00:00.000Z",
"acer chromebook 11 (cb3-131, c735)": "2021-01-01T00:00:00.000Z",
"acer chromebook 11 (cb311-8h, cb311-8ht)": "2023-11-01T00:00:00.000Z",
"acer chromebook 11 n7 (c731, c731t)": "2022-01-01T00:00:00.000Z",
"acer chromebook 13 (cb5-311)": "2019-09-01T00:00:00.000Z",
"acer chromebook 13 (cb713-1w)": "2024-06-01T00:00:00.000Z",
"acer chromebook 13(cb5-311, c810)": "2019-09-01T00:00:00.000Z",
"acer chromebook 14 (cb3-431)": "2021-06-01T00:00:00.000Z",
"acer chromebook 14 for work (cp5-471)": "2022-11-01T00:00:00.000Z",
"acer chromebook 15 (c910 / cb5-571)": "2020-06-01T00:00:00.000Z",
"acer chromebook 15 (cb3-531)": "2020-06-01T00:00:00.000Z",
"acer chromebook 15 (cb3-532)": "2021-08-01T00:00:00.000Z",
"acer chromebook 15 (cb315-1h,cb315-1ht)": "2023-11-01T00:00:00.000Z",
"acer chromebook 15 (cb5-571, c910)": "2020-06-01T00:00:00.000Z",
"acer chromebook 15 (cb515-1h,cb515-1ht)": "2023-11-01T00:00:00.000Z",
"acer chromebook 311": "2025-06-01T00:00:00.000Z",
"acer chromebook 311 (c721, c733, c733u, c733t)": "2025-06-01T00:00:00.000Z",
"acer chromebook 315": "2025-06-01T00:00:00.000Z",
"acer chromebook 315 (cb315-2h)": "2025-06-01T00:00:00.000Z",
"acer chromebook 512 (c851, c851t)": "2025-06-01T00:00:00.000Z",
"acer chromebook 514": "2023-11-01T00:00:00.000Z",
"acer chromebook 714 (cb714-1w / cb714-1wt)": "2024-06-01T00:00:00.000Z",
"acer chromebook 715 (cb715-1w / cb715-1wt)": "2024-06-01T00:00:00.000Z",
"acer chromebook r11 (cb5-132t, c738t)": "2021-06-01T00:00:00.000Z",
"acer chromebook r13 (cb5-312t)": "2021-09-01T00:00:00.000Z",
"acer chromebook spin 11 (cp311-h1, cp311-1hn)": "2023-11-01T00:00:00.000Z",
"acer chromebook spin 11 (r751t)": "2023-11-01T00:00:00.000Z",
"acer chromebook spin 13 (cp713-1wn)": "2024-06-01T00:00:00.000Z",
"acer chromebook spin 15 (cp315)": "2023-11-01T00:00:00.000Z",
"acer chromebook spin 311 (r721t)": "2025-06-01T00:00:00.000Z",
"acer chromebook spin 511": "2025-06-01T00:00:00.000Z",
"acer chromebook spin 511 (r752t, r752tn)": "2025-06-01T00:00:00.000Z",
"acer chromebook spin 512 (r851tn)": "2025-06-01T00:00:00.000Z",
"acer chromebook tab 10": "2023-08-01T00:00:00.000Z",
"acer chromebox": "2019-09-01T00:00:00.000Z",
"acer chromebox cxi2": "2020-06-01T00:00:00.000Z",
"acer chromebox cxi2 / cxv2": "2020-06-01T00:00:00.000Z",
"acer chromebox cxi3": "2024-06-01T00:00:00.000Z",
"aopen chromebase commercial": "2020-09-01T00:00:00.000Z",
"aopen chromebase mini": "2022-02-01T00:00:00.000Z",
"aopen chromebox commercial": "2020-09-01T00:00:00.000Z",
"aopen chromebox commercial 2": "2024-06-01T00:00:00.000Z",
"aopen chromebox mini": "2022-02-01T00:00:00.000Z",
"asi chromebook": "2020-06-01T00:00:00.000Z",
"asus chromebit cs10": "2020-11-01T00:00:00.000Z",
"asus chromebook c200": "2019-06-01T00:00:00.000Z",
"asus chromebook c200ma": "2019-06-01T00:00:00.000Z",
"asus chromebook c201pa": "2020-06-01T00:00:00.000Z",
"asus chromebook c202sa": "2021-06-01T00:00:00.000Z",
"asus chromebook c204": "2025-06-01T00:00:00.000Z",
"asus chromebook c213na": "2023-11-01T00:00:00.000Z",
"asus chromebook c223": "2023-11-01T00:00:00.000Z",
"asus chromebook c300": "2019-08-01T00:00:00.000Z",
"asus chromebook c300ma": "2019-08-01T00:00:00.000Z",
"asus chromebook c300sa / c301sa": "2021-06-01T00:00:00.000Z",
"asus chromebook c403": "2023-11-01T00:00:00.000Z",
"asus chromebook c423": "2023-11-01T00:00:00.000Z",
"asus chromebook c523": "2023-11-01T00:00:00.000Z",
"asus chromebook flip c100pa": "2020-07-01T00:00:00.000Z",
"asus chromebook flip c101pa": "2023-08-01T00:00:00.000Z",
"asus chromebook flip c213": "2023-11-01T00:00:00.000Z",
"asus chromebook flip c214": "2025-06-01T00:00:00.000Z",
"asus chromebook flip c302": "2022-11-01T00:00:00.000Z",
"asus chromebook flip c434": "2024-06-01T00:00:00.000Z",
"asus chromebook tablet ct100": "2023-08-01T00:00:00.000Z",
"asus chromebox (cn60)": "2019-09-01T00:00:00.000Z",
"asus chromebox 2 (cn62)": "2021-06-01T00:00:00.000Z",
"asus chromebox 3": "2024-06-01T00:00:00.000Z",
"asus chromebox 3 (cn65)": "2024-06-01T00:00:00.000Z",
"asus chromebox cn60": "2019-09-01T00:00:00.000Z",
"asus chromebox cn62": "2021-06-01T00:00:00.000Z",
"bobicus chromebook 11": "2020-06-01T00:00:00.000Z",
"chromebook 11 (c730 / cb3-111)": "2019-08-01T00:00:00.000Z",
"chromebook 11 (c735)": "2021-01-01T00:00:00.000Z",
"chromebook 15 (cb515 - 1ht / 1h)": "2023-11-01T00:00:00.000Z",
"chromebook 311 (c721)": "2025-06-01T00:00:00.000Z",
"chromebook pcm-116e": "2020-06-01T00:00:00.000Z",
"consumer chromebook": "2020-06-01T00:00:00.000Z",
"cr-48": "2015-12-01T00:00:00.000Z",
"crambo chromebook": "2020-06-01T00:00:00.000Z",
"ctl chromebook j41 / j41t": "2023-11-01T00:00:00.000Z",
"ctl chromebook nl7": "2023-11-01T00:00:00.000Z",
"ctl chromebook nl7t-360 / nl7tw-360": "2023-11-01T00:00:00.000Z",
"ctl chromebook tab tx1": "2023-08-01T00:00:00.000Z",
"ctl chromebook tablet tx1 for education": "2023-08-01T00:00:00.000Z",
"ctl chromebox cbx1": "2024-06-01T00:00:00.000Z",
"ctl j2 / j4 chromebook": "2020-06-01T00:00:00.000Z",
"ctl j5 chromebook": "2021-08-01T00:00:00.000Z",
"ctl n6 education chromebook": "2020-06-01T00:00:00.000Z",
"ctl nl61 chromebook": "2021-08-01T00:00:00.000Z",
"dell chromebook 11": "2019-06-01T00:00:00.000Z",
"dell chromebook 11 (3120)": "2020-06-01T00:00:00.000Z",
"dell chromebook 11 (3180)": "2022-05-01T00:00:00.000Z",
"dell chromebook 11 (5190)": "2023-11-01T00:00:00.000Z",
"dell chromebook 11 2-in-1 (3189)": "2022-05-01T00:00:00.000Z",
"dell chromebook 11 2-in-1 (5190)": "2023-11-01T00:00:00.000Z",
"dell chromebook 13 (3380)": "2022-11-01T00:00:00.000Z",
"dell chromebook 13 (7310)": "2020-09-01T00:00:00.000Z",
"dell chromebook 3100": "2025-06-01T00:00:00.000Z",
"dell chromebook 3100 2-in-1": "2025-06-01T00:00:00.000Z",
"dell chromebook 3400": "2025-06-01T00:00:00.000Z",
"dell chromebox": "2019-09-01T00:00:00.000Z",
"dell inspiron chromebook 14 2-in-1 (7486)": "2024-06-01T00:00:00.000Z",
"edugear chromebook k": "2020-06-01T00:00:00.000Z",
"edugear chromebook m": "2020-06-01T00:00:00.000Z",
"edugear chromebook r": "2020-06-01T00:00:00.000Z",
"edugear cmt chromebook": "2021-08-01T00:00:00.000Z",
"edxis chromebook": "2020-06-01T00:00:00.000Z",
"edxis education chromebook": "2020-06-01T00:00:00.000Z",
"epik 11.6\" chromebook elb1101": "2020-06-01T00:00:00.000Z",
"google chromebook pixel": "2018-06-01T00:00:00.000Z",
"google chromebook pixel (2015)": "2020-06-01T00:00:00.000Z",
"google cr-48": "2015-12-01T00:00:00.000Z",
"google pixel slate": "2024-06-01T00:00:00.000Z",
"google pixelbook": "2024-06-01T00:00:00.000Z",
"haier chromebook 11": "2020-06-01T00:00:00.000Z",
"haier chromebook 11 c": "2021-08-01T00:00:00.000Z",
"haier chromebook 11 g2": "2020-09-01T00:00:00.000Z",
"haier chromebook 11e": "2020-06-01T00:00:00.000Z",
"hexa chromebook pi": "2020-06-01T00:00:00.000Z",
"hisense chromebook 11": "2020-06-01T00:00:00.000Z",
"hp chromebook 11 1100-1199 / hp chromebook 11 g1": "2018-10-01T00:00:00.000Z",
"hp chromebook 11 2000-2099 / hp chromebook 11 g2": "2019-06-01T00:00:00.000Z",
"hp chromebook 11 2100-2199 / hp chromebook 11 g3": "2020-06-01T00:00:00.000Z",
"hp chromebook 11 2200-2299 / hp chromebook 11 g4/g4 ee": "2020-06-01T00:00:00.000Z",
"hp chromebook 11 g1": "2018-10-01T00:00:00.000Z",
"hp chromebook 11 g2": "2019-06-01T00:00:00.000Z",
"hp chromebook 11 g3": "2020-06-01T00:00:00.000Z",
"hp chromebook 11 g4/g4 ee": "2020-06-01T00:00:00.000Z",
"hp chromebook 11 g5": "2021-07-01T00:00:00.000Z",
"hp chromebook 11 g5 / hp chromebook 11-vxxx": "2021-07-01T00:00:00.000Z",
"hp chromebook 11 g5 ee": "2022-01-01T00:00:00.000Z",
"hp chromebook 11 g6 ee": "2023-11-01T00:00:00.000Z",
"hp chromebook 11 g7 ee": "2025-06-01T00:00:00.000Z",
"hp chromebook 11a g6 ee": "2025-06-01T00:00:00.000Z",
"hp chromebook 13 g1": "2022-11-01T00:00:00.000Z",
"hp chromebook 14": "2019-06-01T00:00:00.000Z",
"hp chromebook 14 / hp chromebook 14 g5": "2023-11-01T00:00:00.000Z",
"hp chromebook 14 ak000-099 / hp chromebook 14 g4": "2021-09-01T00:00:00.000Z",
"hp chromebook 14 db0000-db0999": "2025-06-01T00:00:00.000Z",
"hp chromebook 14 g3": "2019-10-01T00:00:00.000Z",
"hp chromebook 14 g4": "2021-09-01T00:00:00.000Z",
"hp chromebook 14 g5": "2023-11-01T00:00:00.000Z",
"hp chromebook 14 x000-x999 / hp chromebook 14 g3": "2019-10-01T00:00:00.000Z",
"hp chromebook 14a g5": "2025-06-01T00:00:00.000Z",
"hp chromebook 15 g1": "2024-06-01T00:00:00.000Z",
"hp chromebook x2 ": "2024-06-01T00:00:00.000Z",
"hp chromebook x360 11 g1 ee": "2023-11-01T00:00:00.000Z",
"hp chromebook x360 11 g2 ee": "2025-06-01T00:00:00.000Z",
"hp chromebook x360 14": "2024-06-01T00:00:00.000Z",
"hp chromebook x360 14 g1": "2024-06-01T00:00:00.000Z",
"hp chromebox cb1-(000-099) / hp chromebox g1/ hp chromebox for meetings": "2019-09-01T00:00:00.000Z",
"hp chromebox g1": "2019-09-01T00:00:00.000Z",
"hp chromebox g2": "2024-06-01T00:00:00.000Z",
"hp pavilion chromebook 14": "2018-02-01T00:00:00.000Z",
"jp sa couto chromebook": "2020-06-01T00:00:00.000Z",
"lava xolo chromebook": "2020-06-01T00:00:00.000Z",
"lenovo 100e chromebook": "2023-11-01T00:00:00.000Z",
"lenovo 100e chromebook 2nd gen": "2025-06-01T00:00:00.000Z",
"lenovo 100e chromebook 2nd gen mtk": "2025-06-01T00:00:00.000Z",
"lenovo 100s chromebook": "2020-09-01T00:00:00.000Z",
"lenovo 14e chromebook": "2025-06-01T00:00:00.000Z",
"lenovo 300e chromebook": "2025-06-01T00:00:00.000Z",
"lenovo 300e chromebook 2nd gen": "2025-06-01T00:00:00.000Z",
"lenovo 300e chromebook 2nd gen mtk": "2025-06-01T00:00:00.000Z",
"lenovo 500e chromebook": "2023-11-01T00:00:00.000Z",
"lenovo 500e chromebook 2nd gen": "2025-06-01T00:00:00.000Z",
"lenovo chromebook c330": "2022-06-01T00:00:00.000Z",
"lenovo chromebook s330": "2022-06-01T00:00:00.000Z",
"lenovo flex 11 chromebook": "2022-06-01T00:00:00.000Z",
"lenovo ideapad c330 chromebook": "2022-06-01T00:00:00.000Z",
"lenovo ideapad s330 chromebook": "2022-06-01T00:00:00.000Z",
"lenovo n20 chromebook": "2019-06-01T00:00:00.000Z",
"lenovo n21 chromebook": "2020-06-01T00:00:00.000Z",
"lenovo n22 chromebook": "2021-06-01T00:00:00.000Z",
"lenovo n23 chromebook": "2021-06-01T00:00:00.000Z",
"lenovo n23 yoga chromebook": "2022-06-01T00:00:00.000Z",
"lenovo n42 chromebook": "2021-06-01T00:00:00.000Z",
"lenovo thinkcentre chromebox": "2020-06-01T00:00:00.000Z",
"lenovo thinkpad 11e 3rd gen chromebook": "2021-06-01T00:00:00.000Z",
"lenovo thinkpad 11e 4th gen chromebook": "2023-11-01T00:00:00.000Z",
"lenovo thinkpad 11e chromebook": "2019-06-01T00:00:00.000Z",
"lenovo thinkpad 11e chromebook (4th gen)/lenovo thinkpad yoga 11e chromebook (4th gen)": "2023-11-01T00:00:00.000Z",
"lenovo thinkpad 13": "2022-11-01T00:00:00.000Z",
"lenovo thinkpad x131e chromebook": "2018-06-01T00:00:00.000Z",
"lenovo yoga c630 chromebook": "2024-06-01T00:00:00.000Z",
"lg chromebase (22cb25s)": "2020-06-01T00:00:00.000Z",
"lg chromebase (22cv241)": "2019-06-01T00:00:00.000Z",
"lumos education chromebook": "2020-06-01T00:00:00.000Z",
"m&a chromebook": "2020-06-01T00:00:00.000Z",
"mecer chromebook": "2020-06-01T00:00:00.000Z",
"mecer v2 chromebook": "2021-08-01T00:00:00.000Z",
"medion chromebook akoya s2013 ": "2020-06-01T00:00:00.000Z",
"medion chromebook s2015": "2020-06-01T00:00:00.000Z",
"multilaser chromebook m11c": "2021-08-01T00:00:00.000Z",
"ncomputing chromebook cx100": "2020-06-01T00:00:00.000Z",
"ncomputing chromebook cx110": "2020-06-01T00:00:00.000Z",
"nexian chromebook 11.6\"": "2020-06-01T00:00:00.000Z",
"pcmerge chromebook al116": "2023-11-01T00:00:00.000Z",
"pcmerge chromebookpcm-116e/pcm-116eb": "2020-06-01T00:00:00.000Z",
"pcmerge chromebookpcm-116t-432b": "2021-08-01T00:00:00.000Z",
"poin2 chromebook 11": "2020-06-01T00:00:00.000Z",
"poin2 chromebook 11c": "2022-11-01T00:00:00.000Z",
"poin2 chromebook 14": "2022-03-01T00:00:00.000Z",
"positivo chromebook c216b": "2021-08-01T00:00:00.000Z",
"positivo chromebook ch1190": "2020-06-01T00:00:00.000Z",
"promethean chromebox": "2024-06-01T00:00:00.000Z",
"prowise 11.6\" entry line chromebook": "2020-06-01T00:00:00.000Z",
"prowise chromebook eduline": "2023-11-01T00:00:00.000Z",
"prowise chromebook entryline": "2020-06-01T00:00:00.000Z",
"prowise chromebook proline": "2021-08-01T00:00:00.000Z",
"prowise proline chromebook": "2021-08-01T00:00:00.000Z",
"rgs education chromebook": "2020-06-01T00:00:00.000Z",
"samsung chromebook": "2018-07-01T00:00:00.000Z",
"samsung chromebook - xe303": "2018-07-01T00:00:00.000Z",
"samsung chromebook 2 11": "2019-06-01T00:00:00.000Z",
"samsung chromebook 2 11 - xe500c12": "2020-06-01T00:00:00.000Z",
"samsung chromebook 2 13": "2019-06-01T00:00:00.000Z",
"samsung chromebook 3": "2021-06-01T00:00:00.000Z",
"samsung chromebook plus": "2023-08-01T00:00:00.000Z",
"samsung chromebook plus (lte)": "2024-06-01T00:00:00.000Z",
"samsung chromebook plus (v2)": "2024-06-01T00:00:00.000Z",
"samsung chromebook pro": "2022-11-01T00:00:00.000Z",
"samsung chromebook series 5": "2016-06-01T00:00:00.000Z",
"samsung chromebook series 5 550": "2017-05-01T00:00:00.000Z",
"samsung chromebox series 3": "2018-03-01T00:00:00.000Z",
"sector 5 e1 rugged chromebook": "2020-06-01T00:00:00.000Z",
"sector 5 e3 chromebook": "2023-11-01T00:00:00.000Z",
"senkatel c1101 chromebook": "2020-06-01T00:00:00.000Z",
"thinkpad 11e chromebook 3rd gen (yoga/clamshell)": "2021-06-01T00:00:00.000Z",
"thinkpad 13 chromebook": "2022-11-01T00:00:00.000Z",
"toshiba chromebook": "2019-06-01T00:00:00.000Z",
"toshiba chromebook 2": "2020-06-01T00:00:00.000Z",
"toshiba chromebook 2 (2015 edition)": "2020-09-01T00:00:00.000Z",
"true idc chromebook": "2020-06-01T00:00:00.000Z",
"true idc chromebook 11": "2020-06-01T00:00:00.000Z",
"videonet chromebook": "2020-06-01T00:00:00.000Z",
"videonet chromebook bl10": "2020-06-01T00:00:00.000Z",
"viewsonic nmp660 chromebox": "2024-06-01T00:00:00.000Z",
"viglen chromebook 11": "2020-06-01T00:00:00.000Z",
"viglen chromebook 11c": "2023-11-01T00:00:00.000Z",
"viglen chromebook 360": "2021-08-01T00:00:00.000Z",
"xolo chromebook": "2020-06-01T00:00:00.000Z"
}

18
src/display.py Normal file
View File

@@ -0,0 +1,18 @@
"""Methods related to display of information to the user."""
import sys
import utils
from var import ERROR_PREFIX
from var import WARNING_PREFIX
def print_error(message):
"""Prints a one-line error message to stderr in a standard format."""
sys.stderr.write(
utils.convertUTF8('\n{0}{1}\n'.format(ERROR_PREFIX, message)))
def print_warning(message):
"""Prints a one-line warning message to stderr in a standard format."""
sys.stderr.write(
utils.convertUTF8('\n{0}{1}\n'.format(WARNING_PREFIX, message)))

59
src/display_test.py Normal file
View File

@@ -0,0 +1,59 @@
"""Tests for display."""
import unittest
from unittest.mock import patch
import display
from var import ERROR_PREFIX
from var import WARNING_PREFIX
class DisplayTest(unittest.TestCase):
@patch.object(display.sys.stderr, 'write')
def test_print_error_prints_to_stderr(self, mock_write):
message = 'test error'
display.print_error(message)
printed_message = mock_write.call_args[0][0]
self.assertIn(message, printed_message)
@patch.object(display.sys.stderr, 'write')
def test_print_error_prints_error_prefix(self, mock_write):
message = 'test error'
display.print_error(message)
printed_message = mock_write.call_args[0][0]
self.assertLess(
printed_message.find(ERROR_PREFIX), printed_message.find(message),
'The error prefix does not appear before the error message')
@patch.object(display.sys.stderr, 'write')
def test_print_error_ends_message_with_newline(self, mock_write):
message = 'test error'
display.print_error(message)
printed_message = mock_write.call_args[0][0]
self.assertRegex(printed_message, '\n$',
'The error message does not end in a newline.')
@patch.object(display.sys.stderr, 'write')
def test_print_warning_prints_to_stderr(self, mock_write):
message = 'test warning'
display.print_warning(message)
printed_message = mock_write.call_args[0][0]
self.assertIn(message, printed_message)
@patch.object(display.sys.stderr, 'write')
def test_print_warning_prints_error_prefix(self, mock_write):
message = 'test warning'
display.print_error(message)
printed_message = mock_write.call_args[0][0]
self.assertLess(
printed_message.find(WARNING_PREFIX), printed_message.find(message),
'The warning prefix does not appear before the error message')
@patch.object(display.sys.stderr, 'write')
def test_print_warning_ends_message_with_newline(self, mock_write):
message = 'test warning'
display.print_error(message)
printed_message = mock_write.call_args[0][0]
self.assertRegex(printed_message, '\n$',
'The warning message does not end in a newline.')

174
src/fileutils.py Normal file
View File

@@ -0,0 +1,174 @@
"""Common file operations."""
import io
import os
import sys
import controlflow
import display
from var import GM_Globals
from var import GM_SYS_ENCODING
from var import UTF8_SIG
def _open_file(filename, mode, encoding=None, newline=None):
"""Opens a file with no error handling."""
# Determine which encoding to use
if 'b' in mode:
encoding = None
elif not encoding:
encoding = GM_Globals[GM_SYS_ENCODING]
elif 'r' in mode and encoding.lower().replace('-', '') == 'utf8':
encoding = UTF8_SIG
return open(
os.path.expanduser(filename), mode, newline=newline, encoding=encoding)
def open_file(filename,
mode='r',
encoding=None,
newline=None,
strip_utf_bom=False):
"""Opens a file.
Args:
filename: String, the name of the file to open, or '-' to use stdin/stdout,
to read/write, depending on the mode param, respectively.
mode: String, the common file mode to open the file with. Default is read.
encoding: String, the name of the encoding used to decode or encode the
file. This should only be used in text mode.
newline: See param description in
https://docs.python.org/3.7/library/functions.html#open
strip_utf_bom: Boolean, True if the file being opened should seek past the
UTF Byte Order Mark before being returned.
See more: https://en.wikipedia.org/wiki/UTF-8#Byte_order_mark
Returns:
The opened file.
"""
try:
if filename == '-':
# Read from stdin, rather than a file
if 'r' in mode:
return io.StringIO(str(sys.stdin.read()))
return sys.stdout
# Open a file on disk
f = _open_file(filename, mode, newline=newline, encoding=encoding)
if strip_utf_bom:
utf_bom = u'\ufeff'
has_bom = False
if 'b' in mode:
has_bom = f.read(3).decode('UTF-8') == utf_bom
elif f.encoding and not f.encoding.lower().startswith('utf'):
# Convert UTF BOM into ISO-8859-1 via Bytes
utf8_bom_bytes = utf_bom.encode('UTF-8')
iso_8859_1_bom = utf8_bom_bytes.decode('iso-8859-1').encode(
'iso-8859-1')
has_bom = f.read(3).encode('iso-8859-1', 'replace') == iso_8859_1_bom
else:
has_bom = f.read(1) == utf_bom
if not has_bom:
f.seek(0)
return f
except IOError as e:
controlflow.system_error_exit(6, e)
def close_file(f):
"""Closes a file.
Args:
f: The file to close
Returns:
Boolean, True if the file was successfully closed. False if an error
was encountered while closing.
"""
try:
f.close()
return True
except IOError as e:
display.print_error(e)
return False
def read_file(filename,
mode='r',
encoding=None,
newline=None,
continue_on_error=False,
display_errors=True):
"""Reads a file from disk.
Args:
filename: String, the path of the file to open from disk, or "-" to read
from stdin.
mode: String, the mode in which to open the file.
encoding: String, the name of the encoding used to decode or encode the
file. This should only be used in text mode.
newline: See param description in
https://docs.python.org/3.7/library/functions.html#open
continue_on_error: Boolean, If True, suppresses any IO errors and returns to
the caller without any externalities.
display_errors: Boolean, If True, prints error messages when errors are
encountered and continue_on_error is True.
Returns:
The contents of the file, or stdin if filename == "-". Returns None if
an error is encountered and continue_on_errors is True.
"""
try:
if filename == '-':
# Read from stdin, rather than a file.
return str(sys.stdin.read())
with _open_file(filename, mode, newline=newline, encoding=encoding) as f:
return f.read()
except IOError as e:
if continue_on_error:
if display_errors:
display.print_warning(e)
return None
controlflow.system_error_exit(6, e)
except (LookupError, UnicodeDecodeError, UnicodeError) as e:
controlflow.system_error_exit(2, str(e))
def write_file(filename,
data,
mode='w',
continue_on_error=False,
display_errors=True):
"""Writes data to a file.
Args:
filename: String, the path of the file to write to disk.
data: Serializable data to write to the file.
mode: String, the mode in which to open the file and write to it.
continue_on_error: Boolean, If True, suppresses any IO errors and returns to
the caller without any externalities.
display_errors: Boolean, If True, prints error messages when errors are
encountered and continue_on_error is True.
Returns:
Boolean, True if the write operation succeeded, or False if not.
"""
try:
with _open_file(filename, mode) as f:
f.write(data)
return True
except IOError as e:
if continue_on_error:
if display_errors:
display.print_error(e)
return False
else:
controlflow.system_error_exit(6, e)

234
src/fileutils_test.py Normal file
View File

@@ -0,0 +1,234 @@
"""Tests for fileutils."""
import io
import os
import unittest
from unittest.mock import MagicMock
from unittest.mock import patch
import fileutils
class FileutilsTest(unittest.TestCase):
def setUp(self):
self.fake_path = '/some/path/to/file'
super(FileutilsTest, self).setUp()
@patch.object(fileutils.sys, 'stdin')
def test_open_file_stdin(self, mock_stdin):
mock_stdin.read.return_value = 'some stdin content'
f = fileutils.open_file('-', mode='r')
self.assertIsInstance(f, fileutils.io.StringIO)
self.assertEqual(f.getvalue(), mock_stdin.read.return_value)
def test_open_file_stdout(self):
f = fileutils.open_file('-', mode='w')
self.assertEqual(fileutils.sys.stdout, f)
@patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
def test_open_file_opens_correct_path(self, mock_open):
f = fileutils.open_file(self.fake_path)
self.assertEqual(self.fake_path, mock_open.call_args[0][0])
self.assertEqual(mock_open.return_value, f)
@patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
def test_open_file_expands_user_file_path(self, mock_open):
file_path = '~/some/path/containing/tilde/shortcut/to/home'
fileutils.open_file(file_path)
opened_path = mock_open.call_args[0][0]
home_path = os.environ.get('HOME')
self.assertIsNotNone(home_path)
self.assertIn(home_path, opened_path)
@patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
def test_open_file_opens_correct_mode(self, mock_open):
fileutils.open_file(self.fake_path)
self.assertEqual('r', mock_open.call_args[0][1])
@patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
def test_open_file_encoding_for_binary(self, mock_open):
fileutils.open_file(self.fake_path, mode='b')
self.assertIsNone(mock_open.call_args[1]['encoding'])
@patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
def test_open_file_default_system_encoding(self, mock_open):
fileutils.open_file(self.fake_path)
self.assertEqual(fileutils.GM_Globals[fileutils.GM_SYS_ENCODING],
mock_open.call_args[1]['encoding'])
@patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
def test_open_file_utf8_encoding_specified(self, mock_open):
fileutils.open_file(self.fake_path, encoding='UTF-8')
self.assertEqual(fileutils.UTF8_SIG, mock_open.call_args[1]['encoding'])
def test_open_file_strips_utf_bom_in_utf(self):
bom_prefixed_data = u'\ufefffoobar'
fake_file = io.StringIO(bom_prefixed_data)
mock_open = MagicMock(spec=open, return_value=fake_file)
with patch.object(fileutils, 'open', mock_open):
f = fileutils.open_file(self.fake_path, strip_utf_bom=True)
self.assertEqual('foobar', f.read())
def test_open_file_strips_utf_bom_in_non_utf(self):
bom_prefixed_data = b'\xef\xbb\xbffoobar'.decode('iso-8859-1')
# We need to trick the method under test into believing that a StringIO
# instance is a file with an encoding. Since StringIO does not usually have,
# an encoding, we'll mock it and add our own encoding, but send the other
# methods in use (read and seek) back to the real StringIO object.
real_stringio = io.StringIO(bom_prefixed_data)
mock_file = MagicMock(spec=io.StringIO)
mock_file.read.side_effect = real_stringio.read
mock_file.seek.side_effect = real_stringio.seek
mock_file.encoding = 'iso-8859-1'
mock_open = MagicMock(spec=open, return_value=mock_file)
with patch.object(fileutils, 'open', mock_open):
f = fileutils.open_file(self.fake_path, strip_utf_bom=True)
self.assertEqual('foobar', f.read())
def test_open_file_strips_utf_bom_in_binary(self):
bom_prefixed_data = u'\ufefffoobar'.encode('UTF-8')
fake_file = io.BytesIO(bom_prefixed_data)
mock_open = MagicMock(spec=open, return_value=fake_file)
with patch.object(fileutils, 'open', mock_open):
f = fileutils.open_file(self.fake_path, mode='rb', strip_utf_bom=True)
self.assertEqual(b'foobar', f.read())
def test_open_file_strip_utf_bom_when_no_bom_in_data(self):
no_bom_data = 'This data has no BOM'
fake_file = io.StringIO(no_bom_data)
mock_open = MagicMock(spec=open, return_value=fake_file)
with patch.object(fileutils, 'open', mock_open):
f = fileutils.open_file(self.fake_path, strip_utf_bom=True)
# Since there was no opening BOM, we should be back at the beginning of
# the file.
self.assertEqual(fake_file.tell(), 0)
self.assertEqual(f.read(), no_bom_data)
@patch.object(fileutils, 'open', new_callable=unittest.mock.mock_open)
def test_open_file_exits_on_io_error(self, mock_open):
mock_open.side_effect = IOError('Fake IOError')
with self.assertRaises(SystemExit) as context:
fileutils.open_file(self.fake_path)
self.assertEqual(context.exception.code, 6)
def test_close_file_closes_file_successfully(self):
mock_file = MagicMock()
self.assertTrue(fileutils.close_file(mock_file))
self.assertEqual(mock_file.close.call_count, 1)
def test_close_file_with_error(self):
mock_file = MagicMock()
mock_file.close.side_effect = IOError()
self.assertFalse(fileutils.close_file(mock_file))
self.assertEqual(mock_file.close.call_count, 1)
@patch.object(fileutils.sys, 'stdin')
def test_read_file_from_stdin(self, mock_stdin):
mock_stdin.read.return_value = 'some stdin content'
self.assertEqual(fileutils.read_file('-'), mock_stdin.read.return_value)
@patch.object(fileutils, '_open_file')
def test_read_file_default_params(self, mock_open_file):
fake_content = 'some fake content'
mock_open_file.return_value.__enter__().read.return_value = fake_content
self.assertEqual(fileutils.read_file(self.fake_path), fake_content)
self.assertEqual(mock_open_file.call_args[0][0], self.fake_path)
self.assertEqual(mock_open_file.call_args[0][1], 'r')
self.assertIsNone(mock_open_file.call_args[1]['newline'])
@patch.object(fileutils.display, 'print_warning')
@patch.object(fileutils, '_open_file')
def test_read_file_continues_on_errors_without_displaying(
self, mock_open_file, mock_print_warning):
mock_open_file.side_effect = IOError()
contents = fileutils.read_file(
self.fake_path, continue_on_error=True, display_errors=False)
self.assertIsNone(contents)
self.assertFalse(mock_print_warning.called)
@patch.object(fileutils.display, 'print_warning')
@patch.object(fileutils, '_open_file')
def test_read_file_displays_errors(self, mock_open_file, mock_print_warning):
mock_open_file.side_effect = IOError()
fileutils.read_file(
self.fake_path, continue_on_error=True, display_errors=True)
self.assertTrue(mock_print_warning.called)
@patch.object(fileutils, '_open_file')
def test_read_file_exits_code_6_when_continue_on_error_is_false(
self, mock_open_file):
mock_open_file.side_effect = IOError()
with self.assertRaises(SystemExit) as context:
fileutils.read_file(self.fake_path, continue_on_error=False)
self.assertEqual(context.exception.code, 6)
@patch.object(fileutils, '_open_file')
def test_read_file_exits_code_2_on_lookuperror(self, mock_open_file):
mock_open_file.return_value.__enter__().read.side_effect = LookupError()
with self.assertRaises(SystemExit) as context:
fileutils.read_file(self.fake_path)
self.assertEqual(context.exception.code, 2)
@patch.object(fileutils, '_open_file')
def test_read_file_exits_code_2_on_unicodeerror(self, mock_open_file):
mock_open_file.return_value.__enter__().read.side_effect = UnicodeError()
with self.assertRaises(SystemExit) as context:
fileutils.read_file(self.fake_path)
self.assertEqual(context.exception.code, 2)
@patch.object(fileutils, '_open_file')
def test_read_file_exits_code_2_on_unicodedecodeerror(self, mock_open_file):
fake_decode_error = UnicodeDecodeError('fake-encoding', b'fakebytes', 0, 1,
'testing only')
mock_open_file.return_value.__enter__().read.side_effect = fake_decode_error
with self.assertRaises(SystemExit) as context:
fileutils.read_file(self.fake_path)
self.assertEqual(context.exception.code, 2)
@patch.object(fileutils, '_open_file')
def test_write_file_writes_data_to_file(self, mock_open_file):
fake_data = 'some fake data'
fileutils.write_file(self.fake_path, fake_data)
self.assertEqual(mock_open_file.call_args[0][0], self.fake_path)
self.assertEqual(mock_open_file.call_args[0][1], 'w')
opened_file = mock_open_file.return_value.__enter__()
self.assertTrue(opened_file.write.called)
self.assertEqual(opened_file.write.call_args[0][0], fake_data)
@patch.object(fileutils.display, 'print_error')
@patch.object(fileutils, '_open_file')
def test_write_file_continues_on_errors_without_displaying(
self, mock_open_file, mock_print_error):
mock_open_file.side_effect = IOError()
status = fileutils.write_file(
self.fake_path,
'foo data',
continue_on_error=True,
display_errors=False)
self.assertFalse(status)
self.assertFalse(mock_print_error.called)
@patch.object(fileutils.display, 'print_error')
@patch.object(fileutils, '_open_file')
def test_write_file_displays_errors(self, mock_open_file, mock_print_error):
mock_open_file.side_effect = IOError()
fileutils.write_file(
self.fake_path, 'foo data', continue_on_error=True, display_errors=True)
self.assertTrue(mock_print_error.called)
@patch.object(fileutils, '_open_file')
def test_write_file_exits_code_6_when_continue_on_error_is_false(
self, mock_open_file):
mock_open_file.side_effect = IOError()
with self.assertRaises(SystemExit) as context:
fileutils.write_file(self.fake_path, 'foo data', continue_on_error=False)
self.assertEqual(context.exception.code, 6)
if __name__ == '__main__':
unittest.main()

View File

@@ -10,6 +10,7 @@ OPTIONS:
-d Directory where gam folder will be installed. Default is \$HOME/bin/
-a Architecture to install (i386, x86_64, x86_64_legacy, arm, arm64). Default is to detect your arch with "uname -m".
-o OS we are running (linux, macos). Default is to detect your OS with "uname -s".
-b OS version. Default is to detect on MacOS and Linux.
-l Just upgrade GAM to latest version. Skips project creation and auth.
-p Profile update (true, false). Should script add gam command to environment. Default is true.
-u Admin user email address to use with GAM. Default is to prompt.
@@ -21,20 +22,23 @@ EOF
target_dir="$HOME/bin"
gamarch=$(uname -m)
gamos=$(uname -s)
osversion=""
update_profile=true
upgrade_only=false
gamversion="latest"
adminuser=""
regularuser=""
gam_glibc_ver="2.12" # CentOS 6
gam_glibc_vers="2.27 2.23 2.19 2.15"
gam_macos_vers="10.14.4 10.13.6 10.12.6"
while getopts "hd:a:o:lp:u:r:v:" OPTION
while getopts "hd:a:o:b:lp:u:r:v:" OPTION
do
case $OPTION in
h) usage; exit;;
d) target_dir="$OPTARG";;
a) gamarch="$OPTARG";;
o) gamos="$OPTARG";;
b) osversion="$OPTARG";;
l) upgrade_only=true;;
p) update_profile="$OPTARG";;
u) adminuser="$OPTARG";;
@@ -79,39 +83,64 @@ echo -e '\x1B[0m'
version_gt()
{
test "$(printf '%s\n' "$@" | sort -V | head -n 1)" != "$1"
# MacOS < 10.13 doesn't support sort -V
echo "" | sort -V > /dev/null 2>&1
vsort_failed=$?
if [ "${1}" = "${2}" ]; then
true
elif (( $vsort_failed != 0 )); then
false
else
test "$(printf '%s\n' "$@" | sort -V | head -n 1)" != "$1"
fi
}
case $gamos in
[lL]inux)
gamos="linux"
this_glibc_ver=$(ldd --version | awk '/ldd/{print $NF}')
echo "This Linux distribution uses glibc $this_glibc_ver"
if version_gt $gam_glibc_ver $this_glibc_ver; then
echo_yellow "NOTICE: You are running an older Linux distro than the one GAM was compiled on. A legacy GAM version that should be compatible with your system but may run slower will be installed. For best performance, upgrade to a newer Linux distribution like Debian 9 stable, Ubuntu Xenial 16.04, Fedora 24+ or RedHat Enterprise Linux 8."
gamarch=x86_64_legacy
if [ "$osversion" == "" ]; then
this_glibc_ver=$(ldd --version | awk '/ldd/{print $NF}')
else
this_glibc_ver=$osversion
fi
echo "This Linux distribution uses glibc $this_glibc_ver"
useglibc="legacy"
for gam_glibc_ver in $gam_glibc_vers; do
if version_gt $this_glibc_ver $gam_glibc_ver; then
useglibc="glibc$gam_glibc_ver"
echo_green "Using GAM compiled against $useglibc"
break
fi
done
case $gamarch in
x86_64) gamfile="linux-x86_64.tar.xz";;
x86_64_legacy) gamfile="linux-x86_64-legacy.tar.xz";;
i?86) gamfile="linux-i686.tar.xz";;
arm|armv7l) gamfile="linux-armv7l.tar.xz";;
arm64|aarch64) gamfile="linux-aarch64.tar.xz";;
x86_64) gamfile="linux-x86_64-$useglibc.tar.xz";;
arm64|aarch64) gamfile="linux-arm64-$useglibc.tar.xz";;
*)
echo_red "ERROR: this installer currently only supports i386, x86_64, arm and arm64 Linux. Looks like you're running on $gamarch. Exiting."
echo_red "ERROR: this installer currently only supports x86_64 and arm64 Linux. Looks like you're running on $gamarch. Exiting."
exit
esac
;;
[Mm]ac[Oo][sS]|[Dd]arwin)
osver=$(sw_vers -productVersion | awk -F'.' '{print $2}')
if (( $osver < 13 )); then
echo_red "ERROR: GAM currently requires MacOS 10.13 or newer. You are running MacOS 10.$osver. Please upgrade."
exit
else
echo_green "Good, you're running MacOS 10.$osver..."
fi
gamos="macos"
gamfile="macos-x86_64.tar.xz"
if [ "$osversion" == "" ]; then
this_macos_ver=$(sw_vers -productVersion)
else
this_macos_ver=$osversion
fi
echo "You are running MacOS $this_macos_ver"
use_macos_ver=""
for gam_macos_ver in $gam_macos_vers; do
if version_gt $this_macos_ver $gam_macos_ver; then
use_macos_ver="MacOS$gam_macos_ver"
echo_green "Using GAM compiled on $use_macos_ver"
break
fi
done
if [ "$use_macos_ver" == "" ]; then
echo_red "Sorry, you need to be running at least MacOS $gam_macos_ver to run GAM"
exit
fi
gamfile="macos-x86_64-$use_macos_ver.tar.xz"
;;
*)
echo_red "Sorry, this installer currently only supports Linux and MacOS. Looks like you're runnning on $gamos. Exiting."
@@ -158,11 +187,16 @@ try:
except KeyError:
print('ERROR: assets value not found in JSON value of:\n\n%s' % release)"
pycmd="python"
pycmd="python3"
$pycmd -V >/dev/null 2>&1
rc=$?
if (( $rc != 0 )); then
pycmd="python3"
pycmd="python"
fi
$pycmd -V >/dev/null 2>&1
rc=$?
if (( $rc != 0 )); then
pycmd="python2"
fi
$pycmd -V >/dev/null 2>&1
rc=$?
@@ -202,7 +236,7 @@ fi
if [ "$upgrade_only" = true ]; then
echo_green "Here's information about your GAM upgrade:"
"$target_dir/gam/gam" version
"$target_dir/gam/gam" version extended
rc=$?
if (( $rc != 0 )); then
echo_red "ERROR: Failed running GAM for the first time with $rc. Please report this error to GAM mailing list. Exiting."
@@ -325,7 +359,7 @@ while $project_created; do
done
echo_green "Here's information about your new GAM installation:"
"$target_dir/gam/gam" version
"$target_dir/gam/gam" version extended
rc=$?
if (( $rc != 0 )); then
echo_red "ERROR: Failed running GAM for the first time with $rc. Please report this error to GAM mailing list. Exiting."

5316
src/gam.py

File diff suppressed because it is too large Load Diff

327
src/gapi/__init__.py Normal file
View File

@@ -0,0 +1,327 @@
"""Methods related to execution of GAPI requests."""
import sys
import googleapiclient.errors
import google.auth.exceptions
import httplib2
import controlflow
import display
from gapi import errors
from var import (GC_CA_FILE, GC_Values, GC_TLS_MIN_VERSION, GC_TLS_MAX_VERSION,
GM_Globals, GM_CURRENT_API_SCOPES, GM_CURRENT_API_USER,
GM_EXTRA_ARGS_DICT, GM_OAUTH2SERVICE_ACCOUNT_CLIENT_ID,
MAX_RESULTS_API_EXCEPTIONS, MESSAGE_API_ACCESS_CONFIG,
MESSAGE_API_ACCESS_DENIED, MESSAGE_SERVICE_NOT_APPLICABLE)
def create_http(cache=None,
timeout=None,
override_min_tls=None,
override_max_tls=None):
"""Creates a uniform HTTP transport object.
Args:
cache: The HTTP cache to use.
timeout: The cache timeout, in seconds.
override_min_tls: The minimum TLS version to require. If not provided, the
default is used.
override_max_tls: The maximum TLS version to require. If not provided, the
default is used.
Returns:
httplib2.Http with the specified options.
"""
tls_minimum_version = override_min_tls if override_min_tls else GC_Values[
GC_TLS_MIN_VERSION]
tls_maximum_version = override_max_tls if override_max_tls else GC_Values[
GC_TLS_MAX_VERSION]
return httplib2.Http(
ca_certs=GC_Values[GC_CA_FILE],
tls_maximum_version=tls_maximum_version,
tls_minimum_version=tls_minimum_version,
cache=cache,
timeout=timeout)
def call(service,
function,
silent_errors=False,
soft_errors=False,
throw_reasons=None,
retry_reasons=None,
**kwargs):
"""Executes a single request on a Google service function.
Args:
service: A Google service object for the desired API.
function: String, The name of a service request method to execute.
silent_errors: Bool, If True, error messages are suppressed when
encountered.
soft_errors: Bool, If True, writes non-fatal errors to stderr.
throw_reasons: A list of Google HTTP error reason strings indicating the
errors generated by this request should be re-thrown. All other HTTP
errors are consumed.
retry_reasons: A list of Google HTTP error reason strings indicating which
error should be retried, using exponential backoff techniques, when the
error reason is encountered.
**kwargs: Additional params to pass to the request method.
Returns:
A response object for the corresponding Google API call.
"""
if throw_reasons is None:
throw_reasons = []
if retry_reasons is None:
retry_reasons = []
method = getattr(service, function)
retries = 10
parameters = dict(
list(kwargs.items()) + list(GM_Globals[GM_EXTRA_ARGS_DICT].items()))
for n in range(1, retries + 1):
try:
return method(**parameters).execute()
except googleapiclient.errors.HttpError as e:
http_status, reason, message = errors.get_gapi_error_detail(
e,
soft_errors=soft_errors,
silent_errors=silent_errors,
retry_on_http_error=n < 3)
if http_status == -1:
# The error detail indicated that we should retry this request
# We'll refresh credentials and make another pass
service._http.request.credentials.refresh(create_http())
continue
if http_status == 0:
return None
is_known_error_reason = reason in [r.value for r in errors.ErrorReason]
if is_known_error_reason and errors.ErrorReason(reason) in throw_reasons:
if errors.ErrorReason(reason) in errors.ERROR_REASON_TO_EXCEPTION:
raise errors.ERROR_REASON_TO_EXCEPTION[errors.ErrorReason(reason)](
message)
raise e
if (n != retries) and (is_known_error_reason and errors.ErrorReason(
reason) in errors.DEFAULT_RETRY_REASONS + retry_reasons):
controlflow.wait_on_failure(n, retries, reason)
continue
if soft_errors:
display.print_error('{0}: {1} - {2}{3}'.format(http_status, message,
reason,
['',
': Giving up.'][n > 1]))
return None
controlflow.system_error_exit(
int(http_status), '{0}: {1} - {2}'.format(http_status, message,
reason))
except google.auth.exceptions.RefreshError as e:
handle_oauth_token_error(
e, soft_errors or
errors.ErrorReason.SERVICE_NOT_AVAILABLE in throw_reasons)
if errors.ErrorReason.SERVICE_NOT_AVAILABLE in throw_reasons:
raise errors.GapiServiceNotAvailableError(str(e))
display.print_error('User {0}: {1}'.format(
GM_Globals[GM_CURRENT_API_USER], str(e)))
return None
except ValueError as e:
if service._http.cache is not None:
service._http.cache = None
continue
controlflow.system_error_exit(4, str(e))
except (httplib2.ServerNotFoundError, RuntimeError) as e:
if n != retries:
service._http.connections = {}
controlflow.wait_on_failure(n, retries, str(e))
continue
controlflow.system_error_exit(4, str(e))
except TypeError as e:
controlflow.system_error_exit(4, str(e))
def get_items(service,
function,
items='items',
throw_reasons=None,
retry_reasons=None,
**kwargs):
"""Gets a single page of items from a Google service function that is paged.
Args:
service: A Google service object for the desired API.
function: String, The name of a service request method to execute.
items: String, the name of the resulting "items" field within the service
method's response object.
throw_reasons: A list of Google HTTP error reason strings indicating the
errors generated by this request should be re-thrown. All other HTTP
errors are consumed.
retry_reasons: A list of Google HTTP error reason strings indicating which
error should be retried, using exponential backoff techniques, when the
error reason is encountered.
**kwargs: Additional params to pass to the request method.
Returns:
The list of items in the first page of a response.
"""
results = call(
service,
function,
throw_reasons=throw_reasons,
retry_reasons=retry_reasons,
**kwargs)
if results:
return results.get(items, [])
return []
def _get_max_page_size_for_api_call(service, function, **kwargs):
"""Gets the maximum number of results supported for a single API call.
Args:
service: A Google service object for the desired API.
function: String, The name of the service method to check for max page size.
**kwargs: Additional params that will be passed to the request method.
Returns:
Int, A value from discovery if it exists, otherwise value from
MAX_RESULTS_API_EXCEPTIONS, otherwise None
"""
method = getattr(service, function)
api_id = method(**kwargs).methodId
for resource in service._rootDesc.get('resources', {}).values():
for a_method in resource.get('methods', {}).values():
if a_method.get('id') == api_id:
if not a_method.get('parameters') or a_method['parameters'].get(
'pageSize') or not a_method['parameters'].get('maxResults'):
# Make sure API call supports maxResults. For now we don't care to
# set pageSize since all known pageSize API calls have
# default pageSize == max pageSize.
return None
known_api_max = MAX_RESULTS_API_EXCEPTIONS.get(api_id)
max_results = a_method['parameters']['maxResults'].get(
'maximum', known_api_max)
return {'maxResults': max_results}
return None
def get_all_pages(service,
function,
items='items',
page_message=None,
message_attribute=None,
soft_errors=False,
throw_reasons=None,
retry_reasons=None,
**kwargs):
"""Aggregates and returns all pages of a Google service function response.
All pages of items are aggregated and returned as a single list.
Args:
service: A Google service object for the desired API.
function: String, The name of a service request method to execute.
items: String, the name of the resulting "items" field within the method's
response object. The items in this field will be aggregated across all
pages and returned.
page_message: String, a message to be displayed to the user during paging.
Template strings allow for dynamic content to be inserted during paging.
Supported template strings:
%%total_items%% : The current number of items discovered across all
pages.
%%first_item%% : In conjunction with `message_attribute` arg, will
display a unique property of the first item in the current page.
%%last_item%% : In conjunction with `message_attribute` arg, will
display a unique property of the last item in the current page.
message_attribute: String, the name of a signature field within a single
returned item which identifies that unique item. This field is used with
`page_message` to templatize a paging status message.
soft_errors: Bool, If True, writes non-fatal errors to stderr.
throw_reasons: A list of Google HTTP error reason strings indicating the
errors generated by this request should be re-thrown. All other HTTP
errors are consumed.
retry_reasons: A list of Google HTTP error reason strings indicating which
error should be retried, using exponential backoff techniques, when the
error reason is encountered.
**kwargs: Additional params to pass to the request method.
Returns:
A list of all items received from all paged responses.
"""
if 'maxResults' not in kwargs and 'pageSize' not in kwargs:
page_key = _get_max_page_size_for_api_call(service, function, **kwargs)
if page_key:
kwargs.update(page_key)
all_items = []
page_token = None
total_items = 0
while True:
page = call(
service,
function,
soft_errors=soft_errors,
throw_reasons=throw_reasons,
retry_reasons=retry_reasons,
pageToken=page_token,
**kwargs)
if page:
page_token = page.get('nextPageToken')
page_items = page.get(items, [])
num_page_items = len(page_items)
total_items += num_page_items
all_items.extend(page_items)
else:
page_token = None
num_page_items = 0
# Show a paging message to the user that indicates paging progress
if page_message:
show_message = page_message.replace('%%total_items%%', str(total_items))
if message_attribute:
first_item = page_items[0] if num_page_items > 0 else {}
last_item = page_items[-1] if num_page_items > 1 else first_item
show_message = show_message.replace(
'%%first_item%%', str(first_item.get(message_attribute, '')))
show_message = show_message.replace(
'%%last_item%%', str(last_item.get(message_attribute, '')))
sys.stderr.write('\r')
sys.stderr.flush()
sys.stderr.write(show_message)
if not page_token:
# End the paging status message and return all items.
if page_message and (page_message[-1] != '\n'):
sys.stderr.write('\r\n')
sys.stderr.flush()
return all_items
# TODO: Make this private once all execution related items that use this method
# have been brought into this file
def handle_oauth_token_error(e, soft_errors):
"""On a token error, exits the application and writes a message to stderr.
Args:
e: google.auth.exceptions.RefreshError, The error to handle.
soft_errors: Boolean, if True, suppresses any applicable errors and instead
returns to the caller.
"""
token_error = str(e).replace('.', '')
if token_error in errors.OAUTH2_TOKEN_ERRORS or e.startswith(
'Invalid response'):
if soft_errors:
return
if not GM_Globals[GM_CURRENT_API_USER]:
display.print_error(
MESSAGE_API_ACCESS_DENIED.format(
GM_Globals[GM_OAUTH2SERVICE_ACCOUNT_CLIENT_ID],
','.join(GM_Globals[GM_CURRENT_API_SCOPES])))
controlflow.system_error_exit(12, MESSAGE_API_ACCESS_CONFIG)
else:
controlflow.system_error_exit(
19,
MESSAGE_SERVICE_NOT_APPLICABLE.format(
GM_Globals[GM_CURRENT_API_USER]))
controlflow.system_error_exit(18,
'Authentication Token Error - {0}'.format(e))

538
src/gapi/__init___test.py Normal file
View File

@@ -0,0 +1,538 @@
"""Tests for gapi."""
import json
import unittest
from unittest.mock import MagicMock
from unittest.mock import patch
from gam import SetGlobalVariables
import gapi
from gapi import errors
def create_http_error(status, reason, message):
"""Creates a HttpError object similar to most Google API Errors.
Args:
status: Int, the error's HTTP response status number.
reason: String, a camelCase reason for the HttpError being given.
message: String, a general error message describing the error that occurred.
Returns:
googleapiclient.errors.HttpError
"""
response = {
'status': status,
'content-type': 'application/json',
}
content = {
'error': {
'code': status,
'errors': [{
'reason': str(reason),
'message': message,
}]
}
}
content_bytes = json.dumps(content).encode('UTF-8')
return gapi.googleapiclient.errors.HttpError(response, content_bytes)
class CreateHttpTest(unittest.TestCase):
def setUp(self):
SetGlobalVariables()
super(CreateHttpTest, self).setUp()
def test_create_http_sets_default_values_on_http(self):
http = gapi.create_http()
self.assertIsNone(http.cache)
self.assertIsNone(http.timeout)
self.assertEqual(http.tls_minimum_version,
gapi.GC_Values[gapi.GC_TLS_MIN_VERSION])
self.assertEqual(http.tls_maximum_version,
gapi.GC_Values[gapi.GC_TLS_MAX_VERSION])
self.assertEqual(http.ca_certs, gapi.GC_Values[gapi.GC_CA_FILE])
def test_create_http_sets_tls_min_version(self):
http = gapi.create_http(override_min_tls=1111)
self.assertEqual(http.tls_minimum_version, 1111)
def test_create_http_sets_tls_max_version(self):
http = gapi.create_http(override_max_tls=9999)
self.assertEqual(http.tls_maximum_version, 9999)
def test_create_http_sets_cache(self):
fake_cache = {}
http = gapi.create_http(cache=fake_cache)
self.assertEqual(http.cache, fake_cache)
def test_create_http_sets_cache_timeout(self):
http = gapi.create_http(timeout=1234)
self.assertEqual(http.timeout, 1234)
class GapiTest(unittest.TestCase):
def setUp(self):
SetGlobalVariables()
self.mock_service = MagicMock()
self.mock_method_name = 'mock_method'
self.mock_method = getattr(self.mock_service, self.mock_method_name)
self.simple_3_page_response = [
{
'items': [{
'position': 'page1,item1'
}, {
'position': 'page1,item2'
}, {
'position': 'page1,item3'
}],
'nextPageToken': 'page2'
},
{
'items': [{
'position': 'page2,item1'
}, {
'position': 'page2,item2'
}, {
'position': 'page2,item3'
}],
'nextPageToken': 'page3'
},
{
'items': [{
'position': 'page3,item1'
}, {
'position': 'page3,item2'
}, {
'position': 'page3,item3'
}],
},
]
self.empty_items_response = {'items': []}
super(GapiTest, self).setUp()
def test_call_returns_basic_200_response(self):
response = gapi.call(self.mock_service, self.mock_method_name)
self.assertEqual(response, self.mock_method().execute.return_value)
def test_call_passes_target_method_params(self):
gapi.call(
self.mock_service, self.mock_method_name, my_param_1=1, my_param_2=2)
self.assertEqual(self.mock_method.call_count, 1)
method_kwargs = self.mock_method.call_args[1]
self.assertEqual(method_kwargs.get('my_param_1'), 1)
self.assertEqual(method_kwargs.get('my_param_2'), 2)
@patch.object(gapi.errors, 'get_gapi_error_detail')
def test_call_retries_with_soft_errors(self, mock_error_detail):
mock_error_detail.return_value = (-1, 'aReason', 'some message')
# Make the request fail first, then return the proper response on the retry.
fake_http_error = create_http_error(403, 'aReason', 'unused message')
fake_200_response = MagicMock()
self.mock_method.return_value.execute.side_effect = [
fake_http_error, fake_200_response
]
response = gapi.call(
self.mock_service, self.mock_method_name, soft_errors=True)
self.assertEqual(response, fake_200_response)
self.assertEqual(
self.mock_service._http.request.credentials.refresh.call_count, 1)
self.assertEqual(self.mock_method.return_value.execute.call_count, 2)
def test_call_throws_for_provided_reason(self):
throw_reason = errors.ErrorReason.USER_NOT_FOUND
fake_http_error = create_http_error(404, throw_reason, 'forced throw')
self.mock_method.return_value.execute.side_effect = fake_http_error
gam_exception = errors.ERROR_REASON_TO_EXCEPTION[throw_reason]
with self.assertRaises(gam_exception):
gapi.call(
self.mock_service,
self.mock_method_name,
throw_reasons=[throw_reason])
# Prevent wait_on_failure from performing actual backoff unnecessarily, since
# we're not actually testing over a network connection
@patch.object(gapi.controlflow, 'wait_on_failure')
def test_call_retries_request_for_default_retry_reasons(
self, mock_wait_on_failure):
# Test using one of the default retry reasons
default_throw_reason = errors.ErrorReason.BACKEND_ERROR
self.assertIn(default_throw_reason, errors.DEFAULT_RETRY_REASONS)
fake_http_error = create_http_error(404, default_throw_reason, 'message')
fake_200_response = MagicMock()
# Fail once, then succeed on retry
self.mock_method.return_value.execute.side_effect = [
fake_http_error, fake_200_response
]
response = gapi.call(
self.mock_service, self.mock_method_name, retry_reasons=[])
self.assertEqual(response, fake_200_response)
self.assertEqual(self.mock_method.return_value.execute.call_count, 2)
# Make sure a backoff technique was used for retry.
self.assertEqual(mock_wait_on_failure.call_count, 1)
# Prevent wait_on_failure from performing actual backoff unnecessarily, since
# we're not actually testing over a network connection
@patch.object(gapi.controlflow, 'wait_on_failure')
def test_call_retries_requests_for_provided_retry_reasons(
self, unused_mock_wait_on_failure):
retry_reason1 = errors.ErrorReason.INTERNAL_ERROR
fake_retrieable_error1 = create_http_error(400, retry_reason1,
'Forced Error 1')
retry_reason2 = errors.ErrorReason.SYSTEM_ERROR
fake_retrieable_error2 = create_http_error(400, retry_reason2,
'Forced Error 2')
non_retriable_reason = errors.ErrorReason.SERVICE_NOT_AVAILABLE
fake_non_retriable_error = create_http_error(
400, non_retriable_reason,
'This error should not cause the request to be retried')
# Fail once, then succeed on retry
self.mock_method.return_value.execute.side_effect = [
fake_retrieable_error1, fake_retrieable_error2, fake_non_retriable_error
]
with self.assertRaises(SystemExit):
# The third call should raise the SystemExit when non_retriable_error is
# raised.
gapi.call(
self.mock_service,
self.mock_method_name,
retry_reasons=[retry_reason1, retry_reason2])
self.assertEqual(self.mock_method.return_value.execute.call_count, 3)
def test_call_exits_on_oauth_token_error(self):
# An error with any OAUTH2_TOKEN_ERROR
fake_token_error = gapi.google.auth.exceptions.RefreshError(
errors.OAUTH2_TOKEN_ERRORS[0])
self.mock_method.return_value.execute.side_effect = fake_token_error
with self.assertRaises(SystemExit):
gapi.call(self.mock_service, self.mock_method_name)
def test_call_exits_on_nonretriable_error(self):
error_reason = 'unknownReason'
fake_http_error = create_http_error(500, error_reason,
'Testing unretriable errors')
self.mock_method.return_value.execute.side_effect = fake_http_error
with self.assertRaises(SystemExit):
gapi.call(self.mock_service, self.mock_method_name)
def test_call_exits_on_request_valueerror(self):
self.mock_method.return_value.execute.side_effect = ValueError()
with self.assertRaises(SystemExit):
gapi.call(self.mock_service, self.mock_method_name)
def test_call_clears_bad_http_cache_on_request_failure(self):
self.mock_service._http.cache = 'something that is not None'
fake_200_response = MagicMock()
self.mock_method.return_value.execute.side_effect = [
ValueError(), fake_200_response
]
self.assertIsNotNone(self.mock_service._http.cache)
response = gapi.call(self.mock_service, self.mock_method_name)
self.assertEqual(response, fake_200_response)
# Assert the cache was cleared
self.assertIsNone(self.mock_service._http.cache)
# Prevent wait_on_failure from performing actual backoff unnecessarily, since
# we're not actually testing over a network connection
@patch.object(gapi.controlflow, 'wait_on_failure')
def test_call_retries_requests_with_backoff_on_servernotfounderror(
self, mock_wait_on_failure):
fake_servernotfounderror = gapi.httplib2.ServerNotFoundError()
fake_200_response = MagicMock()
# Fail once, then succeed on retry
self.mock_method.return_value.execute.side_effect = [
fake_servernotfounderror, fake_200_response
]
http_connections = self.mock_service._http.connections
response = gapi.call(self.mock_service, self.mock_method_name)
self.assertEqual(response, fake_200_response)
# HTTP cached connections should be cleared on receiving this error
self.assertNotEqual(http_connections, self.mock_service._http.connections)
self.assertEqual(self.mock_method.return_value.execute.call_count, 2)
# Make sure a backoff technique was used for retry.
self.assertEqual(mock_wait_on_failure.call_count, 1)
def test_get_items_calls_correct_service_function(self):
gapi.get_items(self.mock_service, self.mock_method_name)
self.assertTrue(self.mock_method.called)
def test_get_items_returns_one_page(self):
fake_response = {'items': [{}, {}, {}]}
self.mock_method.return_value.execute.return_value = fake_response
page = gapi.get_items(self.mock_service, self.mock_method_name)
self.assertEqual(page, fake_response['items'])
def test_get_items_non_default_page_field_name(self):
field_name = 'things'
fake_response = {field_name: [{}, {}, {}]}
self.mock_method.return_value.execute.return_value = fake_response
page = gapi.get_items(
self.mock_service, self.mock_method_name, items=field_name)
self.assertEqual(page, fake_response[field_name])
def test_get_items_passes_additional_kwargs_to_service(self):
gapi.get_items(
self.mock_service, self.mock_method_name, my_param_1=1, my_param_2=2)
self.assertEqual(self.mock_method.call_count, 1)
method_kwargs = self.mock_method.call_args[1]
self.assertEqual(1, method_kwargs.get('my_param_1'))
self.assertEqual(2, method_kwargs.get('my_param_2'))
def test_get_items_returns_empty_list_when_no_items_returned(self):
non_items_response = {'noItemsInThisResponse': {}}
self.mock_method.return_value.execute.return_value = non_items_response
page = gapi.get_items(self.mock_service, self.mock_method_name)
self.assertIsInstance(page, list)
self.assertEqual(0, len(page))
def test_get_all_pages_returns_all_items(self):
page_1 = {'items': ['1-1', '1-2', '1-3'], 'nextPageToken': '2'}
page_2 = {'items': ['2-1', '2-2', '2-3'], 'nextPageToken': '3'}
page_3 = {'items': ['3-1', '3-2', '3-3']}
self.mock_method.return_value.execute.side_effect = [page_1, page_2, page_3]
response_items = gapi.get_all_pages(self.mock_service,
self.mock_method_name)
self.assertListEqual(response_items,
page_1['items'] + page_2['items'] + page_3['items'])
def test_get_all_pages_includes_next_pagetoken_in_request(self):
page_1 = {'items': ['1-1', '1-2', '1-3'], 'nextPageToken': 'someToken'}
page_2 = {'items': ['2-1', '2-2', '2-3']}
self.mock_method.return_value.execute.side_effect = [page_1, page_2]
gapi.get_all_pages(self.mock_service, self.mock_method_name, pageSize=100)
self.assertEqual(self.mock_method.call_count, 2)
call_2_kwargs = self.mock_method.call_args_list[1][1]
self.assertIn('pageToken', call_2_kwargs)
self.assertEqual(call_2_kwargs['pageToken'], page_1['nextPageToken'])
def test_get_all_pages_uses_default_max_page_size(self):
sample_api_id = list(gapi.MAX_RESULTS_API_EXCEPTIONS.keys())[0]
sample_api_max_results = gapi.MAX_RESULTS_API_EXCEPTIONS[sample_api_id]
self.mock_method.return_value.methodId = sample_api_id
self.mock_service._rootDesc = {
'resources': {
'someResource': {
'methods': {
'someMethod': {
'id': sample_api_id,
'parameters': {
'maxResults': {
'maximum': sample_api_max_results
}
}
}
}
}
}
}
self.mock_method.return_value.execute.return_value = self.empty_items_response
gapi.get_all_pages(self.mock_service, self.mock_method_name)
request_method_kwargs = self.mock_method.call_args[1]
self.assertIn('maxResults', request_method_kwargs)
self.assertEqual(request_method_kwargs['maxResults'],
gapi.MAX_RESULTS_API_EXCEPTIONS.get(sample_api_id))
def test_get_all_pages_max_page_size_overrided(self):
self.mock_method.return_value.execute.return_value = self.empty_items_response
gapi.get_all_pages(
self.mock_service, self.mock_method_name, pageSize=123456)
request_method_kwargs = self.mock_method.call_args[1]
self.assertIn('pageSize', request_method_kwargs)
self.assertEqual(123456, request_method_kwargs['pageSize'])
@patch.object(gapi.sys.stderr, 'write')
def test_get_all_pages_prints_paging_message(self, mock_write):
self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
paging_message = 'A simple string displayed during paging'
gapi.get_all_pages(
self.mock_service, self.mock_method_name, page_message=paging_message)
messages_written = [
call_args[0][0] for call_args in mock_write.call_args_list
]
self.assertIn(paging_message, messages_written)
@patch.object(gapi.sys.stderr, 'write')
def test_get_all_pages_prints_paging_message_inline(self, mock_write):
self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
paging_message = 'A simple string displayed during paging'
gapi.get_all_pages(
self.mock_service, self.mock_method_name, page_message=paging_message)
messages_written = [
call_args[0][0] for call_args in mock_write.call_args_list
]
# Make sure a return carriage was written between two pages
paging_message_call_positions = [
i for i, message in enumerate(messages_written)
if message == paging_message
]
self.assertGreater(len(paging_message_call_positions), 1)
printed_between_page_messages = messages_written[
paging_message_call_positions[0]:paging_message_call_positions[1]]
self.assertIn('\r', printed_between_page_messages)
@patch.object(gapi.sys.stderr, 'write')
def test_get_all_pages_ends_paging_message_with_newline(self, mock_write):
self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
paging_message = 'A simple string displayed during paging'
gapi.get_all_pages(
self.mock_service, self.mock_method_name, page_message=paging_message)
messages_written = [
call_args[0][0] for call_args in mock_write.call_args_list
]
last_page_message_index = len(
messages_written) - messages_written[::-1].index(paging_message)
last_carriage_return_index = len(
messages_written) - messages_written[::-1].index('\r\n')
self.assertGreater(last_carriage_return_index, last_page_message_index)
@patch.object(gapi.sys.stderr, 'write')
def test_get_all_pages_prints_attribute_total_items_in_paging_message(
self, mock_write):
self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
paging_message = 'Total number of items discovered: %%total_items%%'
gapi.get_all_pages(
self.mock_service, self.mock_method_name, page_message=paging_message)
messages_written = [
call_args[0][0] for call_args in mock_write.call_args_list
]
page_1_item_count = len(self.simple_3_page_response[0]['items'])
page_1_message = paging_message.replace('%%total_items%%',
str(page_1_item_count))
self.assertIn(page_1_message, messages_written)
page_2_item_count = len(self.simple_3_page_response[1]['items'])
page_2_message = paging_message.replace(
'%%total_items%%', str(page_1_item_count + page_2_item_count))
self.assertIn(page_2_message, messages_written)
page_3_item_count = len(self.simple_3_page_response[2]['items'])
page_3_message = paging_message.replace(
'%%total_items%%',
str(page_1_item_count + page_2_item_count + page_3_item_count))
self.assertIn(page_3_message, messages_written)
# Assert that the template text is always replaced.
for message in messages_written:
self.assertNotIn('%%total_items', message)
@patch.object(gapi.sys.stderr, 'write')
def test_get_all_pages_prints_attribute_first_item_in_paging_message(
self, mock_write):
self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
paging_message = 'First item in page: %%first_item%%'
gapi.get_all_pages(
self.mock_service,
self.mock_method_name,
page_message=paging_message,
message_attribute='position')
messages_written = [
call_args[0][0] for call_args in mock_write.call_args_list
]
page_1_message = paging_message.replace(
'%%first_item%%',
self.simple_3_page_response[0]['items'][0]['position'])
self.assertIn(page_1_message, messages_written)
page_2_message = paging_message.replace(
'%%first_item%%',
self.simple_3_page_response[1]['items'][0]['position'])
self.assertIn(page_2_message, messages_written)
# Assert that the template text is always replaced.
for message in messages_written:
self.assertNotIn('%%first_item', message)
@patch.object(gapi.sys.stderr, 'write')
def test_get_all_pages_prints_attribute_last_item_in_paging_message(
self, mock_write):
self.mock_method.return_value.execute.side_effect = self.simple_3_page_response
paging_message = 'Last item in page: %%last_item%%'
gapi.get_all_pages(
self.mock_service,
self.mock_method_name,
page_message=paging_message,
message_attribute='position')
messages_written = [
call_args[0][0] for call_args in mock_write.call_args_list
]
page_1_message = paging_message.replace(
'%%last_item%%',
self.simple_3_page_response[0]['items'][-1]['position'])
self.assertIn(page_1_message, messages_written)
page_2_message = paging_message.replace(
'%%last_item%%',
self.simple_3_page_response[1]['items'][-1]['position'])
self.assertIn(page_2_message, messages_written)
# Assert that the template text is always replaced.
for message in messages_written:
self.assertNotIn('%%last_item', message)
def test_get_all_pages_prints_all_attributes_in_paging_message(self):
pass
def test_get_all_pages_passes_additional_kwargs_to_service_method(self):
self.mock_method.return_value.execute.return_value = self.empty_items_response
gapi.get_all_pages(
self.mock_service, self.mock_method_name, my_param_1=1, my_param_2=2)
method_kwargs = self.mock_method.call_args[1]
self.assertEqual(method_kwargs.get('my_param_1'), 1)
self.assertEqual(method_kwargs.get('my_param_2'), 2)
@patch.object(gapi, 'call')
def test_get_all_pages_passes_throw_and_retry_reasons(self, mock_call):
throw_for = MagicMock()
retry_for = MagicMock()
mock_call.return_value = self.empty_items_response
gapi.get_all_pages(
self.mock_service,
self.mock_method_name,
throw_reasons=throw_for,
retry_reasons=retry_for)
method_kwargs = mock_call.call_args[1]
self.assertEqual(method_kwargs.get('throw_reasons'), throw_for)
self.assertEqual(method_kwargs.get('retry_reasons'), retry_for)
def test_get_all_pages_non_default_items_field_name(self):
field_name = 'things'
fake_response = {field_name: [{}, {}, {}]}
self.mock_method.return_value.execute.return_value = fake_response
page = gapi.get_all_pages(
self.mock_service, self.mock_method_name, items=field_name)
self.assertEqual(page, fake_response[field_name])
if __name__ == '__main__':
unittest.main()

358
src/gapi/errors.py Normal file
View File

@@ -0,0 +1,358 @@
"""GAPI and OAuth Token related errors methods."""
from enum import Enum
import json
import controlflow
import display # TODO: Change to relative import when gam is setup as a package
from var import UTF8
class GapiAbortedError(Exception):
pass
class GapiAuthErrorError(Exception):
pass
class GapiBadGatewayError(Exception):
pass
class GapiBadRequestError(Exception):
pass
class GapiConditionNotMetError(Exception):
pass
class GapiCyclicMembershipsNotAllowedError(Exception):
pass
class GapiDomainCannotUseApisError(Exception):
pass
class GapiDomainNotFoundError(Exception):
pass
class GapiDuplicateError(Exception):
pass
class GapiFailedPreconditionError(Exception):
pass
class GapiForbiddenError(Exception):
pass
class GapiGatewayTimeoutError(Exception):
pass
class GapiGroupNotFoundError(Exception):
pass
class GapiInvalidError(Exception):
pass
class GapiInvalidArgumentError(Exception):
pass
class GapiInvalidMemberError(Exception):
pass
class GapiMemberNotFoundError(Exception):
pass
class GapiNotFoundError(Exception):
pass
class GapiNotImplementedError(Exception):
pass
class GapiPermissionDeniedError(Exception):
pass
class GapiResourceNotFoundError(Exception):
pass
class GapiServiceNotAvailableError(Exception):
pass
class GapiUserNotFoundError(Exception):
pass
# GAPI Error Reasons
class ErrorReason(Enum):
"""The reason why a non-200 HTTP response was returned from a GAPI."""
ABORTED = 'aborted'
AUTH_ERROR = 'authError'
BACKEND_ERROR = 'backendError'
BAD_GATEWAY = 'badGateway'
BAD_REQUEST = 'badRequest'
CONDITION_NOT_MET = 'conditionNotMet'
CYCLIC_MEMBERSHIPS_NOT_ALLOWED = 'cyclicMembershipsNotAllowed'
DOMAIN_CANNOT_USE_APIS = 'domainCannotUseApis'
DOMAIN_NOT_FOUND = 'domainNotFound'
DUPLICATE = 'duplicate'
FAILED_PRECONDITION = 'failedPrecondition'
FORBIDDEN = 'forbidden'
GATEWAY_TIMEOUT = 'gatewayTimeout'
GROUP_NOT_FOUND = 'groupNotFound'
INTERNAL_ERROR = 'internalError'
INVALID = 'invalid'
INVALID_ARGUMENT = 'invalidArgument'
INVALID_MEMBER = 'invalidMember'
MEMBER_NOT_FOUND = 'memberNotFound'
NOT_FOUND = 'notFound'
NOT_IMPLEMENTED = 'notImplemented'
PERMISSION_DENIED = 'permissionDenied'
QUOTA_EXCEEDED = 'quotaExceeded'
RATE_LIMIT_EXCEEDED = 'rateLimitExceeded'
RESOURCE_NOT_FOUND = 'resourceNotFound'
SERVICE_NOT_AVAILABLE = 'serviceNotAvailable'
SYSTEM_ERROR = 'systemError'
USER_NOT_FOUND = 'userNotFound'
USER_RATE_LIMIT_EXCEEDED = 'userRateLimitExceeded'
def __str__(self):
return str(self.value)
# Common sets of GAPI error reasons
DEFAULT_RETRY_REASONS = [
ErrorReason.QUOTA_EXCEEDED, ErrorReason.RATE_LIMIT_EXCEEDED,
ErrorReason.USER_RATE_LIMIT_EXCEEDED, ErrorReason.BACKEND_ERROR,
ErrorReason.BAD_GATEWAY, ErrorReason.GATEWAY_TIMEOUT,
ErrorReason.INTERNAL_ERROR
]
GMAIL_THROW_REASONS = [ErrorReason.SERVICE_NOT_AVAILABLE]
GROUP_GET_THROW_REASONS = [
ErrorReason.GROUP_NOT_FOUND, ErrorReason.DOMAIN_NOT_FOUND,
ErrorReason.DOMAIN_CANNOT_USE_APIS, ErrorReason.FORBIDDEN,
ErrorReason.BAD_REQUEST
]
GROUP_GET_RETRY_REASONS = [ErrorReason.INVALID, ErrorReason.SYSTEM_ERROR]
MEMBERS_THROW_REASONS = [
ErrorReason.GROUP_NOT_FOUND, ErrorReason.DOMAIN_NOT_FOUND,
ErrorReason.DOMAIN_CANNOT_USE_APIS, ErrorReason.INVALID,
ErrorReason.FORBIDDEN
]
MEMBERS_RETRY_REASONS = [ErrorReason.SYSTEM_ERROR]
# A map of GAPI error reasons to the corresponding GAM Python Exception
ERROR_REASON_TO_EXCEPTION = {
ErrorReason.ABORTED:
GapiAbortedError,
ErrorReason.AUTH_ERROR:
GapiAuthErrorError,
ErrorReason.BAD_GATEWAY:
GapiBadGatewayError,
ErrorReason.BAD_REQUEST:
GapiBadRequestError,
ErrorReason.CONDITION_NOT_MET:
GapiConditionNotMetError,
ErrorReason.CYCLIC_MEMBERSHIPS_NOT_ALLOWED:
GapiCyclicMembershipsNotAllowedError,
ErrorReason.DOMAIN_CANNOT_USE_APIS:
GapiDomainCannotUseApisError,
ErrorReason.DOMAIN_NOT_FOUND:
GapiDomainNotFoundError,
ErrorReason.DUPLICATE:
GapiDuplicateError,
ErrorReason.FAILED_PRECONDITION:
GapiFailedPreconditionError,
ErrorReason.FORBIDDEN:
GapiForbiddenError,
ErrorReason.GATEWAY_TIMEOUT:
GapiGatewayTimeoutError,
ErrorReason.GROUP_NOT_FOUND:
GapiGroupNotFoundError,
ErrorReason.INVALID:
GapiInvalidError,
ErrorReason.INVALID_ARGUMENT:
GapiInvalidArgumentError,
ErrorReason.INVALID_MEMBER:
GapiInvalidMemberError,
ErrorReason.MEMBER_NOT_FOUND:
GapiMemberNotFoundError,
ErrorReason.NOT_FOUND:
GapiNotFoundError,
ErrorReason.NOT_IMPLEMENTED:
GapiNotImplementedError,
ErrorReason.PERMISSION_DENIED:
GapiPermissionDeniedError,
ErrorReason.RESOURCE_NOT_FOUND:
GapiResourceNotFoundError,
ErrorReason.SERVICE_NOT_AVAILABLE:
GapiServiceNotAvailableError,
ErrorReason.USER_NOT_FOUND:
GapiUserNotFoundError,
}
# OAuth Token Errors
OAUTH2_TOKEN_ERRORS = [
'access_denied',
'access_denied: Requested client not authorized',
'internal_failure: Backend Error',
'internal_failure: None',
'invalid_grant',
'invalid_grant: Bad Request',
'invalid_grant: Invalid email or User ID',
'invalid_grant: Not a valid email',
'invalid_grant: Invalid JWT: No valid verifier found for issuer',
'invalid_request: Invalid impersonation prn email address',
'unauthorized_client: Client is unauthorized to retrieve access tokens '
'using this method',
'unauthorized_client: Client is unauthorized to retrieve access tokens '
'using this method, or client not authorized for any of the scopes '
'requested',
'unauthorized_client: Unauthorized client or scope in request',
]
def _create_http_error_dict(status_code, reason, message):
"""Creates a basic error dict similar to most Google API Errors.
Args:
status_code: Int, the error's HTTP response status code.
reason: String, a camelCase reason for the HttpError being given.
message: String, a general error message describing the error that occurred.
Returns:
dict
"""
return {
'error': {
'code': status_code,
'errors': [{
'reason': str(reason),
'message': message,
}]
}
}
def get_gapi_error_detail(e,
soft_errors=False,
silent_errors=False,
retry_on_http_error=False):
"""Extracts error detail from a non-200 GAPI Response.
Args:
e: googleapiclient.HttpError, The HTTP Error received.
soft_errors: Boolean, If true, causes error messages to be surpressed,
rather than sending them to stderr.
silent_errors: Boolean, If true, suppresses and ignores any errors from
being displayed
retry_on_http_error: Boolean, If true, will return -1 as the HTTP Response
code, indicating that the request can be retried. TODO: Remove this param,
as it seems to be outside the scope of this method.
Returns:
A tuple containing the HTTP Response code, GAPI error reason, and error
message.
"""
try:
error = json.loads(e.content.decode(UTF8))
except ValueError:
error_content = e.content.decode(UTF8) if isinstance(e.content,
bytes) else e.content
if (e.resp['status'] == '503') and (
error_content == 'Quota exceeded for the current request'):
return (e.resp['status'], ErrorReason.QUOTA_EXCEEDED.value, error_content)
if (e.resp['status'] == '403') and (
error_content.startswith('Request rate higher than configured')):
return (e.resp['status'], ErrorReason.QUOTA_EXCEEDED.value, error_content)
if (e.resp['status'] == '502') and ('Bad Gateway' in error_content):
return (e.resp['status'], ErrorReason.BAD_GATEWAY.value, error_content)
if (e.resp['status'] == '504') and ('Gateway Timeout' in error_content):
return (e.resp['status'], ErrorReason.GATEWAY_TIMEOUT.value, error_content)
if (e.resp['status'] == '403') and ('Invalid domain.' in error_content):
error = _create_http_error_dict(403, ErrorReason.NOT_FOUND.value,
'Domain not found')
elif (e.resp['status'] == '400') and (
'InvalidSsoSigningKey' in error_content):
error = _create_http_error_dict(400, ErrorReason.INVALID.value,
'InvalidSsoSigningKey')
elif (e.resp['status'] == '400') and ('UnknownError' in error_content):
error = _create_http_error_dict(400, ErrorReason.INVALID.value,
'UnknownError')
elif retry_on_http_error:
return (-1, None, None)
elif soft_errors:
if not silent_errors:
display.print_error(error_content)
return (0, None, None)
else:
controlflow.system_error_exit(5, error_content)
# END: ValueError catch
if 'error' in error:
http_status = error['error']['code']
try:
message = error['error']['errors'][0]['message']
except KeyError:
message = error['error']['message']
else:
if 'error_description' in error:
if error['error_description'] == 'Invalid Value':
message = error['error_description']
http_status = 400
error = _create_http_error_dict(400, ErrorReason.INVALID.value, message)
else:
controlflow.system_error_exit(4, str(error))
else:
controlflow.system_error_exit(4, str(error))
# Extract the error reason
try:
reason = error['error']['errors'][0]['reason']
if reason == 'notFound':
if 'userKey' in message:
reason = ErrorReason.USER_NOT_FOUND.value
elif 'groupKey' in message:
reason = ErrorReason.GROUP_NOT_FOUND.value
elif 'memberKey' in message:
reason = ErrorReason.MEMBER_NOT_FOUND.value
elif 'Domain not found' in message:
reason = ErrorReason.DOMAIN_NOT_FOUND.value
elif 'Resource Not Found' in message:
reason = ErrorReason.RESOURCE_NOT_FOUND.value
elif reason == 'invalid':
if 'userId' in message:
reason = ErrorReason.USER_NOT_FOUND.value
elif 'memberKey' in message:
reason = ErrorReason.INVALID_MEMBER.value
elif reason == 'failedPrecondition':
if 'Bad Request' in message:
reason = ErrorReason.BAD_REQUEST.value
elif 'Mail service not enabled' in message:
reason = ErrorReason.SERVICE_NOT_AVAILABLE.value
elif reason == 'required':
if 'memberKey' in message:
reason = ErrorReason.MEMBER_NOT_FOUND.value
elif reason == 'conditionNotMet':
if 'Cyclic memberships not allowed' in message:
reason = ErrorReason.CYCLIC_MEMBERSHIPS_NOT_ALLOWED.value
except KeyError:
reason = '{0}'.format(http_status)
return (http_status, reason, message)

209
src/gapi/errors_test.py Normal file
View File

@@ -0,0 +1,209 @@
"""Python unit tests for gapi.errors"""
import json
import unittest
from unittest.mock import patch
import googleapiclient.errors
from gapi import errors
def create_simple_http_error(status, reason, message):
content = errors._create_http_error_dict(status, reason, message)
return create_http_error(status, content)
def create_http_error(status, content):
response = {
'status': status,
'content-type': 'application/json',
}
content_as_bytes = json.dumps(content).encode('UTF-8')
return googleapiclient.errors.HttpError(response, content_as_bytes)
class ErrorsTest(unittest.TestCase):
def test_get_gapi_error_detail_quota_exceeded(self):
# TODO: Add test logic once the opening ValueError exception case has a
# repro case (i.e. an Exception type/format that will cause it to raise).
pass
def test_get_gapi_error_detail_invalid_domain(self):
# TODO: Add test logic once the opening ValueError exception case has a
# repro case (i.e. an Exception type/format that will cause it to raise).
pass
def test_get_gapi_error_detail_invalid_signing_key(self):
# TODO: Add test logic once the opening ValueError exception case has a
# repro case (i.e. an Exception type/format that will cause it to raise).
pass
def test_get_gapi_error_detail_unknown_error(self):
# TODO: Add test logic once the opening ValueError exception case has a
# repro case (i.e. an Exception type/format that will cause it to raise).
pass
def test_get_gapi_error_retry_http_error(self):
# TODO: Add test logic once the opening ValueError exception case has a
# repro case (i.e. an Exception type/format that will cause it to raise).
pass
def test_get_gapi_error_prints_soft_errors(self):
# TODO: Add test logic once the opening ValueError exception case has a
# repro case (i.e. an Exception type/format that will cause it to raise).
pass
def test_get_gapi_error_exits_on_unrecoverable_errors(self):
# TODO: Add test logic once the opening ValueError exception case has a
# repro case (i.e. an Exception type/format that will cause it to raise).
pass
def test_get_gapi_error_quota_exceeded_for_current_request(self):
# TODO: Add test logic once the opening ValueError exception case has a
# repro case (i.e. an Exception type/format that will cause it to raise).
pass
def test_get_gapi_error_quota_exceeded_high_request_rate(self):
# TODO: Add test logic once the opening ValueError exception case has a
# repro case (i.e. an Exception type/format that will cause it to raise).
pass
def test_get_gapi_error_extracts_user_not_found(self):
err = create_simple_http_error(404, 'notFound',
'Resource Not Found: userKey.')
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, 404)
self.assertEqual(reason, errors.ErrorReason.USER_NOT_FOUND.value)
self.assertEqual(message, 'Resource Not Found: userKey.')
def test_get_gapi_error_extracts_group_not_found(self):
err = create_simple_http_error(404, 'notFound',
'Resource Not Found: groupKey.')
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, 404)
self.assertEqual(reason, errors.ErrorReason.GROUP_NOT_FOUND.value)
self.assertEqual(message, 'Resource Not Found: groupKey.')
def test_get_gapi_error_extracts_member_not_found(self):
err = create_simple_http_error(404, 'notFound',
'Resource Not Found: memberKey.')
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, 404)
self.assertEqual(reason, errors.ErrorReason.MEMBER_NOT_FOUND.value)
self.assertEqual(message, 'Resource Not Found: memberKey.')
def test_get_gapi_error_extracts_domain_not_found(self):
err = create_simple_http_error(404, 'notFound', 'Domain not found.')
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, 404)
self.assertEqual(reason, errors.ErrorReason.DOMAIN_NOT_FOUND.value)
self.assertEqual(message, 'Domain not found.')
def test_get_gapi_error_extracts_generic_resource_not_found(self):
err = create_simple_http_error(404, 'notFound',
'Resource Not Found: unknownResource.')
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, 404)
self.assertEqual(reason, errors.ErrorReason.RESOURCE_NOT_FOUND.value)
self.assertEqual(message, 'Resource Not Found: unknownResource.')
def test_get_gapi_error_extracts_invalid_userid(self):
err = create_simple_http_error(400, 'invalid', 'Invalid Input: userId')
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, 400)
self.assertEqual(reason, errors.ErrorReason.USER_NOT_FOUND.value)
self.assertEqual(message, 'Invalid Input: userId')
def test_get_gapi_error_extracts_invalid_member(self):
err = create_simple_http_error(400, 'invalid', 'Invalid Input: memberKey')
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, 400)
self.assertEqual(reason, errors.ErrorReason.INVALID_MEMBER.value)
self.assertEqual(message, 'Invalid Input: memberKey')
def test_get_gapi_error_extracts_bad_request(self):
err = create_simple_http_error(400, 'failedPrecondition', 'Bad Request')
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, 400)
self.assertEqual(reason, errors.ErrorReason.BAD_REQUEST.value)
self.assertEqual(message, 'Bad Request')
def test_get_gapi_error_extracts_service_not_available(self):
err = create_simple_http_error(400, 'failedPrecondition',
'Mail service not enabled')
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, 400)
self.assertEqual(reason, errors.ErrorReason.SERVICE_NOT_AVAILABLE.value)
self.assertEqual(message, 'Mail service not enabled')
def test_get_gapi_error_extracts_required_member_not_found(self):
err = create_simple_http_error(400, 'required',
'Missing required field: memberKey')
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, 400)
self.assertEqual(reason, errors.ErrorReason.MEMBER_NOT_FOUND.value)
self.assertEqual(message, 'Missing required field: memberKey')
def test_get_gapi_error_extracts_cyclic_memberships_error(self):
err = create_simple_http_error(400, 'conditionNotMet',
'Cyclic memberships not allowed')
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, 400)
self.assertEqual(reason,
errors.ErrorReason.CYCLIC_MEMBERSHIPS_NOT_ALLOWED.value)
self.assertEqual(message, 'Cyclic memberships not allowed')
def test_get_gapi_error_extracts_single_error_with_message(self):
status_code = 999
response = {'status': status_code}
# This error does not have an "errors" key describing each error.
content = {'error': {'code': status_code, 'message': 'unknown error'}}
content_as_bytes = json.dumps(content).encode('UTF-8')
err = googleapiclient.errors.HttpError(response, content_as_bytes)
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, status_code)
self.assertEqual(reason, str(status_code))
self.assertEqual(message, content['error']['message'])
def test_get_gapi_error_exits_code_4_on_malformed_error_with_unknown_description(
self):
status_code = 999
response = {'status': status_code}
# This error only has an error_description_field and an unknown description.
content = {'error_description': 'something errored'}
content_as_bytes = json.dumps(content).encode('UTF-8')
err = googleapiclient.errors.HttpError(response, content_as_bytes)
with self.assertRaises(SystemExit) as context:
errors.get_gapi_error_detail(err)
self.assertEqual(4, context.exception.code)
def test_get_gapi_error_exits_on_invalid_error_description(self):
status_code = 400
response = {'status': status_code}
content = {'error_description': 'Invalid Value'}
content_as_bytes = json.dumps(content).encode('UTF-8')
err = googleapiclient.errors.HttpError(response, content_as_bytes)
http_status, reason, message = errors.get_gapi_error_detail(err)
self.assertEqual(http_status, status_code)
self.assertEqual(reason, errors.ErrorReason.INVALID.value)
self.assertEqual(message, 'Invalid Value')
def test_get_gapi_error_exits_code_4_on_unexpected_error_contents(self):
status_code = 900
response = {'status': status_code}
content = {'notErrorContentThatIsExpected': 'foo'}
content_as_bytes = json.dumps(content).encode('UTF-8')
err = googleapiclient.errors.HttpError(response, content_as_bytes)
with self.assertRaises(SystemExit) as context:
errors.get_gapi_error_detail(err)
self.assertEqual(4, context.exception.code)
if __name__ == '__main__':
unittest.main()

View File

@@ -1,23 +1,33 @@
# -*- mode: python -*-
a = Analysis(['gam.py'],
hiddenimports=[],
hookspath=None,
excludes=['_tkinter'],
runtime_hooks=None)
for d in a.datas:
if 'pyconfig' in d[0]:
a.datas.remove(d)
break
a.datas += [('httplib2/cacerts.txt', 'cacerts.txt', 'DATA')]
a.datas += [('cloudprint-v2.json', 'cloudprint-v2.json', 'DATA')]
pyz = PYZ(a.pure)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='gam',
debug=False,
strip=None,
upx=False,
console=True )
# -*- mode: python -*-
import sys
sys.modules['FixTk'] = None
a = Analysis(['gam.py'],
hiddenimports=[],
hookspath=None,
excludes=['FixTk', 'tcl', 'tk', '_tkinter', 'tkinter', 'Tkinter'],
runtime_hooks=None)
for d in a.datas:
if 'pyconfig' in d[0]:
a.datas.remove(d)
break
a.datas += [('cloudprint-v2.json', 'cloudprint-v2.json', 'DATA')]
# dynamically determine where httplib2/cacerts.txt lives
import importlib
proot = os.path.dirname(importlib.import_module('httplib2').__file__)
a.datas += [('httplib2/cacerts.txt', os.path.join(proot, 'cacerts.txt'), 'DATA')]
pyz = PYZ(a.pure)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='gam',
debug=False,
strip=None,
upx=False,
console=True )

View File

@@ -1,23 +1,32 @@
# -*- mode: python -*-
a = Analysis(['gam.py'],
hiddenimports=[],
hookspath=None,
excludes=['_tkinter'],
runtime_hooks=None)
for d in a.datas:
if 'pyconfig' in d[0]:
a.datas.remove(d)
break
a.datas += [('httplib2/cacerts.txt', 'cacerts.txt', 'DATA')]
a.datas += [('cloudprint-v2.json', 'cloudprint-v2.json', 'DATA')]
pyz = PYZ(a.pure)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='gam',
debug=False,
strip=None,
upx=False,
console=True )
# -*- mode: python -*-
import sys
sys.modules['FixTk'] = None
a = Analysis(['gam.py'],
hiddenimports=[],
hookspath=None,
excludes=['FixTk', 'tcl', 'tk', '_tkinter', 'tkinter', 'Tkinter'],
runtime_hooks=None)
for d in a.datas:
if 'pyconfig' in d[0]:
a.datas.remove(d)
break
a.datas += [('cloudprint-v2.json', 'cloudprint-v2.json', 'DATA')]
# dynamically determine where httplib2/cacerts.txt lives
import importlib
proot = os.path.dirname(importlib.import_module('httplib2').__file__)
a.datas += [('httplib2/cacerts.txt', os.path.join(proot, 'cacerts.txt'), 'DATA')]
pyz = PYZ(a.pure)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='gam',
debug=False,
strip=None,
upx=False,
console=True )

View File

@@ -7,6 +7,7 @@ contacts.googleapis.com
drive.googleapis.com
gmail.googleapis.com
groupssettings.googleapis.com
iam.googleapis.com
licensing.googleapis.com
plus.googleapis.com
reseller.googleapis.com

View File

@@ -1,7 +1,10 @@
cryptography
python-dateutil
dnspython
google-api-python-client
distro; sys_platform == 'linux'
filelock
google-api-python-client>=1.7.10
google-auth
google-auth-httplib2
google-oauth
oauth2client
passlib
google-auth-oauthlib>=0.4.1
httplib2>=0.13.0
passlib>=1.7.2

View File

@@ -1,6 +0,0 @@
#!/usr/bin/env bash
mypath=$(realpath $0)
export GAM_REAL_PATH=$(dirname $mypath)
staticx_binary="$mypath-staticx"
"$staticx_binary" "$@"

15
src/tools/a_atleast_b.py Executable file
View File

@@ -0,0 +1,15 @@
#!/usr/bin/env python3
#from packaging import version
from distutils.version import LooseVersion
import sys
a = sys.argv[1]
b = sys.argv[2]
#result = version.parse(a) >= version.parse(b)
result = LooseVersion(a) >= LooseVersion(b)
if result:
print('OK: %s is equal or newer than %s' % (a, b))
else:
print('ERROR: %s is older than %s' % (a, b))
sys.exit(not result)

102
src/tools/parse-aue.py Executable file
View File

@@ -0,0 +1,102 @@
#!/usr/bin/env python3
from xml.etree import ElementTree as ET
import requests
from html.parser import HTMLParser
import string
import sys
import json
import dateutil.parser
class MyHTMLParser(HTMLParser):
def handle_starttag(self, tag, attrs):
global next_data_is_oem, next_data_is_td
if tag == 'h2' and attrs == [('class', 'zippy')]:
next_data_is_oem = True
elif tag == 'td':
next_data_is_td = True
def handle_data(self, data):
global oem, next_data_is_oem, next_data_is_td, data_is_date, model, printable, output_rows
if next_data_is_oem:
oem = ''.join(filter(lambda x: x in printable, data))
next_data_is_oem = False
elif next_data_is_td:
if data_is_date:
if model.lower().startswith(oem.lower()):
fullname = model.lower()
else:
fullname = '%s %s' % (oem, model)
fullname = fullname.lower()
date = dateutil.parser.parse(data).replace(day=1).strftime('%Y-%m-%dT00:00:00.000Z')
output_rows[fullname] = date
if fullname in exceptions:
for value in exceptions[fullname]:
output_rows[value] = date
data_is_date = False
else:
model = ''.join(filter(lambda x: x in printable, data))
data_is_date = True
next_data_is_td = False
global oem, next_data_is_oem, next_data_is_td, data_is_date, model, printable, exceptions, output_rows
output_rows = {}
printable = set(string.printable)
exceptions = {
# 'AUE OEM MODEL': ['API MODEL 1', ...]
'acer c7 chromebook': ['acer c7 chromebook (c710)'],
'acer chromebook 11 (c720, c720p)': ['acer c720 chromebook', 'acer c740 chromebook'],
'acer chromebook 11 (cb3-111, c730, c730e)': ['chromebook 11 (c730 / cb3-111)'],
'acer chromebook 11 (cb3-131, c735)': ['chromebook 11 (c735)'],
'acer chromebook 15 (cb515-1h,cb515-1ht)': ['chromebook 15 (cb515 - 1ht / 1h)'],
'acer chromebook 13(cb5-311, c810)': ['acer chromebook 13 (cb5-311)'],
'acer chromebook 15 (cb5-571, c910)': ['acer chromebook 15 (c910 / cb5-571)'],
'acer chromebook 311 (c721, c733, c733u, c733t)': ['acer chromebook 311', 'chromebook 311 (c721)'],
'acer chromebook 315 (cb315-2h)': ['acer chromebook 315'],
'acer chromebook spin 311 (r721t)': ['acer chromebook 311'],
'acer chromebook spin 511 (r752t, r752tn)': ['acer chromebook spin 511'],
'acer chromebox cxi2 / cxv2': ['acer chromebox cxi2'],
'asus chromebook c200': ['asus chromebook c200ma'],
'asus chromebook c201pa': ['asus chromebook c201pa'],
'asus chromebook c204': ['asus chromebook c204'],
'asus chromebook c300': ['asus chromebook c300ma'],
'asus chromebook flip c213': ['asus chromebook c213na'],
'asus chromebox 2 (cn62)': ['asus chromebox cn62'],
'asus chromebox 3 (cn65)': ['asus chromebox 3'],
'asus chromebox (cn60)': ['asus chromebox cn60'],
'ctl chromebook nl7 / nl7t-360 / nl7tw-360': ['ctl chromebook nl7'],
'ctl chromebook tablet tx1 for education': ['ctl chromebook tab tx1'],
'ctl nl61 chromebook': ['mecer v2 chromebook'],
'google cr-48': ['cr-48'],
'haier chromebook 11e': ['chromebook pcm-116e', 'lumos education chromebook'],
'haier chromebook 11': ['true idc chromebook 11', 'xolo chromebook'],
'hisense chromebook 11': ['epik 11.6" chromebook elb1101', 'mecer chromebook', 'videonet chromebook bl10'],
'hp chromebook 11 g1': ['hp chromebook 11 1100-1199 / hp chromebook 11 g1'],
'hp chromebook 11 g2': ['hp chromebook 11 2000-2099 / hp chromebook 11 g2'],
'hp chromebook 11 g3': ['hp chromebook 11 2100-2199 / hp chromebook 11 g3'],
'hp chromebook 11 g4/g4 ee': ['hp chromebook 11 2200-2299 / hp chromebook 11 g4/g4 ee'],
'hp chromebook 11 g5': ['hp chromebook 11 g5 / hp chromebook 11-vxxx'],
'hp chromebook 14a g5': ['hp chromebook 14 db0000-db0999'],
'hp chromebook 14 g3': ['hp chromebook 14 x000-x999 / hp chromebook 14 g3'],
'hp chromebook 14 g4': ['hp chromebook 14 ak000-099 / hp chromebook 14 g4'],
'hp chromebook 14 g5': ['hp chromebook 14 / hp chromebook 14 g5'],
'hp chromebox g1': ['hp chromebox cb1-(000-099) / hp chromebox g1/ hp chromebox for meetings'],
'lenovo ideapad c330 chromebook': ['lenovo chromebook c330'],
'lenovo ideapad s330 chromebook': ['lenovo chromebook s330'],
'lenovo n21 chromebook': ['asi chromebook', 'crambo chromebook', 'jp sa couto chromebook', 'rgs education chromebook', 'true idc chromebook', 'videonet chromebook', 'consumer chromebook'],
'lenovo thinkpad 11e 3rd gen chromebook': ['thinkpad 11e chromebook 3rd gen (yoga/clamshell)'],
'lenovo thinkpad 11e 4th gen chromebook': ['lenovo thinkpad 11e chromebook (4th gen)/lenovo thinkpad yoga 11e chromebook (4th gen)'],
'lenovo thinkpad 13': ['thinkpad 13 chromebook'],
'poin2 chromebook 14': ['poin2 chromebook 11c'],
'prowise chromebook eduline': ['viglen chromebook 11c'],
'prowise chromebook entryline': ['prowise 11.6\" entry line chromebook'],
'prowise chromebook proline': ['prowise proline chromebook'],
'samsung chromebook - xe303': ['samsung chromebook'],
}
next_data_is_oem = False
next_data_is_td = False
data_is_date = False
auepage = requests.get('https://support.google.com/chrome/a/answer/6220366?hl=en')
parser = MyHTMLParser()
parser.feed(auepage.content.decode('utf-8'))
print(json.dumps(output_rows, indent=2, sort_keys=True))

View File

@@ -0,0 +1,12 @@
{
"_class": "OAuth2Credentials",
"_module": "oauth2client.client",
"access_token": "",
"client_id": "118850122376-72t6r2666n5rbjlfebftqat5qjai2def.apps.googleusercontent.com",
"client_secret": "",
"invalid": false,
"refresh_token": "",
"token_expiry": "2010-04-17T15:18:45Z",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"user_agent": ""
}

View File

@@ -0,0 +1,73 @@
export whereibelong=$(pwd)
export dist=$(lsb_release --codename --short)
echo "We are running on Ubuntu $dist"
export LD_LIBRARY_PATH=~/ssl/lib:~/python/lib
cpucount=$(nproc --all)
echo "This device has $cpucount CPUs for compiling..."
sudo apt-get -qq --yes update > /dev/null
sudo apt-get -qq --yes install xz-utils > /dev/null
SSLVER=$(~/ssl/bin/openssl version)
SSLRESULT=$?
PYVER=$(~/python/bin/python3 -V)
PYRESULT=$?
if [ $SSLRESULT -ne 0 ] || [[ "$SSLVER" != "OpenSSL $BUILD_OPENSSL_VERSION "* ]] || [ $PYRESULT -ne 0 ] || [[ "$PYVER" != "Python $BUILD_PYTHON_VERSION"* ]]; then
echo "RUNNING: apt dist-upgrade..."
sudo apt-get -qq --yes dist-upgrade > /dev/null
echo "Installing build tools..."
sudo apt-get -qq --yes install build-essential
echo "Installing deps for python3"
sudo cp -v /etc/apt/sources.list /tmp
sudo chmod a+rwx /tmp/sources.list
echo "deb-src http://archive.ubuntu.com/ubuntu/ $dist main" >> /tmp/sources.list
sudo cp -v /tmp/sources.list /etc/apt
sudo apt-get -qq --yes update > /dev/null
sudo apt-get -qq --yes build-dep python3 > /dev/null
sudo apt-get -qq --yes install zlib1g-dev > /dev/null
sudo apt-get -qq --yes install libffi-dev > /dev/null
# Compile latest OpenSSL
echo "Downloading OpenSSL..."
wget --quiet https://www.openssl.org/source/openssl-$BUILD_OPENSSL_VERSION.tar.gz
echo "Extracting OpenSSL..."
tar xf openssl-$BUILD_OPENSSL_VERSION.tar.gz
cd openssl-$BUILD_OPENSSL_VERSION
echo "Compiling OpenSSL $BUILD_OPENSSL_VERSION..."
./config shared --prefix=$HOME/ssl
echo "Running make for OpenSSL..."
make -j$cpucount -s
echo "Running make install for OpenSSL..."
make install > /dev/null
cd ~
# Compile latest Python
echo "Downloading Python $BUILD_PYTHON_VERSION..."
curl -O https://www.python.org/ftp/python/$BUILD_PYTHON_VERSION/Python-$BUILD_PYTHON_VERSION.tar.xz
echo "Extracting Python..."
tar xf Python-$BUILD_PYTHON_VERSION.tar.xz
cd Python-$BUILD_PYTHON_VERSION
echo "Compiling Python $BUILD_PYTHON_VERSION..."
safe_flags="--with-openssl=$HOME/ssl --enable-shared --prefix=$HOME/python --with-ensurepip=upgrade"
unsafe_flags="--enable-optimizations --with-lto"
echo "running configure with safe and unsafe"
./configure $safe_flags $unsafe_flags > /dev/null
make -j$cpucount PROFILE_TASK="-m test.regrtest --pgo -j$(( $cpucount * 2 ))" -s
echo "Installing Python..."
make install > /dev/null
fi
python=~/python/bin/python3
pip=~/python/bin/pip3
$python -V
cd $whereibelong
echo "Upgrading pip packages..."
$pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 $pip install -U
$pip install --upgrade -r src/requirements.txt
$pip install --upgrade https://github.com/pyinstaller/pyinstaller/archive/develop.tar.gz
mkdir ~/.ruby
export GEM_HOME=~/.ruby
export PATH=$PATH:~/.ruby/bin

View File

@@ -0,0 +1,33 @@
cd src
if [ "$VMTYPE" == "test" ]; then
export gam="$python gam.py"
export gampath=$(readlink -e .)
else
$python -OO -m PyInstaller --clean --noupx --strip -F --distpath=gam $GAMOS-gam.spec
export gam="gam/gam"
export gampath=$(readlink -e gam)
export GAMVERSION=`$gam version simple`
cp LICENSE $gampath
cp whatsnew.txt $gampath
cp GamCommands.txt $gampath
this_glibc_ver=$(ldd --version | awk '/ldd/{print $NF}')
GAM_ARCHIVE=gam-$GAMVERSION-$GAMOS-$PLATFORM-glibc$this_glibc_ver.tar.xz
rm $gampath/lastupdatecheck.txt
tar cfJ $GAM_ARCHIVE gam/
echo "PyInstaller GAM info:"
du -h gam/gam
time $gam version extended
if [[ "$dist" == "precise" ]]; then
GAM_LEGACY_ARCHIVE=gam-$GAMVERSION-$GAMOS-$PLATFORM-legacy.tar.xz
$python -OO -m staticx gam/gam gam/gam-staticx
strip gam/gam-staticx
rm gam/gam
mv gam/gam-staticx gam/gam
tar cfJ $GAM_LEGACY_ARCHIVE gam/
echo "Legacy StaticX GAM info:"
du -h gam/gam
time $gam version extended
fi
fi

View File

@@ -1,12 +1,115 @@
echo "RUNNING: apt update..."
sudo apt-get --yes update > /dev/null
echo "RUNNING: apt dist-upgrade..."
sudo apt-get --yes dist-upgrade > /dev/null
echo "Installing StaticX deps..."
sudo apt-get --yes install binutils patchelf
if [ "$VMTYPE" == "test" ]; then
export python="python"
export pip="pip"
echo "Travis setup Python $TRAVIS_PYTHON_VERSION"
echo "running tests with this version"
else
export whereibelong=$(pwd)
export dist=$(lsb_release --codename --short)
echo "We are running on Ubuntu $dist"
export LD_LIBRARY_PATH=~/ssl/lib:~/python/lib
cpucount=$(nproc --all)
echo "This device has $cpucount CPUs for compiling..."
SSLVER=$(~/ssl/bin/openssl version)
SSLRESULT=$?
PYVER=$(~/python/bin/python3 -V)
PYRESULT=$?
if [ $SSLRESULT -ne 0 ] || [[ "$SSLVER" != "OpenSSL $BUILD_OPENSSL_VERSION "* ]] || [ $PYRESULT -ne 0 ] || [[ "$PYVER" != "Python $BUILD_PYTHON_VERSION"* ]]; then
echo "SSL Result: $SSLRESULT - SSL Ver: $SSLVER - Py Result: $PYRESULT - Py Ver: $PYVER"
if [ $SSLRESULT -ne 0 ]; then
echo "sslresult -ne 0"
fi
if [[ "$SSLVER" != "OpenSSL $BUILD_OPENSSL_VERSION "* ]]; then
echo "sslver not equal to..."
fi
if [ $PYRESULT -ne 0 ]; then
echo "pyresult -ne 0"
fi
if [[ "$PYVER" != "Python $BUILD_PYTHON_VERSION" ]]; then
echo "pyver not equal to..."
fi
cd ~
rm -rf ssl
rm -rf python
mkdir ssl
mkdir python
echo "RUNNING: apt update..."
sudo apt-get -qq --yes update > /dev/null
echo "RUNNING: apt dist-upgrade..."
sudo apt-get -qq --yes dist-upgrade > /dev/null
echo "Installing build tools..."
sudo apt-get -qq --yes install build-essential
echo "Installing deps for python3"
sudo cp -v /etc/apt/sources.list /tmp
sudo chmod a+rwx /tmp/sources.list
echo "deb-src http://archive.ubuntu.com/ubuntu/ $dist main" >> /tmp/sources.list
sudo cp -v /tmp/sources.list /etc/apt
sudo apt-get -qq --yes update > /dev/null
sudo apt-get -qq --yes build-dep python3 > /dev/null
# Compile latest OpenSSL
wget --quiet https://www.openssl.org/source/openssl-$BUILD_OPENSSL_VERSION.tar.gz
echo "Extracting OpenSSL..."
tar xf openssl-$BUILD_OPENSSL_VERSION.tar.gz
cd openssl-$BUILD_OPENSSL_VERSION
echo "Compiling OpenSSL $BUILD_OPENSSL_VERSION..."
./config shared --prefix=$HOME/ssl
echo "Running make for OpenSSL..."
make -j$cpucount -s
echo "Running make install for OpenSSL..."
make install > /dev/null
cd ~
# Compile latest Python
echo "Downloading Python $BUILD_PYTHON_VERSION..."
curl -O https://www.python.org/ftp/python/$BUILD_PYTHON_VERSION/Python-$BUILD_PYTHON_VERSION.tar.xz
echo "Extracting Python..."
tar xf Python-$BUILD_PYTHON_VERSION.tar.xz
cd Python-$BUILD_PYTHON_VERSION
echo "Compiling Python $BUILD_PYTHON_VERSION..."
safe_flags="--with-openssl=$HOME/ssl --enable-shared --prefix=$HOME/python --with-ensurepip=upgrade"
unsafe_flags="--enable-optimizations --with-lto"
if [ ! -e Makefile ]; then
echo "running configure with safe and unsafe"
./configure $safe_flags $unsafe_flags > /dev/null
fi
make -j$cpucount PROFILE_TASK="-m test.regrtest --pgo -j$(( $cpucount * 2 ))" -s
RESULT=$?
echo "First make exited with $RESULT"
if [ $RESULT != 0 ]; then
echo "Trying Python compile again without unsafe flags..."
make clean
./configure $safe_flags > /dev/null
make -j$cpucount -s
echo "Sticking with safe Python for now..."
fi
echo "Installing Python..."
make install > /dev/null
cd ~
fi
python=~/python/bin/python3
pip=~/python/bin/pip3
if [[ "$dist" == "precise" ]]; then
echo "Installing deps for StaticX..."
sudo apt-get install --yes scons
if [ ! -d patchelf-$PATCHELF_VERSION ]; then
echo "Downloading PatchELF $PATCHELF_VERSION"
wget https://nixos.org/releases/patchelf/patchelf-$PATCHELF_VERSION/patchelf-$PATCHELF_VERSION.tar.bz2
tar xf patchelf-$PATCHELF_VERSION.tar.bz2
cd patchelf-$PATCHELF_VERSION
./configure
make
sudo make install
fi
$pip install git+https://github.com/JonathonReinhart/staticx.git@master
fi
cd $whereibelong
fi
echo "Upgrading pip packages..."
pip freeze > upgrades.txt
pip install --upgrade -r upgrades.txt
pip install -r src/requirements.txt
pip install pyinstaller
pip install staticx
$pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 $pip install -U
$pip install --upgrade -r src/requirements.txt
$pip install --upgrade https://github.com/pyinstaller/pyinstaller/archive/develop.tar.gz

View File

@@ -1,22 +1,32 @@
cd src
pyinstaller --clean --debug -F --distpath=gam $GAMOS-gam.spec
gam/gam version
export GAMVERSION=`gam/gam version simple`
cp LICENSE gam
cp whatsnew.txt gam
cp GamCommands.txt gam
GAM_ARCHIVE=gam-$GAMVERSION-$GAMOS-$PLATFORM.tar.xz
tar cfJ $GAM_ARCHIVE gam/
echo "PyInstaller GAM info:"
du -h gam/gam
time gam/gam version
if [ "$VMTYPE" == "test" ]; then
export gam="$python gam.py"
export gampath=$(readlink -e .)
else
$python -OO -m PyInstaller --clean --noupx --strip -F --distpath=gam $GAMOS-gam.spec
export gam="gam/gam"
export gampath=$(readlink -e gam)
export GAMVERSION=`$gam version simple`
cp LICENSE $gampath
cp whatsnew.txt $gampath
cp GamCommands.txt $gampath
this_glibc_ver=$(ldd --version | awk '/ldd/{print $NF}')
GAM_ARCHIVE=gam-$GAMVERSION-$GAMOS-$PLATFORM-glibc$this_glibc_ver.tar.xz
rm $gampath/lastupdatecheck.txt
tar cfJ $GAM_ARCHIVE gam/
echo "PyInstaller GAM info:"
du -h gam/gam
time $gam version extended
GAM_LEGACY_ARCHIVE=gam-$GAMVERSION-$GAMOS-$PLATFORM-legacy.tar.xz
staticx gam/gam gam/gam-staticx
strip gam/gam-staticx
rm gam/gam
cp staticx-gam.sh gam/gam
tar cfJ $GAM_LEGACY_ARCHIVE gam/
echo "Legacy StaticX GAM info:"
du -h gam/gam-staticx
time gam/gam version
if [[ "$dist" == "precise" ]]; then
GAM_LEGACY_ARCHIVE=gam-$GAMVERSION-$GAMOS-$PLATFORM-legacy.tar.xz
$python -OO -m staticx gam/gam gam/gam-staticx
strip gam/gam-staticx
rm gam/gam
mv gam/gam-staticx gam/gam
tar cfJ $GAM_LEGACY_ARCHIVE gam/
echo "Legacy StaticX GAM info:"
du -h gam/gam
time $gam version extended
fi
fi

Binary file not shown.

View File

@@ -1,8 +1,78 @@
brew update
brew upgrade python3
mypath=$HOME
whereibelong=$(pwd)
#echo "Brew installing xz..."
#brew install xz > /dev/null
#brew upgrade
cd ~
if [ ! -f python-$BUILD_PYTHON_VERSION-macosx10.9.pkg ]; then
wget --quiet https://www.python.org/ftp/python/$BUILD_PYTHON_VERSION/python-$BUILD_PYTHON_VERSION-macosx10.9.pkg
fi
sudo installer -pkg python-$BUILD_PYTHON_VERSION-macosx10.9.pkg -target /
#brew install openssl@1.1
#brew upgrade python
export python=python3
export pip=pip3
echo "Python location:"
which $python
# Compile latest OpenSSL
#if [ ! -d openssl-$BUILD_OPENSSL_VERSION ]; then
# wget --quiet https://www.openssl.org/source/openssl-$BUILD_OPENSSL_VERSION.tar.gz
# echo "Extracting OpenSSL..."
# tar xf openssl-$BUILD_OPENSSL_VERSION.tar.gz
#fi
#cd openssl-$BUILD_OPENSSL_VERSION
#echo "Compiling OpenSSL $BUILD_OPENSSL_VERSION..."
#./config shared --prefix=$mypath/ssl
#echo "Running make for OpenSSL..."
#make -j$cpucount -s
#echo "Running make install for OpenSSL..."
#make install > /dev/null
#export LD_LIBRARY_PATH=~/ssl/lib
#cd ~
# Compile latest Python
#if [ ! -d Python-$BUILD_PYTHON_VERSION ]; then
# wget --quiet https://www.python.org/ftp/python/$BUILD_PYTHON_VERSION/Python-$BUILD_PYTHON_VERSION.tar.xz
# echo "Extracting Python..."
# tar xf Python-$BUILD_PYTHON_VERSION.tar.xz
#fi
#cd Python-$BUILD_PYTHON_VERSION
#echo "Compiling Python $BUILD_PYTHON_VERSION..."
#safe_flags="--with-openssl=$mypath/ssl --enable-shared --prefix=$mypath/python --with-ensurepip=upgrade"
#unsafe_flags="--enable-optimizations --with-lto"
#if [ ! -e Makefile ]; then
# ./configure $safe_flags $unsafe_flags > /dev/null
#fi
#make -j$cpucount -s
#RESULT=$?
#echo "Make Python exited with $RESULT"
#if [ $RESULT != 0 ]; then
# echo "Trying Python make again without unsafe flags..."
# make clean
# ./configure $safe_flags > /dev/null
# make -j$cpucount -s
#fi
#echo "Installing Python..."
#make install > /dev/null
#cd ~
#export LD_LIBRARY_PATH=~/ssl/lib:~/python/lib
#python=~/python/bin/python3
#pip=~/python/bin/pip3
$python -V
cd $whereibelong
export PATH=/usr/local/opt/python/libexec/bin:$PATH
pip install --upgrade pip
pip freeze > upgrades.txt
pip install --upgrade -r upgrades.txt
pip install -r src/requirements.txt
pip install pyinstaller
$pip install --upgrade pip
$pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 $pip install -U
$pip install --upgrade -r src/requirements.txt
$pip install --upgrade https://github.com/pyinstaller/pyinstaller/archive/develop.tar.gz

View File

@@ -1,9 +1,15 @@
cd src
pyinstaller --clean --debug -F --distpath=gam $GAMOS-gam.spec
gam/gam version
echo "MacOS Version Info According to Python:"
python -c "import platform; print(platform.mac_ver())"
$python -OO -m PyInstaller --clean --noupx --strip -F --distpath=gam $GAMOS-gam.spec
export gam="gam/gam"
export gampath=gam
$gam version extended
export GAMVERSION=`gam/gam version simple`
cp LICENSE gam
cp whatsnew.txt gam
cp GamCommands.txt gam
GAM_ARCHIVE=gam-$GAMVERSION-$GAMOS-$PLATFORM.tar.xz
MACOSVERSION=$(defaults read loginwindow SystemVersionStampAsString)
GAM_ARCHIVE=gam-$GAMVERSION-$GAMOS-$PLATFORM-MacOS$MACOSVERSION.tar.xz
rm gam/lastupdatecheck.txt
tar cfJ $GAM_ARCHIVE gam/

View File

@@ -0,0 +1 @@
export GAM_CSV_ROW_FILTER='{"type": "regex:^GOOGLE"}'

15
src/travis/svars-write.py Normal file
View File

@@ -0,0 +1,15 @@
import sys
import json
import os
cfg = json.load(sys.stdin)
cfg['client_secret'] = os.getenv('client_secret')
jid = os.getenv('jid')
cfg['refresh_token'] = os.getenv('refresh_%s' % jid)
vmtype = os.getenv('VMTYPE')
if vmtype == 'test':
out_file = 'oauth2.txt'
else:
out_file = 'gam/oauth2.txt'
with open(out_file, 'w') as f:
json.dump(cfg, f)

View File

@@ -1,9 +1,34 @@
powershell Install-WindowsFeature Net-Framework-Core
echo "Installing Net-Framework-Core..."
export mypath=$(pwd)
cd ~
until powershell Install-WindowsFeature Net-Framework-Core; do echo "trying again..."; done
cinst -y --forcex86 python3
export PATH=$PATH:/c/Python37/scripts
cinst -y wixtoolset
pip install --upgrade pip
pip freeze > upgrades.txt
pip install --upgrade -r upgrades.txt
pip install -r src/requirements.txt
pip install pyinstaller
until cinst -y wixtoolset; do echo "trying again..."; done
export PATH=$PATH:/c/Python38/scripts
cd $mypath
export python=/c/Python38/python.exe
export pip=/c/Python38/scripts/pip.exe
$pip install --upgrade pip
$pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 $pip install -U
$pip install --upgrade -r src/requirements.txt
#$pip install --upgrade pyinstaller
# Install PyInstaller from source and build bootloader
# to try and avoid getting flagged as malware since
# lots of malware uses PyInstaller default bootloader
# https://stackoverflow.com/questions/53584395/how-to-recompile-the-bootloader-of-pyinstaller
echo "Downloading PyInstaller..."
wget --quiet https://github.com/pyinstaller/pyinstaller/archive/develop.tar.gz
tar xf develop.tar.gz
cd pyinstaller-develop/bootloader
echo "bootloader before:"
md5sum ../PyInstaller/bootloader/Windows-32bit/*
$python ./waf all --target-arch=32bit
echo "bootloader after:"
md5sum ../PyInstaller/bootloader/Windows-32bit/*
echo "PATH: $PATH"
cd ..
$python setup.py install
echo "cd to $mypath"
cd $mypath

View File

@@ -1,7 +1,10 @@
cd src
pyinstaller --clean -F --distpath=gam $GAMOS-gam.spec
gam/gam version
export GAMVERSION=`gam/gam version simple`
pyinstaller --clean --noupx -F --distpath=gam $GAMOS-gam.spec
export gam="gam/gam"
export gampath=$(readlink -e gam)
$gam version extended
export GAMVERSION=`$gam version simple`
rm gam/lastupdatecheck.txt
cp LICENSE gam
cp GamCommands.txt gam
cp whatsnew.txt gam

View File

@@ -1,9 +1,46 @@
powershell Install-WindowsFeature Net-Framework-Core
cinst -y $CINST_ARGS python3
export PATH=$PATH:/c/Python37/scripts
cinst -y wixtoolset
pip install --upgrade pip
pip freeze > upgrades.txt
pip install --upgrade -r upgrades.txt
pip install -r src/requirements.txt
pip install pyinstaller
echo "Installing Net-Framework-Core..."
export mypath=$(pwd)
until powershell Install-WindowsFeature Net-Framework-Core; do echo "trying again..."; done
cd ~
#export exefile=Win64OpenSSL_Light-${BUILD_OPENSSL_VERSION//./_}.exe
#if [ ! -e $exefile ]; then
# echo "Downloading $exefile..."
# wget --quiet https://slproweb.com/download/$exefile
#fi
#echo "Installing $exefile..."
#powershell ".\\${exefile} /silent /sp- /suppressmsgboxes /DIR=C:\\ssl"
cinst -y python3
until cinst -y wixtoolset; do echo "trying again..."; done
#until cp -v /c/ssl/libcrypto-1_1-x64.dll /c/Python37/DLLs/libcrypto-1_1.dll; do echo "trying again..."; done
#until cp -v /c/ssl/libssl-1_1-x64.dll /c/Python37/DLLs/libssl-1_1.dll; do echo "trying again..."; done
export PATH=$PATH:/c/Python38/scripts
cd $mypath
export python=/c/Python38/python.exe
export pip=/c/Python38/scripts/pip.exe
$pip install --upgrade pip
$pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 $pip install -U
$pip install --upgrade -r src/requirements.txt
#$pip install --upgrade pyinstaller
# Install PyInstaller from source and build bootloader
# to try and avoid getting flagged as malware since
# lots of malware uses PyInstaller default bootloader
# https://stackoverflow.com/questions/53584395/how-to-recompile-the-bootloader-of-pyinstaller
echo "Downloading PyInstaller..."
#wget --quiet https://github.com/pyinstaller/pyinstaller/releases/download/v$PYINSTALLER_VERSION/PyInstaller-$PYINSTALLER_VERSION.tar.gz
wget --quiet https://github.com/pyinstaller/pyinstaller/archive/develop.tar.gz
#tar xf PyInstaller-$PYINSTALLER_VERSION.tar.gz
tar xf develop.tar.gz
#cd PyInstaller-$PYINSTALLER_VERSION/bootloader
cd pyinstaller-develop/bootloader
echo "bootloader before:"
md5sum ../PyInstaller/bootloader/Windows-64bit/*
$python ./waf all --target-arch=64bit
echo "bootloader after:"
md5sum ../PyInstaller/bootloader/Windows-64bit/*
echo "PATH: $PATH"
cd ..
$python setup.py install
echo "cd to $mypath..."
#until cp -v /c/ssl/*.dll /c/Python37/DLLs; do echo "trying again..."; done
cd $mypath

View File

@@ -1,7 +1,12 @@
cd src
pyinstaller --clean -F --distpath=gam $GAMOS-gam.spec
gam/gam version
export GAMVERSION=`gam/gam version simple`
echo "compiling GAM with pyinstaller..."
pyinstaller --clean --noupx -F --distpath=gam $GAMOS-gam.spec
export gam="gam/gam"
export gampath=$(readlink -e gam)
echo "running compiled GAM..."
$gam version
export GAMVERSION=`$gam version simple`
rm gam/lastupdatecheck.txt
cp LICENSE gam
cp GamCommands.txt gam
cp whatsnew.txt gam

View File

@@ -1,24 +1,20 @@
import collections
import datetime
import re
import sys
from html.entities import name2codepoint
from html.parser import HTMLParser
from var import GM_Globals, GM_WINDOWS, GM_SYS_ENCODING, ONE_KILO_BYTES, ONE_MEGA_BYTES, ONE_GIGA_BYTES
ONE_KILO_BYTES = 1000
ONE_MEGA_BYTES = 1000000
ONE_GIGA_BYTES = 1000000000
def convertUTF8(data):
if isinstance(data, str):
return data
if isinstance(data, str):
if GM_Globals[GM_WINDOWS]:
return data
return data.encode(GM_Globals[GM_SYS_ENCODING])
if isinstance(data, collections.Mapping):
return dict(list(map(convertUTF8, iter(data.items()))))
if isinstance(data, collections.Iterable):
return type(data)(list(map(convertUTF8, data)))
return data
class _DeHTMLParser(HTMLParser):
def __init__(self):
HTMLParser.__init__(self)
self.__text = []
@@ -34,7 +30,7 @@ class _DeHTMLParser(HTMLParser):
if cp:
self.__text.append(chr(cp))
else:
self.__text.append('&'+name)
self.__text.append('&' + name)
def handle_starttag(self, tag, attrs):
if tag == 'p':
@@ -57,12 +53,14 @@ class _DeHTMLParser(HTMLParser):
self.__text.append('\n\n')
def text(self):
return re.sub(r'\n{2}\n+', '\n\n', re.sub(r'\n +', '\n', ''.join(self.__text))).strip()
return re.sub(r'\n{2}\n+', '\n\n',
re.sub(r'\n +', '\n', ''.join(self.__text))).strip()
def dehtml(text):
try:
parser = _DeHTMLParser()
parser.feed(text.encode('utf-8'))
parser.feed(str(text))
parser.close()
return parser.text()
except:
@@ -70,8 +68,20 @@ def dehtml(text):
print_exc(file=sys.stderr)
return text
def indentMultiLineText(message, n=0):
return message.replace('\n', '\n{0}'.format(' '*n)).rstrip()
return message.replace('\n', '\n{0}'.format(' ' * n)).rstrip()
def formatTimestampYMD(timestamp):
return datetime.datetime.fromtimestamp(int(timestamp)/1000).strftime('%Y-%m-%d')
def formatTimestampYMDHMS(timestamp):
return datetime.datetime.fromtimestamp(int(timestamp)/1000).strftime('%Y-%m-%d %H:%M:%S')
def formatTimestampYMDHMSF(timestamp):
return str(datetime.datetime.fromtimestamp(int(timestamp)/1000))
def formatFileSize(fileSize):
if fileSize == 0:
@@ -79,10 +89,11 @@ def formatFileSize(fileSize):
if fileSize < ONE_KILO_BYTES:
return '1kb'
if fileSize < ONE_MEGA_BYTES:
return '{0}kb'.format(fileSize/ONE_KILO_BYTES)
return '{0}kb'.format(fileSize // ONE_KILO_BYTES)
if fileSize < ONE_GIGA_BYTES:
return '{0}mb'.format(fileSize/ONE_MEGA_BYTES)
return '{0}gb'.format(fileSize/ONE_GIGA_BYTES)
return '{0}mb'.format(fileSize // ONE_MEGA_BYTES)
return '{0}gb'.format(fileSize // ONE_GIGA_BYTES)
def formatMilliSeconds(millis):
seconds, millis = divmod(millis, 1000)

View File

@@ -1,27 +1,27 @@
import os
import ssl
import string
import sys
import platform
import re
gam_author = 'Jay Lee <jay0lee@gmail.com>'
gam_version = '4.80'
gam_version = '4.97'
gam_license = 'Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)'
GAM_URL = 'https://git.io/gam'
GAM_INFO = 'GAM {0} - {1} / {2} / Python {3}.{4}.{5} {6} / {7} {8} /'.format(gam_version, GAM_URL,
gam_author,
sys.version_info[0], sys.version_info[1],
sys.version_info[2], sys.version_info[3],
platform.platform(), platform.machine())
gam_author,
sys.version_info[0], sys.version_info[1],
sys.version_info[2], sys.version_info[3],
platform.platform(), platform.machine())
GAM_RELEASES = 'https://github.com/jay0lee/GAM/releases'
GAM_WIKI = 'https://github.com/jay0lee/GAM/wiki'
GAM_ALL_RELEASES = 'https://api.github.com/repos/jay0lee/GAM/releases'
GAM_LATEST_RELEASE = GAM_ALL_RELEASES+'/latest'
GAM_PROJECT_APIS = 'https://raw.githubusercontent.com/jay0lee/GAM/master/src/project-apis.txt'
GAM_PROJECT_FILEPATH = 'https://raw.githubusercontent.com/jay0lee/GAM/master/'
TRUE = 'true'
FALSE = 'false'
true_values = ['on', 'yes', 'enabled', 'true', '1']
false_values = ['off', 'no', 'disabled', 'false', '0']
usergroup_types = ['user', 'users',
@@ -30,22 +30,17 @@ usergroup_types = ['user', 'users',
'ou_and_children', 'ou_and_child', 'ou_and_children_ns', 'ou_and_child_ns', 'ou_and_children_susp', 'ou_and_child_susp',
'query', 'queries', 'license', 'licenses', 'licence', 'licences', 'file', 'csv', 'csvfile', 'all',
'cros', 'cros_sn', 'crosquery', 'crosqueries', 'crosfile', 'croscsv', 'croscsvfile']
ERROR = 'ERROR'
ERROR_PREFIX = ERROR+': '
WARNING = 'WARNING'
WARNING_PREFIX = WARNING+': '
DEFAULT_CHARSET = ['mbcs', 'utf-8'][os.name != 'nt']
ONE_KILO_BYTES = 1000
ONE_MEGA_BYTES = 1000000
ONE_GIGA_BYTES = 1000000000
FN_CLIENT_SECRETS_JSON = 'client_secrets.json'
ERROR_PREFIX = 'ERROR: '
WARNING_PREFIX = 'WARNING: '
UTF8 = 'utf-8'
UTF8_SIG = 'utf-8-sig'
FN_EXTRA_ARGS_TXT = 'extra-args.txt'
FN_LAST_UPDATE_CHECK_TXT = 'lastupdatecheck.txt'
FN_OAUTH2SERVICE_JSON = 'oauth2service.json'
FN_OAUTH2_TXT = 'oauth2.txt'
MY_CUSTOMER = 'my_customer'
# See https://support.google.com/drive/answer/37603
MAX_GOOGLE_SHEET_CELLS = 5000000
MAX_LOCAL_GOOGLE_TIME_OFFSET = 30
SKUS = {
'1010010001': {
'product': '101001', 'aliases': ['identity', 'cloudidentity'], 'displayName': 'Cloud Identity'},
@@ -55,6 +50,12 @@ SKUS = {
'product': '101031', 'aliases': ['gsefe', 'e4e', 'gsuiteenterpriseeducation'], 'displayName': 'G Suite Enterprise for Education'},
'1010310003': {
'product': '101031', 'aliases': ['gsefes', 'e4es', 'gsuiteenterpriseeducationstudent'], 'displayName': 'G Suite Enterprise for Education Student'},
'1010330003': {
'product': '101033', 'aliases': ['gvstarter', 'voicestarter', 'googlevoicestarter'], 'displayName': 'Google Voice Starter'},
'1010330004': {
'product': '101033', 'aliases': ['gvstandard', 'voicestandard', 'googlevoicestandard'], 'displayName': 'Google Voice Standard'},
'1010330002': {
'product': '101033', 'aliases': ['gvpremier', 'voicepremier', 'googlevoicepremier'], 'displayName': 'Google Voice Premier'},
'Google-Apps': {
'product': 'Google-Apps', 'aliases': ['standard', 'free'], 'displayName': 'G Suite Free/Standard'},
'Google-Apps-For-Business': {
@@ -64,11 +65,15 @@ SKUS = {
'Google-Apps-For-Postini': {
'product': 'Google-Apps', 'aliases': ['gams', 'postini', 'gsuitegams', 'gsuitepostini', 'gsuitemessagesecurity'], 'displayName': 'G Suite Message Security'},
'Google-Apps-Lite': {
'product': 'Google-Apps', 'aliases': ['gal', 'lite', 'gsuitelite'], 'displayName': 'G Suite Lite'},
'product': 'Google-Apps', 'aliases': ['gal', 'gsl', 'lite', 'gsuitelite'], 'displayName': 'G Suite Lite'},
'Google-Apps-Unlimited': {
'product': 'Google-Apps', 'aliases': ['gau', 'unlimited', 'gsuitebusiness'], 'displayName': 'G Suite Business'},
'product': 'Google-Apps', 'aliases': ['gau', 'gsb', 'unlimited', 'gsuitebusiness'], 'displayName': 'G Suite Business'},
'1010020020': {
'product': 'Google-Apps', 'aliases': ['gae', 'enterprise', 'gsuiteenterprise'], 'displayName': 'G Suite Enterprise'},
'product': 'Google-Apps', 'aliases': ['gae', 'gse', 'enterprise', 'gsuiteenterprise'], 'displayName': 'G Suite Enterprise'},
'1010340002': {
'product': '101034', 'aliases': ['gsbau', 'businessarchived', 'gsuitebusinessarchived'], 'displayName': 'G Suite Business Archived'},
'1010340001': {
'product': '101034', 'aliases': ['gseau', 'enterprisearchived', 'gsuiteenterprisearchived'], 'displayName': 'G Suite Enterprise Archived'},
'1010060001': {
'product': 'Google-Apps', 'aliases': ['d4e', 'driveenterprise', 'drive4enterprise'], 'displayName': 'Drive Enterprise'},
'Google-Drive-storage-20GB': {
@@ -99,6 +104,20 @@ SKUS = {
'product': 'Google-Chrome-Device-Management', 'aliases': ['chrome', 'cdm', 'googlechromedevicemanagement'], 'displayName': 'Google Chrome Device Management'}
}
PRODUCTID_NAME_MAPPINGS = {
'101001': 'Cloud Identity Free',
'101005': 'Cloud Identity Premium',
'101006': 'Drive Enterprise',
'101031': 'G Suite Enterprise for Education',
'101033': 'Google Voice',
'101034': 'G Suite Archived',
'Google-Apps': 'G Suite',
'Google-Chrome-Device-Management': 'Google Chrome Device Management',
'Google-Coordinate': 'Google Coordinate',
'Google-Drive-storage': 'Google Drive Storage',
'Google-Vault': 'Google Vault',
}
# Legacy APIs that use v1 discovery. Newer APIs should all use v2.
V1_DISCOVERY_APIS = {
'admin',
@@ -126,6 +145,7 @@ API_VER_MAPPING = {
'drive3': 'v3',
'gmail': 'v1',
'groupssettings': 'v1',
'iam': 'v1',
'licensing': 'v1',
'oauth2': 'v2',
'pubsub': 'v1',
@@ -137,16 +157,18 @@ API_VER_MAPPING = {
'vault': 'v1',
}
USERINFO_EMAIL_SCOPE = 'https://www.googleapis.com/auth/userinfo.email'
API_SCOPE_MAPPING = {
'alertcenter': ['https://www.googleapis.com/auth/apps.alerts',],
'appsactivity': ['https://www.googleapis.com/auth/activity',
'https://www.googleapis.com/auth/drive',],
'https://www.googleapis.com/auth/drive',],
'calendar': ['https://www.googleapis.com/auth/calendar',],
'drive': ['https://www.googleapis.com/auth/drive',],
'drive3': ['https://www.googleapis.com/auth/drive',],
'gmail': ['https://mail.google.com/',
'https://www.googleapis.com/auth/gmail.settings.basic',
'https://www.googleapis.com/auth/gmail.settings.sharing',],
'https://www.googleapis.com/auth/gmail.settings.basic',
'https://www.googleapis.com/auth/gmail.settings.sharing',],
'sheets': ['https://www.googleapis.com/auth/spreadsheets',],
}
@@ -211,6 +233,7 @@ DRIVEFILE_FIELDS_CHOICES_MAP = {
'cancomment': 'canComment',
'canreadrevisions': 'canReadRevisions',
'copyable': 'copyable',
'copyrequireswriterpermission': 'copyRequiresWriterPermission',
'createddate': 'createdDate',
'createdtime': 'createdDate',
'description': 'description',
@@ -360,18 +383,33 @@ GOOGLEDOC_VALID_EXTENSIONS_MAP = {
MIMETYPE_GA_SPREADSHEET: ['.csv', '.ods', '.pdf', '.xlsx', '.zip'],
}
MICROSOFT_FORMATS_LIST = [{'mime': 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', 'ext': '.docx'},
{'mime': 'application/vnd.openxmlformats-officedocument.wordprocessingml.template', 'ext': '.dotx'},
{'mime': 'application/vnd.openxmlformats-officedocument.presentationml.presentation', 'ext': '.pptx'},
{'mime': 'application/vnd.openxmlformats-officedocument.presentationml.template', 'ext': '.potx'},
{'mime': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'ext': '.xlsx'},
{'mime': 'application/vnd.openxmlformats-officedocument.spreadsheetml.template', 'ext': '.xltx'},
{'mime': 'application/msword', 'ext': '.doc'},
{'mime': 'application/msword', 'ext': '.dot'},
{'mime': 'application/vnd.ms-powerpoint', 'ext': '.ppt'},
{'mime': 'application/vnd.ms-powerpoint', 'ext': '.pot'},
{'mime': 'application/vnd.ms-excel', 'ext': '.xls'},
{'mime': 'application/vnd.ms-excel', 'ext': '.xlt'}]
MACOS_CODENAMES = {
6: 'Snow Leopard',
7: 'Lion',
8: 'Mountain Lion',
9: 'Mavericks',
10: 'Yosemite',
11: 'El Capitan',
12: 'Sierra',
13: 'High Sierra',
14: 'Mojave',
15: 'Catalina'
}
_MICROSOFT_FORMATS_LIST = [
{'mime': 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', 'ext': '.docx'},
{'mime': 'application/vnd.openxmlformats-officedocument.wordprocessingml.template', 'ext': '.dotx'},
{'mime': 'application/vnd.openxmlformats-officedocument.presentationml.presentation', 'ext': '.pptx'},
{'mime': 'application/vnd.openxmlformats-officedocument.presentationml.template', 'ext': '.potx'},
{'mime': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'ext': '.xlsx'},
{'mime': 'application/vnd.openxmlformats-officedocument.spreadsheetml.template', 'ext': '.xltx'},
{'mime': 'application/msword', 'ext': '.doc'},
{'mime': 'application/msword', 'ext': '.dot'},
{'mime': 'application/vnd.ms-powerpoint', 'ext': '.ppt'},
{'mime': 'application/vnd.ms-powerpoint', 'ext': '.pot'},
{'mime': 'application/vnd.ms-excel', 'ext': '.xls'},
{'mime': 'application/vnd.ms-excel', 'ext': '.xlt'}
]
DOCUMENT_FORMATS_MAP = {
'csv': [{'mime': 'text/csv', 'ext': '.csv'}],
@@ -386,7 +424,7 @@ DOCUMENT_FORMATS_MAP = {
'mht': [{'mime': 'message/rfc822', 'ext': 'mht'}],
'odp': [{'mime': 'application/vnd.oasis.opendocument.presentation', 'ext': '.odp'}],
'ods': [{'mime': 'application/x-vnd.oasis.opendocument.spreadsheet', 'ext': '.ods'},
{'mime': 'application/vnd.oasis.opendocument.spreadsheet', 'ext': '.ods'}],
{'mime': 'application/vnd.oasis.opendocument.spreadsheet', 'ext': '.ods'}],
'odt': [{'mime': 'application/vnd.oasis.opendocument.text', 'ext': '.odt'}],
'pdf': [{'mime': 'application/pdf', 'ext': '.pdf'}],
'png': [{'mime': 'image/png', 'ext': '.png'}],
@@ -397,20 +435,37 @@ DOCUMENT_FORMATS_MAP = {
'rtf': [{'mime': 'application/rtf', 'ext': '.rtf'}],
'svg': [{'mime': 'image/svg+xml', 'ext': '.svg'}],
'tsv': [{'mime': 'text/tab-separated-values', 'ext': '.tsv'},
{'mime': 'text/tsv', 'ext': '.tsv'}],
{'mime': 'text/tsv', 'ext': '.tsv'}],
'txt': [{'mime': 'text/plain', 'ext': '.txt'}],
'xls': [{'mime': 'application/vnd.ms-excel', 'ext': '.xls'}],
'xlt': [{'mime': 'application/vnd.ms-excel', 'ext': '.xlt'}],
'xlsx': [{'mime': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'ext': '.xlsx'}],
'xltx': [{'mime': 'application/vnd.openxmlformats-officedocument.spreadsheetml.template', 'ext': '.xltx'}],
'zip': [{'mime': 'application/zip', 'ext': '.zip'}],
'ms': MICROSOFT_FORMATS_LIST,
'microsoft': MICROSOFT_FORMATS_LIST,
'micro$oft': MICROSOFT_FORMATS_LIST,
'ms': _MICROSOFT_FORMATS_LIST,
'microsoft': _MICROSOFT_FORMATS_LIST,
'micro$oft': _MICROSOFT_FORMATS_LIST,
'openoffice': [{'mime': 'application/vnd.oasis.opendocument.presentation', 'ext': '.odp'},
{'mime': 'application/x-vnd.oasis.opendocument.spreadsheet', 'ext': '.ods'},
{'mime': 'application/vnd.oasis.opendocument.spreadsheet', 'ext': '.ods'},
{'mime': 'application/vnd.oasis.opendocument.text', 'ext': '.odt'}],
{'mime': 'application/x-vnd.oasis.opendocument.spreadsheet', 'ext': '.ods'},
{'mime': 'application/vnd.oasis.opendocument.spreadsheet', 'ext': '.ods'},
{'mime': 'application/vnd.oasis.opendocument.text', 'ext': '.odt'}],
}
REFRESH_PERM_ERRORS = [
'invalid_grant: reauth related error (rapt_required)', # no way to reauth today
'invalid_grant: Token has been expired or revoked.',
]
DNS_ERROR_CODES_MAP = {
1: 'DNS Query Format Error',
2: 'Server failed to complete the DNS request',
3: 'Domain name does not exist',
4: 'Function not implemented',
5: 'The server refused to answer for the query',
6: 'Name that should not exist, does exist',
7: 'RRset that should not exist, does exist',
8: 'Server not authoritative for the zone',
9: 'Name not in zone'
}
EMAILSETTINGS_OLD_NEW_OLD_FORWARD_ACTION_MAP = {
@@ -454,6 +509,12 @@ RT_CLOSE_PATTERN = re.compile(r'{/RT}')
RT_STRIP_PATTERN = re.compile(r'(?s){RT}.*?{/RT}')
RT_TAG_REPLACE_PATTERN = re.compile(r'{(.*?)}')
LOWERNUMERIC_CHARS = string.ascii_lowercase+string.digits
ALPHANUMERIC_CHARS = LOWERNUMERIC_CHARS+string.ascii_uppercase
URL_SAFE_CHARS = ALPHANUMERIC_CHARS+'-._~'
PASSWORD_SAFE_CHARS = ALPHANUMERIC_CHARS+string.punctuation+' '
FILENAME_SAFE_CHARS = ALPHANUMERIC_CHARS+'-_.() '
FILTER_ADD_LABEL_TO_ARGUMENT_MAP = {
'IMPORTANT': 'important',
'STARRED': 'star',
@@ -499,17 +560,21 @@ CROS_ARGUMENT_TO_PROPERTY_MAP = {
'annotateduser': ['annotatedUser',],
'asset': ['annotatedAssetId',],
'assetid': ['annotatedAssetId',],
'autoupdateexpiration': ['autoUpdateExpiration',],
'bootmode': ['bootMode',],
'cpustatusreports': ['cpuStatusReports',],
'devicefiles': ['deviceFiles',],
'deviceid': ['deviceId',],
'dockmacaddress': ['dockMacAddress',],
'diskvolumereports': ['diskVolumeReports',],
'ethernetmacaddress': ['ethernetMacAddress',],
'ethernetmacaddress0': ['ethernetMacAddress0',],
'firmwareversion': ['firmwareVersion',],
'lastenrollmenttime': ['lastEnrollmentTime',],
'lastsync': ['lastSync',],
'location': ['annotatedLocation',],
'macaddress': ['macAddress',],
'manufacturedate': ['manufactureDate',],
'meid': ['meid',],
'model': ['model',],
'notes': ['notes',],
@@ -551,12 +616,18 @@ CROS_SCALAR_PROPERTY_PRINT_ORDER = [
'osVersion',
'bootMode',
'meid',
'dockMacAddress',
'ethernetMacAddress',
'ethernetMacAddress0',
'macAddress',
'systemRamTotal',
'lastEnrollmentTime',
'orderNumber',
'manufactureDate',
'supportEndDate',
'autoUpdateExpiration',
'guessedAUEDate',
'guessedAUEModel',
'tpmVersionInfo',
'willAutoRenew',
]
@@ -575,28 +646,76 @@ CROS_END_ARGUMENTS = ['end', 'enddate']
CROS_TPM_VULN_VERSIONS = ['41f', '420', '628', '8520',]
CROS_TPM_FIXED_VERSIONS = ['422', '62b', '8521',]
COLLABORATIVE_ACL_CHOICES = {
'members': 'ALL_MEMBERS',
'managersonly': 'MANAGERS_ONLY',
'managers': 'OWNERS_AND_MANAGERS',
'owners': 'OWNERS_ONLY',
'none': 'NONE',
}
COLLABORATIVE_INBOX_ATTRIBUTES = [
'whoCanAddReferences',
'whoCanAssignTopics',
'whoCanEnterFreeFormTags',
'whoCanMarkDuplicate',
'whoCanMarkFavoriteReplyOnAnyTopic',
'whoCanMarkFavoriteReplyOnOwnTopic',
'whoCanMarkNoResponseNeeded',
'whoCanModifyTagsAndCategories',
'whoCanTakeTopics',
'whoCanUnassignTopic',
'whoCanUnmarkFavoriteReplyOnAnyTopic',
'favoriteRepliesOnTop',
]
COLLABORATIVE_INBOX_ATTRIBUTES = {
'whoCanAddReferences': 'acl',
'whoCanAssignTopics': 'acl',
'whoCanEnterFreeFormTags': 'acl',
'whoCanMarkDuplicate': 'acl',
'whoCanMarkFavoriteReplyOnAnyTopic': 'acl',
'whoCanMarkFavoriteReplyOnOwnTopic': 'acl',
'whoCanMarkNoResponseNeeded': 'acl',
'whoCanModifyTagsAndCategories': 'acl',
'whoCanTakeTopics': 'acl',
'whoCanUnassignTopic': 'acl',
'whoCanUnmarkFavoriteReplyOnAnyTopic': 'acl',
'favoriteRepliesOnTop': True,
}
GROUP_SETTINGS_LIST_ATTRIBUTES = set([
# ACL choices
'whoCanAdd',
'whoCanApproveMembers',
'whoCanApproveMessages',
'whoCanAssignTopics',
'whoCanAssistContent',
'whoCanBanUsers',
'whoCanContactOwner',
'whoCanDeleteAnyPost',
'whoCanDeleteTopics',
'whoCanDiscoverGroup',
'whoCanEnterFreeFormTags',
'whoCanHideAbuse',
'whoCanInvite',
'whoCanJoin',
'whoCanLeaveGroup',
'whoCanLockTopics',
'whoCanMakeTopicsSticky',
'whoCanMarkDuplicate',
'whoCanMarkFavoriteReplyOnAnyTopic',
'whoCanMarkFavoriteReplyOnOwnTopic',
'whoCanMarkNoResponseNeeded',
'whoCanModerateContent',
'whoCanModerateMembers',
'whoCanModifyMembers',
'whoCanModifyTagsAndCategories',
'whoCanMoveTopicsIn',
'whoCanMoveTopicsOut',
'whoCanPostAnnouncements',
'whoCanPostMessage',
'whoCanTakeTopics',
'whoCanUnassignTopic',
'whoCanUnmarkFavoriteReplyOnAnyTopic',
'whoCanViewGroup',
'whoCanViewMembership',
# Miscellaneous hoices
'messageModerationLevel',
'replyTo',
'spamModerationLevel',
])
GROUP_SETTINGS_BOOLEAN_ATTRIBUTES = set([
'allowExternalMembers',
'allowGoogleCommunication',
'allowWebPosting',
'archiveOnly',
'enableCollaborativeInbox',
'favoriteRepliesOnTop',
'includeCustomFooter',
'includeInGlobalAddressList',
'isArchived',
'membersCanPostAsTheGroup',
'sendMessageDenyNotification',
'showInGroupDirectory',
])
#
# Global variables
@@ -640,12 +759,18 @@ GM_CACHE_DISCOVERY_ONLY = 'gcdo'
GM_MAP_BUILDING_ID_TO_NAME = 'bi2n'
# Dictionary mapping Building Name to ID
GM_MAP_BUILDING_NAME_TO_ID = 'bn2i'
#
_DEFAULT_CHARSET = UTF8
_FN_CLIENT_SECRETS_JSON = 'client_secrets.json'
_FN_OAUTH2SERVICE_JSON = 'oauth2service.json'
_FN_OAUTH2_TXT = 'oauth2.txt'
#
GM_Globals = {
GM_SYSEXITRC: 0,
GM_GAM_PATH: None,
GM_WINDOWS: os.name == 'nt',
GM_SYS_ENCODING: DEFAULT_CHARSET,
GM_SYS_ENCODING: _DEFAULT_CHARSET,
GM_EXTRA_ARGS_DICT: {'prettyPrint': False},
GM_CURRENT_API_SERVICES: {},
GM_CURRENT_API_USER: None,
@@ -666,8 +791,6 @@ GM_Globals = {
#
# Global variables defined by environment variables/signal files
#
# When retrieving lists of Google Drive activities from API, how many should be retrieved in each chunk
GC_ACTIVITY_MAX_RESULTS = 'activity_max_results'
# Automatically generate gam batch command if number of users specified in gam users xxx command exceeds this number
# Default: 0, don't automatically generate gam batch commands
GC_AUTO_BATCH_MIN = 'auto_batch_min'
@@ -687,25 +810,21 @@ GC_CONFIG_DIR = 'config_dir'
GC_CUSTOMER_ID = 'customer_id'
# If debug_level > 0: extra_args[u'prettyPrint'] = True, httplib2.debuglevel = gam_debug_level, appsObj.debug = True
GC_DEBUG_LEVEL = 'debug_level'
# When retrieving lists of ChromeOS/Mobile devices from API, how many should be retrieved in each chunk
GC_DEVICE_MAX_RESULTS = 'device_max_results'
# ID Token decoded from OAuth 2.0 refresh token response. Includes hd (domain) and email of authorized user
GC_DECODED_ID_TOKEN = 'decoded_id_token'
# Domain obtained from gam.cfg or oauth2.txt
GC_DOMAIN = 'domain'
# Google Drive download directory
GC_DRIVE_DIR = 'drive_dir'
# When retrieving lists of Drive files/folders from API, how many should be retrieved in each chunk
GC_DRIVE_MAX_RESULTS = 'drive_max_results'
# When retrieving lists of Google Group members from API, how many should be retrieved in each chunk
GC_MEMBER_MAX_RESULTS = 'member_max_results'
# If no_browser is False, writeCSVfile won't open a browser when todrive is set
# and doRequestOAuth prints a link and waits for the verification code when oauth2.txt is being created
GC_NO_BROWSER = 'no_browser'
# oauth_browser forces usage of web server OAuth flow that proved problematic.
GC_OAUTH_BROWSER = 'oauth_browser'
# Disable GAM API caching
GC_NO_CACHE = 'no_cache'
# Disable GAM update check
GC_NO_UPDATE_CHECK = 'no_update_check'
# Disable SSL certificate validation
GC_NO_VERIFY_SSL = 'no_verify_ssl'
# Number of threads for gam batch
GC_NUM_THREADS = 'num_threads'
# Path to oauth2.txt
@@ -720,43 +839,47 @@ GC_SHOW_COUNTS_MIN = 'show_counts_min'
GC_SHOW_GETTINGS = 'show_gettings'
# GAM config directory containing json discovery files
GC_SITE_DIR = 'site_dir'
# When retrieving lists of Users from API, how many should be retrieved in each chunk
GC_USER_MAX_RESULTS = 'user_max_results'
# CSV Columns GAM should show on CSV output
GC_CSV_HEADER_FILTER = 'csv_header_filter'
# CSV Rows GAM should filter
GC_CSV_ROW_FILTER = 'csv_row_filter'
# Minimum TLS Version required for HTTPS connections
GC_TLS_MIN_VERSION = 'tls_min_ver'
# Maximum TLS Version used for HTTPS connections
GC_TLS_MAX_VERSION = 'tls_max_ver'
# Path to certificate authority file for validating TLS hosts
GC_CA_FILE = 'ca_file'
tls_min = "TLSv1_2" if hasattr(ssl.SSLContext(), "minimum_version") else None
GC_Defaults = {
GC_ACTIVITY_MAX_RESULTS: 100,
GC_AUTO_BATCH_MIN: 0,
GC_BATCH_SIZE: 50,
GC_CACHE_DIR: '',
GC_CACHE_DISCOVERY_ONLY: True,
GC_CHARSET: DEFAULT_CHARSET,
GC_CLIENT_SECRETS_JSON: FN_CLIENT_SECRETS_JSON,
GC_CHARSET: _DEFAULT_CHARSET,
GC_CLIENT_SECRETS_JSON: _FN_CLIENT_SECRETS_JSON,
GC_CONFIG_DIR: '',
GC_CUSTOMER_ID: MY_CUSTOMER,
GC_DEBUG_LEVEL: 0,
GC_DEVICE_MAX_RESULTS: 500,
GC_DECODED_ID_TOKEN: '',
GC_DOMAIN: '',
GC_DRIVE_DIR: '',
GC_DRIVE_MAX_RESULTS: 1000,
GC_MEMBER_MAX_RESULTS: 200,
GC_NO_BROWSER: False,
GC_NO_CACHE: False,
GC_NO_UPDATE_CHECK: False,
GC_NO_VERIFY_SSL: False,
GC_NUM_THREADS: 25,
GC_OAUTH2_TXT: FN_OAUTH2_TXT,
GC_OAUTH2SERVICE_JSON: FN_OAUTH2SERVICE_JSON,
GC_OAUTH_BROWSER: False,
GC_OAUTH2_TXT: _FN_OAUTH2_TXT,
GC_OAUTH2SERVICE_JSON: _FN_OAUTH2SERVICE_JSON,
GC_SECTION: '',
GC_SHOW_COUNTS_MIN: 0,
GC_SHOW_GETTINGS: True,
GC_SITE_DIR: '',
GC_USER_MAX_RESULTS: 500,
GC_CSV_HEADER_FILTER: '',
GC_CSV_ROW_FILTER: '',
GC_TLS_MIN_VERSION: tls_min,
GC_TLS_MAX_VERSION: None,
GC_CA_FILE: None,
}
GC_Values = {}
@@ -766,15 +889,16 @@ GC_TYPE_CHOICE = 'choi'
GC_TYPE_DIRECTORY = 'dire'
GC_TYPE_EMAIL = 'emai'
GC_TYPE_FILE = 'file'
GC_TYPE_HEADERFILTER = 'heaf'
GC_TYPE_INTEGER = 'inte'
GC_TYPE_LANGUAGE = 'lang'
GC_TYPE_ROWFILTER = 'rowf'
GC_TYPE_STRING = 'stri'
GC_VAR_TYPE = 'type'
GC_VAR_LIMITS = 'lmit'
GC_VAR_INFO = {
GC_ACTIVITY_MAX_RESULTS: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (1, 500)},
GC_AUTO_BATCH_MIN: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (0, None)},
GC_BATCH_SIZE: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (1, 1000)},
GC_CACHE_DIR: {GC_VAR_TYPE: GC_TYPE_DIRECTORY},
@@ -784,39 +908,32 @@ GC_VAR_INFO = {
GC_CONFIG_DIR: {GC_VAR_TYPE: GC_TYPE_DIRECTORY},
GC_CUSTOMER_ID: {GC_VAR_TYPE: GC_TYPE_STRING},
GC_DEBUG_LEVEL: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (0, None)},
GC_DEVICE_MAX_RESULTS: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (1, 1000)},
GC_DECODED_ID_TOKEN: {GC_VAR_TYPE: GC_TYPE_STRING},
GC_DOMAIN: {GC_VAR_TYPE: GC_TYPE_STRING},
GC_DRIVE_DIR: {GC_VAR_TYPE: GC_TYPE_DIRECTORY},
GC_DRIVE_MAX_RESULTS: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (1, 1000)},
GC_MEMBER_MAX_RESULTS: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (1, 10000)},
GC_NO_BROWSER: {GC_VAR_TYPE: GC_TYPE_BOOLEAN},
GC_NO_CACHE: {GC_VAR_TYPE: GC_TYPE_BOOLEAN},
GC_NO_UPDATE_CHECK: {GC_VAR_TYPE: GC_TYPE_BOOLEAN},
GC_NO_VERIFY_SSL: {GC_VAR_TYPE: GC_TYPE_BOOLEAN},
GC_NUM_THREADS: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (1, None)},
GC_OAUTH_BROWSER: {GC_VAR_TYPE: GC_TYPE_BOOLEAN},
GC_OAUTH2_TXT: {GC_VAR_TYPE: GC_TYPE_FILE},
GC_OAUTH2SERVICE_JSON: {GC_VAR_TYPE: GC_TYPE_FILE},
GC_SECTION: {GC_VAR_TYPE: GC_TYPE_STRING},
GC_SHOW_COUNTS_MIN: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (0, None)},
GC_SHOW_GETTINGS: {GC_VAR_TYPE: GC_TYPE_BOOLEAN},
GC_SITE_DIR: {GC_VAR_TYPE: GC_TYPE_DIRECTORY},
GC_USER_MAX_RESULTS: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (1, 500)},
GC_CSV_HEADER_FILTER: {GC_VAR_TYPE: GC_TYPE_STRING},
GC_CSV_ROW_FILTER: {GC_VAR_TYPE: GC_TYPE_STRING},
GC_CSV_HEADER_FILTER: {GC_VAR_TYPE: GC_TYPE_HEADERFILTER},
GC_CSV_ROW_FILTER: {GC_VAR_TYPE: GC_TYPE_ROWFILTER},
GC_TLS_MIN_VERSION: {GC_VAR_TYPE: GC_TYPE_STRING},
GC_TLS_MAX_VERSION: {GC_VAR_TYPE: GC_TYPE_STRING},
GC_CA_FILE: {GC_VAR_TYPE: GC_TYPE_FILE},
}
# Google API constants
NEVER_TIME = '1970-01-01T00:00:00.000Z'
NEVER_START_DATE = '1970-01-01'
NEVER_END_DATE = '1969-12-31'
ROLE_MANAGER = 'MANAGER'
ROLE_MEMBER = 'MEMBER'
ROLE_OWNER = 'OWNER'
ROLE_USER = 'USER'
ROLE_MANAGER_MEMBER = ','.join([ROLE_MANAGER, ROLE_MEMBER])
ROLE_MANAGER_OWNER = ','.join([ROLE_MANAGER, ROLE_OWNER])
ROLE_MANAGER_MEMBER_OWNER = ','.join([ROLE_MANAGER, ROLE_MEMBER, ROLE_OWNER])
ROLE_MEMBER_OWNER = ','.join([ROLE_MEMBER, ROLE_OWNER])
PROJECTION_CHOICES_MAP = {'basic': 'BASIC', 'full': 'FULL',}
SORTORDER_CHOICES_MAP = {'ascending': 'ASCENDING', 'descending': 'DESCENDING',}
#
@@ -824,68 +941,22 @@ CLEAR_NONE_ARGUMENT = ['clear', 'none',]
#
MESSAGE_API_ACCESS_CONFIG = 'API access is configured in your Control Panel under: Security-Show more-Advanced settings-Manage API client access'
MESSAGE_API_ACCESS_DENIED = 'API access Denied.\n\nPlease make sure the Client ID: {0} is authorized for the API Scope(s): {1}'
MESSAGE_CONSOLE_AUTHORIZATION_PROMPT = '\nGo to the following link in your browser:\n\n\t{url}\n'
MESSAGE_CONSOLE_AUTHORIZATION_CODE = 'Enter verification code: '
MESSAGE_GAM_EXITING_FOR_UPDATE = 'GAM is now exiting so that you can overwrite this old version with the latest release'
MESSAGE_GAM_OUT_OF_MEMORY = 'GAM has run out of memory. If this is a large G Suite instance, you should use a 64-bit version of GAM on Windows or a 64-bit version of Python on other systems.'
MESSAGE_HEADER_NOT_FOUND_IN_CSV_HEADERS = 'Header "{0}" not found in CSV headers of "{1}".'
MESSAGE_HIT_CONTROL_C_TO_UPDATE = '\n\nHit CTRL+C to visit the GAM website and download the latest release or wait 15 seconds continue with this boring old version. GAM won\'t bother you with this announcement for 1 week or you can create a file named noupdatecheck.txt in the same location as gam.py or gam.exe and GAM won\'t ever check for updates.'
MESSAGE_INVALID_JSON = 'The file {0} has an invalid format.'
MESSAGE_LOCAL_SERVER_AUTHORIZATION_PROMPT = '\nYour browser has been opened to visit:\n\n\t{url}\n\nIf your browser is on a different machine then press CTRL+C and create a file called nobrowser.txt in the same folder as GAM.\n'
MESSAGE_LOCAL_SERVER_SUCCESS = 'The authentication flow has completed. You may close this browser window and return to GAM.'
MESSAGE_NO_DISCOVERY_INFORMATION = 'No online discovery doc and {0} does not exist locally'
MESSAGE_NO_PYTHON_SSL = 'You don\'t have the Python SSL module installed so we can\'t verify SSL Certificates. You can fix this by installing the Python SSL module or you can live on the edge and turn SSL validation off by creating a file named noverifyssl.txt in the same location as gam.exe / gam.py'
MESSAGE_NO_TRANSFER_LACK_OF_DISK_SPACE = 'Cowardly refusing to perform migration due to lack of target drive space. Source size: {0}mb Target Free: {1}mb'
MESSAGE_REQUEST_COMPLETED_NO_FILES = 'Request completed but no results/files were returned, try requesting again'
MESSAGE_REQUEST_NOT_COMPLETE = 'Request needs to be completed before downloading, current status is: {0}'
MESSAGE_RESULTS_TOO_LARGE_FOR_GOOGLE_SPREADSHEET = 'Results are too large for Google Spreadsheets. Uploading as a regular CSV file.'
MESSAGE_SERVICE_NOT_APPLICABLE = 'Service not applicable for this address: {0}. Please make sure service is enabled for user and run\n\ngam user <user> check serviceaccount\n\nfor further instructions'
MESSAGE_INSTRUCTIONS_OAUTH2SERVICE_JSON = 'Please run\n\ngam create project\ngam user <user> check serviceaccount\n\nto create and configure a service account.'
MESSAGE_OAUTH2SERVICE_JSON_INVALID = 'The file {0} is missing required keys (client_email, client_id or private_key). Please remove it and recreate with the commands:\n\ngam create project\ngam user <user> check serviceaccount'
# oauth errors
OAUTH2_TOKEN_ERRORS = [
'access_denied',
'access_denied: Requested client not authorized',
'internal_failure: Backend Error',
'invalid_grant: Bad Request',
'invalid_grant: Invalid email or User ID',
'invalid_grant: Not a valid email',
'invalid_request: Invalid impersonation prn email address',
'unauthorized_client: Client is unauthorized to retrieve access tokens using this method',
'unauthorized_client: Unauthorized client or scope in request',
]
#
# callGAPI throw reasons
GAPI_ABORTED = 'aborted'
GAPI_AUTH_ERROR = 'authError'
GAPI_BACKEND_ERROR = 'backendError'
GAPI_BAD_REQUEST = 'badRequest'
GAPI_CONDITION_NOT_MET = 'conditionNotMet'
GAPI_CYCLIC_MEMBERSHIPS_NOT_ALLOWED = 'cyclicMembershipsNotAllowed'
GAPI_DOMAIN_CANNOT_USE_APIS = 'domainCannotUseApis'
GAPI_DOMAIN_NOT_FOUND = 'domainNotFound'
GAPI_DUPLICATE = 'duplicate'
GAPI_FAILED_PRECONDITION = 'failedPrecondition'
GAPI_FORBIDDEN = 'forbidden'
GAPI_GROUP_NOT_FOUND = 'groupNotFound'
GAPI_INTERNAL_ERROR = 'internalError'
GAPI_INVALID = 'invalid'
GAPI_INVALID_ARGUMENT = 'invalidArgument'
GAPI_INVALID_MEMBER = 'invalidMember'
GAPI_MEMBER_NOT_FOUND = 'memberNotFound'
GAPI_NOT_FOUND = 'notFound'
GAPI_NOT_IMPLEMENTED = 'notImplemented'
GAPI_PERMISSION_DENIED = 'permissionDenied'
GAPI_QUOTA_EXCEEDED = 'quotaExceeded'
GAPI_RATE_LIMIT_EXCEEDED = 'rateLimitExceeded'
GAPI_RESOURCE_NOT_FOUND = 'resourceNotFound'
GAPI_SERVICE_NOT_AVAILABLE = 'serviceNotAvailable'
GAPI_SYSTEM_ERROR = 'systemError'
GAPI_USER_NOT_FOUND = 'userNotFound'
GAPI_USER_RATE_LIMIT_EXCEEDED = 'userRateLimitExceeded'
#
GAPI_DEFAULT_RETRY_REASONS = [GAPI_QUOTA_EXCEEDED, GAPI_RATE_LIMIT_EXCEEDED, GAPI_USER_RATE_LIMIT_EXCEEDED, GAPI_BACKEND_ERROR, GAPI_INTERNAL_ERROR]
GAPI_GMAIL_THROW_REASONS = [GAPI_SERVICE_NOT_AVAILABLE]
GAPI_GROUP_GET_THROW_REASONS = [GAPI_GROUP_NOT_FOUND, GAPI_DOMAIN_NOT_FOUND, GAPI_DOMAIN_CANNOT_USE_APIS, GAPI_FORBIDDEN, GAPI_BAD_REQUEST]
GAPI_GROUP_GET_RETRY_REASONS = [GAPI_INVALID, GAPI_SYSTEM_ERROR]
GAPI_MEMBERS_THROW_REASONS = [GAPI_GROUP_NOT_FOUND, GAPI_DOMAIN_NOT_FOUND, GAPI_DOMAIN_CANNOT_USE_APIS, GAPI_INVALID, GAPI_FORBIDDEN]
GAPI_MEMBERS_RETRY_REASONS = [GAPI_SYSTEM_ERROR]
MESSAGE_UPDATE_GAM_TO_64BIT = "You're running a 32-bit version of GAM on a 64-bit version of Windows, upgrade to a windows-x86_64 version of GAM"
MESSAGE_YOUR_SYSTEM_TIME_DIFFERS_FROM_GOOGLE_BY = 'Your system time differs from Google by %s'
USER_ADDRESS_TYPES = ['home', 'work', 'other']
USER_EMAIL_TYPES = ['home', 'work', 'other']
@@ -1096,3 +1167,19 @@ LANGUAGE_CODES_MAP = {
'ur': 'ur', 'uz': 'uz', 'vi': 'vi', 'wo': 'wo', 'xh': 'xh', 'yi': 'yi', 'yo': 'yo', #Urdu, Uzbek, Vietnamese, Wolof, Xhosa, Yiddish, Yoruba
'zh-cn': 'zh-CN', 'zh-hk': 'zh-HK', 'zh-tw': 'zh-TW', 'zu': 'zu', #Chinese (Simplified), Chinese (Hong Kong/Traditional), Chinese (Taiwan/Traditional), Zulu
}
# maxResults exception values for API list calls. Should only be listed if:
# - discovery doc does not specify maximum value (we use maximum value if it
# exists, not this)
# - actual max API returns with maxResults=<bigNum> > default API returns
# when maxResults isn't specified (we should use default otherwise by not
# setting maxResults)
MAX_RESULTS_API_EXCEPTIONS = {
'calendar.acl.list': 250,
'calendar.calendarList.list': 250,
'calendar.events.list': 2500,
'calendar.settings.list': 250,
'directory.chromeosdevices.list': 200,
'drive.files.list': 1000,
}

View File

@@ -1,3 +1,5 @@
This file has been deprecated. For the latest features see https://git.io/gam-releases notes
GAM 4.65
- Manage email delivery settings for group members
- Add operatingSystemType to user posix config (Roman)

View File

@@ -10,7 +10,7 @@ del /q /f *.wixpdb
set WIXVERSION=3.11
c:\python27-32\scripts\pyinstaller --clean -F --distpath=gam windows-gam.spec
c:\python37-32\scripts\pyinstaller --clean -F --distpath=gam windows-gam.spec
xcopy LICENSE gam\
xcopy whatsnew.txt gam\
xcopy gam-setup.bat gam\
@@ -18,7 +18,7 @@ xcopy GamCommands.txt gam\
del gam\w9xpopen.exe
"%ProgramFiles%\7-Zip\7z.exe" a -tzip gam-%1-windows.zip gam\ -xr!.svn
c:\python27-64\scripts\pyinstaller --clean -F --distpath=gam-64 windows-gam.spec
c:\python37-64\scripts\pyinstaller --clean -F --distpath=gam-64 windows-gam.spec
xcopy LICENSE gam-64\
xcopy whatsnew.txt gam-64\
xcopy gam-setup.bat gam-64\

View File

@@ -1,24 +1,33 @@
# -*- mode: python -*-
a = Analysis(['gam.py'],
pathex=['C:\\Users\\jlee\\Documents\\GitHub\\GAM'],
hiddenimports=[],
hookspath=None,
excludes=['_tkinter'],
runtime_hooks=None)
for d in a.datas:
if 'pyconfig' in d[0]:
a.datas.remove(d)
break
a.datas += [('httplib2/cacerts.txt', 'cacerts.txt', 'DATA')]
a.datas += [('cloudprint-v2.json', 'cloudprint-v2.json', 'DATA')]
pyz = PYZ(a.pure)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='gam.exe',
debug=False,
strip=None,
upx=True,
console=True )
# -*- mode: python -*-
import sys
sys.modules['FixTk'] = None
a = Analysis(['gam.py'],
pathex=['C:\\Users\\jlee\\Documents\\GitHub\\GAM'],
hiddenimports=[],
hookspath=None,
excludes=['FixTk', 'tcl', 'tk', '_tkinter', 'tkinter', 'Tkinter'],
runtime_hooks=None)
for d in a.datas:
if 'pyconfig' in d[0]:
a.datas.remove(d)
break
a.datas += [('cloudprint-v2.json', 'cloudprint-v2.json', 'DATA')]
# dynamically determine where httplib2/cacerts.txt lives
import importlib
proot = os.path.dirname(importlib.import_module('httplib2').__file__)
a.datas += [('httplib2/cacerts.txt', os.path.join(proot, 'cacerts.txt'), 'DATA')]
pyz = PYZ(a.pure)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='gam.exe',
debug=False,
strip=None,
upx=True,
console=True )