GAM 4.23

* Update calendar to allow access to user's secondary calendars

* Code cleanup

* Code cleanup

* Code cleanup

README.md

@@ -13,7 +13,7 @@ The GAM documentation is hosted in the [GitHub Wiki]
 # Mailing List / Discussion group
 The GAM mailing list / discussion group is hosted on [Google Groups].  You can join the list and interact via email, or just post from the web itself.
 # Author
-GAM is maintained by <a href="mailto:jay0lee@gmail.com">Jay Lee</a>. Please direct "how do I?" questions to the mailing list.
+GAM is maintained by <a href="mailto:jay0lee@gmail.com">Jay Lee</a>. Please direct "how do I?" questions to [Google Groups].
 
 [GAM release]: https://git.io/gamreleases
 [GitHub Releases]: https://github.com/jay0lee/GAM/releases

src/GamCommands.txt

@@ -1,6 +1,8 @@
 This document describes the GAM command line syntax in  modified BNF, see https://en.wikipedia.org/wiki/Backus-Naur_Form
+Skip the History section and start reading at Introduction.
+
 Items on the command line are space separated, when an actual space character is required, it will be indicated by <Space>.
-If an item contains spaces, it should be surrounded by " or '.
+If an item contains spaces, it should be surrounded by ".
 
 [] optional item
 () group items
@@ -13,21 +15,37 @@ Primitives
 <Number> ::= <Digit>+
 <Hex> ::= <Digit>|a|b|c|d|e|f|A|B|C|D|E|F
 <Space> ::= an actual space character
-<String> ::= a string of characters, surrounded by " or ' if it contains spaces
+<String> ::= a string of characters, surrounded by " if it contains spaces
 <TrueValues> ::= true|on|yes|enabled|1
 <FalseValues>= false|off|no|disabled|0
 <DataTransferService> ::= googledrive|gdrive|drive|"drive and docs"
-<ProductID> ::= Google-Apps|Google-Drive-storage|Google-Vault|Google-Coordinate
-<SKUID> ::= gafb|gafw|basic|gsuite-basic|
-	    gafg|gsuite-government|
-	    gal|lite|gsuite-lite|
-	    gams|postini|gsuite-gams|
-	    gau|unlimited|gsuite-business|
-	    drive-20gb|drive20gb|20gb|drive-50gb|drive50gb|50gb|drive-200gb|drive200gb|200gb|drive-400gb|drive400gb|400gb|
-	    drive-1tb|drive1tb|1tb|drive-2tb|drive2tb|2tb|drive-4tb|drive4tb|4tb|drive-8tb|drive8tb|8tb|drive-16tb|drive16tb|16tb|
-	    vault|
-	    vfe|
-	    coordinate
+<ProductID> ::=
+	Google-Apps|
+	Google-Chrome-Device-Management|
+	Google-Coordinate|
+	Google-Drive-storage|
+	Google-Vault
+<SKUID> ::=
+	free|standard|Google-Apps|
+	gafb|gafw|basic|gsuitebasic|Google-Apps-For-Business|
+	gafg|gsuitegovernment|gsuitegov|Google-Apps-For-Government|
+	gams|postini|gsuitegams|gsuitepostini|gsuitemessagesecurity|Google-Apps-For-Postini|
+	gal|lite|gsuitelite|Google-Apps-Lite|
+	gau|unlimited|gsuitebusiness|Google-Apps-Unlimited|
+	gae|enterprise|gsuiteenterprise|1010020020|
+	chrome|cdm|googlechromedevicemanagement|Google-Chrome-Device-Management|
+	coordinate|googlecoordinate|Google-Coordinate|
+	drive20gb|20gb|googledrivestorage20gb|Google-Drive-storage-20GB|
+	drive50gb|50gb|googledrivestorage50gb|Google-Drive-storage-50GB|
+	drive200gb|200gb|googledrivestorage200gb|Google-Drive-storage-200GB|
+	drive400gb|400gb|googledrivestorage400gb|Google-Drive-storage-400GB|
+	drive1tb|1tb|googledrivestorage1tb|Google-Drive-storage-1TB|
+	drive2tb|2tb|googledrivestorage2tb|Google-Drive-storage-2TB|
+	drive4tb|4tb|googledrivestorage4tb|Google-Drive-storage-4TB|
+	drive8tb|8tb|googledrivestorage8tb|Google-Drive-storage-8TB|
+	drive16tb|16tb|googledrivestorage16tb|Google-Drive-storage-16TB|
+	vault|googlevault|Google-Vault|
+	vfe|googlevaultformeremployee|Google-Vault-Former-Employee
 <Charset> ::= ascii|mbcs|utf-8|utf-8-sig|utf-16|<String>
 <FileFormat> ::= csv|html|txt|tsv|jpeg|jpg|png|svg|pdf|rtf|pptx|xlsx|docx|odt|ods|openoffice|ms|microsoft|micro$oft
 <Language> ::= ar|bn|bg|ca|zh-CN|zh-TW|hr|cs|da|nl|en|en-GB|et|fi|fr|de|el|gu|iw|is|in|it|ja|kn|ko|lv|lt|ms|ml|mr|no|or|fa|pl|pt-BR|pt-PT|ro|ru|sr|sk|sl|es|sv|tl|ta|te|th|tr|uk|ur|vi
@@ -67,7 +85,7 @@ Named items
 <CrOSItem> ::= <CrOSID>|(query:<QueryCrOS>)
 <DestEmailAddress> ::= <EmailAddress>
 <DomainAlias> ::= <String>
-<DriveFileACLRole> :: =commenter|editor|owner|reader|writer
+<DriveFileACLRole> :: =commenter|editor|organizer|owner|reader|writer
 <DriveFileID> ::= <String>
 <DriveFileURL> :: = https://docs.google.com/a/<DomainName>/document/d/<DriveFileID>/<String>
 <DriveFileItem> ::= <DriveFileID>|<DriveFileURL>
@@ -109,7 +127,7 @@ Named items
 <QueryCalendar> ::= <String>
 <QueryContact> ::= <String> See: https://developers.google.com/google-apps/contacts/v3/reference#contacts-query-parameters-reference
 <QueryCrOS> ::= <String> See: https://support.google.com/chrome/a/answer/1698333?hl=en
-<QueryDriveFile> :: = <String> See: https://developers.google.com/drive/v2/web/search-parameters 
+<QueryDriveFile> :: = <String> See: https://developers.google.com/drive/v2/web/search-parameters
 <QueryGmail> ::= <String> See: https://support.google.com/mail/answer/7190
 <QueryMobile> ::= <String> See: https://support.google.com/a/answer/1408863?hl=en#search
 <QueryPrinter> ::= <String> See: https://developers.google.com/cloud-print/docs/appInterfaces#search
@@ -121,13 +139,34 @@ Named items
 <RoleAssignmentID> ::= <String>
 <SchemaName> ::= <String>
 <Section> ::= <String>
+<S/MIMEID> ::= <String>	
 <StudentItem> ::= <EmailAddress>|<UniqueID>|<String>
+<TeamDriveID> ::= <String>
 <Timezone> ::= <String>
 <Title> ::= <String>
 <URI> ::= <String>
 <URL> ::= <String>
 <UserItem> ::= <EmailAddress>|<UniqueID>|<String>
 
+<CourseFieldName> ::=
+	alternatelink|
+	coursegroupemail|
+	coursematerialsets|
+	coursestate|
+	creationtime|
+	description|
+	descriptionheading|
+	enrollmentcode|
+	guardiansenabled|
+	id|
+	name|
+	ownerid|
+	room|
+	section|
+	teacherfolder|
+	teachergroupemail|
+	updatetime
+<CourseFieldNameList> ::= "<CourseFieldName>(,<CourseFieldName>)*"
 
 <CrOSFieldName> ::=
 	activetimeranges|timeranges|
@@ -231,6 +270,7 @@ Named items
 	customreplyto|
 	defaultmessagedenynotificationtext|
 	description|
+	directmemberscount|
 	email|
 	id|
 	includeinglobaladdresslist|gal|
@@ -289,6 +329,8 @@ Named items
 	ipwhitelisted|
 	isdelegatedadmin|admin|isadmin|
 	ismailboxsetup|
+	is2svenforced|
+	is2svenrolled|
 	lastlogintime|
 	noneditablealiases|aliases|nicknames|
 	notes|note|
@@ -308,35 +350,35 @@ Named Lists
 Lists can be in the following formats
 Items, separated by commas, without spaces or commas in the items themselves: item(,item)*
 Items, separated by spaces, without spaces or commas in the items themselves: "item( item)*"
-Items, separated by commas, with spaces or commas in the items themselves: "'it em'(,'it em')*"
-Items, separated by spaces, with spaces or commas in the items themselves: "'it em'( 'it em')*"
+Items, separated by commas, with spaces or commas in the items themselves: "'it em'(,'it,em')*"
+Items, separated by spaces, with spaces or commas in the items themselves: "'it em'( 'it,em')*"
 
-<ACLList> ::== '<ACLScope>(,<ACLScope>)*'
-<CalendarList> ::= '<CalendarItem>(,<CalendarItem>)*'
-<CourseAliasList> ::= '<CourseAlias>(,<CourseAlias>)*'
-<CourseIDList> ::= '<CourseID>(,<CourseID>)*'
-<CrOSFieldNameList> ::= '<CrOSFieldName>(,<CrOSFieldName>)*'
-<CrOSList> ::= '<CrOSID>(,<CrOSID>)*'
-<DriveFileList> ::= '<DriveFileItem>(,<DriveFileItem>)*'
-<EmailAddressList> ::= '<EmailAddress>(,<EmailAddress>)*'
-<EventIDList> ::= '<EventID>(,<EventID>)*'
-<FileFormatList> ::= '<FileFormat>(,<FileFormat)*'
-<FilterIDList> ::= '<FilterID>(,<FilterID>)*'
-<GroupFieldNameList> ::= '<GroupFieldName>(,<GroupFieldName>)*'
-<GroupList> ::= '<GroupItem>(,<GroupItem>)*'
-<GuardianStateList> ::= '<GuardianState>(,<GuardianState>)*'
-<LabelNameList> ::= '<LabelName>(,<LabelName)*'
-<MembersFieldNameList> ::= '<MembersFieldName>(,<MembersFieldName>)*'
-<MobileList> ::= '<MobileId>(,<MobileId>)*'
-<OrgUnitList> ::== '<OrgUnitPath>(,<OrgUnitPath>)*'
-<PrinterIDList> ::= '<PrinterID>(,<PrinterID>)*'
-<ProductIDList> ::= '(<ProductID>|SKUID>)(,<ProductID>|SKUID>)*'
-<PrintJobIDList> ::= '<PrintJobID>(,<PrintJobID>)*'
-<ResourceIDList> ::= '<ResourceID>(,<ResourceID>)*'
-<SKUIDList> ='<SKUID>(,<SKUID>)*'
-<SchemaNameList> ::= '<SchemaName>(,<SchemaName>)*'
-<UserFieldNameList> ::= '<UserFieldName>(,<UserFieldName>)*'
-<UserList> ::= '<UserItem>(,<UserItem>)*'
+<ACLList> ::== "<ACLScope>(,<ACLScope>)*"
+<CalendarList> ::= "<CalendarItem>(,<CalendarItem>)*"
+<CourseAliasList> ::= "<CourseAlias>(,<CourseAlias>)*"
+<CourseIDList> ::= "<CourseID>(,<CourseID>)*"
+<CrOSFieldNameList> ::= "<CrOSFieldName>(,<CrOSFieldName>)*"
+<CrOSList> ::= "<CrOSID>(,<CrOSID>)*"
+<DriveFileList> ::= "<DriveFileItem>(,<DriveFileItem>)*"
+<EmailAddressList> ::= "<EmailAddress>(,<EmailAddress>)*"
+<EventIDList> ::= "<EventID>(,<EventID>)*"
+<FileFormatList> ::= "<FileFormat>(,<FileFormat)*"
+<FilterIDList> ::= "<FilterID>(,<FilterID>)*"
+<GroupFieldNameList> ::= "<GroupFieldName>(,<GroupFieldName>)*"
+<GroupList> ::= "<GroupItem>(,<GroupItem>)*"
+<GuardianStateList> ::= "<GuardianState>(,<GuardianState>)*"
+<LabelNameList> ::= "<LabelName>(,<LabelName)*"
+<MembersFieldNameList> ::= "<MembersFieldName>(,<MembersFieldName>)*"
+<MobileList> ::= "<MobileId>(,<MobileId>)*"
+<OrgUnitList> ::== "<OrgUnitPath>(,<OrgUnitPath>)*"
+<PrinterIDList> ::= "<PrinterID>(,<PrinterID>)*"
+<ProductIDList> ::= "(<ProductID>|SKUID>)(,<ProductID>|SKUID>)*"
+<PrintJobIDList> ::= "<PrintJobID>(,<PrintJobID>)*"
+<ResourceIDList> ::= "<ResourceID>(,<ResourceID>)*"
+<SKUIDList> ="<SKUID>(,<SKUID>)*"
+<SchemaNameList> ::= "<SchemaName>(,<SchemaName>)*"
+<UserFieldNameList> ::= "<UserFieldName>(,<UserFieldName>)*"
+<UserList> ::= "<UserItem>(,<UserItem>)*"
 
 Specify a collection of ChromeOS devices by directly specifying them
 <CrOSTypeEntity> ::=
@@ -438,7 +480,7 @@ Item attributes
 	(defaultdisplayname <String>)|(description <String>)|(displayname <String>)|(firmware <String>)|(gcpversion <String>)|
 	(istosaccepted <Boolean>)|(manufacturer <String>)|(model <String>)|(name <String>)|(ownerid <EmailAddress>)|(proxy <String>)|(public <Boolean>)|
 	(quotaenabled <Boolean>)|(status <Number>)|(type <String>)|(uuid <String>)|
-	(setupurl <URL>)|(supporturl <URL>)|(updateurl <URL>) 
+	(setupurl <URL>)|(supporturl <URL>)|(updateurl <URL>)
 
 <SchemaFieldDefinition> ::=
 	field <FieldName> (type bool|date|double|email|int64|phone|string) [multivalued|multivalue] [indexed] [restricted] [range <Number> <Number>] endfield
@@ -459,7 +501,7 @@ Item attributes
 	(im|ims clear|(type work|home|other|(custom <String>) protocol aim|gtalk|icq|jabber|msn|net_meeting|qq|skype|yahoo|(custom_protocol <String>) <String> [notprimary|primary]))|
 	(ipwhitelisted <Boolean>)|
 	(lastname|familyname <String>)|
-	(note|notes clear|([text_plain|text_html] <String>|(file <FileName>)))|
+	(note|notes clear|([text_plain|text_html] <String>|(file <FileName> [charset <CharSet>])))|
 	(organization|organizations clear|([type domain_only|school|unknown|work] [customtype <String>] [name <String>] [title <String>] [department <String>] [symbol <String>]
 		[costcenter <String>]  [location <String>] [description <String>] [domain <String>] notprimary|primary))|
 	(org|ou|orgunitpath <OrgUnitPath>)
@@ -469,7 +511,7 @@ Item attributes
 	(relation|relations clear|(spouse|child|mother|father|parent|brother|sister|friend|relative|domestic_partner|manager|assistant|referred_by|partner|<String> <String>))|
 	(suspended <Boolean>)|
 	(website|websites clear|(home_page|blog|profile|work|home|other|ftp|reservations|app_install_page|<String> <URL> [notprimary|primary]))|
-	(<SchemaName>.<FieldName> [multivalued|multivalue|value [type work|home|other|(custom <String>)]] <String>)
+	(<SchemaName>.<FieldName> [multivalued|multivalue|value|multinonempty [type work|home|other|(custom <String>)]] <String>)
 
 gam version [check] [simple]
 gam help
@@ -481,7 +523,7 @@ You can make substitutions in <GAMArgumentList> with values from the CSV file.
 An argument containing exactly ~xxx is replaced by the value of field xxx from the CSV file
 An argument containing instances of ~~xxx~~ has xxx replaced by the value of field xxx from the CSV file
 
-Example: gam csv Users.csv gam update user '~primaryEmail' address type work unstructured '~~Street~~, ~~City~~, ~~State~~ ~~ZIP~~'
+Example: gam csv Users.csv gam update user "~primaryEmail" address type work unstructured "~~Street~~, ~~City~~, ~~State~~ ~~ZIP~~"
 Each user (~primaryEmail, e.g. foo@bar.com) would have their work address updated
 
 gam create project [<EmailAddress>]
@@ -494,6 +536,27 @@ gam <UserTypeEntity> check serviceaccount
 
 gam whatis <EmailItem>
 
+gam create resoldcustomer <CustomerDomain> (customer_auth_token <String>)
+	(email|alternate_email <EmailAddress>) (name|organization_name <String>) (contact|contact_name <String>) [phone|phone_number <String>]
+	[address|address1 <String>] [address2 <String>] [address3 <String>]
+	[locality|city <String>] [region|state <String>] [postal|postal_code <String>] [country|country_code <String>]
+gam update resoldcustomer <CustomerID> [customer_auth_token <String>]
+	[email|alternate_email <EmailAddress>] [name|organization_name <String>] [contact|contact_name <String>] [phone|phone_number <String>]
+	[address|address1 <String>] [address2 <String>] [address3 <String>]
+	[locality|city <String>] [region|state <String>] [postal|postal_code <String>] [country|country_code <String>]
+gam info resoldcustomer <CustomerID>
+
+gam create resoldsubscription <CustomerID> (sku <SKUID>)
+	 (plan annual_monthly_pay|annual_yearly_pay|flexible|trial) (seats <NumberOfSeats> <MaximumNumberOfSeats>)
+	 [customer_auth_token <String>] [deal <String>] [purchaseorderid <String>]
+gam update resoldsubscription <CustomerID> <SKUID>
+	activate|suspend|startpaidservice|
+	(renewal auto_renew_monthly_pay|auto_renew_yearly_pay|cancel|renew_current_users_monthly_pay|renew_current_users_yearly_pay|switch_to_pay_as_you_go)|
+	(seats <NumberOfSeats> [<MaximumNumberOfSeats>])|
+	(plan annual_monthly_pay|annual_yearly_pay|flexible|trial [deal <String>] [purchaseorderid <String>] [seats <NumberOfSeats> [<MaximumNumberOfSeats>]])
+gam delete resoldsubscription <CustomerID> <SKUID> cancel|downgrade|transfer_to_direct
+gam info resoldsubscriptions <CustomerID> [customer_auth_token <String>]
+
 gam report users|user [todrive]
 	[date <Date>] [(user all|<UserItem>)] [filter|filters <String>] [fields|parameters <String>]
 gam report customers|customer|domain [todrive]
@@ -552,11 +615,11 @@ gam calendar <CalendarItem> deleteevent (id|eventid <EventID>)* (query|eventquer
 gam calendar <CalendarItem> wipe
 
 gam update cros <CrOSItem> (<CrOSAttributes>+)|(action deprovision_same_model_replace|deprovision_different_model_replace|deprovision_retiring_device|disable|reenable [acknowledge_device_touch_requirement])
-gam info cros <CrOSItem> [nolists] [listlimit <Number>]
+gam info cros <CrOSItem> [nolists] [listlimit <Number>] [start <Date>] [end <Date>]
 	[basic|full|allfields] <CrOSFieldName>* [fields <CrOSFieldNameList>]
 
-gam print cros [todrive] [query <QueryCrOS>]
-	[orderby <CrOSOrderByFieldName> [ascending|descending]] [nolists] [listlimit <Number>]
+gam print cros [todrive] [query <QueryCrOS>] [limittoou <OrgUnitItem>]
+	[orderby <CrOSOrderByFieldName> [ascending|descending]] [nolists] [listlimit <Number>] [start <Date>] [end <Date>]
 	[basic|full|allfields] <CrOSFieldName>* [fields <CrOSFieldNameList>]
 gam <CrOSTypeEntity> print
 
@@ -570,11 +633,27 @@ Prints a header row and selected fields for specified CrOS devices.
 
 The basic argument yields these column headers: deviceId,annotatedAssetId,annotatedLocation,annotatedUser,lastSync,notes,serialNumber,status
 The full argument yields all column headers including two headers, recentUsers and activeTimeRanges,
-that repeat with two subvalues each, yielding a large number of columns that make the output hard to process.
+that repeat with two/four subvalues each, yielding a large number of columns that make the output hard to process.
 The nolists argument suppresses these two headers; if you want these headers in a more manageable form use the following arguments.
-The listlimit <Number> argument limits the number of repetitions to <Number>. If <Number> equals zero, there is no limit.
-If recentusers is specified as a field, each pair of values for recentUsers is put on a separate row with all of the other headers.
-If timeranges is specified as a field, each  pair of values for activeTimeRanges is put on a separate row with all of the other headers.
+If recentusers is specified, for each recent user, the columns recentUsers.email and recentUsers.type are output on a separate row
+with all of the other headers.
+If timeranges is specified, for each time range entry, the columns activeTimeRanges.date, activeTimeRange.activeTime,
+activeTimeRanges.duration and activeTimeRanges.minutes are output on a separate row with all of the other headers.
+The listlimit <Number> argument limits the number of repetitions to <Number>; if not specified or <Number> equals zero, there is no limit.
+The start <Date> and end <Date> arguments filter the time ranges.
+
+gam print crosactivity [todrive] [query <QueryCrOS>] [limittoou <OrgUnitItem>]
+	[recentusers] [timeranges] [both] [listlimit <Number>] [start <Date>] [end <Date>] [delimiter <String>]
+
+The basic column headers are: deviceId,annotatedAssetId,annotatedLocation,serialNumber,orgUnitPath.
+If recentusers is specified, all of the recent users email addresses, separated by the delimiter <String>,
+with header recentUsers.email, are output with the basic headers.
+If timeranges is specified, for each time range entry, activeTimeRanges.date and activeTimeRanges.duration and activeTimeRanges.minutes,
+are output on a separate row with the basic headers.
+The default is to include both recentusers and timeranges.
+The listlimit <Number> argument limits the number of recent users and time ranges to <Number>; if not specified or <Number> equals zero, there is no limit.
+The start <Date> and end <Date> arguments filter the time ranges.
+Delimiter defaults to comma.
 
 gam update mobile <MobileItem> <MobileAttributes>+
 gam delete mobile <MobileItem>
@@ -617,7 +696,7 @@ gam info schema <SchemaName>
 gam show schema|schemas
 gam print schema|schemas
 
-gam create user <EmailAddress> <UserAttrubutes>*
+gam create user <EmailAddress> <UserAttributes>*
 gam update user <UserItem> <UserAttributes>* [clearschema <SchemaName>] [clearschema <SchemaName>.<FieldName>]
 gam delete user <UserItem>
 gam undelete user <UserItem> [org|ou <OrgUnitPath>]
@@ -626,7 +705,7 @@ gam info user [<UserItem>] [noaliases] [nogroups] [nolicenses|nolicences] [nosch
 gam print users [todrive] ([domain <DomainName>] [query <QueryUser>] [deleted_only|only_deleted])
 	[groups] [license|licenses|licence|licences] [emailpart|emailparts|username]
 	[orderby <UserOrderByFieldName> [ascending|descending]] [userview]
-	[basic|full|allfields | <UserFieldName>* | fields <UserFieldNameList>] [schemas|custom all|<SchemaNameList>]
+	[allfields|basic|full | ((<UserFieldName>* | fields <UserFieldNameList>) [schemas|custom all|<SchemaNameList>])]
 gam <UserTypeEntity> print
 
 Summary of printing:
@@ -644,6 +723,7 @@ gam update course <CourseID> <CourseAttributes>+
 gam delete course <CourseID>
 gam info course <CourseID>
 gam print courses [todrive] [teacher <UserItem>] [student <UserItem>] [alias|aliases] [delimiter <String>]
+	[show all|students|teachers] [countsonly] [fields <CourseFieldNameList>] [skipfields <CourseFieldNameList>]
 
 gam course <CourseID> add alias <CourseAlias>
 gam course <CourseID> delete alias <CourseAlias>
@@ -693,7 +773,7 @@ gam <UserTypeEntity> delete|del backupcodes|backupcode|verificationcodes
 gam <UserTypeEntity> show backupcodes|backupcode|verificationcodes
 
 gam <UserTypeEntity> add calendar <CalendarItem> <CalendarAttributes>*
-gam <UserTypeEntity> update calendar <CalendarItem> <CalendarAttributes>+
+gam <UserTypeEntity> update calendar <CalendarItem>|primary <CalendarAttributes>+
 gam <UserTypeEntity> delete|del calendar <CalendarItem>
 gam <UserTypeEntity> show calendars
 gam <UserTypeEntity> info calendar <CalendarItem>|primary
@@ -721,9 +801,9 @@ gam <UserTypeEntity> delete|del emptydrivefolders
 gam <UserTypeEntity> empty drivetrash
 
 gam <UserTypeEntity> add drivefileacl <DriveFileID> anyone|(user <UserItem>)|(group <GroupItem>)|(domain <DomainName>)
-	(role <DriveFileACLRole>) [withlink] [sendemail] [emailmessage <String>]
+	(role <DriveFileACLRole>) [withlink|discoverable] [sendemail] [emailmessage <String>]
 gam <UserTypeEntity> update drivefileacl <DriveFileID> <PermissionID>
-	(role <DriveFileACLRole>) [withlink] [transferownership <Boolean>]
+	(role <DriveFileACLRole>) [withlink|discoverable] [removeexpiration]
 gam <UserTypeEntity> delete|del drivefileacl <DriveFileID> <PermissionID>
 gam <UserTypeEntity> show drivefileacl <DriveFileID>
 
@@ -731,9 +811,9 @@ gam <UserTypeEntity> delete|del alias|aliases
 
 gam <UserTypeEntity> delete|del group|groups
 
-gam <UserTypeEntity> add license <SKUID>
-gam <UserTypeEntity> update license <SKUID> [from] <SKUID>
-gam <UserTypeEntity> delete|del license <SKUID>
+gam <UserTypeEntity> add license <SKUID> [product|productid <ProductID>]
+gam <UserTypeEntity> update license <SKUID> [product|productid <ProductID>] [from] <SKUID>
+gam <UserTypeEntity> delete|del license <SKUID> [product|productid <ProductID>]
 
 gam <UserTypeEntity> update photo <FileNamePattern>
 gam <UserTypeEntity> delete|del photo
@@ -747,7 +827,7 @@ gam <UserTypeEntity> show tokens|token [clientid <ClientID>]
 gam <UserTypeEntity> print tokens|token [todrive] [clientid <ClientID>]
 gam print tokens|token [todrive] [clientid <ClientID>] [<UserTypeEntity>]
 
-gam <UserTypeEntity> update user <UserAttrubutes>
+gam <UserTypeEntity> update user <UserAttributes>
 
 gam <UserTypeEntity> deprovision|deprov
 
@@ -802,9 +882,21 @@ gam <UserTypeEntity> show sendas [format]
 gam <UserTypeEntity> info sendas <EmailAddress> [format]
 gam <UserTypeEntity> print sendas [todrive]
 
+gam <UserTypeEntity> add smime file <FileName> [password <Password>] [sendas|sendasemail <EmailAddress>] [default]
+gam <UserTypeEntity> update smime [id <S/MIMEID>] [sendas|sendasemail <EmailAddress>] [default]
+gam <UserTypeEntity> delete smime [id <S/MIMEID>] [sendas|sendasemail <EmailAddress>]
+gam <UserTypeEntity> show smime [primaryonly]
+gam <UserTypeEntity> print smime [todrive] [primaryonly]
+
 gam <UserTypeEntity> signature|sig <String>|(file <FileName> [charset <Charset>]) (replace <Tag> <String>)* [html] [name <String>] [replyto <EmailAddress>] [default] [treatasalias <Boolean>]
 gam <UserTypeEntity> show signature|sig [format]
 
+gam <UserTypeEntity> add teamdrive <Name>
+gam <UserTypeEntity> update teamdrive <TeamDriveID> [name <Name>]
+gam <UserTypeEntity> delete teamdrive <TeamDriveID>
+gam <UserTypeEntity> show teamdrives
+gam <UserTypeEntity> print teamdrives [todrive]
+
 gam <UserTypeEntity> vacation <FalseValues>
 gam <UserTypeEntity> vacation <TrueValues> subject <String> (message <String>)|(file <FileName> [charset <CharSet>]) (replace <Tag> <String>)* [html]
 	[contactsonly] [domainonly] [startdate <Date>] [enddate <Date>]

src/gam-install.sh

@@ -10,6 +10,7 @@ OPTIONS:
    -d      Directory where gam folder will be installed. Default is \$HOME/bin/
    -a      Architecture to install (i386, x86_64, arm). Default is to detect your arch with "uname -m".
    -o      OS we are running (linux, macos). Default is to detect your OS with "uname -s".
+   -l      Just upgrade GAM to latest version. Skips project creation and auth.
    -p      Profile update (true, false). Should script add gam command to environment. Default is true.
    -u      Admin user email address to use with GAM. Default is to prompt.
    -r      Regular user email address. Used to test service account access to user data. Default is to prompt.
@@ -21,20 +22,22 @@ target_dir="$HOME/bin"
 gamarch=$(uname -m)
 gamos=$(uname -s)
 update_profile=true
+upgrade_only=false
 gamversion="latest"
 adminuser=""
 regularuser=""
-while getopts "hd:a:o:p:u:r:v:" OPTION
+while getopts "hd:a:o:lp:u:r:v:" OPTION
 do
      case $OPTION in
          h) usage; exit;;
-         d) target_dir=$OPTARG;;
-         a) gamarch=$OPTARG;;
-         o) gamos=$OPTARG;;
-         p) update_profile=$OPTARG;;
-         u) adminuser=$OPTARG;;
-         r) regularuser=$OPTARG;;
-         v) gamversion=$OPTARG;;
+         d) target_dir="$OPTARG";;
+         a) gamarch="$OPTARG";;
+         o) gamos="$OPTARG";;
+         l) upgrade_only=true;;
+         p) update_profile="$OPTARG";;
+         u) adminuser="$OPTARG";;
+         r) regularuser="$OPTARG";;
+         v) gamversion="$OPTARG";;
          ?) usage; exit;;
      esac
 done
@@ -154,10 +157,10 @@ echo_yellow "Downloading file $name from $browser_download_url to $temp_archive_
 # Save archive to temp w/o losing our path
 (cd $temp_archive_dir && curl -O -L $browser_download_url)
 
-mkdir -p $target_dir
+mkdir -p "$target_dir"
 
 echo_yellow "Extracting archive to $target_dir"
-tar xf $temp_archive_dir/$name -C $target_dir
+tar xf $temp_archive_dir/$name -C "$target_dir"
 rc=$?
 if (( $rc != 0 )); then
   echo_red "ERROR: extracting the GAM archive with tar failed with error $rc. Exiting."
@@ -166,6 +169,19 @@ else
   echo_green "Finished extracting GAM archive."
 fi
 
+if [ "$upgrade_only" = true ]; then
+  echo_green "Here's information about your GAM upgrade:"
+  "$target_dir/gam/gam" version
+  rc=$?
+  if (( $rc != 0 )); then
+    echo_red "ERROR: Failed running GAM for the first time with $rc. Please report this error to GAM mailing list. Exiting."
+    exit
+  fi
+
+  echo_green "GAM upgrade complete!"
+  exit
+fi
+
 # Update profile to add gam command
 if [ "$update_profile" = true ]; then
   alias_line="alias gam=\"$target_dir/gam/gam\""
@@ -185,7 +201,7 @@ while true; do
       break
       ;;
     [Nn]*)
-      touch $target_dir/gam/nobrowser.txt > /dev/null 2>&1
+      touch "$target_dir/gam/nobrowser.txt" > /dev/null 2>&1
       break
       ;;
     *)
@@ -203,14 +219,14 @@ while true; do
       if [ "$adminuser" == "" ]; then
         read -p "Please enter your G Suite admin email address: " adminuser
       fi
-      $target_dir/gam/gam create project $adminuser
+      "$target_dir/gam/gam" create project $adminuser
       rc=$?
       if (( $rc == 0 )); then
         echo_green "Project creation complete."
         project_created=true
         break
       else
-        echo_red "Projection creation failed. Trying again. Say N to skip projection creation."
+        echo_red "Project creation failed. Trying again. Say N to skip project creation."
       fi
       ;;
     [Nn]*)
@@ -228,7 +244,7 @@ while $project_created; do
   read -p "Are you ready to authorize GAM to perform G Suite management operations as your admin account? (yes or no) " yn
   case $yn in
     [Yy]*)
-      $target_dir/gam/gam oauth create $adminuser
+      "$target_dir/gam/gam" oauth create $adminuser
       rc=$?
       if (( $rc == 0 )); then
         echo_green "Admin authorization complete."
@@ -257,7 +273,7 @@ while $project_created; do
         read -p "Please enter the email address of a regular G Suite user: " regularuser
       fi
       echo_yellow "Great! Checking service account scopes.This will fail the first time. Follow the steps to authorize and retry. It can take a few minutes for scopes to PASS after they've been authorized in the admin console."
-      $target_dir/gam/gam user $adminuser check serviceaccount
+      "$target_dir/gam/gam" user $adminuser check serviceaccount
       rc=$?
       if (( $rc == 0 )); then
         echo_green "Service account authorization complete."
@@ -278,7 +294,7 @@ while $project_created; do
 done
 
 echo_green "Here's information about your new GAM installation:"
-$target_dir/gam/gam version
+"$target_dir/gam/gam" version
 rc=$?
 if (( $rc != 0 )); then
   echo_red "ERROR: Failed running GAM for the first time with $rc. Please report this error to GAM mailing list. Exiting."

src/googleapiclient/__init__.py

@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-__version__ = "1.6.1"
+__version__ = "1.6.2"
 
 # Set default logging handler to avoid "No handler found" warnings.
 import logging

src/googleapiclient/_auth.py

@@ -14,8 +14,6 @@
 
 """Helpers for authentication using oauth2client or google-auth."""
 
-import httplib2
-
 try:
     import google.auth
     import google.auth.credentials
@@ -31,6 +29,8 @@ try:
 except ImportError:  # pragma: NO COVER
     HAS_OAUTH2CLIENT = False
 
+from googleapiclient.http import build_http
+
 
 def default_credentials():
     """Returns Application Default Credentials."""
@@ -86,6 +86,7 @@ def authorized_http(credentials):
     """
     if HAS_GOOGLE_AUTH and isinstance(
             credentials, google.auth.credentials.Credentials):
-        return google_auth_httplib2.AuthorizedHttp(credentials)
+        return google_auth_httplib2.AuthorizedHttp(credentials,
+                                                   http=build_http())
     else:
-        return credentials.authorize(httplib2.Http())
+        return credentials.authorize(build_http())

src/googleapiclient/discovery.py

@@ -61,6 +61,7 @@ from googleapiclient.errors import MediaUploadSizeError
 from googleapiclient.errors import UnacceptableMimeTypeError
 from googleapiclient.errors import UnknownApiNameOrVersion
 from googleapiclient.errors import UnknownFileType
+from googleapiclient.http import build_http
 from googleapiclient.http import BatchHttpRequest
 from googleapiclient.http import HttpMock
 from googleapiclient.http import HttpMockSequence
@@ -97,6 +98,7 @@ V2_DISCOVERY_URI = ('https://{api}.googleapis.com/$discovery/rest?'
                     'version={apiVersion}')
 DEFAULT_METHOD_DOC = 'A description of how to use this function'
 HTTP_PAYLOAD_METHODS = frozenset(['PUT', 'POST', 'PATCH'])
+
 _MEDIA_SIZE_BIT_SHIFTS = {'KB': 10, 'MB': 20, 'GB': 30, 'TB': 40}
 BODY_PARAMETER_DEFAULT_VALUE = {
     'description': 'The request body.',
@@ -115,6 +117,7 @@ MEDIA_MIME_TYPE_PARAMETER_DEFAULT_VALUE = {
     'type': 'string',
     'required': False,
 }
+_PAGE_TOKEN_NAMES = ('pageToken', 'nextPageToken')
 
 # Parameters accepted by the stack, but not visible via discovery.
 # TODO(dhermes): Remove 'userip' in 'v2'.
@@ -213,7 +216,10 @@ def build(serviceName,
       'apiVersion': version
       }
 
-  discovery_http = http if http is not None else httplib2.Http()
+  if http is None:
+    discovery_http = build_http()
+  else:
+    discovery_http = http
 
   for discovery_url in (discoveryServiceUrl, V2_DISCOVERY_URI,):
     requested_url = uritemplate.expand(discovery_url, params)
@@ -328,6 +334,10 @@ def build_from_document(
   if http is not None and credentials is not None:
     raise ValueError('Arguments http and credentials are mutually exclusive.')
 
+  if developerKey is not None and credentials is not None:
+    raise ValueError(
+      'Arguments developerKey and credentials are mutually exclusive.')
+
   if isinstance(service, six.string_types):
     service = json.loads(service)
 
@@ -350,8 +360,9 @@ def build_from_document(
     scopes = list(
       service.get('auth', {}).get('oauth2', {}).get('scopes', {}).keys())
 
-    # If so, then the we need to setup authentication.
-    if scopes:
+    # If so, then the we need to setup authentication if no developerKey is
+    # specified.
+    if scopes and not developerKey:
       # If the user didn't pass in credentials, attempt to acquire application
       # default credentials.
       if credentials is None:
@@ -366,7 +377,7 @@ def build_from_document(
     # If the service doesn't require scopes then there is no need for
     # authentication.
     else:
-      http = httplib2.Http()
+      http = build_http()
 
   if model is None:
     features = service.get('features', [])
@@ -718,7 +729,11 @@ def createMethod(methodName, methodDesc, rootDesc, schema):
 
     for name in parameters.required_params:
       if name not in kwargs:
-        raise TypeError('Missing required parameter "%s"' % name)
+        # temporary workaround for non-paging methods incorrectly requiring
+        # page token parameter (cf. drive.changes.watch vs. drive.changes.list)
+        if name not in _PAGE_TOKEN_NAMES or _findPageTokenName(
+            _methodProperties(methodDesc, schema, 'response')):
+          raise TypeError('Missing required parameter "%s"' % name)
 
     for name, regex in six.iteritems(parameters.pattern_params):
       if name in kwargs:
@@ -921,13 +936,20 @@ def createMethod(methodName, methodDesc, rootDesc, schema):
   return (methodName, method)
 
 
-def createNextMethod(methodName):
+def createNextMethod(methodName,
+                     pageTokenName='pageToken',
+                     nextPageTokenName='nextPageToken',
+                     isPageTokenParameter=True):
   """Creates any _next methods for attaching to a Resource.
 
   The _next methods allow for easy iteration through list() responses.
 
   Args:
     methodName: string, name of the method to use.
+    pageTokenName: string, name of request page token field.
+    nextPageTokenName: string, name of response page token field.
+    isPageTokenParameter: Boolean, True if request page token is a query
+        parameter, False if request page token is a field of the request body.
   """
   methodName = fix_method_name(methodName)
 
@@ -945,24 +967,24 @@ Returns:
     # Retrieve nextPageToken from previous_response
     # Use as pageToken in previous_request to create new request.
 
-    if 'nextPageToken' not in previous_response or not previous_response['nextPageToken']:
+    nextPageToken = previous_response.get(nextPageTokenName, None)
+    if not nextPageToken:
       return None
 
     request = copy.copy(previous_request)
 
-    pageToken = previous_response['nextPageToken']
-    parsed = list(urlparse(request.uri))
-    q = parse_qsl(parsed[4])
-
-    # Find and remove old 'pageToken' value from URI
-    newq = [(key, value) for (key, value) in q if key != 'pageToken']
-    newq.append(('pageToken', pageToken))
-    parsed[4] = urlencode(newq)
-    uri = urlunparse(parsed)
-
-    request.uri = uri
-
-    logger.info('URL being requested: %s %s' % (methodName,uri))
+    if isPageTokenParameter:
+        # Replace pageToken value in URI
+        request.uri = _add_query_parameter(
+            request.uri, pageTokenName, nextPageToken)
+        logger.info('Next page request URL: %s %s' % (methodName, request.uri))
+    else:
+        # Replace pageToken value in request body
+        model = self._model
+        body = model.deserialize(request.body)
+        body[pageTokenName] = nextPageToken
+        request.body = model.serialize(body)
+        logger.info('Next page request body: %s %s' % (methodName, body))
 
     return request
 
@@ -1110,19 +1132,59 @@ class Resource(object):
                                method.__get__(self, self.__class__))
 
   def _add_next_methods(self, resourceDesc, schema):
-    # Add _next() methods
-    # Look for response bodies in schema that contain nextPageToken, and methods
-    # that take a pageToken parameter.
-    if 'methods' in resourceDesc:
-      for methodName, methodDesc in six.iteritems(resourceDesc['methods']):
-        if 'response' in methodDesc:
-          responseSchema = methodDesc['response']
-          if '$ref' in responseSchema:
-            responseSchema = schema.get(responseSchema['$ref'])
-          hasNextPageToken = 'nextPageToken' in responseSchema.get('properties',
-                                                                   {})
-          hasPageToken = 'pageToken' in methodDesc.get('parameters', {})
-          if hasNextPageToken and hasPageToken:
-            fixedMethodName, method = createNextMethod(methodName + '_next')
-            self._set_dynamic_attr(fixedMethodName,
-                                   method.__get__(self, self.__class__))
+    # Add _next() methods if and only if one of the names 'pageToken' or
+    # 'nextPageToken' occurs among the fields of both the method's response
+    # type either the method's request (query parameters) or request body.
+    if 'methods' not in resourceDesc:
+      return
+    for methodName, methodDesc in six.iteritems(resourceDesc['methods']):
+      nextPageTokenName = _findPageTokenName(
+          _methodProperties(methodDesc, schema, 'response'))
+      if not nextPageTokenName:
+        continue
+      isPageTokenParameter = True
+      pageTokenName = _findPageTokenName(methodDesc.get('parameters', {}))
+      if not pageTokenName:
+        isPageTokenParameter = False
+        pageTokenName = _findPageTokenName(
+            _methodProperties(methodDesc, schema, 'request'))
+      if not pageTokenName:
+        continue
+      fixedMethodName, method = createNextMethod(
+          methodName + '_next', pageTokenName, nextPageTokenName,
+          isPageTokenParameter)
+      self._set_dynamic_attr(fixedMethodName,
+                             method.__get__(self, self.__class__))
+
+
+def _findPageTokenName(fields):
+  """Search field names for one like a page token.
+
+  Args:
+    fields: container of string, names of fields.
+
+  Returns:
+    First name that is either 'pageToken' or 'nextPageToken' if one exists,
+    otherwise None.
+  """
+  return next((tokenName for tokenName in _PAGE_TOKEN_NAMES
+              if tokenName in fields), None)
+
+def _methodProperties(methodDesc, schema, name):
+  """Get properties of a field in a method description.
+
+  Args:
+    methodDesc: object, fragment of deserialized discovery document that
+      describes the method.
+    schema: object, mapping of schema names to schema descriptions.
+    name: string, name of top-level field in method description.
+
+  Returns:
+    Object representing fragment of deserialized discovery document
+    corresponding to 'properties' field of object corresponding to named field
+    in method description, if it exists, otherwise empty dict.
+  """
+  desc = methodDesc.get(name, {})
+  if '$ref' in desc:
+    desc = schema.get(desc['$ref'], {})
+  return desc.get('properties', {})

src/googleapiclient/http.py

@@ -80,6 +80,8 @@ MAX_URI_LENGTH = 2048
 
 _TOO_MANY_REQUESTS = 429
 
+DEFAULT_HTTP_TIMEOUT_SEC = 60
+
 
 def _should_retry_response(resp_status, content):
   """Determines whether a response should be retried.
@@ -815,6 +817,7 @@ class HttpRequest(object):
     if 'content-length' not in self.headers:
       self.headers['content-length'] = str(self.body_size)
     # If the request URI is too long then turn it into a POST request.
+    # Assume that a GET request never contains a request body.
     if len(self.uri) > MAX_URI_LENGTH and self.method == 'GET':
       self.method = 'POST'
       self.headers['x-http-method-override'] = 'GET'
@@ -1732,3 +1735,21 @@ def tunnel_patch(http):
 
   http.request = new_request
   return http
+
+
+def build_http():
+  """Builds httplib2.Http object
+
+  Returns:
+  A httplib2.Http object, which is used to make http requests, and which has timeout set by default.
+  To override default timeout call
+
+    socket.setdefaulttimeout(timeout_in_sec)
+
+  before interacting with this method.
+  """
+  if socket.getdefaulttimeout() is not None:
+    http_timeout = socket.getdefaulttimeout()
+  else:
+    http_timeout = DEFAULT_HTTP_TIMEOUT_SEC
+  return httplib2.Http(timeout=http_timeout)

src/googleapiclient/sample_tools.py

@@ -23,10 +23,10 @@ __all__ = ['init']
 
 
 import argparse
-import httplib2
 import os
 
 from googleapiclient import discovery
+from googleapiclient.http import build_http
 from oauth2client import client
 from oauth2client import file
 from oauth2client import tools
@@ -88,7 +88,7 @@ def init(argv, name, version, doc, filename, scope=None, parents=[], discovery_f
   credentials = storage.get()
   if credentials is None or credentials.invalid:
     credentials = tools.run_flow(flow, storage, flags)
-  http = credentials.authorize(http = httplib2.Http())
+  http = credentials.authorize(http=build_http())
 
   if discovery_filename is None:
     # Construct a service object via the discovery service.

src/googleapiclient/schema.py

@@ -161,13 +161,14 @@ class Schemas(object):
     # Return with trailing comma and newline removed.
     return self._prettyPrintSchema(schema, dent=1)[:-2]
 
-  def get(self, name):
+  def get(self, name, default=None):
     """Get deserialized JSON schema from the schema name.
 
     Args:
       name: string, Schema name.
+      default: object, return value if name not found.
     """
-    return self.schemas[name]
+    return self.schemas.get(name, default)
 
 
 class _SchemaToStruct(object):

src/httplib2/__init__.py

@@ -23,7 +23,7 @@ __contributors__ = ["Thomas Broyer (t.broyer@ltgt.net)",
                     "Louis Nyffenegger",
                     "Alex Yu"]
 __license__ = "MIT"
-__version__ = "0.9.2"
+__version__ = "0.10.3"
 
 import re
 import sys
@@ -65,42 +65,54 @@ except ImportError:
         socks = None
 
 # Build the appropriate socket wrapper for ssl
+ssl = None
+ssl_SSLError = None
+ssl_CertificateError = None
 try:
-    import ssl # python 2.6
-    ssl_SSLError = ssl.SSLError
-    def _ssl_wrap_socket(sock, key_file, cert_file, disable_validation,
-                         ca_certs, ssl_version, hostname):
-        if disable_validation:
-            cert_reqs = ssl.CERT_NONE
-        else:
-            cert_reqs = ssl.CERT_REQUIRED
-        if ssl_version is None:
-            ssl_version = ssl.PROTOCOL_SSLv23
+    import ssl  # python 2.6
+except ImportError:
+    pass
+if ssl is not None:
+    ssl_SSLError = getattr(ssl, 'SSLError', None)
+    ssl_CertificateError = getattr(ssl, 'CertificateError', None)
 
-        if hasattr(ssl, 'SSLContext'): # Python 2.7.9
-            context = ssl.SSLContext(ssl_version)
-            context.verify_mode = cert_reqs
-            context.check_hostname = (cert_reqs != ssl.CERT_NONE)
-            if cert_file:
-                context.load_cert_chain(cert_file, key_file)
-            if ca_certs:
-                context.load_verify_locations(ca_certs)
-            return context.wrap_socket(sock, server_hostname=hostname)
-        else:
-            return ssl.wrap_socket(sock, keyfile=key_file, certfile=cert_file,
-                                   cert_reqs=cert_reqs, ca_certs=ca_certs,
-                                   ssl_version=ssl_version)
-except (AttributeError, ImportError):
-    ssl_SSLError = None
-    def _ssl_wrap_socket(sock, key_file, cert_file, disable_validation,
-                         ca_certs, ssl_version, hostname):
-        if not disable_validation:
-            raise CertificateValidationUnsupported(
-                    "SSL certificate validation is not supported without "
-                    "the ssl module installed. To avoid this error, install "
-                    "the ssl module, or explicity disable validation.")
-        ssl_sock = socket.ssl(sock, key_file, cert_file)
-        return httplib.FakeSocket(sock, ssl_sock)
+
+def _ssl_wrap_socket(sock, key_file, cert_file, disable_validation,
+                     ca_certs, ssl_version, hostname):
+    if disable_validation:
+        cert_reqs = ssl.CERT_NONE
+    else:
+        cert_reqs = ssl.CERT_REQUIRED
+    if ssl_version is None:
+        ssl_version = ssl.PROTOCOL_SSLv23
+
+    if hasattr(ssl, 'SSLContext'):  # Python 2.7.9
+        context = ssl.SSLContext(ssl_version)
+        context.verify_mode = cert_reqs
+        context.check_hostname = (cert_reqs != ssl.CERT_NONE)
+        if cert_file:
+            context.load_cert_chain(cert_file, key_file)
+        if ca_certs:
+            context.load_verify_locations(ca_certs)
+        return context.wrap_socket(sock, server_hostname=hostname)
+    else:
+        return ssl.wrap_socket(sock, keyfile=key_file, certfile=cert_file,
+                               cert_reqs=cert_reqs, ca_certs=ca_certs,
+                               ssl_version=ssl_version)
+
+
+def _ssl_wrap_socket_unsupported(sock, key_file, cert_file, disable_validation,
+                                 ca_certs, ssl_version, hostname):
+    if not disable_validation:
+        raise CertificateValidationUnsupported(
+                "SSL certificate validation is not supported without "
+                "the ssl module installed. To avoid this error, install "
+                "the ssl module, or explicity disable validation.")
+    ssl_sock = socket.ssl(sock, key_file, cert_file)
+    return httplib.FakeSocket(sock, ssl_sock)
+
+if ssl is None:
+    _ssl_wrap_socket = _ssl_wrap_socket_unsupported
 
 
 if sys.version_info >= (2,3):
@@ -269,8 +281,8 @@ def safename(filename):
     filename = re_slash.sub(",", filename)
 
     # limit length of filename
-    if len(filename)>64:
-        filename=filename[:64]
+    if len(filename)>200:
+        filename=filename[:200]
     return ",".join((filename, filemd5))
 
 NORMALIZE_SPACE = re.compile(r'(?:\r\n)?[ \t]+')
@@ -1066,7 +1078,7 @@ class HTTPSConnectionWithTimeout(httplib.HTTPSConnection):
                         raise CertificateHostnameMismatch(
                             'Server presented certificate that does not match '
                             'host %s: %s' % (hostname, cert), hostname, cert)
-            except ssl_SSLError, e:
+            except (ssl_SSLError, ssl_CertificateError, CertificateHostnameMismatch), e:
                 if sock:
                     sock.close()
                 if self.sock:
@@ -1076,7 +1088,7 @@ class HTTPSConnectionWithTimeout(httplib.HTTPSConnection):
                 # to get at more detailed error information, in particular
                 # whether the error is due to certificate validation or
                 # something else (such as SSL protocol mismatch).
-                if e.errno == ssl.SSL_ERROR_SSL:
+                if getattr(e, 'errno', None) == ssl.SSL_ERROR_SSL:
                     raise SSLHandshakeError(e)
                 else:
                     raise
@@ -1155,18 +1167,11 @@ try:
               server_software.startswith('Development/')):
         raise NotRunningAppEngineEnvironment()
 
-    try:
-        from google.appengine.api import apiproxy_stub_map
-        if apiproxy_stub_map.apiproxy.GetStub('urlfetch') is None:
-            raise ImportError  # Bail out; we're not actually running on App Engine.
-        from google.appengine.api.urlfetch import fetch
-        from google.appengine.api.urlfetch import InvalidURLError
-    except (ImportError, AttributeError):
-        from google3.apphosting.api import apiproxy_stub_map
-        if apiproxy_stub_map.apiproxy.GetStub('urlfetch') is None:
-            raise ImportError  # Bail out; we're not actually running on App Engine.
-        from google3.apphosting.api.urlfetch import fetch
-        from google3.apphosting.api.urlfetch import InvalidURLError
+    from google.appengine.api import apiproxy_stub_map
+    if apiproxy_stub_map.apiproxy.GetStub('urlfetch') is None:
+        raise ImportError  # Bail out; we're not actually running on App Engine.
+    from google.appengine.api.urlfetch import fetch
+    from google.appengine.api.urlfetch import InvalidURLError
 
     # Update the connection classes to use the Googel App Engine specific ones.
     SCHEME_TO_CONNECTION = {

src/oauth2client/__init__.py

@@ -14,7 +14,7 @@
 
 """Client library for using OAuth2, especially with Google APIs."""
 
-__version__ = '4.0.0'
+__version__ = '4.1.0'
 
 GOOGLE_AUTH_URI = 'https://accounts.google.com/o/oauth2/v2/auth'
 GOOGLE_DEVICE_URI = 'https://accounts.google.com/o/oauth2/device/code'

src/oauth2client/_helpers.py

@@ -251,8 +251,8 @@ def validate_file(filename):
         raise IOError(_SYM_LINK_MESSAGE.format(filename))
     elif os.path.isdir(filename):
         raise IOError(_IS_DIR_MESSAGE.format(filename))
-    #elif not os.path.isfile(filename):
-    #    warnings.warn(_MISSING_FILE_MESSAGE.format(filename))
+    elif not os.path.isfile(filename):
+        warnings.warn(_MISSING_FILE_MESSAGE.format(filename))
 
 
 def _parse_pem_key(raw_key_input):

src/oauth2client/_pkce.py

@@ -38,7 +38,7 @@ def code_verifier(n_bytes=64):
     Returns:
         Bytestring, representing urlsafe base64-encoded random data.
     """
-    verifier = base64.urlsafe_b64encode(os.urandom(n_bytes))
+    verifier = base64.urlsafe_b64encode(os.urandom(n_bytes)).rstrip(b'=')
     # https://tools.ietf.org/html/rfc7636#section-4.1
     # minimum length of 43 characters and a maximum length of 128 characters.
     if len(verifier) < 43:
@@ -60,6 +60,8 @@ def code_challenge(verifier):
             code_verifier().
 
     Returns:
-        Bytestring, representing a urlsafe base64-encoded sha256 hash digest.
+        Bytestring, representing a urlsafe base64-encoded sha256 hash digest,
+            without '=' padding.
     """
-    return base64.urlsafe_b64encode(hashlib.sha256(verifier).digest())
+    digest = hashlib.sha256(verifier).digest()
+    return base64.urlsafe_b64encode(digest).rstrip(b'=')

src/oauth2client/client.py

@@ -108,7 +108,7 @@ except ValueError:  # pragma: NO COVER
     GCE_METADATA_TIMEOUT = 3
 
 _SERVER_SOFTWARE = 'SERVER_SOFTWARE'
-_GCE_METADATA_URI = 'http://169.254.169.254'
+_GCE_METADATA_URI = 'http://' + os.getenv('GCE_METADATA_IP', '169.254.169.254')
 _METADATA_FLAVOR_HEADER = 'metadata-flavor'  # lowercase header
 _DESIRED_METADATA_FLAVOR = 'Google'
 _GCE_HEADERS = {_METADATA_FLAVOR_HEADER: _DESIRED_METADATA_FLAVOR}
@@ -271,7 +271,7 @@ class Credentials(object):
                 to_serialize[key] = val.decode('utf-8')
             if isinstance(val, set):
                 to_serialize[key] = list(val)
-        return json.dumps(to_serialize, indent=4, sort_keys=True)
+        return json.dumps(to_serialize)
 
     def to_json(self):
         """Creating a JSON representation of an instance of Credentials.
@@ -451,7 +451,7 @@ class OAuth2Credentials(Credentials):
     def __init__(self, access_token, client_id, client_secret, refresh_token,
                  token_expiry, token_uri, user_agent, revoke_uri=None,
                  id_token=None, token_response=None, scopes=None,
-                 token_info_uri=None):
+                 token_info_uri=None, id_token_jwt=None):
         """Create an instance of OAuth2Credentials.
 
         This constructor is not usually called by the user, instead
@@ -474,8 +474,11 @@ class OAuth2Credentials(Credentials):
                             because some providers (e.g. wordpress.com) include
                             extra fields that clients may want.
             scopes: list, authorized scopes for these credentials.
-          token_info_uri: string, the URI for the token info endpoint. Defaults
-                          to None; scopes can not be refreshed if this is None.
+            token_info_uri: string, the URI for the token info endpoint.
+                            Defaults to None; scopes can not be refreshed if
+                            this is None.
+            id_token_jwt: string, the encoded and signed identity JWT. The
+                          decoded version of this is stored in id_token.
 
         Notes:
             store: callable, A callable that when passed a Credential
@@ -493,6 +496,7 @@ class OAuth2Credentials(Credentials):
         self.user_agent = user_agent
         self.revoke_uri = revoke_uri
         self.id_token = id_token
+        self.id_token_jwt = id_token_jwt
         self.token_response = token_response
         self.scopes = set(_helpers.string_to_scopes(scopes or []))
         self.token_info_uri = token_info_uri
@@ -621,6 +625,7 @@ class OAuth2Credentials(Credentials):
             data['user_agent'],
             revoke_uri=data.get('revoke_uri', None),
             id_token=data.get('id_token', None),
+            id_token_jwt=data.get('id_token_jwt', None),
             token_response=data.get('token_response', None),
             scopes=data.get('scopes', None),
             token_info_uri=data.get('token_info_uri', None))
@@ -786,8 +791,10 @@ class OAuth2Credentials(Credentials):
                 self.token_expiry = None
             if 'id_token' in d:
                 self.id_token = _extract_id_token(d['id_token'])
+                self.id_token_jwt = d['id_token']
             else:
                 self.id_token = None
+                self.id_token_jwt = None
             # On temporary refresh errors, the user does not actually have to
             # re-authorize, so we unflag here.
             self.invalid = False
@@ -1771,7 +1778,7 @@ class DeviceFlowInfo(collections.namedtuple('DeviceFlowInfo', (
 def _oauth2_web_server_flow_params(kwargs):
     """Configures redirect URI parameters for OAuth2WebServerFlow."""
     params = {
-#        'access_type': 'offline',
+        'access_type': 'offline',
         'response_type': 'code',
     }
 
@@ -2059,15 +2066,17 @@ class OAuth2WebServerFlow(Flow):
                 token_expiry = delta + _UTCNOW()
 
             extracted_id_token = None
+            id_token_jwt = None
             if 'id_token' in d:
                 extracted_id_token = _extract_id_token(d['id_token'])
+                id_token_jwt = d['id_token']
 
             logger.info('Successfully retrieved access token')
             return OAuth2Credentials(
                 access_token, self.client_id, self.client_secret,
                 refresh_token, token_expiry, self.token_uri, self.user_agent,
                 revoke_uri=self.revoke_uri, id_token=extracted_id_token,
-                token_response=d, scopes=self.scope,
+                id_token_jwt=id_token_jwt, token_response=d, scopes=self.scope,
                 token_info_uri=self.token_info_uri)
         else:
             logger.info('Failed to retrieve access token: %s', content)

src/oauth2client/contrib/_fcntl_opener.py

@@ -1,81 +0,0 @@
-# Copyright 2016 Google Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import errno
-import fcntl
-import time
-
-from oauth2client.contrib import locked_file
-
-
-class _FcntlOpener(locked_file._Opener):
-    """Open, lock, and unlock a file using fcntl.lockf."""
-
-    def open_and_lock(self, timeout, delay):
-        """Open the file and lock it.
-
-        Args:
-            timeout: float, How long to try to lock for.
-            delay: float, How long to wait between retries
-
-        Raises:
-            AlreadyLockedException: if the lock is already acquired.
-            IOError: if the open fails.
-            CredentialsFileSymbolicLinkError: if the file is a symbolic
-                                              link.
-        """
-        if self._locked:
-            raise locked_file.AlreadyLockedException(
-                'File {0} is already locked'.format(self._filename))
-        start_time = time.time()
-
-        locked_file.validate_file(self._filename)
-        try:
-            self._fh = open(self._filename, self._mode)
-        except IOError as e:
-            # If we can't access with _mode, try _fallback_mode and
-            # don't lock.
-            if e.errno in (errno.EPERM, errno.EACCES):
-                self._fh = open(self._filename, self._fallback_mode)
-                return
-
-        # We opened in _mode, try to lock the file.
-        while True:
-            try:
-                fcntl.lockf(self._fh.fileno(), fcntl.LOCK_EX)
-                self._locked = True
-                return
-            except IOError as e:
-                # If not retrying, then just pass on the error.
-                if timeout == 0:
-                    raise
-                if e.errno != errno.EACCES:
-                    raise
-                # We could not acquire the lock. Try again.
-                if (time.time() - start_time) >= timeout:
-                    locked_file.logger.warn('Could not lock %s in %s seconds',
-                                            self._filename, timeout)
-                    if self._fh:
-                        self._fh.close()
-                    self._fh = open(self._filename, self._fallback_mode)
-                    return
-                time.sleep(delay)
-
-    def unlock_and_close(self):
-        """Close and unlock the file using the fcntl.lockf primitive."""
-        if self._locked:
-            fcntl.lockf(self._fh.fileno(), fcntl.LOCK_UN)
-        self._locked = False
-        if self._fh:
-            self._fh.close()

src/oauth2client/contrib/_metadata.py

@@ -19,6 +19,7 @@ See https://cloud.google.com/compute/docs/metadata
 
 import datetime
 import json
+import os
 
 from six.moves import http_client
 from six.moves.urllib import parse as urlparse
@@ -28,7 +29,8 @@ from oauth2client import client
 from oauth2client import transport
 
 
-METADATA_ROOT = 'http://metadata.google.internal/computeMetadata/v1/'
+METADATA_ROOT = 'http://{}/computeMetadata/v1/'.format(
+    os.getenv('GCE_METADATA_ROOT', 'metadata.google.internal'))
 METADATA_HEADERS = {'Metadata-Flavor': 'Google'}
 
 
@@ -38,7 +40,7 @@ def get(http, path, root=METADATA_ROOT, recursive=None):
     Args:
         http: an object to be used to make HTTP requests.
         path: A string indicating the resource to retrieve. For example,
-            'instance/service-accounts/defualt'
+            'instance/service-accounts/default'
         root: A string indicating the full path to the metadata server root.
         recursive: A boolean indicating whether to do a recursive query of
             metadata. See

src/oauth2client/contrib/_win32_opener.py

@@ -1,106 +0,0 @@
-# Copyright 2016 Google Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import errno
-import time
-
-import pywintypes
-import win32con
-import win32file
-
-from oauth2client.contrib import locked_file
-
-
-class _Win32Opener(locked_file._Opener):
-    """Open, lock, and unlock a file using windows primitives."""
-
-    # Error #33:
-    #  'The process cannot access the file because another process'
-    FILE_IN_USE_ERROR = 33
-
-    # Error #158:
-    #  'The segment is already unlocked.'
-    FILE_ALREADY_UNLOCKED_ERROR = 158
-
-    def open_and_lock(self, timeout, delay):
-        """Open the file and lock it.
-
-        Args:
-            timeout: float, How long to try to lock for.
-            delay: float, How long to wait between retries
-
-        Raises:
-            AlreadyLockedException: if the lock is already acquired.
-            IOError: if the open fails.
-            CredentialsFileSymbolicLinkError: if the file is a symbolic
-                                              link.
-        """
-        if self._locked:
-            raise locked_file.AlreadyLockedException(
-                'File {0} is already locked'.format(self._filename))
-        start_time = time.time()
-
-        locked_file.validate_file(self._filename)
-        try:
-            self._fh = open(self._filename, self._mode)
-        except IOError as e:
-            # If we can't access with _mode, try _fallback_mode
-            # and don't lock.
-            if e.errno == errno.EACCES:
-                self._fh = open(self._filename, self._fallback_mode)
-                return
-
-        # We opened in _mode, try to lock the file.
-        while True:
-            try:
-                hfile = win32file._get_osfhandle(self._fh.fileno())
-                win32file.LockFileEx(
-                    hfile,
-                    (win32con.LOCKFILE_FAIL_IMMEDIATELY |
-                     win32con.LOCKFILE_EXCLUSIVE_LOCK), 0, -0x10000,
-                    pywintypes.OVERLAPPED())
-                self._locked = True
-                return
-            except pywintypes.error as e:
-                if timeout == 0:
-                    raise
-
-                # If the error is not that the file is already
-                # in use, raise.
-                if e[0] != _Win32Opener.FILE_IN_USE_ERROR:
-                    raise
-
-                # We could not acquire the lock. Try again.
-                if (time.time() - start_time) >= timeout:
-                    locked_file.logger.warn('Could not lock %s in %s seconds',
-                                            self._filename, timeout)
-                    if self._fh:
-                        self._fh.close()
-                    self._fh = open(self._filename, self._fallback_mode)
-                    return
-                time.sleep(delay)
-
-    def unlock_and_close(self):
-        """Close and unlock the file using the win32 primitive."""
-        if self._locked:
-            try:
-                hfile = win32file._get_osfhandle(self._fh.fileno())
-                win32file.UnlockFileEx(hfile, 0, -0x10000,
-                                       pywintypes.OVERLAPPED())
-            except pywintypes.error as e:
-                if e[0] != _Win32Opener.FILE_ALREADY_UNLOCKED_ERROR:
-                    raise
-        self._locked = False
-        if self._fh:
-            self._fh.close()

src/oauth2client/contrib/devshell.py

@@ -37,6 +37,7 @@ class CommunicationError(Error):
 class NoDevshellServer(Error):
     """Error when no Developer Shell server can be contacted."""
 
+
 # The request for credential information to the Developer Shell client socket
 # is always an empty PBLite-formatted JSON object, so just define it as a
 # constant.

src/oauth2client/contrib/django_util/models.py

@@ -19,6 +19,7 @@ import pickle
 
 from django.db import models
 from django.utils import encoding
+import jsonpickle
 
 import oauth2client
 
@@ -48,7 +49,12 @@ class CredentialsField(models.Field):
         elif isinstance(value, oauth2client.client.Credentials):
             return value
         else:
-            return pickle.loads(base64.b64decode(encoding.smart_bytes(value)))
+            try:
+                return jsonpickle.decode(
+                    base64.b64decode(encoding.smart_bytes(value)).decode())
+            except ValueError:
+                return pickle.loads(
+                    base64.b64decode(encoding.smart_bytes(value)))
 
     def get_prep_value(self, value):
         """Overrides ``models.Field`` method. This is used to convert
@@ -58,7 +64,8 @@ class CredentialsField(models.Field):
         if value is None:
             return None
         else:
-            return encoding.smart_text(base64.b64encode(pickle.dumps(value)))
+            return encoding.smart_text(
+                base64.b64encode(jsonpickle.encode(value).encode()))
 
     def value_to_string(self, obj):
         """Convert the field value from the provided model to a string.

src/oauth2client/contrib/flask_util.py

@@ -176,6 +176,7 @@ try:
     from flask import request
     from flask import session
     from flask import url_for
+    import markupsafe
 except ImportError:  # pragma: NO COVER
     raise ImportError('The flask utilities require flask 0.9 or newer.')
 
@@ -388,6 +389,7 @@ class UserOAuth2(object):
         if 'error' in request.args:
             reason = request.args.get(
                 'error_description', request.args.get('error', ''))
+            reason = markupsafe.escape(reason)
             return ('Authorization failed: {0}'.format(reason),
                     httplib.BAD_REQUEST)
 

src/oauth2client/contrib/locked_file.py

@@ -1,234 +0,0 @@
-# Copyright 2014 Google Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Locked file interface that should work on Unix and Windows pythons.
-
-This module first tries to use fcntl locking to ensure serialized access
-to a file, then falls back on a lock file if that is unavialable.
-
-Usage::
-
-    f = LockedFile('filename', 'r+b', 'rb')
-    f.open_and_lock()
-    if f.is_locked():
-      print('Acquired filename with r+b mode')
-      f.file_handle().write('locked data')
-    else:
-      print('Acquired filename with rb mode')
-    f.unlock_and_close()
-
-"""
-
-from __future__ import print_function
-
-import errno
-import logging
-import os
-import time
-
-from oauth2client import util
-
-
-__author__ = 'cache@google.com (David T McWherter)'
-
-logger = logging.getLogger(__name__)
-
-
-class CredentialsFileSymbolicLinkError(Exception):
-    """Credentials files must not be symbolic links."""
-
-
-class AlreadyLockedException(Exception):
-    """Trying to lock a file that has already been locked by the LockedFile."""
-    pass
-
-
-def validate_file(filename):
-    if os.path.islink(filename):
-        raise CredentialsFileSymbolicLinkError(
-            'File: {0} is a symbolic link.'.format(filename))
-
-
-class _Opener(object):
-    """Base class for different locking primitives."""
-
-    def __init__(self, filename, mode, fallback_mode):
-        """Create an Opener.
-
-        Args:
-            filename: string, The pathname of the file.
-            mode: string, The preferred mode to access the file with.
-            fallback_mode: string, The mode to use if locking fails.
-        """
-        self._locked = False
-        self._filename = filename
-        self._mode = mode
-        self._fallback_mode = fallback_mode
-        self._fh = None
-        self._lock_fd = None
-
-    def is_locked(self):
-        """Was the file locked."""
-        return self._locked
-
-    def file_handle(self):
-        """The file handle to the file. Valid only after opened."""
-        return self._fh
-
-    def filename(self):
-        """The filename that is being locked."""
-        return self._filename
-
-    def open_and_lock(self, timeout, delay):
-        """Open the file and lock it.
-
-        Args:
-            timeout: float, How long to try to lock for.
-            delay: float, How long to wait between retries.
-        """
-        pass
-
-    def unlock_and_close(self):
-        """Unlock and close the file."""
-        pass
-
-
-class _PosixOpener(_Opener):
-    """Lock files using Posix advisory lock files."""
-
-    def open_and_lock(self, timeout, delay):
-        """Open the file and lock it.
-
-        Tries to create a .lock file next to the file we're trying to open.
-
-        Args:
-            timeout: float, How long to try to lock for.
-            delay: float, How long to wait between retries.
-
-        Raises:
-            AlreadyLockedException: if the lock is already acquired.
-            IOError: if the open fails.
-            CredentialsFileSymbolicLinkError if the file is a symbolic link.
-        """
-        if self._locked:
-            raise AlreadyLockedException(
-                'File {0} is already locked'.format(self._filename))
-        self._locked = False
-
-        validate_file(self._filename)
-        try:
-            self._fh = open(self._filename, self._mode)
-        except IOError as e:
-            # If we can't access with _mode, try _fallback_mode and don't lock.
-            if e.errno == errno.EACCES:
-                self._fh = open(self._filename, self._fallback_mode)
-                return
-
-        lock_filename = self._posix_lockfile(self._filename)
-        start_time = time.time()
-        while True:
-            try:
-                self._lock_fd = os.open(lock_filename,
-                                        os.O_CREAT | os.O_EXCL | os.O_RDWR)
-                self._locked = True
-                break
-
-            except OSError as e:
-                if e.errno != errno.EEXIST:
-                    raise
-                if (time.time() - start_time) >= timeout:
-                    logger.warn('Could not acquire lock %s in %s seconds',
-                                lock_filename, timeout)
-                    # Close the file and open in fallback_mode.
-                    if self._fh:
-                        self._fh.close()
-                    self._fh = open(self._filename, self._fallback_mode)
-                    return
-                time.sleep(delay)
-
-    def unlock_and_close(self):
-        """Unlock a file by removing the .lock file, and close the handle."""
-        if self._locked:
-            lock_filename = self._posix_lockfile(self._filename)
-            os.close(self._lock_fd)
-            os.unlink(lock_filename)
-            self._locked = False
-            self._lock_fd = None
-        if self._fh:
-            self._fh.close()
-
-    def _posix_lockfile(self, filename):
-        """The name of the lock file to use for posix locking."""
-        return '{0}.lock'.format(filename)
-
-
-class LockedFile(object):
-    """Represent a file that has exclusive access."""
-
-    @util.positional(4)
-    def __init__(self, filename, mode, fallback_mode, use_native_locking=True):
-        """Construct a LockedFile.
-
-        Args:
-            filename: string, The path of the file to open.
-            mode: string, The mode to try to open the file with.
-            fallback_mode: string, The mode to use if locking fails.
-            use_native_locking: bool, Whether or not fcntl/win32 locking is
-                                used.
-        """
-        opener = None
-        if not opener and use_native_locking:
-            try:
-                from oauth2client.contrib._win32_opener import _Win32Opener
-                opener = _Win32Opener(filename, mode, fallback_mode)
-            except ImportError:
-                try:
-                    from oauth2client.contrib._fcntl_opener import _FcntlOpener
-                    opener = _FcntlOpener(filename, mode, fallback_mode)
-                except ImportError:
-                    pass
-
-        if not opener:
-            opener = _PosixOpener(filename, mode, fallback_mode)
-
-        self._opener = opener
-
-    def filename(self):
-        """Return the filename we were constructed with."""
-        return self._opener._filename
-
-    def file_handle(self):
-        """Return the file_handle to the opened file."""
-        return self._opener.file_handle()
-
-    def is_locked(self):
-        """Return whether we successfully locked the file."""
-        return self._opener.is_locked()
-
-    def open_and_lock(self, timeout=0, delay=0.05):
-        """Open the file, trying to lock it.
-
-        Args:
-            timeout: float, The number of seconds to try to acquire the lock.
-            delay: float, The number of seconds to wait between retry attempts.
-
-        Raises:
-            AlreadyLockedException: if the lock is already acquired.
-            IOError: if the open fails.
-        """
-        self._opener.open_and_lock(timeout, delay)
-
-    def unlock_and_close(self):
-        """Unlock and close a file."""
-        self._opener.unlock_and_close()

src/oauth2client/contrib/multistore_file.py

@@ -1,505 +0,0 @@
-# Copyright 2014 Google Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Multi-credential file store with lock support.
-
-This module implements a JSON credential store where multiple
-credentials can be stored in one file. That file supports locking
-both in a single process and across processes.
-
-The credential themselves are keyed off of:
-
-* client_id
-* user_agent
-* scope
-
-The format of the stored data is like so::
-
-    {
-      'file_version': 1,
-      'data': [
-          {
-              'key': {
-                  'clientId': '<client id>',
-                  'userAgent': '<user agent>',
-                  'scope': '<scope>'
-              },
-              'credential': {
-                  # JSON serialized Credentials.
-              }
-          }
-      ]
-    }
-
-"""
-
-import errno
-import json
-import logging
-import os
-import threading
-
-from oauth2client import client
-from oauth2client import util
-from oauth2client.contrib import locked_file
-
-__author__ = 'jbeda@google.com (Joe Beda)'
-
-logger = logging.getLogger(__name__)
-
-logger.warning(
-    'The oauth2client.contrib.multistore_file module has been deprecated and '
-    'will be removed in the next release of oauth2client. Please migrate to '
-    'multiprocess_file_storage.')
-
-# A dict from 'filename'->_MultiStore instances
-_multistores = {}
-_multistores_lock = threading.Lock()
-
-
-class Error(Exception):
-    """Base error for this module."""
-
-
-class NewerCredentialStoreError(Error):
-    """The credential store is a newer version than supported."""
-
-
-def _dict_to_tuple_key(dictionary):
-    """Converts a dictionary to a tuple that can be used as an immutable key.
-
-    The resulting key is always sorted so that logically equivalent
-    dictionaries always produce an identical tuple for a key.
-
-    Args:
-        dictionary: the dictionary to use as the key.
-
-    Returns:
-        A tuple representing the dictionary in it's naturally sorted ordering.
-    """
-    return tuple(sorted(dictionary.items()))
-
-
-@util.positional(4)
-def get_credential_storage(filename, client_id, user_agent, scope,
-                           warn_on_readonly=True):
-    """Get a Storage instance for a credential.
-
-    Args:
-        filename: The JSON file storing a set of credentials
-        client_id: The client_id for the credential
-        user_agent: The user agent for the credential
-        scope: string or iterable of strings, Scope(s) being requested
-        warn_on_readonly: if True, log a warning if the store is readonly
-
-    Returns:
-        An object derived from client.Storage for getting/setting the
-        credential.
-    """
-    # Recreate the legacy key with these specific parameters
-    key = {'clientId': client_id, 'userAgent': user_agent,
-           'scope': util.scopes_to_string(scope)}
-    return get_credential_storage_custom_key(
-        filename, key, warn_on_readonly=warn_on_readonly)
-
-
-@util.positional(2)
-def get_credential_storage_custom_string_key(filename, key_string,
-                                             warn_on_readonly=True):
-    """Get a Storage instance for a credential using a single string as a key.
-
-    Allows you to provide a string as a custom key that will be used for
-    credential storage and retrieval.
-
-    Args:
-        filename: The JSON file storing a set of credentials
-        key_string: A string to use as the key for storing this credential.
-        warn_on_readonly: if True, log a warning if the store is readonly
-
-    Returns:
-        An object derived from client.Storage for getting/setting the
-        credential.
-    """
-    # Create a key dictionary that can be used
-    key_dict = {'key': key_string}
-    return get_credential_storage_custom_key(
-        filename, key_dict, warn_on_readonly=warn_on_readonly)
-
-
-@util.positional(2)
-def get_credential_storage_custom_key(filename, key_dict,
-                                      warn_on_readonly=True):
-    """Get a Storage instance for a credential using a dictionary as a key.
-
-    Allows you to provide a dictionary as a custom key that will be used for
-    credential storage and retrieval.
-
-    Args:
-        filename: The JSON file storing a set of credentials
-        key_dict: A dictionary to use as the key for storing this credential.
-                  There is no ordering of the keys in the dictionary. Logically
-                  equivalent dictionaries will produce equivalent storage keys.
-        warn_on_readonly: if True, log a warning if the store is readonly
-
-    Returns:
-        An object derived from client.Storage for getting/setting the
-        credential.
-    """
-    multistore = _get_multistore(filename, warn_on_readonly=warn_on_readonly)
-    key = _dict_to_tuple_key(key_dict)
-    return multistore._get_storage(key)
-
-
-@util.positional(1)
-def get_all_credential_keys(filename, warn_on_readonly=True):
-    """Gets all the registered credential keys in the given Multistore.
-
-    Args:
-        filename: The JSON file storing a set of credentials
-        warn_on_readonly: if True, log a warning if the store is readonly
-
-    Returns:
-        A list of the credential keys present in the file.  They are returned
-        as dictionaries that can be passed into
-        get_credential_storage_custom_key to get the actual credentials.
-    """
-    multistore = _get_multistore(filename, warn_on_readonly=warn_on_readonly)
-    multistore._lock()
-    try:
-        return multistore._get_all_credential_keys()
-    finally:
-        multistore._unlock()
-
-
-@util.positional(1)
-def _get_multistore(filename, warn_on_readonly=True):
-    """A helper method to initialize the multistore with proper locking.
-
-    Args:
-        filename: The JSON file storing a set of credentials
-        warn_on_readonly: if True, log a warning if the store is readonly
-
-    Returns:
-        A multistore object
-    """
-    filename = os.path.expanduser(filename)
-    _multistores_lock.acquire()
-    try:
-        multistore = _multistores.setdefault(
-            filename, _MultiStore(filename, warn_on_readonly=warn_on_readonly))
-    finally:
-        _multistores_lock.release()
-    return multistore
-
-
-class _MultiStore(object):
-    """A file backed store for multiple credentials."""
-
-    @util.positional(2)
-    def __init__(self, filename, warn_on_readonly=True):
-        """Initialize the class.
-
-        This will create the file if necessary.
-        """
-        self._file = locked_file.LockedFile(filename, 'r+', 'r')
-        self._thread_lock = threading.Lock()
-        self._read_only = False
-        self._warn_on_readonly = warn_on_readonly
-
-        self._create_file_if_needed()
-
-        # Cache of deserialized store. This is only valid after the
-        # _MultiStore is locked or _refresh_data_cache is called. This is
-        # of the form of:
-        #
-        # ((key, value), (key, value)...) -> OAuth2Credential
-        #
-        # If this is None, then the store hasn't been read yet.
-        self._data = None
-
-    class _Storage(client.Storage):
-        """A Storage object that can read/write a single credential."""
-
-        def __init__(self, multistore, key):
-            self._multistore = multistore
-            self._key = key
-
-        def acquire_lock(self):
-            """Acquires any lock necessary to access this Storage.
-
-            This lock is not reentrant.
-            """
-            self._multistore._lock()
-
-        def release_lock(self):
-            """Release the Storage lock.
-
-            Trying to release a lock that isn't held will result in a
-            RuntimeError.
-            """
-            self._multistore._unlock()
-
-        def locked_get(self):
-            """Retrieve credential.
-
-            The Storage lock must be held when this is called.
-
-            Returns:
-                oauth2client.client.Credentials
-            """
-            credential = self._multistore._get_credential(self._key)
-            if credential:
-                credential.set_store(self)
-            return credential
-
-        def locked_put(self, credentials):
-            """Write a credential.
-
-            The Storage lock must be held when this is called.
-
-            Args:
-                credentials: Credentials, the credentials to store.
-            """
-            self._multistore._update_credential(self._key, credentials)
-
-        def locked_delete(self):
-            """Delete a credential.
-
-            The Storage lock must be held when this is called.
-
-            Args:
-                credentials: Credentials, the credentials to store.
-            """
-            self._multistore._delete_credential(self._key)
-
-    def _create_file_if_needed(self):
-        """Create an empty file if necessary.
-
-        This method will not initialize the file. Instead it implements a
-        simple version of "touch" to ensure the file has been created.
-        """
-        if not os.path.exists(self._file.filename()):
-            old_umask = os.umask(0o177)
-            try:
-                open(self._file.filename(), 'a+b').close()
-            finally:
-                os.umask(old_umask)
-
-    def _lock(self):
-        """Lock the entire multistore."""
-        self._thread_lock.acquire()
-        try:
-            self._file.open_and_lock()
-        except (IOError, OSError) as e:
-            if e.errno == errno.ENOSYS:
-                logger.warn('File system does not support locking the '
-                            'credentials file.')
-            elif e.errno == errno.ENOLCK:
-                logger.warn('File system is out of resources for writing the '
-                            'credentials file (is your disk full?).')
-            elif e.errno == errno.EDEADLK:
-                logger.warn('Lock contention on multistore file, opening '
-                            'in read-only mode.')
-            elif e.errno == errno.EACCES:
-                logger.warn('Cannot access credentials file.')
-            else:
-                raise
-        if not self._file.is_locked():
-            self._read_only = True
-            if self._warn_on_readonly:
-                logger.warn('The credentials file (%s) is not writable. '
-                            'Opening in read-only mode. Any refreshed '
-                            'credentials will only be '
-                            'valid for this run.', self._file.filename())
-
-        if os.path.getsize(self._file.filename()) == 0:
-            logger.debug('Initializing empty multistore file')
-            # The multistore is empty so write out an empty file.
-            self._data = {}
-            self._write()
-        elif not self._read_only or self._data is None:
-            # Only refresh the data if we are read/write or we haven't
-            # cached the data yet. If we are readonly, we assume is isn't
-            # changing out from under us and that we only have to read it
-            # once. This prevents us from whacking any new access keys that
-            # we have cached in memory but were unable to write out.
-            self._refresh_data_cache()
-
-    def _unlock(self):
-        """Release the lock on the multistore."""
-        self._file.unlock_and_close()
-        self._thread_lock.release()
-
-    def _locked_json_read(self):
-        """Get the raw content of the multistore file.
-
-        The multistore must be locked when this is called.
-
-        Returns:
-            The contents of the multistore decoded as JSON.
-        """
-        assert self._thread_lock.locked()
-        self._file.file_handle().seek(0)
-        return json.load(self._file.file_handle())
-
-    def _locked_json_write(self, data):
-        """Write a JSON serializable data structure to the multistore.
-
-        The multistore must be locked when this is called.
-
-        Args:
-            data: The data to be serialized and written.
-        """
-        assert self._thread_lock.locked()
-        if self._read_only:
-            return
-        self._file.file_handle().seek(0)
-        json.dump(data, self._file.file_handle(),
-                  sort_keys=True, indent=2, separators=(',', ': '))
-        self._file.file_handle().truncate()
-
-    def _refresh_data_cache(self):
-        """Refresh the contents of the multistore.
-
-        The multistore must be locked when this is called.
-
-        Raises:
-            NewerCredentialStoreError: Raised when a newer client has written
-            the store.
-        """
-        self._data = {}
-        try:
-            raw_data = self._locked_json_read()
-        except Exception:
-            logger.warn('Credential data store could not be loaded. '
-                        'Will ignore and overwrite.')
-            return
-
-        version = 0
-        try:
-            version = raw_data['file_version']
-        except Exception:
-            logger.warn('Missing version for credential data store. It may be '
-                        'corrupt or an old version. Overwriting.')
-        if version > 1:
-            raise NewerCredentialStoreError(
-                'Credential file has file_version of {0}. '
-                'Only file_version of 1 is supported.'.format(version))
-
-        credentials = []
-        try:
-            credentials = raw_data['data']
-        except (TypeError, KeyError):
-            pass
-
-        for cred_entry in credentials:
-            try:
-                key, credential = self._decode_credential_from_json(cred_entry)
-                self._data[key] = credential
-            except:
-                # If something goes wrong loading a credential, just ignore it
-                logger.info('Error decoding credential, skipping',
-                            exc_info=True)
-
-    def _decode_credential_from_json(self, cred_entry):
-        """Load a credential from our JSON serialization.
-
-        Args:
-            cred_entry: A dict entry from the data member of our format
-
-        Returns:
-            (key, cred) where the key is the key tuple and the cred is the
-            OAuth2Credential object.
-        """
-        raw_key = cred_entry['key']
-        key = _dict_to_tuple_key(raw_key)
-        credential = None
-        credential = client.Credentials.new_from_json(
-            json.dumps(cred_entry['credential']))
-        return (key, credential)
-
-    def _write(self):
-        """Write the cached data back out.
-
-        The multistore must be locked.
-        """
-        raw_data = {'file_version': 1}
-        raw_creds = []
-        raw_data['data'] = raw_creds
-        for (cred_key, cred) in self._data.items():
-            raw_key = dict(cred_key)
-            raw_cred = json.loads(cred.to_json())
-            raw_creds.append({'key': raw_key, 'credential': raw_cred})
-        self._locked_json_write(raw_data)
-
-    def _get_all_credential_keys(self):
-        """Gets all the registered credential keys in the multistore.
-
-        Returns:
-            A list of dictionaries corresponding to all the keys currently
-            registered
-        """
-        return [dict(key) for key in self._data.keys()]
-
-    def _get_credential(self, key):
-        """Get a credential from the multistore.
-
-        The multistore must be locked.
-
-        Args:
-            key: The key used to retrieve the credential
-
-        Returns:
-            The credential specified or None if not present
-        """
-        return self._data.get(key, None)
-
-    def _update_credential(self, key, cred):
-        """Update a credential and write the multistore.
-
-        This must be called when the multistore is locked.
-
-        Args:
-            key: The key used to retrieve the credential
-            cred: The OAuth2Credential to update/set
-        """
-        self._data[key] = cred
-        self._write()
-
-    def _delete_credential(self, key):
-        """Delete a credential and write the multistore.
-
-        This must be called when the multistore is locked.
-
-        Args:
-            key: The key used to retrieve the credential
-        """
-        try:
-            del self._data[key]
-        except KeyError:
-            pass
-        self._write()
-
-    def _get_storage(self, key):
-        """Get a Storage object to get/set a credential.
-
-        This Storage is a 'view' into the multistore.
-
-        Args:
-            key: The key used to retrieve the credential
-
-        Returns:
-            A Storage object that can be used to get/set this cred
-        """
-        return self._Storage(self, key)

src/oauth2client/tools.py

@@ -92,6 +92,7 @@ def _CreateArgumentParser():
         help='Set the logging level of detail.')
     return parser
 
+
 # argparser is an ArgumentParser that contains command-line options expected
 # by tools.run(). Pass it in as part of the 'parents' argument to your own
 # ArgumentParser.
@@ -217,16 +218,6 @@ def run_flow(flow, storage, flags=None, http=None):
     flow.redirect_uri = oauth_callback
     authorize_url = flow.step1_get_authorize_url()
 
-    if flags.short_url:
-        try:
-            from googleapiclient.discovery import build
-            service = build('urlshortener', 'v1', http=http)
-            url_result = service.url().insert(body={'longUrl': authorize_url},
-              key=u'AIzaSyBlmgbii8QfJSYmC9VTMOfqrAt5Vj5wtzE').execute()
-            authorize_url = url_result['id']
-        except:
-          pass
-
     if not flags.noauth_local_webserver:
         import webbrowser
         webbrowser.open(authorize_url, new=1, autoraise=True)

src/oauth2client/util.py

@@ -1,206 +0,0 @@
-# Copyright 2014 Google Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Common utility library."""
-
-import functools
-import inspect
-import logging
-
-import six
-from six.moves import urllib
-
-
-__author__ = [
-    'rafek@google.com (Rafe Kaplan)',
-    'guido@google.com (Guido van Rossum)',
-]
-
-__all__ = [
-    'positional',
-    'POSITIONAL_WARNING',
-    'POSITIONAL_EXCEPTION',
-    'POSITIONAL_IGNORE',
-]
-
-logger = logging.getLogger(__name__)
-
-POSITIONAL_WARNING = 'WARNING'
-POSITIONAL_EXCEPTION = 'EXCEPTION'
-POSITIONAL_IGNORE = 'IGNORE'
-POSITIONAL_SET = frozenset([POSITIONAL_WARNING, POSITIONAL_EXCEPTION,
-                            POSITIONAL_IGNORE])
-
-positional_parameters_enforcement = POSITIONAL_WARNING
-
-
-def positional(max_positional_args):
-    """A decorator to declare that only the first N arguments my be positional.
-
-    This decorator makes it easy to support Python 3 style keyword-only
-    parameters. For example, in Python 3 it is possible to write::
-
-        def fn(pos1, *, kwonly1=None, kwonly1=None):
-            ...
-
-    All named parameters after ``*`` must be a keyword::
-
-        fn(10, 'kw1', 'kw2')  # Raises exception.
-        fn(10, kwonly1='kw1')  # Ok.
-
-    Example
-    ^^^^^^^
-
-    To define a function like above, do::
-
-        @positional(1)
-        def fn(pos1, kwonly1=None, kwonly2=None):
-            ...
-
-    If no default value is provided to a keyword argument, it becomes a
-    required keyword argument::
-
-        @positional(0)
-        def fn(required_kw):
-            ...
-
-    This must be called with the keyword parameter::
-
-        fn()  # Raises exception.
-        fn(10)  # Raises exception.
-        fn(required_kw=10)  # Ok.
-
-    When defining instance or class methods always remember to account for
-    ``self`` and ``cls``::
-
-        class MyClass(object):
-
-            @positional(2)
-            def my_method(self, pos1, kwonly1=None):
-                ...
-
-            @classmethod
-            @positional(2)
-            def my_method(cls, pos1, kwonly1=None):
-                ...
-
-    The positional decorator behavior is controlled by
-    ``util.positional_parameters_enforcement``, which may be set to
-    ``POSITIONAL_EXCEPTION``, ``POSITIONAL_WARNING`` or
-    ``POSITIONAL_IGNORE`` to raise an exception, log a warning, or do
-    nothing, respectively, if a declaration is violated.
-
-    Args:
-        max_positional_arguments: Maximum number of positional arguments. All
-                                  parameters after the this index must be
-                                  keyword only.
-
-    Returns:
-        A decorator that prevents using arguments after max_positional_args
-        from being used as positional parameters.
-
-    Raises:
-        TypeError: if a key-word only argument is provided as a positional
-                   parameter, but only if
-                   util.positional_parameters_enforcement is set to
-                   POSITIONAL_EXCEPTION.
-    """
-
-    def positional_decorator(wrapped):
-        @functools.wraps(wrapped)
-        def positional_wrapper(*args, **kwargs):
-            if len(args) > max_positional_args:
-                plural_s = ''
-                if max_positional_args != 1:
-                    plural_s = 's'
-                message = ('{function}() takes at most {args_max} positional '
-                           'argument{plural} ({args_given} given)'.format(
-                               function=wrapped.__name__,
-                               args_max=max_positional_args,
-                               args_given=len(args),
-                               plural=plural_s))
-                if positional_parameters_enforcement == POSITIONAL_EXCEPTION:
-                    raise TypeError(message)
-                elif positional_parameters_enforcement == POSITIONAL_WARNING:
-                    logger.warning(message)
-            return wrapped(*args, **kwargs)
-        return positional_wrapper
-
-    if isinstance(max_positional_args, six.integer_types):
-        return positional_decorator
-    else:
-        args, _, _, defaults = inspect.getargspec(max_positional_args)
-        return positional(len(args) - len(defaults))(max_positional_args)
-
-
-def scopes_to_string(scopes):
-    """Converts scope value to a string.
-
-    If scopes is a string then it is simply passed through. If scopes is an
-    iterable then a string is returned that is all the individual scopes
-    concatenated with spaces.
-
-    Args:
-        scopes: string or iterable of strings, the scopes.
-
-    Returns:
-        The scopes formatted as a single string.
-    """
-    if isinstance(scopes, six.string_types):
-        return scopes
-    else:
-        return ' '.join(scopes)
-
-
-def string_to_scopes(scopes):
-    """Converts stringifed scope value to a list.
-
-    If scopes is a list then it is simply passed through. If scopes is an
-    string then a list of each individual scope is returned.
-
-    Args:
-        scopes: a string or iterable of strings, the scopes.
-
-    Returns:
-        The scopes in a list.
-    """
-    if not scopes:
-        return []
-    if isinstance(scopes, six.string_types):
-        return scopes.split(' ')
-    else:
-        return scopes
-
-
-def _add_query_parameter(url, name, value):
-    """Adds a query parameter to a url.
-
-    Replaces the current value if it already exists in the URL.
-
-    Args:
-        url: string, url to add the query parameter to.
-        name: string, query parameter name.
-        value: string, query parameter value.
-
-    Returns:
-        Updated query parameter. Does not update the url if value is None.
-    """
-    if value is None:
-        return url
-    else:
-        parsed = list(urllib.parse.urlparse(url))
-        q = dict(urllib.parse.parse_qsl(parsed[4]))
-        q[name] = value
-        parsed[4] = urllib.parse.urlencode(q)
-        return urllib.parse.urlunparse(parsed)

src/project-apis.txt

@@ -8,4 +8,5 @@ gmail.googleapis.com
 groupssettings.googleapis.com
 licensing.googleapis.com
 plus.googleapis.com
+reseller.googleapis.com
 siteverification.googleapis.com

src/utils.py

@@ -1,9 +1,9 @@
-from var import GM_Globals, GM_WINDOWS, GM_SYS_ENCODING
 import collections
 import re
 import sys
 from htmlentitydefs import name2codepoint
 from HTMLParser import HTMLParser
+from var import GM_Globals, GM_WINDOWS, GM_SYS_ENCODING
 
 def convertUTF8(data):
   if isinstance(data, str):

src/var.py

@@ -4,18 +4,21 @@ import platform
 import re
 
 gam_author = u'Jay Lee <jay0lee@gmail.com>'
-gam_version = u'4.11'
+gam_version = u'4.23'
 gam_license = u'Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)'
 
-GAM_URL = u'http://git.io/gam'
+GAM_URL = u'https://git.io/gam'
 GAM_INFO = u'GAM {0} - {1} / {2} / Python {3}.{4}.{5} {6} / {7} {8} /'.format(gam_version, GAM_URL,
-  gam_author, sys.version_info[0], sys.version_info[1], sys.version_info[2], sys.version_info[3],
-  platform.platform(), platform.machine())
+                                                                              gam_author,
+                                                                              sys.version_info[0], sys.version_info[1],
+                                                                              sys.version_info[2], sys.version_info[3],
+                                                                              platform.platform(), platform.machine())
 
 GAM_RELEASES = u'https://github.com/jay0lee/GAM/releases'
 GAM_WIKI = u'https://github.com/jay0lee/GAM/wiki'
 GAM_ALL_RELEASES = u'https://api.github.com/repos/jay0lee/GAM/releases'
 GAM_LATEST_RELEASE = GAM_ALL_RELEASES+u'/latest'
+GAM_PROJECT_APIS = u'https://raw.githubusercontent.com/jay0lee/GAM/master/src/project-apis.txt'
 
 TRUE = u'true'
 FALSE = u'false'
@@ -40,40 +43,46 @@ FN_OAUTH2SERVICE_JSON = u'oauth2service.json'
 FN_OAUTH2_TXT = u'oauth2.txt'
 MY_CUSTOMER = u'my_customer'
 SKUS = {
+  u'Google-Apps': {
+    u'product': u'Google-Apps', u'aliases': [u'standard', u'free'], u'displayName': u'G Suite Free/Standard'},
   u'Google-Apps-For-Business': {
-   u'product': u'Google-Apps', u'aliases': [u'gafb', u'gafw', u'basic', u'gsuite-basic']},
+    u'product': u'Google-Apps', u'aliases': [u'gafb', u'gafw', u'basic', u'gsuitebasic'], u'displayName': u'G Suite Basic'},
   u'Google-Apps-For-Government': {
-   u'product': u'Google-Apps', u'aliases': [u'gafg', u'gsuite-government']},
+    u'product': u'Google-Apps', u'aliases': [u'gafg', u'gsuitegovernment', u'gsuitegov'], u'displayName': u'G Suite Government'},
   u'Google-Apps-For-Postini': {
-   u'product': u'Google-Apps', u'aliases': [u'gams', u'postini', u'gsuite-gams']},
+    u'product': u'Google-Apps', u'aliases': [u'gams', u'postini', u'gsuitegams', u'gsuitepostini', u'gsuitemessagesecurity'], u'displayName': u'G Suite Message Security'},
   u'Google-Apps-Lite': {
-   u'product': u'Google-Apps', u'aliases': [u'gal', u'lite', u'gsuite-lite']},
+    u'product': u'Google-Apps', u'aliases': [u'gal', u'lite', u'gsuitelite'], u'displayName': u'G Suite Lite'},
   u'Google-Apps-Unlimited': {
-   u'product': u'Google-Apps', u'aliases': [u'gau', u'unlimited', u'gsuite-business']},
+    u'product': u'Google-Apps', u'aliases': [u'gau', u'unlimited', u'gsuitebusiness'], u'displayName': u'G Suite Business'},
+  u'1010020020': {
+    u'product': u'Google-Apps', u'aliases': [u'gae', u'enterprise', u'gsuiteenterprise'], u'displayName': u'G Suite Enterprise'},
   u'Google-Drive-storage-20GB': {
-   u'product': u'Google-Drive-storage', u'aliases': [u'drive-20gb', u'drive20gb', u'20gb']},
+    u'product': u'Google-Drive-storage', u'aliases': [u'drive20gb', u'20gb', u'googledrivestorage20gb'], u'displayName': u'Google Drive Storage 20GB'},
   u'Google-Drive-storage-50GB': {
-   u'product': u'Google-Drive-storage', u'aliases': [u'drive-50gb', u'drive50gb', u'50gb']},
+    u'product': u'Google-Drive-storage', u'aliases': [u'drive50gb', u'50gb', u'googledrivestorage50gb'], u'displayName': u'Google Drive Storage 50GB'},
   u'Google-Drive-storage-200GB': {
-   u'product': u'Google-Drive-storage', u'aliases': [u'drive-200gb', u'drive200gb', u'200gb']},
+    u'product': u'Google-Drive-storage', u'aliases': [u'drive200gb', u'200gb', u'googledrivestorage200gb'], u'displayName': u'Google Drive Storage 200GB'},
   u'Google-Drive-storage-400GB': {
-   u'product': u'Google-Drive-storage', u'aliases': [u'drive-400gb', u'drive400gb', u'400gb']},
+    u'product': u'Google-Drive-storage', u'aliases': [u'drive400gb', u'400gb', u'googledrivestorage400gb'], u'displayName': u'Google Drive Storage 400GB'},
   u'Google-Drive-storage-1TB': {
-   u'product': u'Google-Drive-storage', u'aliases': [u'drive-1tb', u'drive1tb', u'1tb']},
+    u'product': u'Google-Drive-storage', u'aliases': [u'drive1tb', u'1tb', u'googledrivestorage1tb'], u'displayName': u'Google Drive Storage 1TB'},
   u'Google-Drive-storage-2TB': {
-   u'product': u'Google-Drive-storage', u'aliases': [u'drive-2tb', u'drive2tb', u'2tb']},
+    u'product': u'Google-Drive-storage', u'aliases': [u'drive2tb', u'2tb', u'googledrivestorage2tb'], u'displayName': u'Google Drive Storage 2TB'},
   u'Google-Drive-storage-4TB': {
-   u'product': u'Google-Drive-storage', u'aliases': [u'drive-4tb', u'drive4tb', u'4tb']},
+    u'product': u'Google-Drive-storage', u'aliases': [u'drive4tb', u'4tb', u'googledrivestorage4tb'], u'displayName': u'Google Drive Storage 4TB'},
   u'Google-Drive-storage-8TB': {
-   u'product': u'Google-Drive-storage', u'aliases': [u'drive-8tb', u'drive8tb', u'8tb']},
+    u'product': u'Google-Drive-storage', u'aliases': [u'drive8tb', u'8tb', u'googledrivestorage8tb'], u'displayName': u'Google Drive Storage 8TB'},
   u'Google-Drive-storage-16TB': {
-   u'product': u'Google-Drive-storage', u'aliases': [u'drive-16tb', u'drive16tb', u'16tb']},
+    u'product': u'Google-Drive-storage', u'aliases': [u'drive16tb', u'16tb', u'googledrivestorage16tb'], u'displayName': u'Google Drive Storage 16TB'},
   u'Google-Vault': {
-   u'product': u'Google-Vault', u'aliases': [u'vault']},
+    u'product': u'Google-Vault', u'aliases': [u'vault', u'googlevault'], u'displayName': u'Google Vault'},
   u'Google-Vault-Former-Employee': {
-   u'product': u'Google-Vault', u'aliases': [u'vfe']},
+    u'product': u'Google-Vault', u'aliases': [u'vfe', u'googlevaultformeremployee'], u'displayName': u'Google Vault Former Employee'},
   u'Google-Coordinate': {
-   u'product': u'Google-Coordinate', u'aliases': [u'coordinate']}
+    u'product': u'Google-Coordinate', u'aliases': [u'coordinate', u'googlecoordinate'], u'displayName': u'Google Coordinate'},
+  u'Google-Chrome-Device-Management': {
+    u'product': u'Google-Chrome-Device-Management', u'aliases': [u'chrome', u'cdm', u'googlechromedevicemanagement'], u'displayName': u'Google Chrome Device Management'}
   }
 
 API_VER_MAPPING = {
@@ -84,6 +93,7 @@ API_VER_MAPPING = {
   u'datatransfer': u'datatransfer_v1',
   u'directory': u'directory_v1',
   u'drive': u'v2',
+  u'drive3': u'v3',
   u'email-settings': u'v2',
   u'gmail': u'v1',
   u'groupssettings': u'v1',
@@ -91,7 +101,9 @@ API_VER_MAPPING = {
   u'oauth2': u'v2',
   u'plus': u'v1',
   u'reports': u'reports_v1',
+  u'reseller': u'v1',
   u'siteVerification': u'v1',
+  u'urlshortener': u'v1',
   }
 
 API_SCOPE_MAPPING = {
@@ -99,15 +111,18 @@ API_SCOPE_MAPPING = {
                     u'https://www.googleapis.com/auth/drive'],
   u'calendar': [u'https://www.googleapis.com/auth/calendar',],
   u'drive': [u'https://www.googleapis.com/auth/drive',],
+  u'drive3': [u'https://www.googleapis.com/auth/drive',],
   u'gmail': [u'https://mail.google.com/',
              u'https://www.googleapis.com/auth/gmail.settings.basic',
              u'https://www.googleapis.com/auth/gmail.settings.sharing',],
   u'plus': [u'https://www.googleapis.com/auth/plus.me',],
 }
 
-ADDRESS_FIELDS_PRINT_ORDER = [u'contactName', u'organizationName', 
-   u'addressLine1', u'addressLine2', u'addressLine3', u'locality',
-   u'region', u'postalCode', u'countryCode']
+ADDRESS_FIELDS_PRINT_ORDER = [
+  u'contactName', u'organizationName',
+  u'addressLine1', u'addressLine2', u'addressLine3',
+  u'locality', u'region', u'postalCode', u'countryCode',
+  ]
 
 ADDRESS_FIELDS_ARGUMENT_MAP = {
   u'contact': u'contactName', u'contactname': u'contactName',
@@ -139,7 +154,7 @@ PRINTJOB_DESCENDINGORDER_MAP = {
   u'TITLE': u'TITLE_DESC',
   }
 
-PRINTJOBS_DEFAULT_JOB_LIMIT = 25
+PRINTJOBS_DEFAULT_JOB_LIMIT = 0
 PRINTJOBS_DEFAULT_MAX_RESULTS = 100
 
 CALENDAR_REMINDER_METHODS = [u'email', u'sms', u'popup',]
@@ -246,8 +261,11 @@ DRIVEFILE_ORDERBY_CHOICES_MAP = {
   u'viewedbymedate': u'lastViewedByMeDate',
   }
 
-DELETE_DRIVEFILE_FUNCTION_TO_ACTION_MAP = {u'delete': u'purging',
-   u'trash': u'trashing', u'untrash': u'untrashing',}
+DELETE_DRIVEFILE_FUNCTION_TO_ACTION_MAP = {
+  u'delete': u'purging',
+  u'trash': u'trashing',
+  u'untrash': u'untrashing',
+  }
 
 DRIVEFILE_LABEL_CHOICES_MAP = {
   u'restricted': u'restricted',
@@ -392,8 +410,10 @@ FILTER_CRITERIA_CHOICES_MAP = {
   u'subject': u'subject',
   u'to': u'to',
   }
-FILTER_ACTION_CHOICES = [u'archive', u'forward', u'important', u'label',
-   u'markread', u'neverspam', u'notimportant', u'star', u'trash',]
+FILTER_ACTION_CHOICES = [
+  u'archive', u'forward', u'important', u'label',
+  u'markread', u'neverspam', u'notimportant', u'star', u'trash',
+  ]
 
 CROS_ARGUMENT_TO_PROPERTY_MAP = {
   u'activetimeranges': [u'activeTimeRanges.activeTime', u'activeTimeRanges.date'],
@@ -425,7 +445,9 @@ CROS_ARGUMENT_TO_PROPERTY_MAP = {
   u'supportenddate': [u'supportEndDate',],
   u'tag': [u'annotatedAssetId',],
   u'timeranges': [u'activeTimeRanges.activeTime', u'activeTimeRanges.date'],
+  u'times': [u'activeTimeRanges.activeTime', u'activeTimeRanges.date'],
   u'user': [u'annotatedUser',],
+  u'users': [u'recentUsers.email', u'recentUsers.type'],
   u'willautorenew': [u'willAutoRenew',],
   }
 
@@ -454,6 +476,11 @@ CROS_SCALAR_PROPERTY_PRINT_ORDER = [
   u'willAutoRenew',
   ]
 
+CROS_RECENT_USERS_ARGUMENTS = [u'recentusers', u'users']
+CROS_ACTIVE_TIME_RANGES_ARGUMENTS = [u'timeranges', u'activetimeranges', u'times']
+CROS_START_ARGUMENTS = [u'start', u'startdate', u'oldestdate']
+CROS_END_ARGUMENTS = [u'end', u'enddate']
+
 #
 # Global variables
 #
@@ -486,6 +513,10 @@ GM_MAP_ROLE_ID_TO_NAME = u'ri2n'
 GM_MAP_ROLE_NAME_TO_ID = u'rn2i'
 # Dictionary mapping User ID to Name
 GM_MAP_USER_ID_TO_NAME = u'ui2n'
+# GAM cache directory. If no_cache is True, this variable will be set to None
+GM_CACHE_DIR = u'gacd'
+# Reset GAM cache directory after discovery
+GM_CACHE_DISCOVERY_ONLY = u'gcdo'
 #
 GM_Globals = {
   GM_SYSEXITRC: 0,
@@ -502,6 +533,8 @@ GM_Globals = {
   GM_MAP_ROLE_ID_TO_NAME: None,
   GM_MAP_ROLE_NAME_TO_ID: None,
   GM_MAP_USER_ID_TO_NAME: None,
+  GM_CACHE_DIR: None,
+  GM_CACHE_DISCOVERY_ONLY: True,
   }
 #
 # Global variables defined by environment variables/signal files
@@ -515,6 +548,8 @@ GC_AUTO_BATCH_MIN = u'auto_batch_min'
 GC_BATCH_SIZE = u'batch_size'
 # GAM cache directory. If no_cache is specified, this variable will be set to None
 GC_CACHE_DIR = u'cache_dir'
+# GAM cache discovery only. If no_cache is False, only API discovery calls will be cached
+GC_CACHE_DISCOVERY_ONLY = u'cache_discovery_only'
 # Character set of batch, csv, data files
 GC_CHARSET = u'charset'
 # Path to client_secrets.json
@@ -564,6 +599,7 @@ GC_Defaults = {
   GC_AUTO_BATCH_MIN: 0,
   GC_BATCH_SIZE: 50,
   GC_CACHE_DIR: u'',
+  GC_CACHE_DISCOVERY_ONLY: True,
   GC_CHARSET: DEFAULT_CHARSET,
   GC_CLIENT_SECRETS_JSON: FN_CLIENT_SECRETS_JSON,
   GC_CONFIG_DIR: u'',
@@ -573,16 +609,16 @@ GC_Defaults = {
   GC_DOMAIN: u'',
   GC_DRIVE_DIR: u'',
   GC_DRIVE_MAX_RESULTS: 1000,
-  GC_NO_BROWSER: FALSE,
-  GC_NO_CACHE: FALSE,
-  GC_NO_UPDATE_CHECK: FALSE,
-  GC_NO_VERIFY_SSL: FALSE,
+  GC_NO_BROWSER: False,
+  GC_NO_CACHE: False,
+  GC_NO_UPDATE_CHECK: False,
+  GC_NO_VERIFY_SSL: False,
   GC_NUM_THREADS: 25,
   GC_OAUTH2_TXT: FN_OAUTH2_TXT,
   GC_OAUTH2SERVICE_JSON: FN_OAUTH2SERVICE_JSON,
   GC_SECTION: u'',
   GC_SHOW_COUNTS_MIN: 0,
-  GC_SHOW_GETTINGS: TRUE,
+  GC_SHOW_GETTINGS: True,
   GC_SITE_DIR: u'',
   GC_USER_MAX_RESULTS: 500,
   }
@@ -606,6 +642,7 @@ GC_VAR_INFO = {
   GC_AUTO_BATCH_MIN: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (0, None)},
   GC_BATCH_SIZE: {GC_VAR_TYPE: GC_TYPE_INTEGER, GC_VAR_LIMITS: (1, 1000)},
   GC_CACHE_DIR: {GC_VAR_TYPE: GC_TYPE_DIRECTORY},
+  GC_CACHE_DISCOVERY_ONLY: {GC_VAR_TYPE: GC_TYPE_BOOLEAN},
   GC_CHARSET: {GC_VAR_TYPE: GC_TYPE_STRING},
   GC_CLIENT_SECRETS_JSON: {GC_VAR_TYPE: GC_TYPE_FILE},
   GC_CONFIG_DIR: {GC_VAR_TYPE: GC_TYPE_DIRECTORY},
@@ -663,9 +700,17 @@ MESSAGE_SERVICE_NOT_APPLICABLE = u'Service not applicable for this address: {0}.
 MESSAGE_INSTRUCTIONS_OAUTH2SERVICE_JSON = u'Please run\n\ngam create project\ngam user <user> check serviceaccount\n\nto create and configure a service account.'
 MESSAGE_OAUTH2SERVICE_JSON_INVALID = u'The file {0} is missing required keys (client_email, client_id or private_key). Please remove it and recreate with the commands:\n\ngam create project\ngam user <user> check serviceaccount'
 # oauth errors
-OAUTH2_TOKEN_ERRORS = [u'access_denied', u'unauthorized_client: Unauthorized client or scope in request.', u'access_denied: Requested client not authorized.',
-                       u'invalid_grant: Not a valid email.', u'invalid_grant: Invalid email or User ID', u'invalid_grant: Bad Request',
-                       u'invalid_request: Invalid impersonation prn email address.', u'internal_failure: Backend Error']
+OAUTH2_TOKEN_ERRORS = [
+  u'access_denied',
+  u'access_denied: Requested client not authorized',
+  u'internal_failure: Backend Error',
+  u'invalid_grant: Bad Request',
+  u'invalid_grant: Invalid email or User ID',
+  u'invalid_grant: Not a valid email',
+  u'invalid_request: Invalid impersonation prn email address',
+  u'unauthorized_client: Client is unauthorized to retrieve access tokens using this method',
+  u'unauthorized_client: Unauthorized client or scope in request',
+  ]
 #
 # callGAPI throw reasons
 GAPI_BACKEND_ERROR = u'backendError'
@@ -683,4 +728,3 @@ GAPI_USER_RATE_LIMIT_EXCEEDED = u'userRateLimitExceeded'
 GAPI_DEFAULT_RETRY_REASONS = [GAPI_QUOTA_EXCEEDED, GAPI_RATE_LIMIT_EXCEEDED, GAPI_USER_RATE_LIMIT_EXCEEDED, GAPI_BACKEND_ERROR, GAPI_INTERNAL_ERROR]
 GAPI_GMAIL_THROW_REASONS = [GAPI_SERVICE_NOT_AVAILABLE]
 GAPI_GPLUS_THROW_REASONS = [GAPI_SERVICE_NOT_AVAILABLE]
-

src/whatsnew.txt

@@ -1,3 +1,32 @@
+GAM 4.22
+ - Validate client id and secret from user input (reduces user errors on create project)
+ - Ross - optimize "gam print printjobs"
+ - Ross- countsonly option for gam print courses
+
+GAM 4.21
+ - Drive v3 fixes/updates by Ross
+ - "gam print crosactivty" command outputs active users and times
+ - SMime and calendar ACL fixes by Ross
+ - standardized cros info/print functionality by Ross
+
+GAM 4.2
+ - Create, Update, Delete and List Team Drives
+ - Start moving to Drive API v3
+ - Disable GAM cache by default to prevent errors (Ross)
+ - Use service accounts for all Calendar, Drive and Gmail operations to reduce scopes
+ - Fix "Unknown" errors due to a scope issue (may require "gam oauth revoke" and re-authentication)
+ - "gam info domain" shows basic user / license sums again
+ - "gam report customer" now shows more browser usage stats
+ - Fix project creation ToS error (Ross)
+
+GAM 4.12
+ - Realtime 2SV user status in gam info user and gam print users. Thanks hajdbo!
+ - Reseller API support. Create and manage customers and subscriptions.
+ - Delete or modify Gmail threads. Improved message delete/modify performance.
+ - Support for the new G Suite Enterprise license
+ - Manage S/MIME certificates for G Suite Enterprise users
+ - Many fixes and improvements by Ross.
+
 GAM 4.11
  - Allow newlines in calendar event descriptions (Ross)
  - All HTTP requests should now honor SSL verify setting, debug, etc

src/windows-build.bat

@@ -5,8 +5,8 @@ rmdir /q /s dist
 del /q /f gam-%1-windows.zip
 del /q /f gam-%1-windows-x64.zip
 del /q /f gam-%1-windows-x64.msi
-del /q /f gam.wixobj
-del /q /f gam.wixpdb
+del /q /f *.wixobj
+del /q /f *.wixpdb
 
 c:\python27-32\scripts\pyinstaller --clean -F --distpath=gam windows-gam.spec
 xcopy LICENSE gam\
@@ -25,4 +25,5 @@ xcopy GamCommands.txt gam-64\
 
 set GAMVERSION=%1
 "%ProgramFiles(x86)%\WiX Toolset v3.10\bin\candle.exe" -arch x64 gam.wxs
-"%ProgramFiles(x86)%\WiX Toolset v3.10\bin\light.exe" -ext "%ProgramFiles(x86)%\WiX Toolset v3.10\bin\WixUIExtension.dll" gam.wixobj -o gam-%1-windows-x64.msi
+"%ProgramFiles(x86)%\WiX Toolset v3.10\bin\light.exe" -ext "%ProgramFiles(x86)%\WiX Toolset v3.10\bin\WixUIExtension.dll" gam.wixobj -o gam-%1-windows-x64.msi
+del /q /f gam-%1-windows-x64.wixpdb