deepblue/GoogleDriveManagement
1
0
Fork 0
mirror of https://github.com/GAM-team/GAM.git synced 2026-06-28 18:01:36 +00:00
Code Issues Actions 9 Packages Projects Releases Wiki Activity

Updated device commmands, Update use of \ in CSV files

Browse Source
This commit is contained in:
Ross Scroggs
2023-12-01 18:04:18 -08:00
parent b0f05c2dea
commit a64d6f1215
20 changed files with 297 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/Bulk-Processing.md
View File

@@ -69,13 +69,13 @@ gam redirect stdout ./NewStudents.out redirect stderr ./NewStudents.err tbatch N
## CSV files
```
gam csv <FileName>|-|(gsheet <UserGoogleSheet>)|(gdoc <UserGoogleDoc>) [charset <Charset>] [warnifnodata]
[columndelimiter <Character>] [quotechar <Character>] [fields <FieldNameList>]
[columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>] [fields <FieldNameList>]
(matchfield|skipfield <FieldName> <RegularExpression>)* [showcmds [<Boolean>]]
[maxrows <Integer>]
gam <GAMArgumentList>
gam loop <FileName>|-|(gsheet <UserGoogleSheet>)|(gdoc <UserGoogleDoc>) [charset <Charset>] [warnifnodata]
[columndelimiter <Character>] [quotechar <Character>] [fields <FieldNameList>]
[columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>] [fields <FieldNameList>]
(matchfield|skipfield <FieldName> <RegularExpression>)* [showcmds [<Boolean>]]
[maxrows <Integer>]
gam <GAMArgumentList>
@@ -87,6 +87,7 @@ gam loop <FileName>|-|(gsheet <UserGoogleSheet>)|(gdoc <UserGoogleDoc>) [charset
* `gsheet <UserGoogleSheet>` - A Google Sheet and the one or more columns that contain data
* `gdoc <UserGoogleDoc>` - A Google Doc and the one or more columns that contain data
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings.
* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected

94
docs/CSV-Special-Characters.md Normal file
View File

@@ -0,0 +1,94 @@
# CSV Special Characters
- [Python CSV documentation](https://docs.python.org/3/library/csv.html#dialects-and-formatting-parameters)
## Python variables that control CSV file reading/writing:
```
Dialect.delimiter
A one-character string used to separate fields.
It defaults to ','.
Dialect.doublequote
Controls how instances of quotechar appearing inside a field should themselves be quoted.
When True, the character is doubled. When False, the escapechar is used as a prefix to the quotechar.
It defaults to True.
Dialect.escapechar
A one-character string used by the writer to escape the delimiter if quoting is set to QUOTE_NONE and the quotechar if doublequote is False.
On reading, the escapechar removes any special meaning from the following character.
It defaults to None, which disables escaping.
Dialect.lineterminator
The string used to terminate lines produced by the writer.
It defaults to '\r\n'.
The reader is hard-coded to recognise either '\r' or '\n' as end-of-line, and ignores lineterminator.
Dialect.quotechar
A one-character string used to quote fields containing special characters, such as the delimiter or quotechar, or which contain new-line characters.
It defaults to '"'.
Dialect.quoting
Controls when quotes should be generated by the writer and recognised by the reader. It can take on any of the QUOTE_* constants (see section Module Contents).
It defaults to QUOTE_MINIMAL.
```
## GAM variables that control CSV file reading/writing:
```
csv_input_column_delimiter = , - Dialect.delimiter
csv_input_no_escape_char = true - Dialect.escapechar is set to None if true, '\' if false
csv_input_quote_char = " - Dialect.quotechar
csv_output_column_delimiter = , - Dialect.delimiter
csv_output_no_escape_char = false - Dialect.escapechar is set to None if true, '\' if false
csv_output_line_terminator = lf - Dialect.lineterminator
csv_output_quote_char = " - Dialect.quotechar
todrive_no_escape_char = true - Dialect.escapechar is set to None if true, '\' if false
```
GAM sets Dialect.doublequote to true and Dialect.quoting to QUOTE_MINIMAL; there are no variables to change these values.
## Examples
### Local file, default settings
With these settings, here are examples of how field values are mapped on output to a local file:
```
csv_output_column_delimiter = ,
csv_output_no_escape_char = false
csv_output_quote_char = "
```
| Input | Output |
|-------|--------|
| abc def | abc def |
| abc,def | "abc,def" |
| abc"def | "abc""def" |
| abc\def | abc\\\\def |
### Local file, modified settings
With these settings, here are examples of how field values are mapped on output to a local file:
```
csv_output_column_delimiter = ,
csv_output_no_escape_char = true
csv_output_quote_char = "
```
| Input | Output |
|-------|--------|
| abc def | abc def |
| abc,def | "abc,def" |
| abc"def | "abc""def" |
| abc\def | abc\def |
### todrive, default settings
With these settings, here are examples of how field values are mapped on output to todrive
```
csv_output_column_delimiter = ,
todrive_no_escape_char = true
csv_output_quote_char = "
```
| Input | Output |
|-------|--------|
| abc def | abc def |
| abc,def | "abc,def" |
| abc"def | "abc""def" |
| abc\def | abc\def |

3
docs/Cloud-Identity-Devices.md
View File

@@ -36,8 +36,9 @@
See: https://support.google.com/a/answer/7549103
<QueryDeviceList> ::= "<QueryDevice>(,<QueryDevice>)*"
<DeviceID> ::= devices/<String>
<DeviceIDList> ::= "<DeviceID>(,<DeviceID>)*"
<DeviceEntity> ::=
<DeviceIDList> |
<DeviceIDList> | devicesn <String> |
(query:<QueryDevice>)|(query <QueryDevice>)
<DeviceType> ::= android|chrome_os|google_sync|linux|mac_os|windows
<DeviceUserID> ::= devices/<String>/deviceUsers/<String>

20
docs/Collections-of-ChromeOS-Devices.md
View File

@@ -90,7 +90,7 @@
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>])|
@@ -100,7 +100,7 @@
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>])|
@@ -117,7 +117,7 @@
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>])|
@@ -128,7 +128,7 @@
(gdoc <UserGoogleDoc>)|
(gcscsv <StorageBucketObjectName>)|
(gcsdoc <StorageBucketObjectName>))
[charset <Charset>] [columndelimiter <Character>] [quotechar <Character>] [fields <FieldNameList>])
[charset <Charset>] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>] [fields <FieldNameList>])
keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
(matchfield|skipfield <FieldName> <RegularExpression>)*
@@ -263,7 +263,7 @@ croscsvfile
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>]
@@ -276,6 +276,7 @@ croscsvfile
* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain ChromeOS deviceIds
* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
@@ -290,7 +291,7 @@ croscsvfile_sn
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>]
@@ -303,6 +304,7 @@ croscsvfile_sn
* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain ChromeOS serial numbers
* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
@@ -334,7 +336,7 @@ csvdatafile
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>]
@@ -348,6 +350,7 @@ csvdatafile
* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain ChromeOS deviceIds
* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
@@ -363,7 +366,7 @@ csvkmd
(gdoc <UserGoogleDoc>)|
(gcscsv <StorageBucketObjectName>)|
(gcsdoc <StorageBucketObjectName>))
[charset <Charset>] [columndelimiter <Character>] [quotechar <Character>] [fields <FieldNameList>])
[charset <Charset>] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>] [fields <FieldNameList>])
keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
(matchfield|skipfield <FieldName> <RegularExpression>)*
@@ -376,6 +379,7 @@ csvkmd
* `gdoc <UserGoogleDoc>` - A Google Doc containing rows with columns of the type of item specified
* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings

6
docs/Collections-of-Items.md
View File

@@ -63,7 +63,7 @@ A CSV file with one or more columns per row that contain Items.
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>]
@@ -75,6 +75,7 @@ A CSV file with one or more columns per row that contain Items.
* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain Items
* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
@@ -90,7 +91,7 @@ A CSV file with a key column that contains an Item and optional subkey and data
(gdoc <UserGoogleDoc>)|
(gcscsv <StorageBucketObjectName>)|
(gcsdoc <StorageBucketObjectName>))
[charset <Charset>] [columndelimiter <Character>] [quotechar <Character>] [fields <FieldNameList>])
[charset <Charset>] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>] [fields <FieldNameList>])
keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
(matchfield|skipfield <FieldName> <RegularExpression>)*
@@ -102,6 +103,7 @@ A CSV file with a key column that contains an Item and optional subkey and data
* `gcscsv <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object containing rows with columns of items
* `gcsdoc <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object containing rows with columns of items
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings

21
docs/Collections-of-Users.md
View File

@@ -127,7 +127,7 @@
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>])|
@@ -148,7 +148,7 @@
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>])|
@@ -161,7 +161,7 @@
(gdoc <UserGoogleDoc>)|
(gcscsv <StorageBucketObjectName>)|
(gcsdoc <StorageBucketObjectName>))
[charset <Charset>] [columndelimiter <Character>] [quotechar <Character>] [fields <FieldNameList>])
[charset <Charset>] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>] [fields <FieldNameList>])
keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
(matchfield|skipfield <FieldName> <RegularExpression>)*
@@ -360,7 +360,7 @@ csvfile
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>]
@@ -373,7 +373,8 @@ csvfile
* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns that contain Users
* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote character is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
@@ -408,7 +409,7 @@ csvdatafile
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>]
@@ -422,7 +423,8 @@ csvdatafile
* `gcsdoc(:<FieldName>)+ <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object and the one or more columns contain the type of item specified
* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote character is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
* `(matchfield|skipfield <FieldName> <RegularExpression>)*` - The criteria to select rows from the CSV file; can be used multiple times; if not specified, all rows are selected
@@ -439,7 +441,7 @@ csvkmd
(gdoc <UserGoogleDoc>)|
(gcscsv <StorageBucketObjectName>)|
(gcsdoc <StorageBucketObjectName>))
[charset <Charset>] [columndelimiter <Character>] [quotechar <Character>] [fields <FieldNameList>])
[charset <Charset>] [columndelimiter <Character>] [noescapechar <Boolean>][quotechar <Character>] [fields <FieldNameList>])
keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
subkeyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>]
(matchfield|skipfield <FieldName> <RegularExpression>)*
@@ -454,7 +456,8 @@ csvkmd
* `gcsdoc <StorageBucketObjectName>` - A Google Cloud Storage Bucket Object with columns of the type of item specified
* `warnifnodata` - Issue message 'No CSV file data found' and exit with return code 60 if there is no data selected from the file
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote character is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `endcsv` - Use this option to signal the end of the csvfile parameters in the case that the next argument on the command line is `fields` but is specifying the output field list for the command not column headings
* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
* `(keyfield <FieldName> [keypattern <RegularExpression>] [keyvalue <String>] [delimiter <Character>])+`

23
docs/GamUpdates.md
View File

@@ -10,6 +10,29 @@ Add the `-s` option to the end of the above commands to suppress creating the `g
See [Downloads](https://github.com/taers232c/GAMADV-XTD3/wiki/Downloads) for Windows or other options, including manual installation
### 6.66.02
Updated device commmands to handle the following error caused by an invalid query.
```
ERROR: 400: invalidArgument - Request contains an invalid argument.
```
Added fields `deviceid` and `hostname` to `<DeviceFieldName>`.
### 6.66.01
Added the following variables to gam.cfg that allow control over whether `\` is used as an escape character
when reading/writing CSV files.
```
csv_input_no_escape_char - default value True
csv_output_no_escape_char - default value False
todrive_no_escape_char - default value True
```
When the value is True, `\` is ignored as an escape character; when the value is False,
`\\` on input is converted to `\`, `\` on output is converted to `\\`.
* See: https://github.com/taers232c/GAMADV-XTD3/wiki/CSV-Special-Characters
### 6.66.00
Added support for `Focus Time` and `Out of Office` status events in user's primary calendars.

4
docs/How-to-Upgrade-from-Standard-GAM.md
View File

@@ -334,7 +334,7 @@ writes the credentials into the file oauth2.txt.
admin@server:/Users/admin/bin/gamadv-xtd3$ rm -f /Users/admin/GAMConfig/oauth2.txt
admin@server:/Users/admin/bin/gamadv-xtd3$ ./gam version
WARNING: Config File: /Users/admin/GAMConfig/gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: /Users/admin/GAMConfig/oauth2.txt, Not Found
GAMADV-XTD3 6.66.00 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.66.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.10.8 64-bit final
MacOS High Sierra 10.13.6 x86_64
@@ -1002,7 +1002,7 @@ writes the credentials into the file oauth2.txt.
C:\GAMADV-XTD3>del C:\GAMConfig\oauth2.txt
C:\GAMADV-XTD3>gam version
WARNING: Config File: C:\GAMConfig\gam.cfg, Section: DEFAULT, Item: oauth2_txt, Value: C:\GAMConfig\oauth2.txt, Not Found
GAMADV-XTD3 6.66.00 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.66.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.0 64-bit final
Windows-10-10.0.17134 AMD64

9
docs/Meta-Commands-and-File-Redirection.md
View File

@@ -112,7 +112,7 @@ You can redirect stdout and stderr to null and stderr can be redirected to stdou
```
<Redirect> ::=
redirect csv <FileName> [multiprocess] [append] [noheader] [charset <Charset>]
[columndelimiter <Character>] [quotechar <Character>]
[columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[timestampcolumn <String>]
[todrive <ToDriveAttribute>*] |
redirect stdout <FileName> [multiprocess] [append] |
@@ -141,8 +141,11 @@ subsequent GAM commands specify `append noheader`.
The `charset <Charset>` subargument sets the character set of the CSV file; the default is the value of `charset`
in `gam.cfg` which defaults to UTF-8.
The `columndelimiter <Character>` sets the intercolumn delimiter of the CSV file; the default value
is the value of csv_output_column_delimiter` in `gam.cfg` which defaults to comma.
The `columndelimiter <Character>` subargument sets the intercolumn delimiter of the CSV file; the default value
is the value of `csv_output_column_delimiter` in `gam.cfg` which defaults to comma.
The `noescapechar <Boolean>` subargument controls whether `\` is used as an escape character when writing the CSV file; the default value
is the value of `csv_output_no_escape_char` in `gam.cfg` which defaults to False.
The `quotechar <Character>` subargument sets the character used to quote fields in the CSV file
that contaim special charactere; the default value is the value of `csv_output_quote_char` in `gam.cfg`

2
docs/Tag-Replace.md
View File

@@ -167,7 +167,7 @@ gam create user <EmailAddress> <UserAttribute>*
gam update user <UserItem> <UserAttribute>
[updateprimaryemail <RegularExpression> <EmailReplacement>]
[updateoufromgroup <FileName> [charset <CharSet>]
[columndelimiter <Character>] [quotechar <Character>]
[columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[fields <FieldNameList>] [keyfield <FieldName>] [datafield <FieldName>]]
[clearschema <SchemaName>] [clearschema <SchemaName>.<FieldName>]
[createifnotfound] [notify <EmailAddress>] [subject <String>]

11
docs/Todrive.md
View File

@@ -132,6 +132,9 @@ todrive_nobrowser
todrive_noemail
Enable/disable sending an email when todrive is specified
Default: True
todrive_no_escape_char
When writing a CSV file to Google Drive, should `\` be ignored as an escape character.
Default: True
todrive_parent
Parent folder for CSV files when todrive is specified;
can be id:<DriveFolderID> or <DriveFolderName>
@@ -180,6 +183,7 @@ direct the uploaded file to a particular user and location and add a timestamp t
(tdlocale <Locale>)|
(tdnobrowser [<Boolean>])|
(tdnoemail [<Boolean>])|
(tdnoescapechar [<Boolean>])|
(tdparent (id:<DriveFolderID>)|<DriveFolderName>)|
(tdretaintitle [<Boolean>])|
(tdshare <EmailAddress> commenter|reader|writer)|
@@ -233,6 +237,9 @@ If `tdfileid <DriveFileID>` is not specified, a new file is created.
* `tdnobrowser` - If False, a browser is opened to view the file uploaded to Google Drive; if not specified, the `todrive_nobrowser` value from gam.cfg is used.
* `tdnoemail` - If False, an email is sent to `tduser` informing them of name and URL of the uploaded file; if not specified, the `todrive_noemail` value from gam.cfg is used.
## Escape character
* `tdnoescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `todrive_no_escape_char` from `gam.cfg` will be used
## Local copy
* `tdlocalcopy` - Should a local copy of the CSV file be saved in addition to the file uploaded to Google Drive; if not specified, the `todrive_localcopy` value from gam.cfg is used.
@@ -265,7 +272,7 @@ If `tdfileid <DriveFileID>` is not specified, a new file is created.
You can specify `todrive` options in conjunction with `redirect csv`.
```
redirect csv <FileName> [multiprocess] [append] [noheader] [charset <Charset>]
[columndelimiter <Character>] [quotechar <Character>]
[columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[todrive <ToDriveAttribute>*]
```
If you are doing `redirect csv <FileName> multiprocess`, it is more efficient to specify `todrive <ToDriveAttribute>*` as part of
@@ -274,6 +281,8 @@ the redirect as verification of the `todrive` settings, which can invole several
`columndelimiter <Character>` and `quotechar <Character>` will not generally be used with `todrive` as
Google Sheets only recognizes `,` as the column delimiter and `"` as the quote character.
`noescapechar true` will generally be used with `todrive` as Google Sheets does not recognize `\\` as an escaped `\`.
## Examples
Generate a list of user IDs and names, title the file "User IDs and Names", upload it to the "GAM Reports" folder of usermgr@domain.com, add a timestamp to the title.
```

4
docs/Users-Calendars-Events.md
View File

@@ -48,7 +48,7 @@
(gdoc <UserGoogleDoc>)|
(gcscsv <StorageBucketObjectName>)|
(gcsdoc <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
<CSVFileSelector> ::=
@@ -57,7 +57,7 @@
(gdoc(:<FieldName>)+ <UserGoogleDoc>)|
(gcscsv(:<FieldName>)+ <StorageBucketObjectName>)|
(gcsdoc(:<FieldName>)+ <StorageBucketObjectName>))
[warnifnodata] [columndelimiter <Character>] [quotechar <Character>]
[warnifnodata] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[endcsv|(fields <FieldNameList>)]
(matchfield|skipfield <FieldName> <RegularExpression>)*
[delimiter <Character>]

2
docs/Users-Drive-Activity-Settings.md
View File

@@ -78,7 +78,7 @@ gam <UserTypeEntity> print|show driveactivity [v2] [todrive <ToDriveAttributes>*
yesterday|today|thismonth|(previousmonths <Integer>)]
[action|actions [not] <DriveActivityActionList>]
[consolidationstrategy legacy|none]
[idmapfile <FileName>|(gsheet <UserGoogleSheet>) [charset <String>] [columndelimiter <Character>] [quotechar <Character>]]
[idmapfile <FileName>|(gsheet <UserGoogleSheet>) [charset <String>] [columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]]
[formatjson [quotechar <Character>]]
```
By default, Drive Activity API v2 is used; the `v2` option is ignored.

9
docs/Users.md
View File

@@ -598,7 +598,7 @@ gam update user <UserItem> [ignorenullpassword] <UserAttribute>*
[verifynotinvitable|alwaysevict] [noactionifalias]
[updateprimaryemail <RegularExpression> <EmailReplacement>]
[updateoufromgroup <FileName> [charset <CharSet>]
[columndelimiter <Character>] [quotechar <Character>]
[columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[fields <FieldNameList>] [keyfield <FieldName>] [datafield <FieldName>]]
[clearschema <SchemaName>] [clearschema <SchemaName>.<FieldName>]
[createifnotfound] [notfoundpassword random|<Password>]
@@ -619,7 +619,7 @@ gam update users <UserTypeEntity> [ignorenullpassword] <UserAttribute>*
[verifynotinvitable|alwaysevict] [noactionifalias]
[updateprimaryemail <RegularExpression> <EmailReplacement>]
[updateoufromgroup <FileName> [charset <CharSet>]
[columndelimiter <Character>] [quotechar <Character>]
[columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[fields <FieldNameList>] [keyfield <FieldName>] [datafield <FieldName>]]
[clearschema <SchemaName>] [clearschema <SchemaName>.<FieldName>]
[createifnotfound] [notfoundpassword random|<Password>]
@@ -640,7 +640,7 @@ gam <UserTypeEntity> update users [ignorenullpassword] <UserAttribute>*
[verifynotinvitable|alwaysevict] [noactionifalias]
[updateprimaryemail <RegularExpression> <EmailReplacement>]
[updateoufromgroup <FileName> [charset <CharSet>]
[columndelimiter <Character>] [quotechar <Character>]
[columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[fields <FieldNameList>] [keyfield <FieldName>] [datafield <FieldName>]]
[clearschema <SchemaName>] [clearschema <SchemaName>.<FieldName>]
[createifnotfound] [notfoundpassword random|<Password>]
@@ -814,11 +814,12 @@ No update is performed if a user does not belong to any group in the CSV file or
```
[updateoufromgroup <FileName> [charset <CharSet>]
[columndelimiter <Character>] [quotechar <Character>]
[columndelimiter <Character>] [noescapechar <Boolean>] [quotechar <Character>]
[fields <FieldNameList>] [keyfield <FieldName>] [datafield <FieldName>]]
```
* `<FileName>` - A CSV file containing rows with columns of items
* `columndelimiter <Character>` - Columns are separated by `<Character>`; if not specified, the value of `csv_input_column_delimiter` from `gam.cfg` will be used
* `noescapechar <Boolean>` - Should `\` be ignored as an escape character; if not specified, the value of `csv_input_no_escape_char` from `gam.cfg` will be used
* `quotechar <Character>` - The column quote characer is `<Character>`; if not specified, the value of `csv_input_quote_char` from `gam.cfg` will be used
* `fields <FieldNameList>` - The column headings of a CSV file that does not contain column headings
* `keyfield <FieldName>` - The column heading of the group column; the default is Group

12
docs/Version-and-Help.md
View File

@@ -4,7 +4,7 @@
Print the current version of Gam with details
```
gam version
GAMADV-XTD3 6.66.00 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.66.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.0 64-bit final
MacOS Monterey 12.7 x86_64
@@ -16,7 +16,7 @@ Time: 2023-06-02T21:10:00-07:00
Print the current version of Gam with details and time offset information
```
gam version timeoffset
GAMADV-XTD3 6.66.00 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.66.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.0 64-bit final
MacOS Monterey 12.7 x86_64
@@ -28,7 +28,7 @@ Your system time differs from www.googleapis.com by less than 1 second
Print the current version of Gam with extended details and SSL information
```
gam version extended
GAMADV-XTD3 6.66.00 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
GAMADV-XTD3 6.66.02 - https://github.com/taers232c/GAMADV-XTD3 - pythonsource
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.0 64-bit final
MacOS Monterey 12.7 x86_64
@@ -65,7 +65,7 @@ MacOS High Sierra 10.13.6 x86_64
Path: /Users/Admin/bin/gamadv-xtd3
Version Check:
Current: 5.35.08
Latest: 6.66.00
Latest: 6.66.02
echo $?
1
```
@@ -73,7 +73,7 @@ echo $?
Print the current version number without details
```
gam version simple
6.66.00
6.66.02
```
In Linux/MacOS you can do:
```
@@ -83,7 +83,7 @@ echo $VER
Print the current version of Gam and address of this Wiki
```
gam help
GAM 6.66.00 - https://github.com/taers232c/GAMADV-XTD3
GAM 6.66.02 - https://github.com/taers232c/GAMADV-XTD3
Ross Scroggs <ross.scroggs@gmail.com>
Python 3.12.0 64-bit final
MacOS Monterey 12.7 x86_64

1
docs/_Sidebar.md
View File

@@ -43,6 +43,7 @@ Command Processing
* [Command Line Parsing](Command-Line-Parsing)
* [Command Logging and Progress](Command-Logging-Progress)
* [Command data from Google Docs/Sheets/Storage](Command-Data-From-Google-Docs-Sheets-Storage)
* [CSV Special Characters](CSV-Special-Characters)
* [CSV Input Filtering](CSV-Input-Filtering)
* [CSV Output Filtering](CSV-Output-Filtering)
* [Meta Commands and File Redirection](Meta-Commands-and-File-Redirection)

19
docs/gam.cfg.md
View File

@@ -148,6 +148,10 @@ csv_input_column_delimiter
All places where an input CSV file can be specified have an
argument columndelimiter <String> that can override this value.
Default: ','
csv_input_no_escape_char
When reading a CSV file, should `\` be ignored as an escape character.
Set this to False if the input file data was written using `\` as an escape character.
Default: True
csv_input_quote_char
A one-character string used to quote fields containing special characters,
such as the csv_input_column_delimiter or csv_input_quote_char, or newline characters.
@@ -209,11 +213,15 @@ csv_output_header_force
for inclusion in the CSV file written by a gam print command
Default: ''
csv_output_line_terminator
Allowed values: cr, lf, crlf
p Allowed values: cr, lf, crlf
Designates character(s) used to terminate the lines of a CSV file.
For Linux and Mac OS, this would typically be lf.
For Windows, this would typically be crlf.
Default: lf
csv_output_no_escape_char
When writing a CSV file, should `\` be ignored as an escape character.
Set this to True if the output file data is to be read by a non-Python program.
Default: False
csv_output_quote_char
A one-character string used to quote fields containing special characters,
such as the csv_output_column_delimiter or csv_output_quote_char
@@ -527,6 +535,9 @@ todrive_nobrowser
todrive_noemail
Enable/disable sending an email when todrive is specified
Default: True
todrive_no_escape_char
When writing a CSV file to Google Drive, should `\` be ignored as an escape character.
Default: True
todrive_parent
Parent folder for CSV files when todrive is specified;
can be id:<DriveFolderID> or <DriveFolderName>
@@ -605,6 +616,7 @@ Section: DEFAULT
config_dir = /Users/admin/.gam
contact_max_results = 100
csv_input_column_delimiter = ,
csv_input_no_escape_char = true
csv_input_quote_char = '"'
csv_input_row_drop_filter = ''
csv_input_row_drop_filter = ''
@@ -619,6 +631,7 @@ Section: DEFAULT
csv_output_header_filter = ''
csv_output_header_force = ''
csv_output_line_terminator = lf
csv_output_no_escape_char = false
csv_output_quote_char = '"'
csv_output_row_drop_filter = ''
csv_output_row_drop_filter_mode = anymatch
@@ -689,6 +702,7 @@ Section: DEFAULT
todrive_locale = ''
todrive_nobrowser = false
todrive_noemail = true
todrive_no_escape_char = true
todrive_parent = root
todrive_sheet_timeformat = ''
todrive_sheet_timestamp = false
@@ -797,6 +811,7 @@ clock_skew_in_seconds = 10
config_dir = /Users/admin/.gam
contact_max_results = 100
csv_input_column_delimiter = ,
csv_input_no_escape_char = true
csv_input_quote_char = '"'
csv_input_row_drop_filter = ''
csv_input_row_filter = ''
@@ -807,6 +822,7 @@ csv_output_header_drop_filter = ''
csv_output_header_filter = ''
csv_output_header_force = ''
csv_output_line_terminator = lf
csv_output_no_escape_char = false
csv_output_quote_char = '"'
csv_output_row_drop_filter =
csv_output_row_filter = ''
@@ -873,6 +889,7 @@ todrive_localcopy = false
todrive_locale = ''
todrive_nobrowser = false
todrive_noemail = true
todrive_no_escape_char = true
todrive_parent = root
todrive_sheet_timeformat = ''
todrive_sheet_timestamp = false