deepblue/GoogleDriveManagement
1
0
Fork 0
mirror of https://github.com/GAM-team/GAM.git synced 2025-07-07 21:23:34 +00:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity

Add existing GAM 3.21 code

Browse Source
This commit is contained in:
Jay Lee
2014-06-28 10:14:05 -04:00
parent 8a904a81f5
commit d0356796e9
486 changed files with 158994 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

547
LICENSE.txt Normal file
View File

@ -0,0 +1,547 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
APACHE HTTP SERVER SUBCOMPONENTS:
The Apache HTTP Server includes a number of subcomponents with
separate copyright notices and license terms. Your use of the source
code for the these subcomponents is subject to the terms and
conditions of the following licenses.
For the mod_mime_magic component:
/*
* mod_mime_magic: MIME type lookup via file magic numbers
* Copyright (c) 1996-1997 Cisco Systems, Inc.
*
* This software was submitted by Cisco Systems to the Apache Group in July
* 1997. Future revisions and derivatives of this source code must
* acknowledge Cisco Systems as the original contributor of this module.
* All other licensing and usage conditions are those of the Apache Group.
*
* Some of this code is derived from the free version of the file command
* originally posted to comp.sources.unix. Copyright info for that program
* is included below as required.
* ---------------------------------------------------------------------------
* - Copyright (c) Ian F. Darwin, 1987. Written by Ian F. Darwin.
*
* This software is not subject to any license of the American Telephone and
* Telegraph Company or of the Regents of the University of California.
*
* Permission is granted to anyone to use this software for any purpose on any
* computer system, and to alter it and redistribute it freely, subject to
* the following restrictions:
*
* 1. The author is not responsible for the consequences of use of this
* software, no matter how awful, even if they arise from flaws in it.
*
* 2. The origin of this software must not be misrepresented, either by
* explicit claim or by omission. Since few users ever read sources, credits
* must appear in the documentation.
*
* 3. Altered versions must be plainly marked as such, and must not be
* misrepresented as being the original software. Since few users ever read
* sources, credits must appear in the documentation.
*
* 4. This notice may not be removed or altered.
* -------------------------------------------------------------------------
*
*/
For the modules\mappers\mod_imagemap.c component:
"macmartinized" polygon code copyright 1992 by Eric Haines, erich@eye.com
For the server\util_md5.c component:
/************************************************************************
* NCSA HTTPd Server
* Software Development Group
* National Center for Supercomputing Applications
* University of Illinois at Urbana-Champaign
* 605 E. Springfield, Champaign, IL 61820
* httpd@ncsa.uiuc.edu
*
* Copyright (C) 1995, Board of Trustees of the University of Illinois
*
************************************************************************
*
* md5.c: NCSA HTTPd code which uses the md5c.c RSA Code
*
* Original Code Copyright (C) 1994, Jeff Hostetler, Spyglass, Inc.
* Portions of Content-MD5 code Copyright (C) 1993, 1994 by Carnegie Mellon
* University (see Copyright below).
* Portions of Content-MD5 code Copyright (C) 1991 Bell Communications
* Research, Inc. (Bellcore) (see Copyright below).
* Portions extracted from mpack, John G. Myers - jgm+@cmu.edu
* Content-MD5 Code contributed by Martin Hamilton (martin@net.lut.ac.uk)
*
*/
/* these portions extracted from mpack, John G. Myers - jgm+@cmu.edu */
/* (C) Copyright 1993,1994 by Carnegie Mellon University
* All Rights Reserved.
*
* Permission to use, copy, modify, distribute, and sell this software
* and its documentation for any purpose is hereby granted without
* fee, provided that the above copyright notice appear in all copies
* and that both that copyright notice and this permission notice
* appear in supporting documentation, and that the name of Carnegie
* Mellon University not be used in advertising or publicity
* pertaining to distribution of the software without specific,
* written prior permission. Carnegie Mellon University makes no
* representations about the suitability of this software for any
* purpose. It is provided "as is" without express or implied
* warranty.
*
* CARNEGIE MELLON UNIVERSITY DISCLAIMS ALL WARRANTIES WITH REGARD TO
* THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
* AND FITNESS, IN NO EVENT SHALL CARNEGIE MELLON UNIVERSITY BE LIABLE
* FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
* AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING
* OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS
* SOFTWARE.
*/
/*
* Copyright (c) 1991 Bell Communications Research, Inc. (Bellcore)
*
* Permission to use, copy, modify, and distribute this material
* for any purpose and without fee is hereby granted, provided
* that the above copyright notice and this permission notice
* appear in all copies, and that the name of Bellcore not be
* used in advertising or publicity pertaining to this
* material without the specific, prior written permission
* of an authorized representative of Bellcore. BELLCORE
* MAKES NO REPRESENTATIONS ABOUT THE ACCURACY OR SUITABILITY
* OF THIS MATERIAL FOR ANY PURPOSE. IT IS PROVIDED "AS IS",
* WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES.
*/
For the srclib\apr\include\apr_md5.h component:
/*
* This is work is derived from material Copyright RSA Data Security, Inc.
*
* The RSA copyright statement and Licence for that original material is
* included below. This is followed by the Apache copyright statement and
* licence for the modifications made to that material.
*/
/* Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
rights reserved.
License to copy and use this software is granted provided that it
is identified as the "RSA Data Security, Inc. MD5 Message-Digest
Algorithm" in all material mentioning or referencing this software
or this function.
License is also granted to make and use derivative works provided
that such works are identified as "derived from the RSA Data
Security, Inc. MD5 Message-Digest Algorithm" in all material
mentioning or referencing the derived work.
RSA Data Security, Inc. makes no representations concerning either
the merchantability of this software or the suitability of this
software for any particular purpose. It is provided "as is"
without express or implied warranty of any kind.
These notices must be retained in any copies of any part of this
documentation and/or software.
*/
For the srclib\apr\passwd\apr_md5.c component:
/*
* This is work is derived from material Copyright RSA Data Security, Inc.
*
* The RSA copyright statement and Licence for that original material is
* included below. This is followed by the Apache copyright statement and
* licence for the modifications made to that material.
*/
/* MD5C.C - RSA Data Security, Inc., MD5 message-digest algorithm
*/
/* Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
rights reserved.
License to copy and use this software is granted provided that it
is identified as the "RSA Data Security, Inc. MD5 Message-Digest
Algorithm" in all material mentioning or referencing this software
or this function.
License is also granted to make and use derivative works provided
that such works are identified as "derived from the RSA Data
Security, Inc. MD5 Message-Digest Algorithm" in all material
mentioning or referencing the derived work.
RSA Data Security, Inc. makes no representations concerning either
the merchantability of this software or the suitability of this
software for any particular purpose. It is provided "as is"
without express or implied warranty of any kind.
These notices must be retained in any copies of any part of this
documentation and/or software.
*/
/*
* The apr_md5_encode() routine uses much code obtained from the FreeBSD 3.0
* MD5 crypt() function, which is licenced as follows:
* ----------------------------------------------------------------------------
* "THE BEER-WARE LICENSE" (Revision 42):
* <phk@login.dknet.dk> wrote this file. As long as you retain this notice you
* can do whatever you want with this stuff. If we meet some day, and you think
* this stuff is worth it, you can buy me a beer in return. Poul-Henning Kamp
* ----------------------------------------------------------------------------
*/
For the srclib\apr-util\crypto\apr_md4.c component:
* This is derived from material copyright RSA Data Security, Inc.
* Their notice is reproduced below in its entirety.
*
* Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
* rights reserved.
*
* License to copy and use this software is granted provided that it
* is identified as the "RSA Data Security, Inc. MD4 Message-Digest
* Algorithm" in all material mentioning or referencing this software
* or this function.
*
* License is also granted to make and use derivative works provided
* that such works are identified as "derived from the RSA Data
* Security, Inc. MD4 Message-Digest Algorithm" in all material
* mentioning or referencing the derived work.
*
* RSA Data Security, Inc. makes no representations concerning either
* the merchantability of this software or the suitability of this
* software for any particular purpose. It is provided "as is"
* without express or implied warranty of any kind.
*
* These notices must be retained in any copies of any part of this
* documentation and/or software.
*/
For the srclib\apr-util\include\apr_md4.h component:
*
* This is derived from material copyright RSA Data Security, Inc.
* Their notice is reproduced below in its entirety.
*
* Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
* rights reserved.
*
* License to copy and use this software is granted provided that it
* is identified as the "RSA Data Security, Inc. MD4 Message-Digest
* Algorithm" in all material mentioning or referencing this software
* or this function.
*
* License is also granted to make and use derivative works provided
* that such works are identified as "derived from the RSA Data
* Security, Inc. MD4 Message-Digest Algorithm" in all material
* mentioning or referencing the derived work.
*
* RSA Data Security, Inc. makes no representations concerning either
* the merchantability of this software or the suitability of this
* software for any particular purpose. It is provided "as is"
* without express or implied warranty of any kind.
*
* These notices must be retained in any copies of any part of this
* documentation and/or software.
*/
For the srclib\apr-util\test\testmd4.c component:
*
* This is derived from material copyright RSA Data Security, Inc.
* Their notice is reproduced below in its entirety.
*
* Copyright (C) 1990-2, RSA Data Security, Inc. Created 1990. All
* rights reserved.
*
* RSA Data Security, Inc. makes no representations concerning either
* the merchantability of this software or the suitability of this
* software for any particular purpose. It is provided "as is"
* without express or implied warranty of any kind.
*
* These notices must be retained in any copies of any part of this
* documentation and/or software.
*/
For the srclib\apr-util\xml\expat\conftools\install-sh component:
#
# install - install a program, script, or datafile
# This comes from X11R5 (mit/util/scripts/install.sh).
#
# Copyright 1991 by the Massachusetts Institute of Technology
#
# Permission to use, copy, modify, distribute, and sell this software and its
# documentation for any purpose is hereby granted without fee, provided that
# the above copyright notice appear in all copies and that both that
# copyright notice and this permission notice appear in supporting
# documentation, and that the name of M.I.T. not be used in advertising or
# publicity pertaining to distribution of the software without specific,
# written prior permission. M.I.T. makes no representations about the
# suitability of this software for any purpose. It is provided "as is"
# without express or implied warranty.
#
For the test\zb.c component:
/* ZeusBench V1.01
===============
This program is Copyright (C) Zeus Technology Limited 1996.
This program may be used and copied freely providing this copyright notice
is not removed.
This software is provided "as is" and any express or implied waranties,
including but not limited to, the implied warranties of merchantability and
fitness for a particular purpose are disclaimed. In no event shall
Zeus Technology Ltd. be liable for any direct, indirect, incidental, special,
exemplary, or consequential damaged (including, but not limited to,
procurement of substitute good or services; loss of use, data, or profits;
or business interruption) however caused and on theory of liability. Whether
in contract, strict liability or tort (including negligence or otherwise)
arising in any way out of the use of this software, even if advised of the
possibility of such damage.
Written by Adam Twiss (adam@zeus.co.uk). March 1996
Thanks to the following people for their input:
Mike Belshe (mbelshe@netscape.com)
Michael Campanella (campanella@stevms.enet.dec.com)
*/
For the expat xml parser component:
Copyright (c) 1998, 1999, 2000 Thai Open Source Software Center Ltd
and Clark Cooper
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
====================================================================

642
admin-settings-v1.json Normal file
View File

@ -0,0 +1,642 @@
{
"kind": "discovery#restDescription",
"discoveryVersion": "v1",
"id": "admin-settings:v1",
"name": "admin-settings",
"version": "v1",
"revision": "20130823",
"title": "Admin Settings API (read-only calls)",
"description": "Lets you access Google Apps Admin Settings",
"ownerDomain": "google.com",
"ownerName": "Google",
"icons": {
"x16": "http://www.google.com/images/icons/product/search-16.gif",
"x32": "http://www.google.com/images/icons/product/search-32.gif"
},
"documentationLink": "https://developers.google.com/admin-sdk/admin-settings",
"protocol": "rest",
"baseUrl": "https://apps-apis.google.com/",
"basePath": "/a/feeds/domain/2.0/",
"rootUrl": "https://apps-apis.google.com/",
"servicePath": "/a/feeds/domain/2.0/",
"parameters": {
"v": {
"type": "string",
"description": "GData Version",
"default": "2.0",
"enum": [
"2.0"
],
"enumDescriptions": [
"GData 2.0"
],
"location": "query"
},
"alt": {
"type": "string",
"description": "Data format for the response.",
"default": "json",
"enum": [
"json"
],
"enumDescriptions": [
"Responses with Content-Type of application/json"
],
"location": "query"
},
"quotaUser": {
"type": "string",
"description": "Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters. Overrides userIp if both are provided.",
"location": "query"
},
"prettyPrint": {
"type": "boolean",
"description": "Returns response with indentations and line breaks.",
"default": "true",
"location": "query"
}
},
"auth": {
"oauth2": {
"scopes": {
"https://apps-apis.google.com/a/feeds/domain/": {
"description": "Manage domain settings"
}
}
}
},
"schemas": {
"DefaultLanguage": {
"id": "DefaultLanguage",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "default language value"
}
}
}
}
},
"OrganizationName": {
"id": "OrganizationName",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "organization name value"
}
}
}
}
},
"MaximumNumberOfUsers": {
"id": "MaximumNumberOfUsers",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "maximum number of users value"
}
}
}
}
},
"CurrentNumberOfUsers": {
"id": "CurrentNumbersOfUsers",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "current number of users value"
}
}
}
}
},
"IsVerified": {
"id": "IsVerified",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "boolean",
"description": "current verification status value"
}
}
}
}
},
"Edition": {
"id": "Edition",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "domain edition value"
}
}
}
}
},
"CustomerPIN": {
"id": "CustomerPIN",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "customer pin value"
}
}
}
}
},
"CreationTime": {
"id": "CreationTime",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "domain creation time value"
}
}
}
}
},
"CountryCode": {
"id": "CountryCode",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "domain country code value"
}
}
}
}
},
"AdminSecondaryEmail": {
"id": "AdminSecondaryEmail",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "admin secondary email value"
}
}
}
}
},
"MXVerification": {
"id": "MXVerification",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "domain mx verification value"
}
}
}
}
},
"SSOGeneral": {
"id": "SSOGeneral",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "domain sso general value"
}
}
}
}
},
"SSOSigningKey": {
"id": "SSOSigningKey",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "domain sso signing key value"
}
}
}
}
},
"UserEmailMigrationEnabled": {
"id": "UserEmailMigrationEnabled",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "user email migration enabled value"
}
}
}
}
},
"OutboundGateway": {
"id": "OutboundGateway",
"type": "object",
"properties": {
"apps$property": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "property name"
},
"value": {
"type": "string",
"description": "domain outbound gateway value"
}
}
}
}
}
},
"resources": {
"defaultLanguage": {
"methods": {
"get": {
"id": "admin-settings.defaultLanguage.get",
"path": "{domainName}/general/defaultLanguage",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "DefaultLanguage"
}
}
}
},
"organizationName": {
"methods": {
"get": {
"id": "admin-settings.organizationName.get",
"path": "{domainName}/general/organizationName",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "OrganizationName"
}
}
}
},
"maximumNumberOfUsers": {
"methods": {
"get": {
"id": "admin-settings.maximumNumberOfUsers.get",
"path": "{domainName}/general/maximumNumberOfUsers",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "MaximumNumberOfUsers"
}
}
}
},
"currentNumberOfUsers": {
"methods": {
"get": {
"id": "admin-settings.currentNumberOfUsers.get",
"path": "{domainName}/general/currentNumberOfUsers",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "CurrentNumberOfUsers"
}
}
}
},
"isVerified": {
"methods": {
"get": {
"id": "admin-settings.isVerified.get",
"path": "{domainName}/accountInformation/isVerified",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "IsVerified"
}
}
}
},
"edition": {
"methods": {
"get": {
"id": "admin-settings.edition.get",
"path": "{domainName}/accountInformation/edition",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "Edition"
}
}
}
},
"customerPIN": {
"methods": {
"get": {
"id": "admin-settings.customerPIN.get",
"path": "{domainName}/accountInformation/customerPIN",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "CustomerPIN"
}
}
}
},
"creationTime": {
"methods": {
"get": {
"id": "admin-settings.creationTime.get",
"path": "{domainName}/accountInformation/creationTime",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "CreationTime"
}
}
}
},
"countryCode": {
"methods": {
"get": {
"id": "admin-settings.countryCode.get",
"path": "{domainName}/accountInformation/countryCode",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "CountryCode"
}
}
}
},
"adminSecondaryEmail": {
"methods": {
"get": {
"id": "admin-settings.adminSecondaryEmail.get",
"path": "{domainName}/accountInformation/adminSecondaryEmail",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "AdminSecondaryEmail"
}
}
}
},
"mxVerification": {
"methods": {
"get": {
"id": "admin-settings.mxVerification.get",
"path": "{domainName}/verification/mx",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "MXVerification"
}
}
}
},
"ssoGeneral": {
"methods": {
"get": {
"id": "admin-settings.ssoGeneral.get",
"path": "{domainName}/sso/general",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "SSOGeneral"
}
}
}
},
"ssoSigningKey": {
"methods": {
"get": {
"id": "admin-settings.ssoSigningKey.get",
"path": "{domainName}/sso/signingkey",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "SSOSigningKey"
}
}
}
},
"userEmailMigrationEnabled": {
"methods": {
"get": {
"id": "admin-settings.userEmailMigrationEnabled.get",
"path": "{domainName}/email/migration",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "UserEmailMigrationEnabled"
}
}
}
},
"outboundGateway": {
"methods": {
"get": {
"id": "admin-settings.outboundGateway.get",
"path": "{domainName}/email/gateway",
"httpMethod": "GET",
"parameters": {
"domainName": {
"type": "string",
"required": "true",
"location": "path"
}
},
"response": {
"$ref": "OutboundGateway"
}
}
}
}
}
}

15
apiclient/__init__.py Normal file
View File

@ -0,0 +1,15 @@
# Copyright (C) 2012 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
__version__ = "1.2"

285
apiclient/channel.py Normal file
View File

@ -0,0 +1,285 @@
"""Channel notifications support.
Classes and functions to support channel subscriptions and notifications
on those channels.
Notes:
- This code is based on experimental APIs and is subject to change.
- Notification does not do deduplication of notification ids, that's up to
the receiver.
- Storing the Channel between calls is up to the caller.
Example setting up a channel:
# Create a new channel that gets notifications via webhook.
channel = new_webhook_channel("https://example.com/my_web_hook")
# Store the channel, keyed by 'channel.id'. Store it before calling the
# watch method because notifications may start arriving before the watch
# method returns.
...
resp = service.objects().watchAll(
bucket="some_bucket_id", body=channel.body()).execute()
channel.update(resp)
# Store the channel, keyed by 'channel.id'. Store it after being updated
# since the resource_id value will now be correct, and that's needed to
# stop a subscription.
...
An example Webhook implementation using webapp2. Note that webapp2 puts
headers in a case insensitive dictionary, as headers aren't guaranteed to
always be upper case.
id = self.request.headers[X_GOOG_CHANNEL_ID]
# Retrieve the channel by id.
channel = ...
# Parse notification from the headers, including validating the id.
n = notification_from_headers(channel, self.request.headers)
# Do app specific stuff with the notification here.
if n.resource_state == 'sync':
# Code to handle sync state.
elif n.resource_state == 'exists':
# Code to handle the exists state.
elif n.resource_state == 'not_exists':
# Code to handle the not exists state.
Example of unsubscribing.
service.channels().stop(channel.body())
"""
import datetime
import uuid
from apiclient import errors
from oauth2client import util
# The unix time epoch starts at midnight 1970.
EPOCH = datetime.datetime.utcfromtimestamp(0)
# Map the names of the parameters in the JSON channel description to
# the parameter names we use in the Channel class.
CHANNEL_PARAMS = {
'address': 'address',
'id': 'id',
'expiration': 'expiration',
'params': 'params',
'resourceId': 'resource_id',
'resourceUri': 'resource_uri',
'type': 'type',
'token': 'token',
}
X_GOOG_CHANNEL_ID = 'X-GOOG-CHANNEL-ID'
X_GOOG_MESSAGE_NUMBER = 'X-GOOG-MESSAGE-NUMBER'
X_GOOG_RESOURCE_STATE = 'X-GOOG-RESOURCE-STATE'
X_GOOG_RESOURCE_URI = 'X-GOOG-RESOURCE-URI'
X_GOOG_RESOURCE_ID = 'X-GOOG-RESOURCE-ID'
def _upper_header_keys(headers):
new_headers = {}
for k, v in headers.iteritems():
new_headers[k.upper()] = v
return new_headers
class Notification(object):
"""A Notification from a Channel.
Notifications are not usually constructed directly, but are returned
from functions like notification_from_headers().
Attributes:
message_number: int, The unique id number of this notification.
state: str, The state of the resource being monitored.
uri: str, The address of the resource being monitored.
resource_id: str, The unique identifier of the version of the resource at
this event.
"""
@util.positional(5)
def __init__(self, message_number, state, resource_uri, resource_id):
"""Notification constructor.
Args:
message_number: int, The unique id number of this notification.
state: str, The state of the resource being monitored. Can be one
of "exists", "not_exists", or "sync".
resource_uri: str, The address of the resource being monitored.
resource_id: str, The identifier of the watched resource.
"""
self.message_number = message_number
self.state = state
self.resource_uri = resource_uri
self.resource_id = resource_id
class Channel(object):
"""A Channel for notifications.
Usually not constructed directly, instead it is returned from helper
functions like new_webhook_channel().
Attributes:
type: str, The type of delivery mechanism used by this channel. For
example, 'web_hook'.
id: str, A UUID for the channel.
token: str, An arbitrary string associated with the channel that
is delivered to the target address with each event delivered
over this channel.
address: str, The address of the receiving entity where events are
delivered. Specific to the channel type.
expiration: int, The time, in milliseconds from the epoch, when this
channel will expire.
params: dict, A dictionary of string to string, with additional parameters
controlling delivery channel behavior.
resource_id: str, An opaque id that identifies the resource that is
being watched. Stable across different API versions.
resource_uri: str, The canonicalized ID of the watched resource.
"""
@util.positional(5)
def __init__(self, type, id, token, address, expiration=None,
params=None, resource_id="", resource_uri=""):
"""Create a new Channel.
In user code, this Channel constructor will not typically be called
manually since there are functions for creating channels for each specific
type with a more customized set of arguments to pass.
Args:
type: str, The type of delivery mechanism used by this channel. For
example, 'web_hook'.
id: str, A UUID for the channel.
token: str, An arbitrary string associated with the channel that
is delivered to the target address with each event delivered
over this channel.
address: str, The address of the receiving entity where events are
delivered. Specific to the channel type.
expiration: int, The time, in milliseconds from the epoch, when this
channel will expire.
params: dict, A dictionary of string to string, with additional parameters
controlling delivery channel behavior.
resource_id: str, An opaque id that identifies the resource that is
being watched. Stable across different API versions.
resource_uri: str, The canonicalized ID of the watched resource.
"""
self.type = type
self.id = id
self.token = token
self.address = address
self.expiration = expiration
self.params = params
self.resource_id = resource_id
self.resource_uri = resource_uri
def body(self):
"""Build a body from the Channel.
Constructs a dictionary that's appropriate for passing into watch()
methods as the value of body argument.
Returns:
A dictionary representation of the channel.
"""
result = {
'id': self.id,
'token': self.token,
'type': self.type,
'address': self.address
}
if self.params:
result['params'] = self.params
if self.resource_id:
result['resourceId'] = self.resource_id
if self.resource_uri:
result['resourceUri'] = self.resource_uri
if self.expiration:
result['expiration'] = self.expiration
return result
def update(self, resp):
"""Update a channel with information from the response of watch().
When a request is sent to watch() a resource, the response returned
from the watch() request is a dictionary with updated channel information,
such as the resource_id, which is needed when stopping a subscription.
Args:
resp: dict, The response from a watch() method.
"""
for json_name, param_name in CHANNEL_PARAMS.iteritems():
value = resp.get(json_name)
if value is not None:
setattr(self, param_name, value)
def notification_from_headers(channel, headers):
"""Parse a notification from the webhook request headers, validate
the notification, and return a Notification object.
Args:
channel: Channel, The channel that the notification is associated with.
headers: dict, A dictionary like object that contains the request headers
from the webhook HTTP request.
Returns:
A Notification object.
Raises:
errors.InvalidNotificationError if the notification is invalid.
ValueError if the X-GOOG-MESSAGE-NUMBER can't be converted to an int.
"""
headers = _upper_header_keys(headers)
channel_id = headers[X_GOOG_CHANNEL_ID]
if channel.id != channel_id:
raise errors.InvalidNotificationError(
'Channel id mismatch: %s != %s' % (channel.id, channel_id))
else:
message_number = int(headers[X_GOOG_MESSAGE_NUMBER])
state = headers[X_GOOG_RESOURCE_STATE]
resource_uri = headers[X_GOOG_RESOURCE_URI]
resource_id = headers[X_GOOG_RESOURCE_ID]
return Notification(message_number, state, resource_uri, resource_id)
@util.positional(2)
def new_webhook_channel(url, token=None, expiration=None, params=None):
"""Create a new webhook Channel.
Args:
url: str, URL to post notifications to.
token: str, An arbitrary string associated with the channel that
is delivered to the target address with each notification delivered
over this channel.
expiration: datetime.datetime, A time in the future when the channel
should expire. Can also be None if the subscription should use the
default expiration. Note that different services may have different
limits on how long a subscription lasts. Check the response from the
watch() method to see the value the service has set for an expiration
time.
params: dict, Extra parameters to pass on channel creation. Currently
not used for webhook channels.
"""
expiration_ms = 0
if expiration:
delta = expiration - EPOCH
expiration_ms = delta.microseconds/1000 + (
delta.seconds + delta.days*24*3600)*1000
if expiration_ms < 0:
expiration_ms = 0
return Channel('web_hook', str(uuid.uuid4()),
token, url, expiration=expiration_ms,
params=params)

963
apiclient/discovery.py Normal file
View File

@ -0,0 +1,963 @@
# Copyright (C) 2010 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Client for discovery based APIs.
A client library for Google's discovery based APIs.
"""
__author__ = 'jcgregorio@google.com (Joe Gregorio)'
__all__ = [
'build',
'build_from_document',
'fix_method_name',
'key2param',
]
# Standard library imports
import copy
from email.mime.multipart import MIMEMultipart
from email.mime.nonmultipart import MIMENonMultipart
import keyword
import logging
import mimetypes
import os
import re
import urllib
import urlparse
try:
from urlparse import parse_qsl
except ImportError:
from cgi import parse_qsl
# Third-party imports
import httplib2
import mimeparse
import uritemplate
# Local imports
from apiclient.errors import HttpError
from apiclient.errors import InvalidJsonError
from apiclient.errors import MediaUploadSizeError
from apiclient.errors import UnacceptableMimeTypeError
from apiclient.errors import UnknownApiNameOrVersion
from apiclient.errors import UnknownFileType
from apiclient.http import HttpRequest
from apiclient.http import MediaFileUpload
from apiclient.http import MediaUpload
from apiclient.model import JsonModel
from apiclient.model import MediaModel
from apiclient.model import RawModel
from apiclient.schema import Schemas
from oauth2client.anyjson import simplejson
from oauth2client.util import _add_query_parameter
from oauth2client.util import positional
# The client library requires a version of httplib2 that supports RETRIES.
httplib2.RETRIES = 1
logger = logging.getLogger(__name__)
URITEMPLATE = re.compile('{[^}]*}')
VARNAME = re.compile('[a-zA-Z0-9_-]+')
if httplib2.debuglevel > 0:
prettyPrint = 'true'
else:
prettyPrint = 'false'
DISCOVERY_URI = ('https://www.googleapis.com/discovery/v1/apis/'
'{api}/{apiVersion}/rest?prettyPrint=%s' % prettyPrint)
DEFAULT_METHOD_DOC = 'A description of how to use this function'
HTTP_PAYLOAD_METHODS = frozenset(['PUT', 'POST', 'PATCH'])
_MEDIA_SIZE_BIT_SHIFTS = {'KB': 10, 'MB': 20, 'GB': 30, 'TB': 40}
BODY_PARAMETER_DEFAULT_VALUE = {
'description': 'The request body.',
'type': 'object',
'required': True,
}
MEDIA_BODY_PARAMETER_DEFAULT_VALUE = {
'description': ('The filename of the media request body, or an instance '
'of a MediaUpload object.'),
'type': 'string',
'required': False,
}
# Parameters accepted by the stack, but not visible via discovery.
# TODO(dhermes): Remove 'userip' in 'v2'.
STACK_QUERY_PARAMETERS = frozenset(['trace', 'pp', 'userip', 'strict'])
STACK_QUERY_PARAMETER_DEFAULT_VALUE = {'type': 'string', 'location': 'query'}
# Library-specific reserved words beyond Python keywords.
RESERVED_WORDS = frozenset(['body'])
def fix_method_name(name):
"""Fix method names to avoid reserved word conflicts.
Args:
name: string, method name.
Returns:
The name with a '_' prefixed if the name is a reserved word.
"""
if keyword.iskeyword(name) or name in RESERVED_WORDS:
return name + '_'
else:
return name
def key2param(key):
"""Converts key names into parameter names.
For example, converting "max-results" -> "max_results"
Args:
key: string, the method key name.
Returns:
A safe method name based on the key name.
"""
result = []
key = list(key)
if not key[0].isalpha():
result.append('x')
for c in key:
if c.isalnum():
result.append(c)
else:
result.append('_')
return ''.join(result)
@positional(2)
def build(serviceName,
version,
http=None,
discoveryServiceUrl=DISCOVERY_URI,
developerKey=None,
model=None,
requestBuilder=HttpRequest):
"""Construct a Resource for interacting with an API.
Construct a Resource object for interacting with an API. The serviceName and
version are the names from the Discovery service.
Args:
serviceName: string, name of the service.
version: string, the version of the service.
http: httplib2.Http, An instance of httplib2.Http or something that acts
like it that HTTP requests will be made through.
discoveryServiceUrl: string, a URI Template that points to the location of
the discovery service. It should have two parameters {api} and
{apiVersion} that when filled in produce an absolute URI to the discovery
document for that service.
developerKey: string, key obtained from
https://code.google.com/apis/console.
model: apiclient.Model, converts to and from the wire format.
requestBuilder: apiclient.http.HttpRequest, encapsulator for an HTTP
request.
Returns:
A Resource object with methods for interacting with the service.
"""
params = {
'api': serviceName,
'apiVersion': version
}
if http is None:
http = httplib2.Http()
requested_url = uritemplate.expand(discoveryServiceUrl, params)
# REMOTE_ADDR is defined by the CGI spec [RFC3875] as the environment
# variable that contains the network address of the client sending the
# request. If it exists then add that to the request for the discovery
# document to avoid exceeding the quota on discovery requests.
if 'REMOTE_ADDR' in os.environ:
requested_url = _add_query_parameter(requested_url, 'userIp',
os.environ['REMOTE_ADDR'])
logger.info('URL being requested: %s' % requested_url)
resp, content = http.request(requested_url)
if resp.status == 404:
raise UnknownApiNameOrVersion("name: %s version: %s" % (serviceName,
version))
if resp.status >= 400:
raise HttpError(resp, content, uri=requested_url)
try:
service = simplejson.loads(content)
except ValueError, e:
logger.error('Failed to parse as JSON: ' + content)
raise InvalidJsonError()
return build_from_document(content, base=discoveryServiceUrl, http=http,
developerKey=developerKey, model=model, requestBuilder=requestBuilder)
@positional(1)
def build_from_document(
service,
base=None,
future=None,
http=None,
developerKey=None,
model=None,
requestBuilder=HttpRequest):
"""Create a Resource for interacting with an API.
Same as `build()`, but constructs the Resource object from a discovery
document that is it given, as opposed to retrieving one over HTTP.
Args:
service: string or object, the JSON discovery document describing the API.
The value passed in may either be the JSON string or the deserialized
JSON.
base: string, base URI for all HTTP requests, usually the discovery URI.
This parameter is no longer used as rootUrl and servicePath are included
within the discovery document. (deprecated)
future: string, discovery document with future capabilities (deprecated).
http: httplib2.Http, An instance of httplib2.Http or something that acts
like it that HTTP requests will be made through.
developerKey: string, Key for controlling API usage, generated
from the API Console.
model: Model class instance that serializes and de-serializes requests and
responses.
requestBuilder: Takes an http request and packages it up to be executed.
Returns:
A Resource object with methods for interacting with the service.
"""
# future is no longer used.
future = {}
if isinstance(service, basestring):
service = simplejson.loads(service)
base = urlparse.urljoin(service['rootUrl'], service['servicePath'])
schema = Schemas(service)
if model is None:
features = service.get('features', [])
model = JsonModel('dataWrapper' in features)
return Resource(http=http, baseUrl=base, model=model,
developerKey=developerKey, requestBuilder=requestBuilder,
resourceDesc=service, rootDesc=service, schema=schema)
def _cast(value, schema_type):
"""Convert value to a string based on JSON Schema type.
See http://tools.ietf.org/html/draft-zyp-json-schema-03 for more details on
JSON Schema.
Args:
value: any, the value to convert
schema_type: string, the type that value should be interpreted as
Returns:
A string representation of 'value' based on the schema_type.
"""
if schema_type == 'string':
if type(value) == type('') or type(value) == type(u''):
return value
else:
return str(value)
elif schema_type == 'integer':
return str(int(value))
elif schema_type == 'number':
return str(float(value))
elif schema_type == 'boolean':
return str(bool(value)).lower()
else:
if type(value) == type('') or type(value) == type(u''):
return value
else:
return str(value)
def _media_size_to_long(maxSize):
"""Convert a string media size, such as 10GB or 3TB into an integer.
Args:
maxSize: string, size as a string, such as 2MB or 7GB.
Returns:
The size as an integer value.
"""
if len(maxSize) < 2:
return 0L
units = maxSize[-2:].upper()
bit_shift = _MEDIA_SIZE_BIT_SHIFTS.get(units)
if bit_shift is not None:
return long(maxSize[:-2]) << bit_shift
else:
return long(maxSize)
def _media_path_url_from_info(root_desc, path_url):
"""Creates an absolute media path URL.
Constructed using the API root URI and service path from the discovery
document and the relative path for the API method.
Args:
root_desc: Dictionary; the entire original deserialized discovery document.
path_url: String; the relative URL for the API method. Relative to the API
root, which is specified in the discovery document.
Returns:
String; the absolute URI for media upload for the API method.
"""
return '%(root)supload/%(service_path)s%(path)s' % {
'root': root_desc['rootUrl'],
'service_path': root_desc['servicePath'],
'path': path_url,
}
def _fix_up_parameters(method_desc, root_desc, http_method):
"""Updates parameters of an API method with values specific to this library.
Specifically, adds whatever global parameters are specified by the API to the
parameters for the individual method. Also adds parameters which don't
appear in the discovery document, but are available to all discovery based
APIs (these are listed in STACK_QUERY_PARAMETERS).
SIDE EFFECTS: This updates the parameters dictionary object in the method
description.
Args:
method_desc: Dictionary with metadata describing an API method. Value comes
from the dictionary of methods stored in the 'methods' key in the
deserialized discovery document.
root_desc: Dictionary; the entire original deserialized discovery document.
http_method: String; the HTTP method used to call the API method described
in method_desc.
Returns:
The updated Dictionary stored in the 'parameters' key of the method
description dictionary.
"""
parameters = method_desc.setdefault('parameters', {})
# Add in the parameters common to all methods.
for name, description in root_desc.get('parameters', {}).iteritems():
parameters[name] = description
# Add in undocumented query parameters.
for name in STACK_QUERY_PARAMETERS:
parameters[name] = STACK_QUERY_PARAMETER_DEFAULT_VALUE.copy()
# Add 'body' (our own reserved word) to parameters if the method supports
# a request payload.
if http_method in HTTP_PAYLOAD_METHODS and 'request' in method_desc:
body = BODY_PARAMETER_DEFAULT_VALUE.copy()
body.update(method_desc['request'])
parameters['body'] = body
return parameters
def _fix_up_media_upload(method_desc, root_desc, path_url, parameters):
"""Updates parameters of API by adding 'media_body' if supported by method.
SIDE EFFECTS: If the method supports media upload and has a required body,
sets body to be optional (required=False) instead. Also, if there is a
'mediaUpload' in the method description, adds 'media_upload' key to
parameters.
Args:
method_desc: Dictionary with metadata describing an API method. Value comes
from the dictionary of methods stored in the 'methods' key in the
deserialized discovery document.
root_desc: Dictionary; the entire original deserialized discovery document.
path_url: String; the relative URL for the API method. Relative to the API
root, which is specified in the discovery document.
parameters: A dictionary describing method parameters for method described
in method_desc.
Returns:
Triple (accept, max_size, media_path_url) where:
- accept is a list of strings representing what content types are
accepted for media upload. Defaults to empty list if not in the
discovery document.
- max_size is a long representing the max size in bytes allowed for a
media upload. Defaults to 0L if not in the discovery document.
- media_path_url is a String; the absolute URI for media upload for the
API method. Constructed using the API root URI and service path from
the discovery document and the relative path for the API method. If
media upload is not supported, this is None.
"""
media_upload = method_desc.get('mediaUpload', {})
accept = media_upload.get('accept', [])
max_size = _media_size_to_long(media_upload.get('maxSize', ''))
media_path_url = None
if media_upload:
media_path_url = _media_path_url_from_info(root_desc, path_url)
parameters['media_body'] = MEDIA_BODY_PARAMETER_DEFAULT_VALUE.copy()
if 'body' in parameters:
parameters['body']['required'] = False
return accept, max_size, media_path_url
def _fix_up_method_description(method_desc, root_desc):
"""Updates a method description in a discovery document.
SIDE EFFECTS: Changes the parameters dictionary in the method description with
extra parameters which are used locally.
Args:
method_desc: Dictionary with metadata describing an API method. Value comes
from the dictionary of methods stored in the 'methods' key in the
deserialized discovery document.
root_desc: Dictionary; the entire original deserialized discovery document.
Returns:
Tuple (path_url, http_method, method_id, accept, max_size, media_path_url)
where:
- path_url is a String; the relative URL for the API method. Relative to
the API root, which is specified in the discovery document.
- http_method is a String; the HTTP method used to call the API method
described in the method description.
- method_id is a String; the name of the RPC method associated with the
API method, and is in the method description in the 'id' key.
- accept is a list of strings representing what content types are
accepted for media upload. Defaults to empty list if not in the
discovery document.
- max_size is a long representing the max size in bytes allowed for a
media upload. Defaults to 0L if not in the discovery document.
- media_path_url is a String; the absolute URI for media upload for the
API method. Constructed using the API root URI and service path from
the discovery document and the relative path for the API method. If
media upload is not supported, this is None.
"""
path_url = method_desc['path']
http_method = method_desc['httpMethod']
method_id = method_desc['id']
parameters = _fix_up_parameters(method_desc, root_desc, http_method)
# Order is important. `_fix_up_media_upload` needs `method_desc` to have a
# 'parameters' key and needs to know if there is a 'body' parameter because it
# also sets a 'media_body' parameter.
accept, max_size, media_path_url = _fix_up_media_upload(
method_desc, root_desc, path_url, parameters)
return path_url, http_method, method_id, accept, max_size, media_path_url
# TODO(dhermes): Convert this class to ResourceMethod and make it callable
class ResourceMethodParameters(object):
"""Represents the parameters associated with a method.
Attributes:
argmap: Map from method parameter name (string) to query parameter name
(string).
required_params: List of required parameters (represented by parameter
name as string).
repeated_params: List of repeated parameters (represented by parameter
name as string).
pattern_params: Map from method parameter name (string) to regular
expression (as a string). If the pattern is set for a parameter, the
value for that parameter must match the regular expression.
query_params: List of parameters (represented by parameter name as string)
that will be used in the query string.
path_params: Set of parameters (represented by parameter name as string)
that will be used in the base URL path.
param_types: Map from method parameter name (string) to parameter type. Type
can be any valid JSON schema type; valid values are 'any', 'array',
'boolean', 'integer', 'number', 'object', or 'string'. Reference:
http://tools.ietf.org/html/draft-zyp-json-schema-03#section-5.1
enum_params: Map from method parameter name (string) to list of strings,
where each list of strings is the list of acceptable enum values.
"""
def __init__(self, method_desc):
"""Constructor for ResourceMethodParameters.
Sets default values and defers to set_parameters to populate.
Args:
method_desc: Dictionary with metadata describing an API method. Value
comes from the dictionary of methods stored in the 'methods' key in
the deserialized discovery document.
"""
self.argmap = {}
self.required_params = []
self.repeated_params = []
self.pattern_params = {}
self.query_params = []
# TODO(dhermes): Change path_params to a list if the extra URITEMPLATE
# parsing is gotten rid of.
self.path_params = set()
self.param_types = {}
self.enum_params = {}
self.set_parameters(method_desc)
def set_parameters(self, method_desc):
"""Populates maps and lists based on method description.
Iterates through each parameter for the method and parses the values from
the parameter dictionary.
Args:
method_desc: Dictionary with metadata describing an API method. Value
comes from the dictionary of methods stored in the 'methods' key in
the deserialized discovery document.
"""
for arg, desc in method_desc.get('parameters', {}).iteritems():
param = key2param(arg)
self.argmap[param] = arg
if desc.get('pattern'):
self.pattern_params[param] = desc['pattern']
if desc.get('enum'):
self.enum_params[param] = desc['enum']
if desc.get('required'):
self.required_params.append(param)
if desc.get('repeated'):
self.repeated_params.append(param)
if desc.get('location') == 'query':
self.query_params.append(param)
if desc.get('location') == 'path':
self.path_params.add(param)
self.param_types[param] = desc.get('type', 'string')
# TODO(dhermes): Determine if this is still necessary. Discovery based APIs
# should have all path parameters already marked with
# 'location: path'.
for match in URITEMPLATE.finditer(method_desc['path']):
for namematch in VARNAME.finditer(match.group(0)):
name = key2param(namematch.group(0))
self.path_params.add(name)
if name in self.query_params:
self.query_params.remove(name)
def createMethod(methodName, methodDesc, rootDesc, schema):
"""Creates a method for attaching to a Resource.
Args:
methodName: string, name of the method to use.
methodDesc: object, fragment of deserialized discovery document that
describes the method.
rootDesc: object, the entire deserialized discovery document.
schema: object, mapping of schema names to schema descriptions.
"""
methodName = fix_method_name(methodName)
(pathUrl, httpMethod, methodId, accept,
maxSize, mediaPathUrl) = _fix_up_method_description(methodDesc, rootDesc)
parameters = ResourceMethodParameters(methodDesc)
def method(self, **kwargs):
# Don't bother with doc string, it will be over-written by createMethod.
for name in kwargs.iterkeys():
if name not in parameters.argmap:
raise TypeError('Got an unexpected keyword argument "%s"' % name)
# Remove args that have a value of None.
keys = kwargs.keys()
for name in keys:
if kwargs[name] is None:
del kwargs[name]
for name in parameters.required_params:
if name not in kwargs:
raise TypeError('Missing required parameter "%s"' % name)
for name, regex in parameters.pattern_params.iteritems():
if name in kwargs:
if isinstance(kwargs[name], basestring):
pvalues = [kwargs[name]]
else:
pvalues = kwargs[name]
for pvalue in pvalues:
if re.match(regex, pvalue) is None:
raise TypeError(
'Parameter "%s" value "%s" does not match the pattern "%s"' %
(name, pvalue, regex))
for name, enums in parameters.enum_params.iteritems():
if name in kwargs:
# We need to handle the case of a repeated enum
# name differently, since we want to handle both
# arg='value' and arg=['value1', 'value2']
if (name in parameters.repeated_params and
not isinstance(kwargs[name], basestring)):
values = kwargs[name]
else:
values = [kwargs[name]]
for value in values:
if value not in enums:
raise TypeError(
'Parameter "%s" value "%s" is not an allowed value in "%s"' %
(name, value, str(enums)))
actual_query_params = {}
actual_path_params = {}
for key, value in kwargs.iteritems():
to_type = parameters.param_types.get(key, 'string')
# For repeated parameters we cast each member of the list.
if key in parameters.repeated_params and type(value) == type([]):
cast_value = [_cast(x, to_type) for x in value]
else:
cast_value = _cast(value, to_type)
if key in parameters.query_params:
actual_query_params[parameters.argmap[key]] = cast_value
if key in parameters.path_params:
actual_path_params[parameters.argmap[key]] = cast_value
body_value = kwargs.get('body', None)
media_filename = kwargs.get('media_body', None)
if self._developerKey:
actual_query_params['key'] = self._developerKey
model = self._model
if methodName.endswith('_media'):
model = MediaModel()
elif 'response' not in methodDesc:
model = RawModel()
headers = {}
headers, params, query, body = model.request(headers,
actual_path_params, actual_query_params, body_value)
expanded_url = uritemplate.expand(pathUrl, params)
url = urlparse.urljoin(self._baseUrl, expanded_url + query)
resumable = None
multipart_boundary = ''
if media_filename:
# Ensure we end up with a valid MediaUpload object.
if isinstance(media_filename, basestring):
(media_mime_type, encoding) = mimetypes.guess_type(media_filename)
if media_mime_type is None:
raise UnknownFileType(media_filename)
if not mimeparse.best_match([media_mime_type], ','.join(accept)):
raise UnacceptableMimeTypeError(media_mime_type)
media_upload = MediaFileUpload(media_filename,
mimetype=media_mime_type)
elif isinstance(media_filename, MediaUpload):
media_upload = media_filename
else:
raise TypeError('media_filename must be str or MediaUpload.')
# Check the maxSize
if maxSize > 0 and media_upload.size() > maxSize:
raise MediaUploadSizeError("Media larger than: %s" % maxSize)
# Use the media path uri for media uploads
expanded_url = uritemplate.expand(mediaPathUrl, params)
url = urlparse.urljoin(self._baseUrl, expanded_url + query)
if media_upload.resumable():
url = _add_query_parameter(url, 'uploadType', 'resumable')
if media_upload.resumable():
# This is all we need to do for resumable, if the body exists it gets
# sent in the first request, otherwise an empty body is sent.
resumable = media_upload
else:
# A non-resumable upload
if body is None:
# This is a simple media upload
headers['content-type'] = media_upload.mimetype()
body = media_upload.getbytes(0, media_upload.size())
url = _add_query_parameter(url, 'uploadType', 'media')
else:
# This is a multipart/related upload.
msgRoot = MIMEMultipart('related')
# msgRoot should not write out it's own headers
setattr(msgRoot, '_write_headers', lambda self: None)
# attach the body as one part
msg = MIMENonMultipart(*headers['content-type'].split('/'))
msg.set_payload(body)
msgRoot.attach(msg)
# attach the media as the second part
msg = MIMENonMultipart(*media_upload.mimetype().split('/'))
msg['Content-Transfer-Encoding'] = 'binary'
payload = media_upload.getbytes(0, media_upload.size())
msg.set_payload(payload)
msgRoot.attach(msg)
body = msgRoot.as_string()
multipart_boundary = msgRoot.get_boundary()
headers['content-type'] = ('multipart/related; '
'boundary="%s"') % multipart_boundary
url = _add_query_parameter(url, 'uploadType', 'multipart')
logger.info('URL being requested: %s' % url)
return self._requestBuilder(self._http,
model.response,
url,
method=httpMethod,
body=body,
headers=headers,
methodId=methodId,
resumable=resumable)
docs = [methodDesc.get('description', DEFAULT_METHOD_DOC), '\n\n']
if len(parameters.argmap) > 0:
docs.append('Args:\n')
# Skip undocumented params and params common to all methods.
skip_parameters = rootDesc.get('parameters', {}).keys()
skip_parameters.extend(STACK_QUERY_PARAMETERS)
all_args = parameters.argmap.keys()
args_ordered = [key2param(s) for s in methodDesc.get('parameterOrder', [])]
# Move body to the front of the line.
if 'body' in all_args:
args_ordered.append('body')
for name in all_args:
if name not in args_ordered:
args_ordered.append(name)
for arg in args_ordered:
if arg in skip_parameters:
continue
repeated = ''
if arg in parameters.repeated_params:
repeated = ' (repeated)'
required = ''
if arg in parameters.required_params:
required = ' (required)'
paramdesc = methodDesc['parameters'][parameters.argmap[arg]]
paramdoc = paramdesc.get('description', 'A parameter')
if '$ref' in paramdesc:
docs.append(
(' %s: object, %s%s%s\n The object takes the'
' form of:\n\n%s\n\n') % (arg, paramdoc, required, repeated,
schema.prettyPrintByName(paramdesc['$ref'])))
else:
paramtype = paramdesc.get('type', 'string')
docs.append(' %s: %s, %s%s%s\n' % (arg, paramtype, paramdoc, required,
repeated))
enum = paramdesc.get('enum', [])
enumDesc = paramdesc.get('enumDescriptions', [])
if enum and enumDesc:
docs.append(' Allowed values\n')
for (name, desc) in zip(enum, enumDesc):
docs.append(' %s - %s\n' % (name, desc))
if 'response' in methodDesc:
if methodName.endswith('_media'):
docs.append('\nReturns:\n The media object as a string.\n\n ')
else:
docs.append('\nReturns:\n An object of the form:\n\n ')
docs.append(schema.prettyPrintSchema(methodDesc['response']))
setattr(method, '__doc__', ''.join(docs))
return (methodName, method)
def createNextMethod(methodName):
"""Creates any _next methods for attaching to a Resource.
The _next methods allow for easy iteration through list() responses.
Args:
methodName: string, name of the method to use.
"""
methodName = fix_method_name(methodName)
def methodNext(self, previous_request, previous_response):
"""Retrieves the next page of results.
Args:
previous_request: The request for the previous page. (required)
previous_response: The response from the request for the previous page. (required)
Returns:
A request object that you can call 'execute()' on to request the next
page. Returns None if there are no more items in the collection.
"""
# Retrieve nextPageToken from previous_response
# Use as pageToken in previous_request to create new request.
if 'nextPageToken' not in previous_response:
return None
request = copy.copy(previous_request)
pageToken = previous_response['nextPageToken']
parsed = list(urlparse.urlparse(request.uri))
q = parse_qsl(parsed[4])
# Find and remove old 'pageToken' value from URI
newq = [(key, value) for (key, value) in q if key != 'pageToken']
newq.append(('pageToken', pageToken))
parsed[4] = urllib.urlencode(newq)
uri = urlparse.urlunparse(parsed)
request.uri = uri
logger.info('URL being requested: %s' % uri)
return request
return (methodName, methodNext)
class Resource(object):
"""A class for interacting with a resource."""
def __init__(self, http, baseUrl, model, requestBuilder, developerKey,
resourceDesc, rootDesc, schema):
"""Build a Resource from the API description.
Args:
http: httplib2.Http, Object to make http requests with.
baseUrl: string, base URL for the API. All requests are relative to this
URI.
model: apiclient.Model, converts to and from the wire format.
requestBuilder: class or callable that instantiates an
apiclient.HttpRequest object.
developerKey: string, key obtained from
https://code.google.com/apis/console
resourceDesc: object, section of deserialized discovery document that
describes a resource. Note that the top level discovery document
is considered a resource.
rootDesc: object, the entire deserialized discovery document.
schema: object, mapping of schema names to schema descriptions.
"""
self._dynamic_attrs = []
self._http = http
self._baseUrl = baseUrl
self._model = model
self._developerKey = developerKey
self._requestBuilder = requestBuilder
self._resourceDesc = resourceDesc
self._rootDesc = rootDesc
self._schema = schema
self._set_service_methods()
def _set_dynamic_attr(self, attr_name, value):
"""Sets an instance attribute and tracks it in a list of dynamic attributes.
Args:
attr_name: string; The name of the attribute to be set
value: The value being set on the object and tracked in the dynamic cache.
"""
self._dynamic_attrs.append(attr_name)
self.__dict__[attr_name] = value
def __getstate__(self):
"""Trim the state down to something that can be pickled.
Uses the fact that the instance variable _dynamic_attrs holds attrs that
will be wiped and restored on pickle serialization.
"""
state_dict = copy.copy(self.__dict__)
for dynamic_attr in self._dynamic_attrs:
del state_dict[dynamic_attr]
del state_dict['_dynamic_attrs']
return state_dict
def __setstate__(self, state):
"""Reconstitute the state of the object from being pickled.
Uses the fact that the instance variable _dynamic_attrs holds attrs that
will be wiped and restored on pickle serialization.
"""
self.__dict__.update(state)
self._dynamic_attrs = []
self._set_service_methods()
def _set_service_methods(self):
self._add_basic_methods(self._resourceDesc, self._rootDesc, self._schema)
self._add_nested_resources(self._resourceDesc, self._rootDesc, self._schema)
self._add_next_methods(self._resourceDesc, self._schema)
def _add_basic_methods(self, resourceDesc, rootDesc, schema):
# Add basic methods to Resource
if 'methods' in resourceDesc:
for methodName, methodDesc in resourceDesc['methods'].iteritems():
fixedMethodName, method = createMethod(
methodName, methodDesc, rootDesc, schema)
self._set_dynamic_attr(fixedMethodName,
method.__get__(self, self.__class__))
# Add in _media methods. The functionality of the attached method will
# change when it sees that the method name ends in _media.
if methodDesc.get('supportsMediaDownload', False):
fixedMethodName, method = createMethod(
methodName + '_media', methodDesc, rootDesc, schema)
self._set_dynamic_attr(fixedMethodName,
method.__get__(self, self.__class__))
def _add_nested_resources(self, resourceDesc, rootDesc, schema):
# Add in nested resources
if 'resources' in resourceDesc:
def createResourceMethod(methodName, methodDesc):
"""Create a method on the Resource to access a nested Resource.
Args:
methodName: string, name of the method to use.
methodDesc: object, fragment of deserialized discovery document that
describes the method.
"""
methodName = fix_method_name(methodName)
def methodResource(self):
return Resource(http=self._http, baseUrl=self._baseUrl,
model=self._model, developerKey=self._developerKey,
requestBuilder=self._requestBuilder,
resourceDesc=methodDesc, rootDesc=rootDesc,
schema=schema)
setattr(methodResource, '__doc__', 'A collection resource.')
setattr(methodResource, '__is_resource__', True)
return (methodName, methodResource)
for methodName, methodDesc in resourceDesc['resources'].iteritems():
fixedMethodName, method = createResourceMethod(methodName, methodDesc)
self._set_dynamic_attr(fixedMethodName,
method.__get__(self, self.__class__))
def _add_next_methods(self, resourceDesc, schema):
# Add _next() methods
# Look for response bodies in schema that contain nextPageToken, and methods
# that take a pageToken parameter.
if 'methods' in resourceDesc:
for methodName, methodDesc in resourceDesc['methods'].iteritems():
if 'response' in methodDesc:
responseSchema = methodDesc['response']
if '$ref' in responseSchema:
responseSchema = schema.get(responseSchema['$ref'])
hasNextPageToken = 'nextPageToken' in responseSchema.get('properties',
{})
hasPageToken = 'pageToken' in methodDesc.get('parameters', {})
if hasNextPageToken and hasPageToken:
fixedMethodName, method = createNextMethod(methodName + '_next')
self._set_dynamic_attr(fixedMethodName,
method.__get__(self, self.__class__))

140
apiclient/errors.py Normal file
View File

@ -0,0 +1,140 @@
#!/usr/bin/python2.4
#
# Copyright (C) 2010 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Errors for the library.
All exceptions defined by the library
should be defined in this file.
"""
__author__ = 'jcgregorio@google.com (Joe Gregorio)'
from oauth2client import util
from oauth2client.anyjson import simplejson
class Error(Exception):
"""Base error for this module."""
pass
class HttpError(Error):
"""HTTP data was invalid or unexpected."""
@util.positional(3)
def __init__(self, resp, content, uri=None):
self.resp = resp
self.content = content
self.uri = uri
def _get_reason(self):
"""Calculate the reason for the error from the response content."""
reason = self.resp.reason
try:
data = simplejson.loads(self.content)
reason = data['error']['message']
except (ValueError, KeyError):
pass
if reason is None:
reason = ''
return reason
def __repr__(self):
if self.uri:
return '<HttpError %s when requesting %s returned "%s">' % (
self.resp.status, self.uri, self._get_reason().strip())
else:
return '<HttpError %s "%s">' % (self.resp.status, self._get_reason())
__str__ = __repr__
class InvalidJsonError(Error):
"""The JSON returned could not be parsed."""
pass
class UnknownFileType(Error):
"""File type unknown or unexpected."""
pass
class UnknownLinkType(Error):
"""Link type unknown or unexpected."""
pass
class UnknownApiNameOrVersion(Error):
"""No API with that name and version exists."""
pass
class UnacceptableMimeTypeError(Error):
"""That is an unacceptable mimetype for this operation."""
pass
class MediaUploadSizeError(Error):
"""Media is larger than the method can accept."""
pass
class ResumableUploadError(HttpError):
"""Error occured during resumable upload."""
pass
class InvalidChunkSizeError(Error):
"""The given chunksize is not valid."""
pass
class InvalidNotificationError(Error):
"""The channel Notification is invalid."""
pass
class BatchError(HttpError):
"""Error occured during batch operations."""
@util.positional(2)
def __init__(self, reason, resp=None, content=None):
self.resp = resp
self.content = content
self.reason = reason
def __repr__(self):
return '<BatchError %s "%s">' % (self.resp.status, self.reason)
__str__ = __repr__
class UnexpectedMethodError(Error):
"""Exception raised by RequestMockBuilder on unexpected calls."""
@util.positional(1)
def __init__(self, methodId=None):
"""Constructor for an UnexpectedMethodError."""
super(UnexpectedMethodError, self).__init__(
'Received unexpected call %s' % methodId)
class UnexpectedBodyError(Error):
"""Exception raised by RequestMockBuilder on unexpected bodies."""
def __init__(self, expected, provided):
"""Constructor for an UnexpectedMethodError."""
super(UnexpectedBodyError, self).__init__(
'Expected: [%s] - Provided: [%s]' % (expected, provided))

0
apiclient/ext/__init__.py Normal file
View File

1609
apiclient/http.py Normal file
View File

File diff suppressed because it is too large Load Diff

172
apiclient/mimeparse.py Normal file
View File

@ -0,0 +1,172 @@
# Copyright (C) 2007 Joe Gregorio
#
# Licensed under the MIT License
"""MIME-Type Parser
This module provides basic functions for handling mime-types. It can handle
matching mime-types against a list of media-ranges. See section 14.1 of the
HTTP specification [RFC 2616] for a complete explanation.
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.1
Contents:
- parse_mime_type(): Parses a mime-type into its component parts.
- parse_media_range(): Media-ranges are mime-types with wild-cards and a 'q'
quality parameter.
- quality(): Determines the quality ('q') of a mime-type when
compared against a list of media-ranges.
- quality_parsed(): Just like quality() except the second parameter must be
pre-parsed.
- best_match(): Choose the mime-type with the highest quality ('q')
from a list of candidates.
"""
__version__ = '0.1.3'
__author__ = 'Joe Gregorio'
__email__ = 'joe@bitworking.org'
__license__ = 'MIT License'
__credits__ = ''
def parse_mime_type(mime_type):
"""Parses a mime-type into its component parts.
Carves up a mime-type and returns a tuple of the (type, subtype, params)
where 'params' is a dictionary of all the parameters for the media range.
For example, the media range 'application/xhtml;q=0.5' would get parsed
into:
('application', 'xhtml', {'q', '0.5'})
"""
parts = mime_type.split(';')
params = dict([tuple([s.strip() for s in param.split('=', 1)])\
for param in parts[1:]
])
full_type = parts[0].strip()
# Java URLConnection class sends an Accept header that includes a
# single '*'. Turn it into a legal wildcard.
if full_type == '*':
full_type = '*/*'
(type, subtype) = full_type.split('/')
return (type.strip(), subtype.strip(), params)
def parse_media_range(range):
"""Parse a media-range into its component parts.
Carves up a media range and returns a tuple of the (type, subtype,
params) where 'params' is a dictionary of all the parameters for the media
range. For example, the media range 'application/*;q=0.5' would get parsed
into:
('application', '*', {'q', '0.5'})
In addition this function also guarantees that there is a value for 'q'
in the params dictionary, filling it in with a proper default if
necessary.
"""
(type, subtype, params) = parse_mime_type(range)
if not params.has_key('q') or not params['q'] or \
not float(params['q']) or float(params['q']) > 1\
or float(params['q']) < 0:
params['q'] = '1'
return (type, subtype, params)
def fitness_and_quality_parsed(mime_type, parsed_ranges):
"""Find the best match for a mime-type amongst parsed media-ranges.
Find the best match for a given mime-type against a list of media_ranges
that have already been parsed by parse_media_range(). Returns a tuple of
the fitness value and the value of the 'q' quality parameter of the best
match, or (-1, 0) if no match was found. Just as for quality_parsed(),
'parsed_ranges' must be a list of parsed media ranges.
"""
best_fitness = -1
best_fit_q = 0
(target_type, target_subtype, target_params) =\
parse_media_range(mime_type)
for (type, subtype, params) in parsed_ranges:
type_match = (type == target_type or\
type == '*' or\
target_type == '*')
subtype_match = (subtype == target_subtype or\
subtype == '*' or\
target_subtype == '*')
if type_match and subtype_match:
param_matches = reduce(lambda x, y: x + y, [1 for (key, value) in \
target_params.iteritems() if key != 'q' and \
params.has_key(key) and value == params[key]], 0)
fitness = (type == target_type) and 100 or 0
fitness += (subtype == target_subtype) and 10 or 0
fitness += param_matches
if fitness > best_fitness:
best_fitness = fitness
best_fit_q = params['q']
return best_fitness, float(best_fit_q)
def quality_parsed(mime_type, parsed_ranges):
"""Find the best match for a mime-type amongst parsed media-ranges.
Find the best match for a given mime-type against a list of media_ranges
that have already been parsed by parse_media_range(). Returns the 'q'
quality parameter of the best match, 0 if no match was found. This function
bahaves the same as quality() except that 'parsed_ranges' must be a list of
parsed media ranges.
"""
return fitness_and_quality_parsed(mime_type, parsed_ranges)[1]
def quality(mime_type, ranges):
"""Return the quality ('q') of a mime-type against a list of media-ranges.
Returns the quality 'q' of a mime-type when compared against the
media-ranges in ranges. For example:
>>> quality('text/html','text/*;q=0.3, text/html;q=0.7,
text/html;level=1, text/html;level=2;q=0.4, */*;q=0.5')
0.7
"""
parsed_ranges = [parse_media_range(r) for r in ranges.split(',')]
return quality_parsed(mime_type, parsed_ranges)
def best_match(supported, header):
"""Return mime-type with the highest quality ('q') from list of candidates.
Takes a list of supported mime-types and finds the best match for all the
media-ranges listed in header. The value of header must be a string that
conforms to the format of the HTTP Accept: header. The value of 'supported'
is a list of mime-types. The list of supported mime-types should be sorted
in order of increasing desirability, in case of a situation where there is
a tie.
>>> best_match(['application/xbel+xml', 'text/xml'],
'text/*;q=0.5,*/*; q=0.1')
'text/xml'
"""
split_header = _filter_blank(header.split(','))
parsed_header = [parse_media_range(r) for r in split_header]
weighted_matches = []
pos = 0
for mime_type in supported:
weighted_matches.append((fitness_and_quality_parsed(mime_type,
parsed_header), pos, mime_type))
pos += 1
weighted_matches.sort()
return weighted_matches[-1][0][1] and weighted_matches[-1][2] or ''
def _filter_blank(i):
for s in i:
if s.strip():
yield s

383
apiclient/model.py Normal file
View File

@ -0,0 +1,383 @@
#!/usr/bin/python2.4
#
# Copyright (C) 2010 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Model objects for requests and responses.
Each API may support one or more serializations, such
as JSON, Atom, etc. The model classes are responsible
for converting between the wire format and the Python
object representation.
"""
__author__ = 'jcgregorio@google.com (Joe Gregorio)'
import logging
import urllib
from apiclient import __version__
from errors import HttpError
from oauth2client.anyjson import simplejson
dump_request_response = False
def _abstract():
raise NotImplementedError('You need to override this function')
class Model(object):
"""Model base class.
All Model classes should implement this interface.
The Model serializes and de-serializes between a wire
format such as JSON and a Python object representation.
"""
def request(self, headers, path_params, query_params, body_value):
"""Updates outgoing requests with a serialized body.
Args:
headers: dict, request headers
path_params: dict, parameters that appear in the request path
query_params: dict, parameters that appear in the query
body_value: object, the request body as a Python object, which must be
serializable.
Returns:
A tuple of (headers, path_params, query, body)
headers: dict, request headers
path_params: dict, parameters that appear in the request path
query: string, query part of the request URI
body: string, the body serialized in the desired wire format.
"""
_abstract()
def response(self, resp, content):
"""Convert the response wire format into a Python object.
Args:
resp: httplib2.Response, the HTTP response headers and status
content: string, the body of the HTTP response
Returns:
The body de-serialized as a Python object.
Raises:
apiclient.errors.HttpError if a non 2xx response is received.
"""
_abstract()
class BaseModel(Model):
"""Base model class.
Subclasses should provide implementations for the "serialize" and
"deserialize" methods, as well as values for the following class attributes.
Attributes:
accept: The value to use for the HTTP Accept header.
content_type: The value to use for the HTTP Content-type header.
no_content_response: The value to return when deserializing a 204 "No
Content" response.
alt_param: The value to supply as the "alt" query parameter for requests.
"""
accept = None
content_type = None
no_content_response = None
alt_param = None
def _log_request(self, headers, path_params, query, body):
"""Logs debugging information about the request if requested."""
if dump_request_response:
logging.info('--request-start--')
logging.info('-headers-start-')
for h, v in headers.iteritems():
logging.info('%s: %s', h, v)
logging.info('-headers-end-')
logging.info('-path-parameters-start-')
for h, v in path_params.iteritems():
logging.info('%s: %s', h, v)
logging.info('-path-parameters-end-')
logging.info('body: %s', body)
logging.info('query: %s', query)
logging.info('--request-end--')
def request(self, headers, path_params, query_params, body_value):
"""Updates outgoing requests with a serialized body.
Args:
headers: dict, request headers
path_params: dict, parameters that appear in the request path
query_params: dict, parameters that appear in the query
body_value: object, the request body as a Python object, which must be
serializable by simplejson.
Returns:
A tuple of (headers, path_params, query, body)
headers: dict, request headers
path_params: dict, parameters that appear in the request path
query: string, query part of the request URI
body: string, the body serialized as JSON
"""
query = self._build_query(query_params)
headers['accept'] = self.accept
headers['accept-encoding'] = 'gzip, deflate'
if 'user-agent' in headers:
headers['user-agent'] += ' '
else:
headers['user-agent'] = ''
headers['user-agent'] += 'google-api-python-client/%s (gzip)' % __version__
if body_value is not None:
headers['content-type'] = self.content_type
body_value = self.serialize(body_value)
self._log_request(headers, path_params, query, body_value)
return (headers, path_params, query, body_value)
def _build_query(self, params):
"""Builds a query string.
Args:
params: dict, the query parameters
Returns:
The query parameters properly encoded into an HTTP URI query string.
"""
if self.alt_param is not None:
params.update({'alt': self.alt_param})
astuples = []
for key, value in params.iteritems():
if type(value) == type([]):
for x in value:
x = x.encode('utf-8')
astuples.append((key, x))
else:
if getattr(value, 'encode', False) and callable(value.encode):
value = value.encode('utf-8')
astuples.append((key, value))
return '?' + urllib.urlencode(astuples)
def _log_response(self, resp, content):
"""Logs debugging information about the response if requested."""
if dump_request_response:
logging.info('--response-start--')
for h, v in resp.iteritems():
logging.info('%s: %s', h, v)
if content:
logging.info(content)
logging.info('--response-end--')
def response(self, resp, content):
"""Convert the response wire format into a Python object.
Args:
resp: httplib2.Response, the HTTP response headers and status
content: string, the body of the HTTP response
Returns:
The body de-serialized as a Python object.
Raises:
apiclient.errors.HttpError if a non 2xx response is received.
"""
self._log_response(resp, content)
# Error handling is TBD, for example, do we retry
# for some operation/error combinations?
if resp.status < 300:
if resp.status == 204:
# A 204: No Content response should be treated differently
# to all the other success states
return self.no_content_response
return self.deserialize(content)
else:
logging.debug('Content from bad request was: %s' % content)
raise HttpError(resp, content)
def serialize(self, body_value):
"""Perform the actual Python object serialization.
Args:
body_value: object, the request body as a Python object.
Returns:
string, the body in serialized form.
"""
_abstract()
def deserialize(self, content):
"""Perform the actual deserialization from response string to Python
object.
Args:
content: string, the body of the HTTP response
Returns:
The body de-serialized as a Python object.
"""
_abstract()
class JsonModel(BaseModel):
"""Model class for JSON.
Serializes and de-serializes between JSON and the Python
object representation of HTTP request and response bodies.
"""
accept = 'application/json'
content_type = 'application/json'
alt_param = 'json'
def __init__(self, data_wrapper=False):
"""Construct a JsonModel.
Args:
data_wrapper: boolean, wrap requests and responses in a data wrapper
"""
self._data_wrapper = data_wrapper
def serialize(self, body_value):
if (isinstance(body_value, dict) and 'data' not in body_value and
self._data_wrapper):
body_value = {'data': body_value}
return simplejson.dumps(body_value)
def deserialize(self, content):
content = content.decode('utf-8')
body = simplejson.loads(content)
if self._data_wrapper and isinstance(body, dict) and 'data' in body:
body = body['data']
return body
@property
def no_content_response(self):
return {}
class RawModel(JsonModel):
"""Model class for requests that don't return JSON.
Serializes and de-serializes between JSON and the Python
object representation of HTTP request, and returns the raw bytes
of the response body.
"""
accept = '*/*'
content_type = 'application/json'
alt_param = None
def deserialize(self, content):
return content
@property
def no_content_response(self):
return ''
class MediaModel(JsonModel):
"""Model class for requests that return Media.
Serializes and de-serializes between JSON and the Python
object representation of HTTP request, and returns the raw bytes
of the response body.
"""
accept = '*/*'
content_type = 'application/json'
alt_param = 'media'
def deserialize(self, content):
return content
@property
def no_content_response(self):
return ''
class ProtocolBufferModel(BaseModel):
"""Model class for protocol buffers.
Serializes and de-serializes the binary protocol buffer sent in the HTTP
request and response bodies.
"""
accept = 'application/x-protobuf'
content_type = 'application/x-protobuf'
alt_param = 'proto'
def __init__(self, protocol_buffer):
"""Constructs a ProtocolBufferModel.
The serialzed protocol buffer returned in an HTTP response will be
de-serialized using the given protocol buffer class.
Args:
protocol_buffer: The protocol buffer class used to de-serialize a
response from the API.
"""
self._protocol_buffer = protocol_buffer
def serialize(self, body_value):
return body_value.SerializeToString()
def deserialize(self, content):
return self._protocol_buffer.FromString(content)
@property
def no_content_response(self):
return self._protocol_buffer()
def makepatch(original, modified):
"""Create a patch object.
Some methods support PATCH, an efficient way to send updates to a resource.
This method allows the easy construction of patch bodies by looking at the
differences between a resource before and after it was modified.
Args:
original: object, the original deserialized resource
modified: object, the modified deserialized resource
Returns:
An object that contains only the changes from original to modified, in a
form suitable to pass to a PATCH method.
Example usage:
item = service.activities().get(postid=postid, userid=userid).execute()
original = copy.deepcopy(item)
item['object']['content'] = 'This is updated.'
service.activities.patch(postid=postid, userid=userid,
body=makepatch(original, item)).execute()
"""
patch = {}
for key, original_value in original.iteritems():
modified_value = modified.get(key, None)
if modified_value is None:
# Use None to signal that the element is deleted
patch[key] = None
elif original_value != modified_value:
if type(original_value) == type({}):
# Recursively descend objects
patch[key] = makepatch(original_value, modified_value)
else:
# In the case of simple types or arrays we just replace
patch[key] = modified_value
else:
# Don't add anything to patch if there's no change
pass
for key in modified:
if key not in original:
patch[key] = modified[key]
return patch

274
apiclient/push.py Normal file
View File

@ -0,0 +1,274 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Push notifications support.
This code is based on experimental APIs and is subject to change.
"""
__author__ = 'afshar@google.com (Ali Afshar)'
import binascii
import collections
import os
import urllib
SUBSCRIBE = 'X-GOOG-SUBSCRIBE'
SUBSCRIPTION_ID = 'X-GOOG-SUBSCRIPTION-ID'
TOPIC_ID = 'X-GOOG-TOPIC-ID'
TOPIC_URI = 'X-GOOG-TOPIC-URI'
CLIENT_TOKEN = 'X-GOOG-CLIENT-TOKEN'
EVENT_TYPE = 'X-GOOG-EVENT-TYPE'
UNSUBSCRIBE = 'X-GOOG-UNSUBSCRIBE'
class InvalidSubscriptionRequestError(ValueError):
"""The request cannot be subscribed."""
def new_token():
"""Gets a random token for use as a client_token in push notifications.
Returns:
str, a new random token.
"""
return binascii.hexlify(os.urandom(32))
class Channel(object):
"""Base class for channel types."""
def __init__(self, channel_type, channel_args):
"""Create a new Channel.
You probably won't need to create this channel manually, since there are
subclassed Channel for each specific type with a more customized set of
arguments to pass. However, you may wish to just create it manually here.
Args:
channel_type: str, the type of channel.
channel_args: dict, arguments to pass to the channel.
"""
self.channel_type = channel_type
self.channel_args = channel_args
def as_header_value(self):
"""Create the appropriate header for this channel.
Returns:
str encoded channel description suitable for use as a header.
"""
return '%s?%s' % (self.channel_type, urllib.urlencode(self.channel_args))
def write_header(self, headers):
"""Write the appropriate subscribe header to a headers dict.
Args:
headers: dict, headers to add subscribe header to.
"""
headers[SUBSCRIBE] = self.as_header_value()
class WebhookChannel(Channel):
"""Channel for registering web hook notifications."""
def __init__(self, url, app_engine=False):
"""Create a new WebhookChannel
Args:
url: str, URL to post notifications to.
app_engine: bool, default=False, whether the destination for the
notifications is an App Engine application.
"""
super(WebhookChannel, self).__init__(
channel_type='web_hook',
channel_args={
'url': url,
'app_engine': app_engine and 'true' or 'false',
}
)
class Headers(collections.defaultdict):
"""Headers for managing subscriptions."""
ALL_HEADERS = set([SUBSCRIBE, SUBSCRIPTION_ID, TOPIC_ID, TOPIC_URI,
CLIENT_TOKEN, EVENT_TYPE, UNSUBSCRIBE])
def __init__(self):
"""Create a new subscription configuration instance."""
collections.defaultdict.__init__(self, str)
def __setitem__(self, key, value):
"""Set a header value, ensuring the key is an allowed value.
Args:
key: str, the header key.
value: str, the header value.
Raises:
ValueError if key is not one of the accepted headers.
"""
normal_key = self._normalize_key(key)
if normal_key not in self.ALL_HEADERS:
raise ValueError('Header name must be one of %s.' % self.ALL_HEADERS)
else:
return collections.defaultdict.__setitem__(self, normal_key, value)
def __getitem__(self, key):
"""Get a header value, normalizing the key case.
Args:
key: str, the header key.
Returns:
String header value.
Raises:
KeyError if the key is not one of the accepted headers.
"""
normal_key = self._normalize_key(key)
if normal_key not in self.ALL_HEADERS:
raise ValueError('Header name must be one of %s.' % self.ALL_HEADERS)
else:
return collections.defaultdict.__getitem__(self, normal_key)
def _normalize_key(self, key):
"""Normalize a header name for use as a key."""
return key.upper()
def items(self):
"""Generator for each header."""
for header in self.ALL_HEADERS:
value = self[header]
if value:
yield header, value
def write(self, headers):
"""Applies the subscription headers.
Args:
headers: dict of headers to insert values into.
"""
for header, value in self.items():
headers[header.lower()] = value
def read(self, headers):
"""Read from headers.
Args:
headers: dict of headers to read from.
"""
for header in self.ALL_HEADERS:
if header.lower() in headers:
self[header] = headers[header.lower()]
class Subscription(object):
"""Information about a subscription."""
def __init__(self):
"""Create a new Subscription."""
self.headers = Headers()
@classmethod
def for_request(cls, request, channel, client_token=None):
"""Creates a subscription and attaches it to a request.
Args:
request: An http.HttpRequest to modify for making a subscription.
channel: A apiclient.push.Channel describing the subscription to
create.
client_token: (optional) client token to verify the notification.
Returns:
New subscription object.
"""
subscription = cls.for_channel(channel=channel, client_token=client_token)
subscription.headers.write(request.headers)
if request.method != 'GET':
raise InvalidSubscriptionRequestError(
'Can only subscribe to requests which are GET.')
request.method = 'POST'
def _on_response(response, subscription=subscription):
"""Called with the response headers. Reads the subscription headers."""
subscription.headers.read(response)
request.add_response_callback(_on_response)
return subscription
@classmethod
def for_channel(cls, channel, client_token=None):
"""Alternate constructor to create a subscription from a channel.
Args:
channel: A apiclient.push.Channel describing the subscription to
create.
client_token: (optional) client token to verify the notification.
Returns:
New subscription object.
"""
subscription = cls()
channel.write_header(subscription.headers)
if client_token is None:
client_token = new_token()
subscription.headers[SUBSCRIPTION_ID] = new_token()
subscription.headers[CLIENT_TOKEN] = client_token
return subscription
def verify(self, headers):
"""Verifies that a webhook notification has the correct client_token.
Args:
headers: dict of request headers for a push notification.
Returns:
Boolean value indicating whether the notification is verified.
"""
new_subscription = Subscription()
new_subscription.headers.read(headers)
return new_subscription.client_token == self.client_token
@property
def subscribe(self):
"""Subscribe header value."""
return self.headers[SUBSCRIBE]
@property
def subscription_id(self):
"""Subscription ID header value."""
return self.headers[SUBSCRIPTION_ID]
@property
def topic_id(self):
"""Topic ID header value."""
return self.headers[TOPIC_ID]
@property
def topic_uri(self):
"""Topic URI header value."""
return self.headers[TOPIC_URI]
@property
def client_token(self):
"""Client Token header value."""
return self.headers[CLIENT_TOKEN]
@property
def event_type(self):
"""Event Type header value."""
return self.headers[EVENT_TYPE]
@property
def unsubscribe(self):
"""Unsuscribe header value."""
return self.headers[UNSUBSCRIBE]

93
apiclient/sample_tools.py Normal file
View File

@ -0,0 +1,93 @@
# Copyright (C) 2013 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Utilities for making samples.
Consolidates a lot of code commonly repeated in sample applications.
"""
__author__ = 'jcgregorio@google.com (Joe Gregorio)'
__all__ = ['init']
import argparse
import httplib2
import os
from apiclient import discovery
from oauth2client import client
from oauth2client import file
from oauth2client import tools
def init(argv, name, version, doc, filename, scope=None, parents=[]):
"""A common initialization routine for samples.
Many of the sample applications do the same initialization, which has now
been consolidated into this function. This function uses common idioms found
in almost all the samples, i.e. for an API with name 'apiname', the
credentials are stored in a file named apiname.dat, and the
client_secrets.json file is stored in the same directory as the application
main file.
Args:
argv: list of string, the command-line parameters of the application.
name: string, name of the API.
version: string, version of the API.
doc: string, description of the application. Usually set to __doc__.
file: string, filename of the application. Usually set to __file__.
parents: list of argparse.ArgumentParser, additional command-line flags.
scope: string, The OAuth scope used.
Returns:
A tuple of (service, flags), where service is the service object and flags
is the parsed command-line flags.
"""
if scope is None:
scope = 'https://www.googleapis.com/auth/' + name
# Parser command-line arguments.
parent_parsers = [tools.argparser]
parent_parsers.extend(parents)
parser = argparse.ArgumentParser(
description=doc,
formatter_class=argparse.RawDescriptionHelpFormatter,
parents=parent_parsers)
flags = parser.parse_args(argv[1:])
# Name of a file containing the OAuth 2.0 information for this
# application, including client_id and client_secret, which are found
# on the API Access tab on the Google APIs
# Console <http://code.google.com/apis/console>.
client_secrets = os.path.join(os.path.dirname(filename),
'client_secrets.json')
# Set up a Flow object to be used if we need to authenticate.
flow = client.flow_from_clientsecrets(client_secrets,
scope=scope,
message=tools.message_if_missing(client_secrets))
# Prepare credentials, and authorize HTTP object with them.
# If the credentials don't exist or are invalid run through the native client
# flow. The Storage object will ensure that if successful the good
# credentials will get written back to a file.
storage = file.Storage(name + '.dat')
credentials = storage.get()
if credentials is None or credentials.invalid:
credentials = tools.run_flow(flow, storage, flags)
http = credentials.authorize(http = httplib2.Http())
# Construct a service object via the discovery service.
service = discovery.build(name, version, http=http)
return (service, flags)

312
apiclient/schema.py Normal file
View File

@ -0,0 +1,312 @@
# Copyright (C) 2010 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Schema processing for discovery based APIs
Schemas holds an APIs discovery schemas. It can return those schema as
deserialized JSON objects, or pretty print them as prototype objects that
conform to the schema.
For example, given the schema:
schema = \"\"\"{
"Foo": {
"type": "object",
"properties": {
"etag": {
"type": "string",
"description": "ETag of the collection."
},
"kind": {
"type": "string",
"description": "Type of the collection ('calendar#acl').",
"default": "calendar#acl"
},
"nextPageToken": {
"type": "string",
"description": "Token used to access the next
page of this result. Omitted if no further results are available."
}
}
}
}\"\"\"
s = Schemas(schema)
print s.prettyPrintByName('Foo')
Produces the following output:
{
"nextPageToken": "A String", # Token used to access the
# next page of this result. Omitted if no further results are available.
"kind": "A String", # Type of the collection ('calendar#acl').
"etag": "A String", # ETag of the collection.
},
The constructor takes a discovery document in which to look up named schema.
"""
# TODO(jcgregorio) support format, enum, minimum, maximum
__author__ = 'jcgregorio@google.com (Joe Gregorio)'
import copy
from oauth2client import util
from oauth2client.anyjson import simplejson
class Schemas(object):
"""Schemas for an API."""
def __init__(self, discovery):
"""Constructor.
Args:
discovery: object, Deserialized discovery document from which we pull
out the named schema.
"""
self.schemas = discovery.get('schemas', {})
# Cache of pretty printed schemas.
self.pretty = {}
@util.positional(2)
def _prettyPrintByName(self, name, seen=None, dent=0):
"""Get pretty printed object prototype from the schema name.
Args:
name: string, Name of schema in the discovery document.
seen: list of string, Names of schema already seen. Used to handle
recursive definitions.
Returns:
string, A string that contains a prototype object with
comments that conforms to the given schema.
"""
if seen is None:
seen = []
if name in seen:
# Do not fall into an infinite loop over recursive definitions.
return '# Object with schema name: %s' % name
seen.append(name)
if name not in self.pretty:
self.pretty[name] = _SchemaToStruct(self.schemas[name],
seen, dent=dent).to_str(self._prettyPrintByName)
seen.pop()
return self.pretty[name]
def prettyPrintByName(self, name):
"""Get pretty printed object prototype from the schema name.
Args:
name: string, Name of schema in the discovery document.
Returns:
string, A string that contains a prototype object with
comments that conforms to the given schema.
"""
# Return with trailing comma and newline removed.
return self._prettyPrintByName(name, seen=[], dent=1)[:-2]
@util.positional(2)
def _prettyPrintSchema(self, schema, seen=None, dent=0):
"""Get pretty printed object prototype of schema.
Args:
schema: object, Parsed JSON schema.
seen: list of string, Names of schema already seen. Used to handle
recursive definitions.
Returns:
string, A string that contains a prototype object with
comments that conforms to the given schema.
"""
if seen is None:
seen = []
return _SchemaToStruct(schema, seen, dent=dent).to_str(self._prettyPrintByName)
def prettyPrintSchema(self, schema):
"""Get pretty printed object prototype of schema.
Args:
schema: object, Parsed JSON schema.
Returns:
string, A string that contains a prototype object with
comments that conforms to the given schema.
"""
# Return with trailing comma and newline removed.
return self._prettyPrintSchema(schema, dent=1)[:-2]
def get(self, name):
"""Get deserialized JSON schema from the schema name.
Args:
name: string, Schema name.
"""
return self.schemas[name]
class _SchemaToStruct(object):
"""Convert schema to a prototype object."""
@util.positional(3)
def __init__(self, schema, seen, dent=0):
"""Constructor.
Args:
schema: object, Parsed JSON schema.
seen: list, List of names of schema already seen while parsing. Used to
handle recursive definitions.
dent: int, Initial indentation depth.
"""
# The result of this parsing kept as list of strings.
self.value = []
# The final value of the parsing.
self.string = None
# The parsed JSON schema.
self.schema = schema
# Indentation level.
self.dent = dent
# Method that when called returns a prototype object for the schema with
# the given name.
self.from_cache = None
# List of names of schema already seen while parsing.
self.seen = seen
def emit(self, text):
"""Add text as a line to the output.
Args:
text: string, Text to output.
"""
self.value.extend([" " * self.dent, text, '\n'])
def emitBegin(self, text):
"""Add text to the output, but with no line terminator.
Args:
text: string, Text to output.
"""
self.value.extend([" " * self.dent, text])
def emitEnd(self, text, comment):
"""Add text and comment to the output with line terminator.
Args:
text: string, Text to output.
comment: string, Python comment.
"""
if comment:
divider = '\n' + ' ' * (self.dent + 2) + '# '
lines = comment.splitlines()
lines = [x.rstrip() for x in lines]
comment = divider.join(lines)
self.value.extend([text, ' # ', comment, '\n'])
else:
self.value.extend([text, '\n'])
def indent(self):
"""Increase indentation level."""
self.dent += 1
def undent(self):
"""Decrease indentation level."""
self.dent -= 1
def _to_str_impl(self, schema):
"""Prototype object based on the schema, in Python code with comments.
Args:
schema: object, Parsed JSON schema file.
Returns:
Prototype object based on the schema, in Python code with comments.
"""
stype = schema.get('type')
if stype == 'object':
self.emitEnd('{', schema.get('description', ''))
self.indent()
if 'properties' in schema:
for pname, pschema in schema.get('properties', {}).iteritems():
self.emitBegin('"%s": ' % pname)
self._to_str_impl(pschema)
elif 'additionalProperties' in schema:
self.emitBegin('"a_key": ')
self._to_str_impl(schema['additionalProperties'])
self.undent()
self.emit('},')
elif '$ref' in schema:
schemaName = schema['$ref']
description = schema.get('description', '')
s = self.from_cache(schemaName, seen=self.seen)
parts = s.splitlines()
self.emitEnd(parts[0], description)
for line in parts[1:]:
self.emit(line.rstrip())
elif stype == 'boolean':
value = schema.get('default', 'True or False')
self.emitEnd('%s,' % str(value), schema.get('description', ''))
elif stype == 'string':
value = schema.get('default', 'A String')
self.emitEnd('"%s",' % str(value), schema.get('description', ''))
elif stype == 'integer':
value = schema.get('default', '42')
self.emitEnd('%s,' % str(value), schema.get('description', ''))
elif stype == 'number':
value = schema.get('default', '3.14')
self.emitEnd('%s,' % str(value), schema.get('description', ''))
elif stype == 'null':
self.emitEnd('None,', schema.get('description', ''))
elif stype == 'any':
self.emitEnd('"",', schema.get('description', ''))
elif stype == 'array':
self.emitEnd('[', schema.get('description'))
self.indent()
self.emitBegin('')
self._to_str_impl(schema['items'])
self.undent()
self.emit('],')
else:
self.emit('Unknown type! %s' % stype)
self.emitEnd('', '')
self.string = ''.join(self.value)
return self.string
def to_str(self, from_cache):
"""Prototype object based on the schema, in Python code with comments.
Args:
from_cache: callable(name, seen), Callable that retrieves an object
prototype for a schema with the given name. Seen is a list of schema
names already seen as we recursively descend the schema definition.
Returns:
Prototype object based on the schema, in Python code with comments.
The lines of the code will all be properly indented.
"""
self.from_cache = from_cache
return self._to_str_impl(self.schema)

1484
atom/__init__.py Normal file
View File

File diff suppressed because it is too large Load Diff

43
atom/auth.py Normal file
View File

@ -0,0 +1,43 @@
#!/usr/bin/env python
#
# Copyright (C) 2009 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This module is used for version 2 of the Google Data APIs.
__author__ = 'j.s@google.com (Jeff Scudder)'
import base64
class BasicAuth(object):
"""Sets the Authorization header as defined in RFC1945"""
def __init__(self, user_id, password):
self.basic_cookie = base64.encodestring(
'%s:%s' % (user_id, password)).strip()
def modify_request(self, http_request):
http_request.headers['Authorization'] = 'Basic %s' % self.basic_cookie
ModifyRequest = modify_request
class NoAuth(object):
def modify_request(self, http_request):
pass

221
atom/client.py Normal file
View File

@ -0,0 +1,221 @@
#!/usr/bin/env python
#
# Copyright (C) 2009 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""AtomPubClient provides CRUD ops. in line with the Atom Publishing Protocol.
"""
__author__ = 'j.s@google.com (Jeff Scudder)'
import atom.http_core
class Error(Exception):
pass
class MissingHost(Error):
pass
class AtomPubClient(object):
host = None
auth_token = None
ssl = False # Whether to force all requests over https
xoauth_requestor_id = None
def __init__(self, http_client=None, host=None, auth_token=None, source=None,
xoauth_requestor_id=None, **kwargs):
"""Creates a new AtomPubClient instance.
Args:
source: The name of your application.
http_client: An object capable of performing HTTP requests through a
request method. This object is used to perform the request
when the AtomPubClient's request method is called. Used to
allow HTTP requests to be directed to a mock server, or use
an alternate library instead of the default of httplib to
make HTTP requests.
host: str The default host name to use if a host is not specified in the
requested URI.
auth_token: An object which sets the HTTP Authorization header when its
modify_request method is called.
"""
self.http_client = http_client or atom.http_core.ProxiedHttpClient()
if host is not None:
self.host = host
if auth_token is not None:
self.auth_token = auth_token
self.xoauth_requestor_id = xoauth_requestor_id
self.source = source
def request(self, method=None, uri=None, auth_token=None,
http_request=None, **kwargs):
"""Performs an HTTP request to the server indicated.
Uses the http_client instance to make the request.
Args:
method: The HTTP method as a string, usually one of 'GET', 'POST',
'PUT', or 'DELETE'
uri: The URI desired as a string or atom.http_core.Uri.
http_request:
auth_token: An authorization token object whose modify_request method
sets the HTTP Authorization header.
Returns:
The results of calling self.http_client.request. With the default
http_client, this is an HTTP response object.
"""
# Modify the request based on the AtomPubClient settings and parameters
# passed in to the request.
http_request = self.modify_request(http_request)
if isinstance(uri, (str, unicode)):
uri = atom.http_core.Uri.parse_uri(uri)
if uri is not None:
uri.modify_request(http_request)
if isinstance(method, (str, unicode)):
http_request.method = method
# Any unrecognized arguments are assumed to be capable of modifying the
# HTTP request.
for name, value in kwargs.iteritems():
if value is not None:
value.modify_request(http_request)
# Default to an http request if the protocol scheme is not set.
if http_request.uri.scheme is None:
http_request.uri.scheme = 'http'
# Override scheme. Force requests over https.
if self.ssl:
http_request.uri.scheme = 'https'
if http_request.uri.path is None:
http_request.uri.path = '/'
# Add the Authorization header at the very end. The Authorization header
# value may need to be calculated using information in the request.
if auth_token:
auth_token.modify_request(http_request)
elif self.auth_token:
self.auth_token.modify_request(http_request)
# Check to make sure there is a host in the http_request.
if http_request.uri.host is None:
raise MissingHost('No host provided in request %s %s' % (
http_request.method, str(http_request.uri)))
# Perform the fully specified request using the http_client instance.
# Sends the request to the server and returns the server's response.
return self.http_client.request(http_request)
Request = request
def get(self, uri=None, auth_token=None, http_request=None, **kwargs):
"""Performs a request using the GET method, returns an HTTP response."""
return self.request(method='GET', uri=uri, auth_token=auth_token,
http_request=http_request, **kwargs)
Get = get
def post(self, uri=None, data=None, auth_token=None, http_request=None,
**kwargs):
"""Sends data using the POST method, returns an HTTP response."""
return self.request(method='POST', uri=uri, auth_token=auth_token,
http_request=http_request, data=data, **kwargs)
Post = post
def put(self, uri=None, data=None, auth_token=None, http_request=None,
**kwargs):
"""Sends data using the PUT method, returns an HTTP response."""
return self.request(method='PUT', uri=uri, auth_token=auth_token,
http_request=http_request, data=data, **kwargs)
Put = put
def delete(self, uri=None, auth_token=None, http_request=None, **kwargs):
"""Performs a request using the DELETE method, returns an HTTP response."""
return self.request(method='DELETE', uri=uri, auth_token=auth_token,
http_request=http_request, **kwargs)
Delete = delete
def modify_request(self, http_request):
"""Changes the HTTP request before sending it to the server.
Sets the User-Agent HTTP header and fills in the HTTP host portion
of the URL if one was not included in the request (for this it uses
the self.host member if one is set). This method is called in
self.request.
Args:
http_request: An atom.http_core.HttpRequest() (optional) If one is
not provided, a new HttpRequest is instantiated.
Returns:
An atom.http_core.HttpRequest() with the User-Agent header set and
if this client has a value in its host member, the host in the request
URL is set.
"""
if http_request is None:
http_request = atom.http_core.HttpRequest()
if self.host is not None and http_request.uri.host is None:
http_request.uri.host = self.host
if self.xoauth_requestor_id is not None:
http_request.uri.query['xoauth_requestor_id'] = self.xoauth_requestor_id
# Set the user agent header for logging purposes.
if self.source:
http_request.headers['User-Agent'] = '%s gdata-py/2.0.14' % self.source
else:
http_request.headers['User-Agent'] = 'gdata-py/2.0.14'
return http_request
ModifyRequest = modify_request
class CustomHeaders(object):
"""Add custom headers to an http_request.
Usage:
>>> custom_headers = atom.client.CustomHeaders(header1='value1',
header2='value2')
>>> client.get(uri, custom_headers=custom_headers)
"""
def __init__(self, **kwargs):
"""Creates a CustomHeaders instance.
Initialize the headers dictionary with the arguments list.
"""
self.headers = kwargs
def modify_request(self, http_request):
"""Changes the HTTP request before sending it to the server.
Adds the custom headers to the HTTP request.
Args:
http_request: An atom.http_core.HttpRequest().
Returns:
An atom.http_core.HttpRequest() with the added custom headers.
"""
for name, value in self.headers.iteritems():
if value is not None:
http_request.headers[name] = value
return http_request

550
atom/core.py Normal file
View File

@ -0,0 +1,550 @@
#!/usr/bin/env python
#
# Copyright (C) 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This module is used for version 2 of the Google Data APIs.
__author__ = 'j.s@google.com (Jeff Scudder)'
import inspect
try:
from xml.etree import cElementTree as ElementTree
except ImportError:
try:
import cElementTree as ElementTree
except ImportError:
try:
from xml.etree import ElementTree
except ImportError:
from elementtree import ElementTree
try:
from xml.dom.minidom import parseString as xmlString
except ImportError:
xmlString = None
STRING_ENCODING = 'utf-8'
class XmlElement(object):
"""Represents an element node in an XML document.
The text member is a UTF-8 encoded str or unicode.
"""
_qname = None
_other_elements = None
_other_attributes = None
# The rule set contains mappings for XML qnames to child members and the
# appropriate member classes.
_rule_set = None
_members = None
text = None
def __init__(self, text=None, *args, **kwargs):
if ('_members' not in self.__class__.__dict__
or self.__class__._members is None):
self.__class__._members = tuple(self.__class__._list_xml_members())
for member_name, member_type in self.__class__._members:
if member_name in kwargs:
setattr(self, member_name, kwargs[member_name])
else:
if isinstance(member_type, list):
setattr(self, member_name, [])
else:
setattr(self, member_name, None)
self._other_elements = []
self._other_attributes = {}
if text is not None:
self.text = text
def _list_xml_members(cls):
"""Generator listing all members which are XML elements or attributes.
The following members would be considered XML members:
foo = 'abc' - indicates an XML attribute with the qname abc
foo = SomeElement - indicates an XML child element
foo = [AnElement] - indicates a repeating XML child element, each instance
will be stored in a list in this member
foo = ('att1', '{http://example.com/namespace}att2') - indicates an XML
attribute which has different parsing rules in different versions of
the protocol. Version 1 of the XML parsing rules will look for an
attribute with the qname 'att1' but verion 2 of the parsing rules will
look for a namespaced attribute with the local name of 'att2' and an
XML namespace of 'http://example.com/namespace'.
"""
members = []
for pair in inspect.getmembers(cls):
if not pair[0].startswith('_') and pair[0] != 'text':
member_type = pair[1]
if (isinstance(member_type, tuple) or isinstance(member_type, list)
or isinstance(member_type, (str, unicode))
or (inspect.isclass(member_type)
and issubclass(member_type, XmlElement))):
members.append(pair)
return members
_list_xml_members = classmethod(_list_xml_members)
def _get_rules(cls, version):
"""Initializes the _rule_set for the class which is used when parsing XML.
This method is used internally for parsing and generating XML for an
XmlElement. It is not recommended that you call this method directly.
Returns:
A tuple containing the XML parsing rules for the appropriate version.
The tuple looks like:
(qname, {sub_element_qname: (member_name, member_class, repeating), ..},
{attribute_qname: member_name})
To give a couple of concrete example, the atom.data.Control _get_rules
with version of 2 will return:
('{http://www.w3.org/2007/app}control',
{'{http://www.w3.org/2007/app}draft': ('draft',
<class 'atom.data.Draft'>,
False)},
{})
Calling _get_rules with version 1 on gdata.data.FeedLink will produce:
('{http://schemas.google.com/g/2005}feedLink',
{'{http://www.w3.org/2005/Atom}feed': ('feed',
<class 'gdata.data.GDFeed'>,
False)},
{'href': 'href', 'readOnly': 'read_only', 'countHint': 'count_hint',
'rel': 'rel'})
"""
# Initialize the _rule_set to make sure there is a slot available to store
# the parsing rules for this version of the XML schema.
# Look for rule set in the class __dict__ proxy so that only the
# _rule_set for this class will be found. By using the dict proxy
# we avoid finding rule_sets defined in superclasses.
# The four lines below provide support for any number of versions, but it
# runs a bit slower then hard coding slots for two versions, so I'm using
# the below two lines.
#if '_rule_set' not in cls.__dict__ or cls._rule_set is None:
# cls._rule_set = []
#while len(cls.__dict__['_rule_set']) < version:
# cls._rule_set.append(None)
# If there is no rule set cache in the class, provide slots for two XML
# versions. If and when there is a version 3, this list will need to be
# expanded.
if '_rule_set' not in cls.__dict__ or cls._rule_set is None:
cls._rule_set = [None, None]
# If a version higher than 2 is requested, fall back to version 2 because
# 2 is currently the highest supported version.
if version > 2:
return cls._get_rules(2)
# Check the dict proxy for the rule set to avoid finding any rule sets
# which belong to the superclass. We only want rule sets for this class.
if cls._rule_set[version-1] is None:
# The rule set for each version consists of the qname for this element
# ('{namespace}tag'), a dictionary (elements) for looking up the
# corresponding class member when given a child element's qname, and a
# dictionary (attributes) for looking up the corresponding class member
# when given an XML attribute's qname.
elements = {}
attributes = {}
if ('_members' not in cls.__dict__ or cls._members is None):
cls._members = tuple(cls._list_xml_members())
for member_name, target in cls._members:
if isinstance(target, list):
# This member points to a repeating element.
elements[_get_qname(target[0], version)] = (member_name, target[0],
True)
elif isinstance(target, tuple):
# This member points to a versioned XML attribute.
if version <= len(target):
attributes[target[version-1]] = member_name
else:
attributes[target[-1]] = member_name
elif isinstance(target, (str, unicode)):
# This member points to an XML attribute.
attributes[target] = member_name
elif issubclass(target, XmlElement):
# This member points to a single occurance element.
elements[_get_qname(target, version)] = (member_name, target, False)
version_rules = (_get_qname(cls, version), elements, attributes)
cls._rule_set[version-1] = version_rules
return version_rules
else:
return cls._rule_set[version-1]
_get_rules = classmethod(_get_rules)
def get_elements(self, tag=None, namespace=None, version=1):
"""Find all sub elements which match the tag and namespace.
To find all elements in this object, call get_elements with the tag and
namespace both set to None (the default). This method searches through
the object's members and the elements stored in _other_elements which
did not match any of the XML parsing rules for this class.
Args:
tag: str
namespace: str
version: int Specifies the version of the XML rules to be used when
searching for matching elements.
Returns:
A list of the matching XmlElements.
"""
matches = []
ignored1, elements, ignored2 = self.__class__._get_rules(version)
if elements:
for qname, element_def in elements.iteritems():
member = getattr(self, element_def[0])
if member:
if _qname_matches(tag, namespace, qname):
if element_def[2]:
# If this is a repeating element, copy all instances into the
# result list.
matches.extend(member)
else:
matches.append(member)
for element in self._other_elements:
if _qname_matches(tag, namespace, element._qname):
matches.append(element)
return matches
GetElements = get_elements
# FindExtensions and FindChildren are provided for backwards compatibility
# to the atom.AtomBase class.
# However, FindExtensions may return more results than the v1 atom.AtomBase
# method does, because get_elements searches both the expected children
# and the unexpected "other elements". The old AtomBase.FindExtensions
# method searched only "other elements" AKA extension_elements.
FindExtensions = get_elements
FindChildren = get_elements
def get_attributes(self, tag=None, namespace=None, version=1):
"""Find all attributes which match the tag and namespace.
To find all attributes in this object, call get_attributes with the tag
and namespace both set to None (the default). This method searches
through the object's members and the attributes stored in
_other_attributes which did not fit any of the XML parsing rules for this
class.
Args:
tag: str
namespace: str
version: int Specifies the version of the XML rules to be used when
searching for matching attributes.
Returns:
A list of XmlAttribute objects for the matching attributes.
"""
matches = []
ignored1, ignored2, attributes = self.__class__._get_rules(version)
if attributes:
for qname, attribute_def in attributes.iteritems():
if isinstance(attribute_def, (list, tuple)):
attribute_def = attribute_def[0]
member = getattr(self, attribute_def)
# TODO: ensure this hasn't broken existing behavior.
#member = getattr(self, attribute_def[0])
if member:
if _qname_matches(tag, namespace, qname):
matches.append(XmlAttribute(qname, member))
for qname, value in self._other_attributes.iteritems():
if _qname_matches(tag, namespace, qname):
matches.append(XmlAttribute(qname, value))
return matches
GetAttributes = get_attributes
def _harvest_tree(self, tree, version=1):
"""Populates object members from the data in the tree Element."""
qname, elements, attributes = self.__class__._get_rules(version)
for element in tree:
if elements and element.tag in elements:
definition = elements[element.tag]
# If this is a repeating element, make sure the member is set to a
# list.
if definition[2]:
if getattr(self, definition[0]) is None:
setattr(self, definition[0], [])
getattr(self, definition[0]).append(_xml_element_from_tree(element,
definition[1], version))
else:
setattr(self, definition[0], _xml_element_from_tree(element,
definition[1], version))
else:
self._other_elements.append(_xml_element_from_tree(element, XmlElement,
version))
for attrib, value in tree.attrib.iteritems():
if attributes and attrib in attributes:
setattr(self, attributes[attrib], value)
else:
self._other_attributes[attrib] = value
if tree.text:
self.text = tree.text
def _to_tree(self, version=1, encoding=None):
new_tree = ElementTree.Element(_get_qname(self, version))
self._attach_members(new_tree, version, encoding)
return new_tree
def _attach_members(self, tree, version=1, encoding=None):
"""Convert members to XML elements/attributes and add them to the tree.
Args:
tree: An ElementTree.Element which will be modified. The members of
this object will be added as child elements or attributes
according to the rules described in _expected_elements and
_expected_attributes. The elements and attributes stored in
other_attributes and other_elements are also added a children
of this tree.
version: int Ingnored in this method but used by VersionedElement.
encoding: str (optional)
"""
qname, elements, attributes = self.__class__._get_rules(version)
encoding = encoding or STRING_ENCODING
# Add the expected elements and attributes to the tree.
if elements:
for tag, element_def in elements.iteritems():
member = getattr(self, element_def[0])
# If this is a repeating element and there are members in the list.
if member and element_def[2]:
for instance in member:
instance._become_child(tree, version)
elif member:
member._become_child(tree, version)
if attributes:
for attribute_tag, member_name in attributes.iteritems():
value = getattr(self, member_name)
if value:
tree.attrib[attribute_tag] = value
# Add the unexpected (other) elements and attributes to the tree.
for element in self._other_elements:
element._become_child(tree, version)
for key, value in self._other_attributes.iteritems():
# I'm not sure if unicode can be used in the attribute name, so for now
# we assume the encoding is correct for the attribute name.
if not isinstance(value, unicode):
value = value.decode(encoding)
tree.attrib[key] = value
if self.text:
if isinstance(self.text, unicode):
tree.text = self.text
else:
tree.text = self.text.decode(encoding)
def to_string(self, version=1, encoding=None, pretty_print=None):
"""Converts this object to XML."""
tree_string = ElementTree.tostring(self._to_tree(version, encoding))
if pretty_print and xmlString is not None:
return xmlString(tree_string).toprettyxml()
return tree_string
ToString = to_string
def __str__(self):
return self.to_string()
def _become_child(self, tree, version=1):
"""Adds a child element to tree with the XML data in self."""
new_child = ElementTree.Element('')
tree.append(new_child)
new_child.tag = _get_qname(self, version)
self._attach_members(new_child, version)
def __get_extension_elements(self):
return self._other_elements
def __set_extension_elements(self, elements):
self._other_elements = elements
extension_elements = property(__get_extension_elements,
__set_extension_elements,
"""Provides backwards compatibility for v1 atom.AtomBase classes.""")
def __get_extension_attributes(self):
return self._other_attributes
def __set_extension_attributes(self, attributes):
self._other_attributes = attributes
extension_attributes = property(__get_extension_attributes,
__set_extension_attributes,
"""Provides backwards compatibility for v1 atom.AtomBase classes.""")
def _get_tag(self, version=1):
qname = _get_qname(self, version)
if qname:
return qname[qname.find('}')+1:]
return None
def _get_namespace(self, version=1):
qname = _get_qname(self, version)
if qname.startswith('{'):
return qname[1:qname.find('}')]
else:
return None
def _set_tag(self, tag):
if isinstance(self._qname, tuple):
self._qname = self._qname.copy()
if self._qname[0].startswith('{'):
self._qname[0] = '{%s}%s' % (self._get_namespace(1), tag)
else:
self._qname[0] = tag
else:
if self._qname is not None and self._qname.startswith('{'):
self._qname = '{%s}%s' % (self._get_namespace(), tag)
else:
self._qname = tag
def _set_namespace(self, namespace):
tag = self._get_tag(1)
if tag is None:
tag = ''
if isinstance(self._qname, tuple):
self._qname = self._qname.copy()
if namespace:
self._qname[0] = '{%s}%s' % (namespace, tag)
else:
self._qname[0] = tag
else:
if namespace:
self._qname = '{%s}%s' % (namespace, tag)
else:
self._qname = tag
tag = property(_get_tag, _set_tag,
"""Provides backwards compatibility for v1 atom.AtomBase classes.""")
namespace = property(_get_namespace, _set_namespace,
"""Provides backwards compatibility for v1 atom.AtomBase classes.""")
# Provided for backwards compatibility to atom.ExtensionElement
children = extension_elements
attributes = extension_attributes
def _get_qname(element, version):
if isinstance(element._qname, tuple):
if version <= len(element._qname):
return element._qname[version-1]
else:
return element._qname[-1]
else:
return element._qname
def _qname_matches(tag, namespace, qname):
"""Logic determines if a QName matches the desired local tag and namespace.
This is used in XmlElement.get_elements and XmlElement.get_attributes to
find matches in the element's members (among all expected-and-unexpected
elements-and-attributes).
Args:
expected_tag: string
expected_namespace: string
qname: string in the form '{xml_namespace}localtag' or 'tag' if there is
no namespace.
Returns:
boolean True if the member's tag and namespace fit the expected tag and
namespace.
"""
# If there is no expected namespace or tag, then everything will match.
if qname is None:
member_tag = None
member_namespace = None
else:
if qname.startswith('{'):
member_namespace = qname[1:qname.index('}')]
member_tag = qname[qname.index('}') + 1:]
else:
member_namespace = None
member_tag = qname
return ((tag is None and namespace is None)
# If there is a tag, but no namespace, see if the local tag matches.
or (namespace is None and member_tag == tag)
# There was no tag, but there was a namespace so see if the namespaces
# match.
or (tag is None and member_namespace == namespace)
# There was no tag, and the desired elements have no namespace, so check
# to see that the member's namespace is None.
or (tag is None and namespace == ''
and member_namespace is None)
# The tag and the namespace both match.
or (tag == member_tag
and namespace == member_namespace)
# The tag matches, and the expected namespace is the empty namespace,
# check to make sure the member's namespace is None.
or (tag == member_tag and namespace == ''
and member_namespace is None))
def parse(xml_string, target_class=None, version=1, encoding=None):
"""Parses the XML string according to the rules for the target_class.
Args:
xml_string: str or unicode
target_class: XmlElement or a subclass. If None is specified, the
XmlElement class is used.
version: int (optional) The version of the schema which should be used when
converting the XML into an object. The default is 1.
encoding: str (optional) The character encoding of the bytes in the
xml_string. Default is 'UTF-8'.
"""
if target_class is None:
target_class = XmlElement
if isinstance(xml_string, unicode):
if encoding is None:
xml_string = xml_string.encode(STRING_ENCODING)
else:
xml_string = xml_string.encode(encoding)
tree = ElementTree.fromstring(xml_string)
return _xml_element_from_tree(tree, target_class, version)
Parse = parse
xml_element_from_string = parse
XmlElementFromString = xml_element_from_string
def _xml_element_from_tree(tree, target_class, version=1):
if target_class._qname is None:
instance = target_class()
instance._qname = tree.tag
instance._harvest_tree(tree, version)
return instance
# TODO handle the namespace-only case
# Namespace only will be used with Google Spreadsheets rows and
# Google Base item attributes.
elif tree.tag == _get_qname(target_class, version):
instance = target_class()
instance._harvest_tree(tree, version)
return instance
return None
class XmlAttribute(object):
def __init__(self, qname, value):
self._qname = qname
self.value = value

338
atom/data.py Normal file
View File

@ -0,0 +1,338 @@
#!/usr/bin/env python
#
# Copyright (C) 2009 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This module is used for version 2 of the Google Data APIs.
__author__ = 'j.s@google.com (Jeff Scudder)'
import atom.core
XML_TEMPLATE = '{http://www.w3.org/XML/1998/namespace}%s'
ATOM_TEMPLATE = '{http://www.w3.org/2005/Atom}%s'
APP_TEMPLATE_V1 = '{http://purl.org/atom/app#}%s'
APP_TEMPLATE_V2 = '{http://www.w3.org/2007/app}%s'
class Name(atom.core.XmlElement):
"""The atom:name element."""
_qname = ATOM_TEMPLATE % 'name'
class Email(atom.core.XmlElement):
"""The atom:email element."""
_qname = ATOM_TEMPLATE % 'email'
class Uri(atom.core.XmlElement):
"""The atom:uri element."""
_qname = ATOM_TEMPLATE % 'uri'
class Person(atom.core.XmlElement):
"""A foundation class which atom:author and atom:contributor extend.
A person contains information like name, email address, and web page URI for
an author or contributor to an Atom feed.
"""
name = Name
email = Email
uri = Uri
class Author(Person):
"""The atom:author element.
An author is a required element in Feed unless each Entry contains an Author.
"""
_qname = ATOM_TEMPLATE % 'author'
class Contributor(Person):
"""The atom:contributor element."""
_qname = ATOM_TEMPLATE % 'contributor'
class Link(atom.core.XmlElement):
"""The atom:link element."""
_qname = ATOM_TEMPLATE % 'link'
href = 'href'
rel = 'rel'
type = 'type'
hreflang = 'hreflang'
title = 'title'
length = 'length'
class Generator(atom.core.XmlElement):
"""The atom:generator element."""
_qname = ATOM_TEMPLATE % 'generator'
uri = 'uri'
version = 'version'
class Text(atom.core.XmlElement):
"""A foundation class from which atom:title, summary, etc. extend.
This class should never be instantiated.
"""
type = 'type'
class Title(Text):
"""The atom:title element."""
_qname = ATOM_TEMPLATE % 'title'
class Subtitle(Text):
"""The atom:subtitle element."""
_qname = ATOM_TEMPLATE % 'subtitle'
class Rights(Text):
"""The atom:rights element."""
_qname = ATOM_TEMPLATE % 'rights'
class Summary(Text):
"""The atom:summary element."""
_qname = ATOM_TEMPLATE % 'summary'
class Content(Text):
"""The atom:content element."""
_qname = ATOM_TEMPLATE % 'content'
src = 'src'
class Category(atom.core.XmlElement):
"""The atom:category element."""
_qname = ATOM_TEMPLATE % 'category'
term = 'term'
scheme = 'scheme'
label = 'label'
class Id(atom.core.XmlElement):
"""The atom:id element."""
_qname = ATOM_TEMPLATE % 'id'
class Icon(atom.core.XmlElement):
"""The atom:icon element."""
_qname = ATOM_TEMPLATE % 'icon'
class Logo(atom.core.XmlElement):
"""The atom:logo element."""
_qname = ATOM_TEMPLATE % 'logo'
class Draft(atom.core.XmlElement):
"""The app:draft element which indicates if this entry should be public."""
_qname = (APP_TEMPLATE_V1 % 'draft', APP_TEMPLATE_V2 % 'draft')
class Control(atom.core.XmlElement):
"""The app:control element indicating restrictions on publication.
The APP control element may contain a draft element indicating whether or
not this entry should be publicly available.
"""
_qname = (APP_TEMPLATE_V1 % 'control', APP_TEMPLATE_V2 % 'control')
draft = Draft
class Date(atom.core.XmlElement):
"""A parent class for atom:updated, published, etc."""
class Updated(Date):
"""The atom:updated element."""
_qname = ATOM_TEMPLATE % 'updated'
class Published(Date):
"""The atom:published element."""
_qname = ATOM_TEMPLATE % 'published'
class LinkFinder(object):
"""An "interface" providing methods to find link elements
Entry elements often contain multiple links which differ in the rel
attribute or content type. Often, developers are interested in a specific
type of link so this class provides methods to find specific classes of
links.
This class is used as a mixin in Atom entries and feeds.
"""
def find_url(self, rel):
"""Returns the URL (as a string) in a link with the desired rel value."""
for link in self.link:
if link.rel == rel and link.href:
return link.href
return None
FindUrl = find_url
def get_link(self, rel):
"""Returns a link object which has the desired rel value.
If you are interested in the URL instead of the link object,
consider using find_url instead.
"""
for link in self.link:
if link.rel == rel and link.href:
return link
return None
GetLink = get_link
def find_self_link(self):
"""Find the first link with rel set to 'self'
Returns:
A str containing the link's href or None if none of the links had rel
equal to 'self'
"""
return self.find_url('self')
FindSelfLink = find_self_link
def get_self_link(self):
return self.get_link('self')
GetSelfLink = get_self_link
def find_edit_link(self):
return self.find_url('edit')
FindEditLink = find_edit_link
def get_edit_link(self):
return self.get_link('edit')
GetEditLink = get_edit_link
def find_edit_media_link(self):
link = self.find_url('edit-media')
# Search for media-edit as well since Picasa API used media-edit instead.
if link is None:
return self.find_url('media-edit')
return link
FindEditMediaLink = find_edit_media_link
def get_edit_media_link(self):
link = self.get_link('edit-media')
if link is None:
return self.get_link('media-edit')
return link
GetEditMediaLink = get_edit_media_link
def find_next_link(self):
return self.find_url('next')
FindNextLink = find_next_link
def get_next_link(self):
return self.get_link('next')
GetNextLink = get_next_link
def find_license_link(self):
return self.find_url('license')
FindLicenseLink = find_license_link
def get_license_link(self):
return self.get_link('license')
GetLicenseLink = get_license_link
def find_alternate_link(self):
return self.find_url('alternate')
FindAlternateLink = find_alternate_link
def get_alternate_link(self):
return self.get_link('alternate')
GetAlternateLink = get_alternate_link
class FeedEntryParent(atom.core.XmlElement, LinkFinder):
"""A super class for atom:feed and entry, contains shared attributes"""
author = [Author]
category = [Category]
contributor = [Contributor]
id = Id
link = [Link]
rights = Rights
title = Title
updated = Updated
def __init__(self, atom_id=None, text=None, *args, **kwargs):
if atom_id is not None:
self.id = atom_id
atom.core.XmlElement.__init__(self, text=text, *args, **kwargs)
class Source(FeedEntryParent):
"""The atom:source element."""
_qname = ATOM_TEMPLATE % 'source'
generator = Generator
icon = Icon
logo = Logo
subtitle = Subtitle
class Entry(FeedEntryParent):
"""The atom:entry element."""
_qname = ATOM_TEMPLATE % 'entry'
content = Content
published = Published
source = Source
summary = Summary
control = Control
class Feed(Source):
"""The atom:feed element which contains entries."""
_qname = ATOM_TEMPLATE % 'feed'
entry = [Entry]
class ExtensionElement(atom.core.XmlElement):
"""Provided for backwards compatibility to the v1 atom.ExtensionElement."""
def __init__(self, tag=None, namespace=None, attributes=None,
children=None, text=None, *args, **kwargs):
if namespace:
self._qname = '{%s}%s' % (namespace, tag)
else:
self._qname = tag
self.children = children or []
self.attributes = attributes or {}
self.text = text
_BecomeChildElement = atom.core.XmlElement._become_child

364
atom/http.py Normal file
View File

@ -0,0 +1,364 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""HttpClients in this module use httplib to make HTTP requests.
This module make HTTP requests based on httplib, but there are environments
in which an httplib based approach will not work (if running in Google App
Engine for example). In those cases, higher level classes (like AtomService
and GDataService) can swap out the HttpClient to transparently use a
different mechanism for making HTTP requests.
HttpClient: Contains a request method which performs an HTTP call to the
server.
ProxiedHttpClient: Contains a request method which connects to a proxy using
settings stored in operating system environment variables then
performs an HTTP call to the endpoint server.
"""
__author__ = 'api.jscudder (Jeff Scudder)'
import types
import os
import httplib
import atom.url
import atom.http_interface
import socket
import base64
import atom.http_core
ssl_imported = False
ssl = None
try:
import ssl
ssl_imported = True
except ImportError:
pass
class ProxyError(atom.http_interface.Error):
pass
class TestConfigurationError(Exception):
pass
DEFAULT_CONTENT_TYPE = 'application/atom+xml; charset=UTF-8'
class HttpClient(atom.http_interface.GenericHttpClient):
# Added to allow old v1 HttpClient objects to use the new
# http_code.HttpClient. Used in unit tests to inject a mock client.
v2_http_client = None
def __init__(self, headers=None):
self.debug = False
self.headers = headers or {}
def request(self, operation, url, data=None, headers=None):
"""Performs an HTTP call to the server, supports GET, POST, PUT, and
DELETE.
Usage example, perform and HTTP GET on http://www.google.com/:
import atom.http
client = atom.http.HttpClient()
http_response = client.request('GET', 'http://www.google.com/')
Args:
operation: str The HTTP operation to be performed. This is usually one
of 'GET', 'POST', 'PUT', or 'DELETE'
data: filestream, list of parts, or other object which can be converted
to a string. Should be set to None when performing a GET or DELETE.
If data is a file-like object which can be read, this method will
read a chunk of 100K bytes at a time and send them.
If the data is a list of parts to be sent, each part will be
evaluated and sent.
url: The full URL to which the request should be sent. Can be a string
or atom.url.Url.
headers: dict of strings. HTTP headers which should be sent
in the request.
"""
all_headers = self.headers.copy()
if headers:
all_headers.update(headers)
# If the list of headers does not include a Content-Length, attempt to
# calculate it based on the data object.
if data and 'Content-Length' not in all_headers:
if isinstance(data, types.StringTypes):
all_headers['Content-Length'] = str(len(data))
else:
raise atom.http_interface.ContentLengthRequired('Unable to calculate '
'the length of the data parameter. Specify a value for '
'Content-Length')
# Set the content type to the default value if none was set.
if 'Content-Type' not in all_headers:
all_headers['Content-Type'] = DEFAULT_CONTENT_TYPE
if self.v2_http_client is not None:
http_request = atom.http_core.HttpRequest(method=operation)
atom.http_core.Uri.parse_uri(str(url)).modify_request(http_request)
http_request.headers = all_headers
if data:
http_request._body_parts.append(data)
return self.v2_http_client.request(http_request=http_request)
if not isinstance(url, atom.url.Url):
if isinstance(url, types.StringTypes):
url = atom.url.parse_url(url)
else:
raise atom.http_interface.UnparsableUrlObject('Unable to parse url '
'parameter because it was not a string or atom.url.Url')
connection = self._prepare_connection(url, all_headers)
if self.debug:
connection.debuglevel = 1
connection.putrequest(operation, self._get_access_url(url),
skip_host=True)
if url.port is not None:
connection.putheader('Host', '%s:%s' % (url.host, url.port))
else:
connection.putheader('Host', url.host)
# Overcome a bug in Python 2.4 and 2.5
# httplib.HTTPConnection.putrequest adding
# HTTP request header 'Host: www.google.com:443' instead of
# 'Host: www.google.com', and thus resulting the error message
# 'Token invalid - AuthSub token has wrong scope' in the HTTP response.
if (url.protocol == 'https' and int(url.port or 443) == 443 and
hasattr(connection, '_buffer') and
isinstance(connection._buffer, list)):
header_line = 'Host: %s:443' % url.host
replacement_header_line = 'Host: %s' % url.host
try:
connection._buffer[connection._buffer.index(header_line)] = (
replacement_header_line)
except ValueError: # header_line missing from connection._buffer
pass
# Send the HTTP headers.
for header_name in all_headers:
connection.putheader(header_name, all_headers[header_name])
connection.endheaders()
# If there is data, send it in the request.
if data:
if isinstance(data, list):
for data_part in data:
_send_data_part(data_part, connection)
else:
_send_data_part(data, connection)
# Return the HTTP Response from the server.
return connection.getresponse()
def _prepare_connection(self, url, headers):
if not isinstance(url, atom.url.Url):
if isinstance(url, types.StringTypes):
url = atom.url.parse_url(url)
else:
raise atom.http_interface.UnparsableUrlObject('Unable to parse url '
'parameter because it was not a string or atom.url.Url')
if url.protocol == 'https':
if not url.port:
return httplib.HTTPSConnection(url.host)
return httplib.HTTPSConnection(url.host, int(url.port))
else:
if not url.port:
return httplib.HTTPConnection(url.host)
return httplib.HTTPConnection(url.host, int(url.port))
def _get_access_url(self, url):
return url.to_string()
class ProxiedHttpClient(HttpClient):
"""Performs an HTTP request through a proxy.
The proxy settings are obtained from enviroment variables. The URL of the
proxy server is assumed to be stored in the environment variables
'https_proxy' and 'http_proxy' respectively. If the proxy server requires
a Basic Auth authorization header, the username and password are expected to
be in the 'proxy-username' or 'proxy_username' variable and the
'proxy-password' or 'proxy_password' variable, or in 'http_proxy' or
'https_proxy' as "protocol://[username:password@]host:port".
After connecting to the proxy server, the request is completed as in
HttpClient.request.
"""
def _prepare_connection(self, url, headers):
proxy_settings = os.environ.get('%s_proxy' % url.protocol)
if not proxy_settings:
# The request was HTTP or HTTPS, but there was no appropriate proxy set.
return HttpClient._prepare_connection(self, url, headers)
else:
#print '!!!!%s' % proxy_settings
proxy_auth = _get_proxy_auth(proxy_settings)
proxy_netloc = _get_proxy_net_location(proxy_settings)
#print '!!!!%s' % proxy_auth
#print '!!!!%s' % proxy_netloc
if url.protocol == 'https':
# Set any proxy auth headers
if proxy_auth:
proxy_auth = 'Proxy-authorization: %s' % proxy_auth
# Construct the proxy connect command.
port = url.port
if not port:
port = '443'
proxy_connect = 'CONNECT %s:%s HTTP/1.0\r\n' % (url.host, port)
# Set the user agent to send to the proxy
if headers and 'User-Agent' in headers:
user_agent = 'User-Agent: %s\r\n' % (headers['User-Agent'])
else:
user_agent = 'User-Agent: python\r\n'
proxy_pieces = '%s%s%s\r\n' % (proxy_connect, proxy_auth, user_agent)
# Find the proxy host and port.
proxy_url = atom.url.parse_url(proxy_netloc)
if not proxy_url.port:
proxy_url.port = '80'
# Connect to the proxy server, very simple recv and error checking
p_sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
p_sock.connect((proxy_url.host, int(proxy_url.port)))
p_sock.sendall(proxy_pieces)
response = ''
# Wait for the full response.
while response.find("\r\n\r\n") == -1:
response += p_sock.recv(8192)
p_status = response.split()[1]
if p_status != str(200):
raise ProxyError('Error status=%s' % str(p_status))
# Trivial setup for ssl socket.
sslobj = None
if ssl_imported:
sslobj = ssl.wrap_socket(p_sock, None, None)
else:
sock_ssl = socket.ssl(p_sock, None, None)
sslobj = httplib.FakeSocket(p_sock, sock_ssl)
# Initalize httplib and replace with the proxy socket.
connection = httplib.HTTPConnection(proxy_url.host)
connection.sock = sslobj
return connection
else:
# If protocol was not https.
# Find the proxy host and port.
proxy_url = atom.url.parse_url(proxy_netloc)
if not proxy_url.port:
proxy_url.port = '80'
if proxy_auth:
headers['Proxy-Authorization'] = proxy_auth.strip()
return httplib.HTTPConnection(proxy_url.host, int(proxy_url.port))
def _get_access_url(self, url):
return url.to_string()
def _get_proxy_auth(proxy_settings):
"""Returns proxy authentication string for header.
Will check environment variables for proxy authentication info, starting with
proxy(_/-)username and proxy(_/-)password before checking the given
proxy_settings for a [protocol://]username:password@host[:port] string.
Args:
proxy_settings: String from http_proxy or https_proxy environment variable.
Returns:
Authentication string for proxy, or empty string if no proxy username was
found.
"""
proxy_username = None
proxy_password = None
proxy_username = os.environ.get('proxy-username')
if not proxy_username:
proxy_username = os.environ.get('proxy_username')
proxy_password = os.environ.get('proxy-password')
if not proxy_password:
proxy_password = os.environ.get('proxy_password')
if not proxy_username:
if '@' in proxy_settings:
protocol_and_proxy_auth = proxy_settings.split('@')[0].split(':')
if len(protocol_and_proxy_auth) == 3:
# 3 elements means we have [<protocol>, //<user>, <password>]
proxy_username = protocol_and_proxy_auth[1].lstrip('/')
proxy_password = protocol_and_proxy_auth[2]
elif len(protocol_and_proxy_auth) == 2:
# 2 elements means we have [<user>, <password>]
proxy_username = protocol_and_proxy_auth[0]
proxy_password = protocol_and_proxy_auth[1]
if proxy_username:
user_auth = base64.encodestring('%s:%s' % (proxy_username,
proxy_password))
return 'Basic %s\r\n' % (user_auth.strip())
else:
return ''
def _get_proxy_net_location(proxy_settings):
"""Returns proxy host and port.
Args:
proxy_settings: String from http_proxy or https_proxy environment variable.
Must be in the form of protocol://[username:password@]host:port
Returns:
String in the form of protocol://host:port
"""
if '@' in proxy_settings:
protocol = proxy_settings.split(':')[0]
netloc = proxy_settings.split('@')[1]
return '%s://%s' % (protocol, netloc)
else:
return proxy_settings
def _send_data_part(data, connection):
if isinstance(data, types.StringTypes):
connection.send(data)
return
# Check to see if data is a file-like object that has a read method.
elif hasattr(data, 'read'):
# Read the file and send it a chunk at a time.
while 1:
binarydata = data.read(100000)
if binarydata == '': break
connection.send(binarydata)
return
else:
# The data object was not a file.
# Try to convert to a string and send the data.
connection.send(str(data))
return

597
atom/http_core.py Normal file
View File

@ -0,0 +1,597 @@
#!/usr/bin/env python
#
# Copyright (C) 2009 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This module is used for version 2 of the Google Data APIs.
# TODO: add proxy handling.
__author__ = 'j.s@google.com (Jeff Scudder)'
import os
import StringIO
import urlparse
import urllib
import httplib
ssl = None
try:
import ssl
except ImportError:
pass
class Error(Exception):
pass
class UnknownSize(Error):
pass
class ProxyError(Error):
pass
MIME_BOUNDARY = 'END_OF_PART'
def get_headers(http_response):
"""Retrieves all HTTP headers from an HTTP response from the server.
This method is provided for backwards compatibility for Python2.2 and 2.3.
The httplib.HTTPResponse object in 2.2 and 2.3 does not have a getheaders
method so this function will use getheaders if available, but if not it
will retrieve a few using getheader.
"""
if hasattr(http_response, 'getheaders'):
return http_response.getheaders()
else:
headers = []
for header in (
'location', 'content-type', 'content-length', 'age', 'allow',
'cache-control', 'content-location', 'content-encoding', 'date',
'etag', 'expires', 'last-modified', 'pragma', 'server',
'set-cookie', 'transfer-encoding', 'vary', 'via', 'warning',
'www-authenticate', 'gdata-version'):
value = http_response.getheader(header, None)
if value is not None:
headers.append((header, value))
return headers
class HttpRequest(object):
"""Contains all of the parameters for an HTTP 1.1 request.
The HTTP headers are represented by a dictionary, and it is the
responsibility of the user to ensure that duplicate field names are combined
into one header value according to the rules in section 4.2 of RFC 2616.
"""
method = None
uri = None
def __init__(self, uri=None, method=None, headers=None):
"""Construct an HTTP request.
Args:
uri: The full path or partial path as a Uri object or a string.
method: The HTTP method for the request, examples include 'GET', 'POST',
etc.
headers: dict of strings The HTTP headers to include in the request.
"""
self.headers = headers or {}
self._body_parts = []
if method is not None:
self.method = method
if isinstance(uri, (str, unicode)):
uri = Uri.parse_uri(uri)
self.uri = uri or Uri()
def add_body_part(self, data, mime_type, size=None):
"""Adds data to the HTTP request body.
If more than one part is added, this is assumed to be a mime-multipart
request. This method is designed to create MIME 1.0 requests as specified
in RFC 1341.
Args:
data: str or a file-like object containing a part of the request body.
mime_type: str The MIME type describing the data
size: int Required if the data is a file like object. If the data is a
string, the size is calculated so this parameter is ignored.
"""
if isinstance(data, str):
size = len(data)
if size is None:
# TODO: support chunked transfer if some of the body is of unknown size.
raise UnknownSize('Each part of the body must have a known size.')
if 'Content-Length' in self.headers:
content_length = int(self.headers['Content-Length'])
else:
content_length = 0
# If this is the first part added to the body, then this is not a multipart
# request.
if len(self._body_parts) == 0:
self.headers['Content-Type'] = mime_type
content_length = size
self._body_parts.append(data)
elif len(self._body_parts) == 1:
# This is the first member in a mime-multipart request, so change the
# _body_parts list to indicate a multipart payload.
self._body_parts.insert(0, 'Media multipart posting')
boundary_string = '\r\n--%s\r\n' % (MIME_BOUNDARY,)
content_length += len(boundary_string) + size
self._body_parts.insert(1, boundary_string)
content_length += len('Media multipart posting')
# Put the content type of the first part of the body into the multipart
# payload.
original_type_string = 'Content-Type: %s\r\n\r\n' % (
self.headers['Content-Type'],)
self._body_parts.insert(2, original_type_string)
content_length += len(original_type_string)
boundary_string = '\r\n--%s\r\n' % (MIME_BOUNDARY,)
self._body_parts.append(boundary_string)
content_length += len(boundary_string)
# Change the headers to indicate this is now a mime multipart request.
self.headers['Content-Type'] = 'multipart/related; boundary="%s"' % (
MIME_BOUNDARY,)
self.headers['MIME-version'] = '1.0'
# Include the mime type of this part.
type_string = 'Content-Type: %s\r\n\r\n' % (mime_type)
self._body_parts.append(type_string)
content_length += len(type_string)
self._body_parts.append(data)
ending_boundary_string = '\r\n--%s--' % (MIME_BOUNDARY,)
self._body_parts.append(ending_boundary_string)
content_length += len(ending_boundary_string)
else:
# This is a mime multipart request.
boundary_string = '\r\n--%s\r\n' % (MIME_BOUNDARY,)
self._body_parts.insert(-1, boundary_string)
content_length += len(boundary_string) + size
# Include the mime type of this part.
type_string = 'Content-Type: %s\r\n\r\n' % (mime_type)
self._body_parts.insert(-1, type_string)
content_length += len(type_string)
self._body_parts.insert(-1, data)
self.headers['Content-Length'] = str(content_length)
# I could add an "append_to_body_part" method as well.
AddBodyPart = add_body_part
def add_form_inputs(self, form_data,
mime_type='application/x-www-form-urlencoded'):
"""Form-encodes and adds data to the request body.
Args:
form_data: dict or sequnce or two member tuples which contains the
form keys and values.
mime_type: str The MIME type of the form data being sent. Defaults
to 'application/x-www-form-urlencoded'.
"""
body = urllib.urlencode(form_data)
self.add_body_part(body, mime_type)
AddFormInputs = add_form_inputs
def _copy(self):
"""Creates a deep copy of this request."""
copied_uri = Uri(self.uri.scheme, self.uri.host, self.uri.port,
self.uri.path, self.uri.query.copy())
new_request = HttpRequest(uri=copied_uri, method=self.method,
headers=self.headers.copy())
new_request._body_parts = self._body_parts[:]
return new_request
def _dump(self):
"""Converts to a printable string for debugging purposes.
In order to preserve the request, it does not read from file-like objects
in the body.
"""
output = 'HTTP Request\n method: %s\n url: %s\n headers:\n' % (
self.method, str(self.uri))
for header, value in self.headers.iteritems():
output += ' %s: %s\n' % (header, value)
output += ' body sections:\n'
i = 0
for part in self._body_parts:
if isinstance(part, (str, unicode)):
output += ' %s: %s\n' % (i, part)
else:
output += ' %s: <file like object>\n' % i
i += 1
return output
def _apply_defaults(http_request):
if http_request.uri.scheme is None:
if http_request.uri.port == 443:
http_request.uri.scheme = 'https'
else:
http_request.uri.scheme = 'http'
class Uri(object):
"""A URI as used in HTTP 1.1"""
scheme = None
host = None
port = None
path = None
def __init__(self, scheme=None, host=None, port=None, path=None, query=None):
"""Constructor for a URI.
Args:
scheme: str This is usually 'http' or 'https'.
host: str The host name or IP address of the desired server.
post: int The server's port number.
path: str The path of the resource following the host. This begins with
a /, example: '/calendar/feeds/default/allcalendars/full'
query: dict of strings The URL query parameters. The keys and values are
both escaped so this dict should contain the unescaped values.
For example {'my key': 'val', 'second': '!!!'} will become
'?my+key=val&second=%21%21%21' which is appended to the path.
"""
self.query = query or {}
if scheme is not None:
self.scheme = scheme
if host is not None:
self.host = host
if port is not None:
self.port = port
if path:
self.path = path
def _get_query_string(self):
param_pairs = []
for key, value in self.query.iteritems():
param_pairs.append('='.join((urllib.quote_plus(key),
urllib.quote_plus(str(value)))))
return '&'.join(param_pairs)
def _get_relative_path(self):
"""Returns the path with the query parameters escaped and appended."""
param_string = self._get_query_string()
if self.path is None:
path = '/'
else:
path = self.path
if param_string:
return '?'.join([path, param_string])
else:
return path
def _to_string(self):
if self.scheme is None and self.port == 443:
scheme = 'https'
elif self.scheme is None:
scheme = 'http'
else:
scheme = self.scheme
if self.path is None:
path = '/'
else:
path = self.path
if self.port is None:
return '%s://%s%s' % (scheme, self.host, self._get_relative_path())
else:
return '%s://%s:%s%s' % (scheme, self.host, str(self.port),
self._get_relative_path())
def __str__(self):
return self._to_string()
def modify_request(self, http_request=None):
"""Sets HTTP request components based on the URI."""
if http_request is None:
http_request = HttpRequest()
if http_request.uri is None:
http_request.uri = Uri()
# Determine the correct scheme.
if self.scheme:
http_request.uri.scheme = self.scheme
if self.port:
http_request.uri.port = self.port
if self.host:
http_request.uri.host = self.host
# Set the relative uri path
if self.path:
http_request.uri.path = self.path
if self.query:
http_request.uri.query = self.query.copy()
return http_request
ModifyRequest = modify_request
def parse_uri(uri_string):
"""Creates a Uri object which corresponds to the URI string.
This method can accept partial URIs, but it will leave missing
members of the Uri unset.
"""
parts = urlparse.urlparse(uri_string)
uri = Uri()
if parts[0]:
uri.scheme = parts[0]
if parts[1]:
host_parts = parts[1].split(':')
if host_parts[0]:
uri.host = host_parts[0]
if len(host_parts) > 1:
uri.port = int(host_parts[1])
if parts[2]:
uri.path = parts[2]
if parts[4]:
param_pairs = parts[4].split('&')
for pair in param_pairs:
pair_parts = pair.split('=')
if len(pair_parts) > 1:
uri.query[urllib.unquote_plus(pair_parts[0])] = (
urllib.unquote_plus(pair_parts[1]))
elif len(pair_parts) == 1:
uri.query[urllib.unquote_plus(pair_parts[0])] = None
return uri
parse_uri = staticmethod(parse_uri)
ParseUri = parse_uri
parse_uri = Uri.parse_uri
ParseUri = Uri.parse_uri
class HttpResponse(object):
status = None
reason = None
_body = None
def __init__(self, status=None, reason=None, headers=None, body=None):
self._headers = headers or {}
if status is not None:
self.status = status
if reason is not None:
self.reason = reason
if body is not None:
if hasattr(body, 'read'):
self._body = body
else:
self._body = StringIO.StringIO(body)
def getheader(self, name, default=None):
if name in self._headers:
return self._headers[name]
else:
return default
def getheaders(self):
return self._headers
def read(self, amt=None):
if self._body is None:
return None
if not amt:
return self._body.read()
else:
return self._body.read(amt)
def _dump_response(http_response):
"""Converts to a string for printing debug messages.
Does not read the body since that may consume the content.
"""
output = 'HttpResponse\n status: %s\n reason: %s\n headers:' % (
http_response.status, http_response.reason)
headers = get_headers(http_response)
if isinstance(headers, dict):
for header, value in headers.iteritems():
output += ' %s: %s\n' % (header, value)
else:
for pair in headers:
output += ' %s: %s\n' % (pair[0], pair[1])
return output
class HttpClient(object):
"""Performs HTTP requests using httplib."""
debug = None
def request(self, http_request):
return self._http_request(http_request.method, http_request.uri,
http_request.headers, http_request._body_parts)
Request = request
def _get_connection(self, uri, headers=None):
"""Opens a socket connection to the server to set up an HTTP request.
Args:
uri: The full URL for the request as a Uri object.
headers: A dict of string pairs containing the HTTP headers for the
request.
"""
connection = None
if uri.scheme == 'https':
if not uri.port:
connection = httplib.HTTPSConnection(uri.host)
else:
connection = httplib.HTTPSConnection(uri.host, int(uri.port))
else:
if not uri.port:
connection = httplib.HTTPConnection(uri.host)
else:
connection = httplib.HTTPConnection(uri.host, int(uri.port))
return connection
def _http_request(self, method, uri, headers=None, body_parts=None):
"""Makes an HTTP request using httplib.
Args:
method: str example: 'GET', 'POST', 'PUT', 'DELETE', etc.
uri: str or atom.http_core.Uri
headers: dict of strings mapping to strings which will be sent as HTTP
headers in the request.
body_parts: list of strings, objects with a read method, or objects
which can be converted to strings using str. Each of these
will be sent in order as the body of the HTTP request.
"""
if isinstance(uri, (str, unicode)):
uri = Uri.parse_uri(uri)
connection = self._get_connection(uri, headers=headers)
if self.debug:
connection.debuglevel = 1
if connection.host != uri.host:
connection.putrequest(method, str(uri))
else:
connection.putrequest(method, uri._get_relative_path())
# Overcome a bug in Python 2.4 and 2.5
# httplib.HTTPConnection.putrequest adding
# HTTP request header 'Host: www.google.com:443' instead of
# 'Host: www.google.com', and thus resulting the error message
# 'Token invalid - AuthSub token has wrong scope' in the HTTP response.
if (uri.scheme == 'https' and int(uri.port or 443) == 443 and
hasattr(connection, '_buffer') and
isinstance(connection._buffer, list)):
header_line = 'Host: %s:443' % uri.host
replacement_header_line = 'Host: %s' % uri.host
try:
connection._buffer[connection._buffer.index(header_line)] = (
replacement_header_line)
except ValueError: # header_line missing from connection._buffer
pass
# Send the HTTP headers.
for header_name, value in headers.iteritems():
connection.putheader(header_name, value)
connection.endheaders()
# If there is data, send it in the request.
if body_parts and filter(lambda x: x != '', body_parts):
for part in body_parts:
_send_data_part(part, connection)
# Return the HTTP Response from the server.
return connection.getresponse()
def _send_data_part(data, connection):
if isinstance(data, (str, unicode)):
# I might want to just allow str, not unicode.
connection.send(data)
return
# Check to see if data is a file-like object that has a read method.
elif hasattr(data, 'read'):
# Read the file and send it a chunk at a time.
while 1:
binarydata = data.read(100000)
if binarydata == '': break
connection.send(binarydata)
return
else:
# The data object was not a file.
# Try to convert to a string and send the data.
connection.send(str(data))
return
class ProxiedHttpClient(HttpClient):
def _get_connection(self, uri, headers=None):
# Check to see if there are proxy settings required for this request.
proxy = None
if uri.scheme == 'https':
proxy = os.environ.get('https_proxy')
elif uri.scheme == 'http':
proxy = os.environ.get('http_proxy')
if not proxy:
return HttpClient._get_connection(self, uri, headers=headers)
# Now we have the URL of the appropriate proxy server.
# Get a username and password for the proxy if required.
proxy_auth = _get_proxy_auth()
if uri.scheme == 'https':
import socket
if proxy_auth:
proxy_auth = 'Proxy-authorization: %s' % proxy_auth
# Construct the proxy connect command.
port = uri.port
if not port:
port = 443
proxy_connect = 'CONNECT %s:%s HTTP/1.0\r\n' % (uri.host, port)
# Set the user agent to send to the proxy
user_agent = ''
if headers and 'User-Agent' in headers:
user_agent = 'User-Agent: %s\r\n' % (headers['User-Agent'])
proxy_pieces = '%s%s%s\r\n' % (proxy_connect, proxy_auth, user_agent)
# Find the proxy host and port.
proxy_uri = Uri.parse_uri(proxy)
if not proxy_uri.port:
proxy_uri.port = '80'
# Connect to the proxy server, very simple recv and error checking
p_sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
p_sock.connect((proxy_uri.host, int(proxy_uri.port)))
p_sock.sendall(proxy_pieces)
response = ''
# Wait for the full response.
while response.find("\r\n\r\n") == -1:
response += p_sock.recv(8192)
p_status = response.split()[1]
if p_status != str(200):
raise ProxyError('Error status=%s' % str(p_status))
# Trivial setup for ssl socket.
sslobj = None
if ssl is not None:
sslobj = ssl.wrap_socket(p_sock, None, None)
else:
sock_ssl = socket.ssl(p_sock, None, Nonesock_)
sslobj = httplib.FakeSocket(p_sock, sock_ssl)
# Initalize httplib and replace with the proxy socket.
connection = httplib.HTTPConnection(proxy_uri.host)
connection.sock = sslobj
return connection
elif uri.scheme == 'http':
proxy_uri = Uri.parse_uri(proxy)
if not proxy_uri.port:
proxy_uri.port = '80'
if proxy_auth:
headers['Proxy-Authorization'] = proxy_auth.strip()
return httplib.HTTPConnection(proxy_uri.host, int(proxy_uri.port))
return None
def _get_proxy_auth():
import base64
proxy_username = os.environ.get('proxy-username')
if not proxy_username:
proxy_username = os.environ.get('proxy_username')
proxy_password = os.environ.get('proxy-password')
if not proxy_password:
proxy_password = os.environ.get('proxy_password')
if proxy_username:
user_auth = base64.b64encode('%s:%s' % (proxy_username,
proxy_password))
return 'Basic %s\r\n' % (user_auth.strip())
else:
return ''

156
atom/http_interface.py Normal file
View File

@ -0,0 +1,156 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""This module provides a common interface for all HTTP requests.
HttpResponse: Represents the server's response to an HTTP request. Provides
an interface identical to httplib.HTTPResponse which is the response
expected from higher level classes which use HttpClient.request.
GenericHttpClient: Provides an interface (superclass) for an object
responsible for making HTTP requests. Subclasses of this object are
used in AtomService and GDataService to make requests to the server. By
changing the http_client member object, the AtomService is able to make
HTTP requests using different logic (for example, when running on
Google App Engine, the http_client makes requests using the App Engine
urlfetch API).
"""
__author__ = 'api.jscudder (Jeff Scudder)'
import StringIO
USER_AGENT = '%s GData-Python 2.0.14+20110902+custom_mods'
class Error(Exception):
pass
class UnparsableUrlObject(Error):
pass
class ContentLengthRequired(Error):
pass
class HttpResponse(object):
def __init__(self, body=None, status=None, reason=None, headers=None):
"""Constructor for an HttpResponse object.
HttpResponse represents the server's response to an HTTP request from
the client. The HttpClient.request method returns a httplib.HTTPResponse
object and this HttpResponse class is designed to mirror the interface
exposed by httplib.HTTPResponse.
Args:
body: A file like object, with a read() method. The body could also
be a string, and the constructor will wrap it so that
HttpResponse.read(self) will return the full string.
status: The HTTP status code as an int. Example: 200, 201, 404.
reason: The HTTP status message which follows the code. Example:
OK, Created, Not Found
headers: A dictionary containing the HTTP headers in the server's
response. A common header in the response is Content-Length.
"""
if body:
if hasattr(body, 'read'):
self._body = body
else:
self._body = StringIO.StringIO(body)
else:
self._body = None
if status is not None:
self.status = int(status)
else:
self.status = None
self.reason = reason
self._headers = headers or {}
def getheader(self, name, default=None):
if name in self._headers:
return self._headers[name]
else:
return default
def read(self, amt=None):
if not amt:
return self._body.read()
else:
return self._body.read(amt)
class GenericHttpClient(object):
debug = False
def __init__(self, http_client, headers=None):
"""
Args:
http_client: An object which provides a request method to make an HTTP
request. The request method in GenericHttpClient performs a
call-through to the contained HTTP client object.
headers: A dictionary containing HTTP headers which should be included
in every HTTP request. Common persistent headers include
'User-Agent'.
"""
self.http_client = http_client
self.headers = headers or {}
def request(self, operation, url, data=None, headers=None):
all_headers = self.headers.copy()
if headers:
all_headers.update(headers)
return self.http_client.request(operation, url, data=data,
headers=all_headers)
def get(self, url, headers=None):
return self.request('GET', url, headers=headers)
def post(self, url, data, headers=None):
return self.request('POST', url, data=data, headers=headers)
def put(self, url, data, headers=None):
return self.request('PUT', url, data=data, headers=headers)
def delete(self, url, headers=None):
return self.request('DELETE', url, headers=headers)
class GenericToken(object):
"""Represents an Authorization token to be added to HTTP requests.
Some Authorization headers included calculated fields (digital
signatures for example) which are based on the parameters of the HTTP
request. Therefore the token is responsible for signing the request
and adding the Authorization header.
"""
def perform_request(self, http_client, operation, url, data=None,
headers=None):
"""For the GenericToken, no Authorization token is set."""
return http_client.request(operation, url, data=data, headers=headers)
def valid_for_scope(self, url):
"""Tells the caller if the token authorizes access to the desired URL.
Since the generic token doesn't add an auth header, it is not valid for
any scope.
"""
return False

132
atom/mock_http.py Normal file
View File

@ -0,0 +1,132 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
__author__ = 'api.jscudder (Jeff Scudder)'
import atom.http_interface
import atom.url
class Error(Exception):
pass
class NoRecordingFound(Error):
pass
class MockRequest(object):
"""Holds parameters of an HTTP request for matching against future requests.
"""
def __init__(self, operation, url, data=None, headers=None):
self.operation = operation
if isinstance(url, (str, unicode)):
url = atom.url.parse_url(url)
self.url = url
self.data = data
self.headers = headers
class MockResponse(atom.http_interface.HttpResponse):
"""Simulates an httplib.HTTPResponse object."""
def __init__(self, body=None, status=None, reason=None, headers=None):
if body and hasattr(body, 'read'):
self.body = body.read()
else:
self.body = body
if status is not None:
self.status = int(status)
else:
self.status = None
self.reason = reason
self._headers = headers or {}
def read(self):
return self.body
class MockHttpClient(atom.http_interface.GenericHttpClient):
def __init__(self, headers=None, recordings=None, real_client=None):
"""An HttpClient which responds to request with stored data.
The request-response pairs are stored as tuples in a member list named
recordings.
The MockHttpClient can be switched from replay mode to record mode by
setting the real_client member to an instance of an HttpClient which will
make real HTTP requests and store the server's response in list of
recordings.
Args:
headers: dict containing HTTP headers which should be included in all
HTTP requests.
recordings: The initial recordings to be used for responses. This list
contains tuples in the form: (MockRequest, MockResponse)
real_client: An HttpClient which will make a real HTTP request. The
response will be converted into a MockResponse and stored in
recordings.
"""
self.recordings = recordings or []
self.real_client = real_client
self.headers = headers or {}
def add_response(self, response, operation, url, data=None, headers=None):
"""Adds a request-response pair to the recordings list.
After the recording is added, future matching requests will receive the
response.
Args:
response: MockResponse
operation: str
url: str
data: str, Currently the data is ignored when looking for matching
requests.
headers: dict of strings: Currently the headers are ignored when
looking for matching requests.
"""
request = MockRequest(operation, url, data=data, headers=headers)
self.recordings.append((request, response))
def request(self, operation, url, data=None, headers=None):
"""Returns a matching MockResponse from the recordings.
If the real_client is set, the request will be passed along and the
server's response will be added to the recordings and also returned.
If there is no match, a NoRecordingFound error will be raised.
"""
if self.real_client is None:
if isinstance(url, (str, unicode)):
url = atom.url.parse_url(url)
for recording in self.recordings:
if recording[0].operation == operation and recording[0].url == url:
return recording[1]
raise NoRecordingFound('No recodings found for %s %s' % (
operation, url))
else:
# There is a real HTTP client, so make the request, and record the
# response.
response = self.real_client.request(operation, url, data=data,
headers=headers)
# TODO: copy the headers
stored_response = MockResponse(body=response, status=response.status,
reason=response.reason)
self.add_response(stored_response, operation, url, data=data,
headers=headers)
return stored_response

323
atom/mock_http_core.py Normal file
View File

@ -0,0 +1,323 @@
#!/usr/bin/env python
#
# Copyright (C) 2009 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This module is used for version 2 of the Google Data APIs.
__author__ = 'j.s@google.com (Jeff Scudder)'
import StringIO
import pickle
import os.path
import tempfile
import atom.http_core
class Error(Exception):
pass
class NoRecordingFound(Error):
pass
class MockHttpClient(object):
debug = None
real_client = None
last_request_was_live = False
# The following members are used to construct the session cache temp file
# name.
# These are combined to form the file name
# /tmp/cache_prefix.cache_case_name.cache_test_name
cache_name_prefix = 'gdata_live_test'
cache_case_name = ''
cache_test_name = ''
def __init__(self, recordings=None, real_client=None):
self._recordings = recordings or []
if real_client is not None:
self.real_client = real_client
def add_response(self, http_request, status, reason, headers=None,
body=None):
response = MockHttpResponse(status, reason, headers, body)
# TODO Scrub the request and the response.
self._recordings.append((http_request._copy(), response))
AddResponse = add_response
def request(self, http_request):
"""Provide a recorded response, or record a response for replay.
If the real_client is set, the request will be made using the
real_client, and the response from the server will be recorded.
If the real_client is None (the default), this method will examine
the recordings and find the first which matches.
"""
request = http_request._copy()
_scrub_request(request)
if self.real_client is None:
self.last_request_was_live = False
for recording in self._recordings:
if _match_request(recording[0], request):
return recording[1]
else:
# Pass along the debug settings to the real client.
self.real_client.debug = self.debug
# Make an actual request since we can use the real HTTP client.
self.last_request_was_live = True
response = self.real_client.request(http_request)
scrubbed_response = _scrub_response(response)
self.add_response(request, scrubbed_response.status,
scrubbed_response.reason,
dict(atom.http_core.get_headers(scrubbed_response)),
scrubbed_response.read())
# Return the recording which we just added.
return self._recordings[-1][1]
raise NoRecordingFound('No recoding was found for request: %s %s' % (
request.method, str(request.uri)))
Request = request
def _save_recordings(self, filename):
recording_file = open(os.path.join(tempfile.gettempdir(), filename),
'wb')
pickle.dump(self._recordings, recording_file)
recording_file.close()
def _load_recordings(self, filename):
recording_file = open(os.path.join(tempfile.gettempdir(), filename),
'rb')
self._recordings = pickle.load(recording_file)
recording_file.close()
def _delete_recordings(self, filename):
full_path = os.path.join(tempfile.gettempdir(), filename)
if os.path.exists(full_path):
os.remove(full_path)
def _load_or_use_client(self, filename, http_client):
if os.path.exists(os.path.join(tempfile.gettempdir(), filename)):
self._load_recordings(filename)
else:
self.real_client = http_client
def use_cached_session(self, name=None, real_http_client=None):
"""Attempts to load recordings from a previous live request.
If a temp file with the recordings exists, then it is used to fulfill
requests. If the file does not exist, then a real client is used to
actually make the desired HTTP requests. Requests and responses are
recorded and will be written to the desired temprary cache file when
close_session is called.
Args:
name: str (optional) The file name of session file to be used. The file
is loaded from the temporary directory of this machine. If no name
is passed in, a default name will be constructed using the
cache_name_prefix, cache_case_name, and cache_test_name of this
object.
real_http_client: atom.http_core.HttpClient the real client to be used
if the cached recordings are not found. If the default
value is used, this will be an
atom.http_core.HttpClient.
"""
if real_http_client is None:
real_http_client = atom.http_core.HttpClient()
if name is None:
self._recordings_cache_name = self.get_cache_file_name()
else:
self._recordings_cache_name = name
self._load_or_use_client(self._recordings_cache_name, real_http_client)
def close_session(self):
"""Saves recordings in the temporary file named in use_cached_session."""
if self.real_client is not None:
self._save_recordings(self._recordings_cache_name)
def delete_session(self, name=None):
"""Removes recordings from a previous live request."""
if name is None:
self._delete_recordings(self._recordings_cache_name)
else:
self._delete_recordings(name)
def get_cache_file_name(self):
return '%s.%s.%s' % (self.cache_name_prefix, self.cache_case_name,
self.cache_test_name)
def _dump(self):
"""Provides debug information in a string."""
output = 'MockHttpClient\n real_client: %s\n cache file name: %s\n' % (
self.real_client, self.get_cache_file_name())
output += ' recordings:\n'
i = 0
for recording in self._recordings:
output += ' recording %i is for: %s %s\n' % (
i, recording[0].method, str(recording[0].uri))
i += 1
return output
def _match_request(http_request, stored_request):
"""Determines whether a request is similar enough to a stored request
to cause the stored response to be returned."""
# Check to see if the host names match.
if (http_request.uri.host is not None
and http_request.uri.host != stored_request.uri.host):
return False
# Check the request path in the URL (/feeds/private/full/x)
elif http_request.uri.path != stored_request.uri.path:
return False
# Check the method used in the request (GET, POST, etc.)
elif http_request.method != stored_request.method:
return False
# If there is a gsession ID in either request, make sure that it is matched
# exactly.
elif ('gsessionid' in http_request.uri.query
or 'gsessionid' in stored_request.uri.query):
if 'gsessionid' not in stored_request.uri.query:
return False
elif 'gsessionid' not in http_request.uri.query:
return False
elif (http_request.uri.query['gsessionid']
!= stored_request.uri.query['gsessionid']):
return False
# Ignores differences in the query params (?start-index=5&max-results=20),
# the body of the request, the port number, HTTP headers, just to name a
# few.
return True
def _scrub_request(http_request):
""" Removes email address and password from a client login request.
Since the mock server saves the request and response in plantext, sensitive
information like the password should be removed before saving the
recordings. At the moment only requests sent to a ClientLogin url are
scrubbed.
"""
if (http_request and http_request.uri and http_request.uri.path and
http_request.uri.path.endswith('ClientLogin')):
# Remove the email and password from a ClientLogin request.
http_request._body_parts = []
http_request.add_form_inputs(
{'form_data': 'client login request has been scrubbed'})
else:
# We can remove the body of the post from the recorded request, since
# the request body is not used when finding a matching recording.
http_request._body_parts = []
return http_request
def _scrub_response(http_response):
return http_response
class EchoHttpClient(object):
"""Sends the request data back in the response.
Used to check the formatting of the request as it was sent. Always responds
with a 200 OK, and some information from the HTTP request is returned in
special Echo-X headers in the response. The following headers are added
in the response:
'Echo-Host': The host name and port number to which the HTTP connection is
made. If no port was passed in, the header will contain
host:None.
'Echo-Uri': The path portion of the URL being requested. /example?x=1&y=2
'Echo-Scheme': The beginning of the URL, usually 'http' or 'https'
'Echo-Method': The HTTP method being used, 'GET', 'POST', 'PUT', etc.
"""
def request(self, http_request):
return self._http_request(http_request.uri, http_request.method,
http_request.headers, http_request._body_parts)
def _http_request(self, uri, method, headers=None, body_parts=None):
body = StringIO.StringIO()
response = atom.http_core.HttpResponse(status=200, reason='OK', body=body)
if headers is None:
response._headers = {}
else:
# Copy headers from the request to the response but convert values to
# strings. Server response headers always come in as strings, so an int
# should be converted to a corresponding string when echoing.
for header, value in headers.iteritems():
response._headers[header] = str(value)
response._headers['Echo-Host'] = '%s:%s' % (uri.host, str(uri.port))
response._headers['Echo-Uri'] = uri._get_relative_path()
response._headers['Echo-Scheme'] = uri.scheme
response._headers['Echo-Method'] = method
for part in body_parts:
if isinstance(part, str):
body.write(part)
elif hasattr(part, 'read'):
body.write(part.read())
body.seek(0)
return response
class SettableHttpClient(object):
"""An HTTP Client which responds with the data given in set_response."""
def __init__(self, status, reason, body, headers):
"""Configures the response for the server.
See set_response for details on the arguments to the constructor.
"""
self.set_response(status, reason, body, headers)
self.last_request = None
def set_response(self, status, reason, body, headers):
"""Determines the response which will be sent for each request.
Args:
status: An int for the HTTP status code, example: 200, 404, etc.
reason: String for the HTTP reason, example: OK, NOT FOUND, etc.
body: The body of the HTTP response as a string or a file-like
object (something with a read method).
headers: dict of strings containing the HTTP headers in the response.
"""
self.response = atom.http_core.HttpResponse(status=status, reason=reason,
body=body)
self.response._headers = headers.copy()
def request(self, http_request):
self.last_request = http_request
return self.response
class MockHttpResponse(atom.http_core.HttpResponse):
def __init__(self, status=None, reason=None, headers=None, body=None):
self._headers = headers or {}
if status is not None:
self.status = status
if reason is not None:
self.reason = reason
if body is not None:
# Instead of using a file-like object for the body, store as a string
# so that reads can be repeated.
if hasattr(body, 'read'):
self._body = body.read()
else:
self._body = body
def read(self):
return self._body

243
atom/mock_service.py Normal file
View File

@ -0,0 +1,243 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""MockService provides CRUD ops. for mocking calls to AtomPub services.
MockService: Exposes the publicly used methods of AtomService to provide
a mock interface which can be used in unit tests.
"""
import atom.service
import pickle
__author__ = 'api.jscudder (Jeffrey Scudder)'
# Recordings contains pairings of HTTP MockRequest objects with MockHttpResponse objects.
recordings = []
# If set, the mock service HttpRequest are actually made through this object.
real_request_handler = None
def ConcealValueWithSha(source):
import sha
return sha.new(source[:-5]).hexdigest()
def DumpRecordings(conceal_func=ConcealValueWithSha):
if conceal_func:
for recording_pair in recordings:
recording_pair[0].ConcealSecrets(conceal_func)
return pickle.dumps(recordings)
def LoadRecordings(recordings_file_or_string):
if isinstance(recordings_file_or_string, str):
atom.mock_service.recordings = pickle.loads(recordings_file_or_string)
elif hasattr(recordings_file_or_string, 'read'):
atom.mock_service.recordings = pickle.loads(
recordings_file_or_string.read())
def HttpRequest(service, operation, data, uri, extra_headers=None,
url_params=None, escape_params=True, content_type='application/atom+xml'):
"""Simulates an HTTP call to the server, makes an actual HTTP request if
real_request_handler is set.
This function operates in two different modes depending on if
real_request_handler is set or not. If real_request_handler is not set,
HttpRequest will look in this module's recordings list to find a response
which matches the parameters in the function call. If real_request_handler
is set, this function will call real_request_handler.HttpRequest, add the
response to the recordings list, and respond with the actual response.
Args:
service: atom.AtomService object which contains some of the parameters
needed to make the request. The following members are used to
construct the HTTP call: server (str), additional_headers (dict),
port (int), and ssl (bool).
operation: str The HTTP operation to be performed. This is usually one of
'GET', 'POST', 'PUT', or 'DELETE'
data: ElementTree, filestream, list of parts, or other object which can be
converted to a string.
Should be set to None when performing a GET or PUT.
If data is a file-like object which can be read, this method will read
a chunk of 100K bytes at a time and send them.
If the data is a list of parts to be sent, each part will be evaluated
and sent.
uri: The beginning of the URL to which the request should be sent.
Examples: '/', '/base/feeds/snippets',
'/m8/feeds/contacts/default/base'
extra_headers: dict of strings. HTTP headers which should be sent
in the request. These headers are in addition to those stored in
service.additional_headers.
url_params: dict of strings. Key value pairs to be added to the URL as
URL parameters. For example {'foo':'bar', 'test':'param'} will
become ?foo=bar&test=param.
escape_params: bool default True. If true, the keys and values in
url_params will be URL escaped when the form is constructed
(Special characters converted to %XX form.)
content_type: str The MIME type for the data being sent. Defaults to
'application/atom+xml', this is only used if data is set.
"""
full_uri = atom.service.BuildUri(uri, url_params, escape_params)
(server, port, ssl, uri) = atom.service.ProcessUrl(service, uri)
current_request = MockRequest(operation, full_uri, host=server, ssl=ssl,
data=data, extra_headers=extra_headers, url_params=url_params,
escape_params=escape_params, content_type=content_type)
# If the request handler is set, we should actually make the request using
# the request handler and record the response to replay later.
if real_request_handler:
response = real_request_handler.HttpRequest(service, operation, data, uri,
extra_headers=extra_headers, url_params=url_params,
escape_params=escape_params, content_type=content_type)
# TODO: need to copy the HTTP headers from the real response into the
# recorded_response.
recorded_response = MockHttpResponse(body=response.read(),
status=response.status, reason=response.reason)
# Insert a tuple which maps the request to the response object returned
# when making an HTTP call using the real_request_handler.
recordings.append((current_request, recorded_response))
return recorded_response
else:
# Look through available recordings to see if one matches the current
# request.
for request_response_pair in recordings:
if request_response_pair[0].IsMatch(current_request):
return request_response_pair[1]
return None
class MockRequest(object):
"""Represents a request made to an AtomPub server.
These objects are used to determine if a client request matches a recorded
HTTP request to determine what the mock server's response will be.
"""
def __init__(self, operation, uri, host=None, ssl=False, port=None,
data=None, extra_headers=None, url_params=None, escape_params=True,
content_type='application/atom+xml'):
"""Constructor for a MockRequest
Args:
operation: str One of 'GET', 'POST', 'PUT', or 'DELETE' this is the
HTTP operation requested on the resource.
uri: str The URL describing the resource to be modified or feed to be
retrieved. This should include the protocol (http/https) and the host
(aka domain). For example, these are some valud full_uris:
'http://example.com', 'https://www.google.com/accounts/ClientLogin'
host: str (optional) The server name which will be placed at the
beginning of the URL if the uri parameter does not begin with 'http'.
Examples include 'example.com', 'www.google.com', 'www.blogger.com'.
ssl: boolean (optional) If true, the request URL will begin with https
instead of http.
data: ElementTree, filestream, list of parts, or other object which can be
converted to a string. (optional)
Should be set to None when performing a GET or PUT.
If data is a file-like object which can be read, the constructor
will read the entire file into memory. If the data is a list of
parts to be sent, each part will be evaluated and stored.
extra_headers: dict (optional) HTTP headers included in the request.
url_params: dict (optional) Key value pairs which should be added to
the URL as URL parameters in the request. For example uri='/',
url_parameters={'foo':'1','bar':'2'} could become '/?foo=1&bar=2'.
escape_params: boolean (optional) Perform URL escaping on the keys and
values specified in url_params. Defaults to True.
content_type: str (optional) Provides the MIME type of the data being
sent.
"""
self.operation = operation
self.uri = _ConstructFullUrlBase(uri, host=host, ssl=ssl)
self.data = data
self.extra_headers = extra_headers
self.url_params = url_params or {}
self.escape_params = escape_params
self.content_type = content_type
def ConcealSecrets(self, conceal_func):
"""Conceal secret data in this request."""
if self.extra_headers.has_key('Authorization'):
self.extra_headers['Authorization'] = conceal_func(
self.extra_headers['Authorization'])
def IsMatch(self, other_request):
"""Check to see if the other_request is equivalent to this request.
Used to determine if a recording matches an incoming request so that a
recorded response should be sent to the client.
The matching is not exact, only the operation and URL are examined
currently.
Args:
other_request: MockRequest The request which we want to check this
(self) MockRequest against to see if they are equivalent.
"""
# More accurate matching logic will likely be required.
return (self.operation == other_request.operation and self.uri ==
other_request.uri)
def _ConstructFullUrlBase(uri, host=None, ssl=False):
"""Puts URL components into the form http(s)://full.host.strinf/uri/path
Used to construct a roughly canonical URL so that URLs which begin with
'http://example.com/' can be compared to a uri of '/' when the host is
set to 'example.com'
If the uri contains 'http://host' already, the host and ssl parameters
are ignored.
Args:
uri: str The path component of the URL, examples include '/'
host: str (optional) The host name which should prepend the URL. Example:
'example.com'
ssl: boolean (optional) If true, the returned URL will begin with https
instead of http.
Returns:
String which has the form http(s)://example.com/uri/string/contents
"""
if uri.startswith('http'):
return uri
if ssl:
return 'https://%s%s' % (host, uri)
else:
return 'http://%s%s' % (host, uri)
class MockHttpResponse(object):
"""Returned from MockService crud methods as the server's response."""
def __init__(self, body=None, status=None, reason=None, headers=None):
"""Construct a mock HTTPResponse and set members.
Args:
body: str (optional) The HTTP body of the server's response.
status: int (optional)
reason: str (optional)
headers: dict (optional)
"""
self.body = body
self.status = status
self.reason = reason
self.headers = headers or {}
def read(self):
return self.body
def getheader(self, header_name):
return self.headers[header_name]

745
atom/service.py Normal file
View File

@ -0,0 +1,745 @@
#!/usr/bin/python
#
# Copyright (C) 2006, 2007, 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""AtomService provides CRUD ops. in line with the Atom Publishing Protocol.
AtomService: Encapsulates the ability to perform insert, update and delete
operations with the Atom Publishing Protocol on which GData is
based. An instance can perform query, insertion, deletion, and
update.
HttpRequest: Function that performs a GET, POST, PUT, or DELETE HTTP request
to the specified end point. An AtomService object or a subclass can be
used to specify information about the request.
"""
__author__ = 'api.jscudder (Jeff Scudder)'
import atom.http_interface
import atom.url
import atom.http
import atom.token_store
import os
import types
import httplib
import urllib
import re
import base64
import socket
import warnings
try:
from xml.etree import cElementTree as ElementTree
except ImportError:
try:
import cElementTree as ElementTree
except ImportError:
try:
from xml.etree import ElementTree
except ImportError:
from elementtree import ElementTree
import atom
class AtomService(object):
"""Performs Atom Publishing Protocol CRUD operations.
The AtomService contains methods to perform HTTP CRUD operations.
"""
# Default values for members
port = 80
ssl = False
# Set the current_token to force the AtomService to use this token
# instead of searching for an appropriate token in the token_store.
current_token = None
auto_store_tokens = True
auto_set_current_token = True
def _get_override_token(self):
return self.current_token
def _set_override_token(self, token):
self.current_token = token
override_token = property(_get_override_token, _set_override_token)
#@atom.v1_deprecated('Please use atom.client.AtomPubClient instead.')
def __init__(self, server=None, additional_headers=None,
application_name='', http_client=None, token_store=None):
"""Creates a new AtomService client.
Args:
server: string (optional) The start of a URL for the server
to which all operations should be directed. Example:
'www.google.com'
additional_headers: dict (optional) Any additional HTTP headers which
should be included with CRUD operations.
http_client: An object responsible for making HTTP requests using a
request method. If none is provided, a new instance of
atom.http.ProxiedHttpClient will be used.
token_store: Keeps a collection of authorization tokens which can be
applied to requests for a specific URLs. Critical methods are
find_token based on a URL (atom.url.Url or a string), add_token,
and remove_token.
"""
self.http_client = http_client or atom.http.ProxiedHttpClient()
self.token_store = token_store or atom.token_store.TokenStore()
self.server = server
self.additional_headers = additional_headers or {}
self.additional_headers['User-Agent'] = atom.http_interface.USER_AGENT % (
application_name,)
# If debug is True, the HTTPConnection will display debug information
self._set_debug(False)
__init__ = atom.v1_deprecated(
'Please use atom.client.AtomPubClient instead.')(
__init__)
def _get_debug(self):
return self.http_client.debug
def _set_debug(self, value):
self.http_client.debug = value
debug = property(_get_debug, _set_debug,
doc='If True, HTTP debug information is printed.')
def use_basic_auth(self, username, password, scopes=None):
if username is not None and password is not None:
if scopes is None:
scopes = [atom.token_store.SCOPE_ALL]
base_64_string = base64.encodestring('%s:%s' % (username, password))
token = BasicAuthToken('Basic %s' % base_64_string.strip(),
scopes=[atom.token_store.SCOPE_ALL])
if self.auto_set_current_token:
self.current_token = token
if self.auto_store_tokens:
return self.token_store.add_token(token)
return True
return False
def UseBasicAuth(self, username, password, for_proxy=False):
"""Sets an Authenticaiton: Basic HTTP header containing plaintext.
Deprecated, use use_basic_auth instead.
The username and password are base64 encoded and added to an HTTP header
which will be included in each request. Note that your username and
password are sent in plaintext.
Args:
username: str
password: str
"""
self.use_basic_auth(username, password)
#@atom.v1_deprecated('Please use atom.client.AtomPubClient for requests.')
def request(self, operation, url, data=None, headers=None,
url_params=None):
if isinstance(url, (str, unicode)):
if url.startswith('http:') and self.ssl:
# Force all requests to be https if self.ssl is True.
url = atom.url.parse_url('https:' + url[5:])
elif not url.startswith('http') and self.ssl:
url = atom.url.parse_url('https://%s%s' % (self.server, url))
elif not url.startswith('http'):
url = atom.url.parse_url('http://%s%s' % (self.server, url))
else:
url = atom.url.parse_url(url)
if url_params:
for name, value in url_params.iteritems():
url.params[name] = value
all_headers = self.additional_headers.copy()
if headers:
all_headers.update(headers)
if isinstance(data, types.StringTypes):
data = data.encode('utf-8')
# If the list of headers does not include a Content-Length, attempt to
# calculate it based on the data object.
if data and 'Content-Length' not in all_headers:
content_length = CalculateDataLength(data)
if content_length:
all_headers['Content-Length'] = str(content_length)
all_headers['GData-Version'] = '2.0'
# Find an Authorization token for this URL if one is available.
if self.override_token:
auth_token = self.override_token
else:
auth_token = self.token_store.find_token(url)
return auth_token.perform_request(self.http_client, operation, url,
data=data, headers=all_headers)
request = atom.v1_deprecated(
'Please use atom.client.AtomPubClient for requests.')(
request)
# CRUD operations
def Get(self, uri, extra_headers=None, url_params=None, escape_params=True):
"""Query the APP server with the given URI
The uri is the portion of the URI after the server value
(server example: 'www.google.com').
Example use:
To perform a query against Google Base, set the server to
'base.google.com' and set the uri to '/base/feeds/...', where ... is
your query. For example, to find snippets for all digital cameras uri
should be set to: '/base/feeds/snippets?bq=digital+camera'
Args:
uri: string The query in the form of a URI. Example:
'/base/feeds/snippets?bq=digital+camera'.
extra_headers: dicty (optional) Extra HTTP headers to be included
in the GET request. These headers are in addition to
those stored in the client's additional_headers property.
The client automatically sets the Content-Type and
Authorization headers.
url_params: dict (optional) Additional URL parameters to be included
in the query. These are translated into query arguments
in the form '&dict_key=value&...'.
Example: {'max-results': '250'} becomes &max-results=250
escape_params: boolean (optional) If false, the calling code has already
ensured that the query will form a valid URL (all
reserved characters have been escaped). If true, this
method will escape the query and any URL parameters
provided.
Returns:
httplib.HTTPResponse The server's response to the GET request.
"""
return self.request('GET', uri, data=None, headers=extra_headers,
url_params=url_params)
def Post(self, data, uri, extra_headers=None, url_params=None,
escape_params=True, content_type='application/atom+xml; charset=UTF-8'):
"""Insert data into an APP server at the given URI.
Args:
data: string, ElementTree._Element, or something with a __str__ method
The XML to be sent to the uri.
uri: string The location (feed) to which the data should be inserted.
Example: '/base/feeds/items'.
extra_headers: dict (optional) HTTP headers which are to be included.
The client automatically sets the Content-Type,
Authorization, and Content-Length headers.
url_params: dict (optional) Additional URL parameters to be included
in the URI. These are translated into query arguments
in the form '&dict_key=value&...'.
Example: {'max-results': '250'} becomes &max-results=250
escape_params: boolean (optional) If false, the calling code has already
ensured that the query will form a valid URL (all
reserved characters have been escaped). If true, this
method will escape the query and any URL parameters
provided.
Returns:
httplib.HTTPResponse Server's response to the POST request.
"""
if extra_headers is None:
extra_headers = {}
if content_type:
extra_headers['Content-Type'] = content_type
return self.request('POST', uri, data=data, headers=extra_headers,
url_params=url_params)
def Put(self, data, uri, extra_headers=None, url_params=None,
escape_params=True, content_type='application/atom+xml; charset=UTF-8'):
"""Updates an entry at the given URI.
Args:
data: string, ElementTree._Element, or xml_wrapper.ElementWrapper The
XML containing the updated data.
uri: string A URI indicating entry to which the update will be applied.
Example: '/base/feeds/items/ITEM-ID'
extra_headers: dict (optional) HTTP headers which are to be included.
The client automatically sets the Content-Type,
Authorization, and Content-Length headers.
url_params: dict (optional) Additional URL parameters to be included
in the URI. These are translated into query arguments
in the form '&dict_key=value&...'.
Example: {'max-results': '250'} becomes &max-results=250
escape_params: boolean (optional) If false, the calling code has already
ensured that the query will form a valid URL (all
reserved characters have been escaped). If true, this
method will escape the query and any URL parameters
provided.
Returns:
httplib.HTTPResponse Server's response to the PUT request.
"""
if extra_headers is None:
extra_headers = {}
if content_type:
extra_headers['Content-Type'] = content_type
return self.request('PUT', uri, data=data, headers=extra_headers,
url_params=url_params)
def Delete(self, uri, extra_headers=None, url_params=None,
escape_params=True):
"""Deletes the entry at the given URI.
Args:
uri: string The URI of the entry to be deleted. Example:
'/base/feeds/items/ITEM-ID'
extra_headers: dict (optional) HTTP headers which are to be included.
The client automatically sets the Content-Type and
Authorization headers.
url_params: dict (optional) Additional URL parameters to be included
in the URI. These are translated into query arguments
in the form '&dict_key=value&...'.
Example: {'max-results': '250'} becomes &max-results=250
escape_params: boolean (optional) If false, the calling code has already
ensured that the query will form a valid URL (all
reserved characters have been escaped). If true, this
method will escape the query and any URL parameters
provided.
Returns:
httplib.HTTPResponse Server's response to the DELETE request.
"""
return self.request('DELETE', uri, data=None, headers=extra_headers,
url_params=url_params)
class BasicAuthToken(atom.http_interface.GenericToken):
def __init__(self, auth_header, scopes=None):
"""Creates a token used to add Basic Auth headers to HTTP requests.
Args:
auth_header: str The value for the Authorization header.
scopes: list of str or atom.url.Url specifying the beginnings of URLs
for which this token can be used. For example, if scopes contains
'http://example.com/foo', then this token can be used for a request to
'http://example.com/foo/bar' but it cannot be used for a request to
'http://example.com/baz'
"""
self.auth_header = auth_header
self.scopes = scopes or []
def perform_request(self, http_client, operation, url, data=None,
headers=None):
"""Sets the Authorization header to the basic auth string."""
if headers is None:
headers = {'Authorization':self.auth_header}
else:
headers['Authorization'] = self.auth_header
return http_client.request(operation, url, data=data, headers=headers)
def __str__(self):
return self.auth_header
def valid_for_scope(self, url):
"""Tells the caller if the token authorizes access to the desired URL.
"""
if isinstance(url, (str, unicode)):
url = atom.url.parse_url(url)
for scope in self.scopes:
if scope == atom.token_store.SCOPE_ALL:
return True
if isinstance(scope, (str, unicode)):
scope = atom.url.parse_url(scope)
if scope == url:
return True
# Check the host and the path, but ignore the port and protocol.
elif scope.host == url.host and not scope.path:
return True
elif scope.host == url.host and scope.path and not url.path:
continue
elif scope.host == url.host and url.path.startswith(scope.path):
return True
return False
def PrepareConnection(service, full_uri):
"""Opens a connection to the server based on the full URI.
This method is deprecated, instead use atom.http.HttpClient.request.
Examines the target URI and the proxy settings, which are set as
environment variables, to open a connection with the server. This
connection is used to make an HTTP request.
Args:
service: atom.AtomService or a subclass. It must have a server string which
represents the server host to which the request should be made. It may also
have a dictionary of additional_headers to send in the HTTP request.
full_uri: str Which is the target relative (lacks protocol and host) or
absolute URL to be opened. Example:
'https://www.google.com/accounts/ClientLogin' or
'base/feeds/snippets' where the server is set to www.google.com.
Returns:
A tuple containing the httplib.HTTPConnection and the full_uri for the
request.
"""
deprecation('calling deprecated function PrepareConnection')
(server, port, ssl, partial_uri) = ProcessUrl(service, full_uri)
if ssl:
# destination is https
proxy = os.environ.get('https_proxy')
if proxy:
(p_server, p_port, p_ssl, p_uri) = ProcessUrl(service, proxy, True)
proxy_username = os.environ.get('proxy-username')
if not proxy_username:
proxy_username = os.environ.get('proxy_username')
proxy_password = os.environ.get('proxy-password')
if not proxy_password:
proxy_password = os.environ.get('proxy_password')
if proxy_username:
user_auth = base64.encodestring('%s:%s' % (proxy_username,
proxy_password))
proxy_authorization = ('Proxy-authorization: Basic %s\r\n' % (
user_auth.strip()))
else:
proxy_authorization = ''
proxy_connect = 'CONNECT %s:%s HTTP/1.0\r\n' % (server, port)
user_agent = 'User-Agent: %s\r\n' % (
service.additional_headers['User-Agent'])
proxy_pieces = (proxy_connect + proxy_authorization + user_agent
+ '\r\n')
#now connect, very simple recv and error checking
p_sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
p_sock.connect((p_server,p_port))
p_sock.sendall(proxy_pieces)
response = ''
# Wait for the full response.
while response.find("\r\n\r\n") == -1:
response += p_sock.recv(8192)
p_status=response.split()[1]
if p_status!=str(200):
raise atom.http.ProxyError('Error status=%s' % p_status)
# Trivial setup for ssl socket.
ssl = socket.ssl(p_sock, None, None)
fake_sock = httplib.FakeSocket(p_sock, ssl)
# Initalize httplib and replace with the proxy socket.
connection = httplib.HTTPConnection(server)
connection.sock=fake_sock
full_uri = partial_uri
else:
connection = httplib.HTTPSConnection(server, port)
full_uri = partial_uri
else:
# destination is http
proxy = os.environ.get('http_proxy')
if proxy:
(p_server, p_port, p_ssl, p_uri) = ProcessUrl(service.server, proxy, True)
proxy_username = os.environ.get('proxy-username')
if not proxy_username:
proxy_username = os.environ.get('proxy_username')
proxy_password = os.environ.get('proxy-password')
if not proxy_password:
proxy_password = os.environ.get('proxy_password')
if proxy_username:
UseBasicAuth(service, proxy_username, proxy_password, True)
connection = httplib.HTTPConnection(p_server, p_port)
if not full_uri.startswith("http://"):
if full_uri.startswith("/"):
full_uri = "http://%s%s" % (service.server, full_uri)
else:
full_uri = "http://%s/%s" % (service.server, full_uri)
else:
connection = httplib.HTTPConnection(server, port)
full_uri = partial_uri
return (connection, full_uri)
def UseBasicAuth(service, username, password, for_proxy=False):
"""Sets an Authenticaiton: Basic HTTP header containing plaintext.
Deprecated, use AtomService.use_basic_auth insread.
The username and password are base64 encoded and added to an HTTP header
which will be included in each request. Note that your username and
password are sent in plaintext. The auth header is added to the
additional_headers dictionary in the service object.
Args:
service: atom.AtomService or a subclass which has an
additional_headers dict as a member.
username: str
password: str
"""
deprecation('calling deprecated function UseBasicAuth')
base_64_string = base64.encodestring('%s:%s' % (username, password))
base_64_string = base_64_string.strip()
if for_proxy:
header_name = 'Proxy-Authorization'
else:
header_name = 'Authorization'
service.additional_headers[header_name] = 'Basic %s' % (base_64_string,)
def ProcessUrl(service, url, for_proxy=False):
"""Processes a passed URL. If the URL does not begin with https?, then
the default value for server is used
This method is deprecated, use atom.url.parse_url instead.
"""
if not isinstance(url, atom.url.Url):
url = atom.url.parse_url(url)
server = url.host
ssl = False
port = 80
if not server:
if hasattr(service, 'server'):
server = service.server
else:
server = service
if not url.protocol and hasattr(service, 'ssl'):
ssl = service.ssl
if hasattr(service, 'port'):
port = service.port
else:
if url.protocol == 'https':
ssl = True
elif url.protocol == 'http':
ssl = False
if url.port:
port = int(url.port)
elif port == 80 and ssl:
port = 443
return (server, port, ssl, url.get_request_uri())
def DictionaryToParamList(url_parameters, escape_params=True):
"""Convert a dictionary of URL arguments into a URL parameter string.
This function is deprcated, use atom.url.Url instead.
Args:
url_parameters: The dictionaty of key-value pairs which will be converted
into URL parameters. For example,
{'dry-run': 'true', 'foo': 'bar'}
will become ['dry-run=true', 'foo=bar'].
Returns:
A list which contains a string for each key-value pair. The strings are
ready to be incorporated into a URL by using '&'.join([] + parameter_list)
"""
# Choose which function to use when modifying the query and parameters.
# Use quote_plus when escape_params is true.
transform_op = [str, urllib.quote_plus][bool(escape_params)]
# Create a list of tuples containing the escaped version of the
# parameter-value pairs.
parameter_tuples = [(transform_op(param), transform_op(value))
for param, value in (url_parameters or {}).items()]
# Turn parameter-value tuples into a list of strings in the form
# 'PARAMETER=VALUE'.
return ['='.join(x) for x in parameter_tuples]
def BuildUri(uri, url_params=None, escape_params=True):
"""Converts a uri string and a collection of parameters into a URI.
This function is deprcated, use atom.url.Url instead.
Args:
uri: string
url_params: dict (optional)
escape_params: boolean (optional)
uri: string The start of the desired URI. This string can alrady contain
URL parameters. Examples: '/base/feeds/snippets',
'/base/feeds/snippets?bq=digital+camera'
url_parameters: dict (optional) Additional URL parameters to be included
in the query. These are translated into query arguments
in the form '&dict_key=value&...'.
Example: {'max-results': '250'} becomes &max-results=250
escape_params: boolean (optional) If false, the calling code has already
ensured that the query will form a valid URL (all
reserved characters have been escaped). If true, this
method will escape the query and any URL parameters
provided.
Returns:
string The URI consisting of the escaped URL parameters appended to the
initial uri string.
"""
# Prepare URL parameters for inclusion into the GET request.
parameter_list = DictionaryToParamList(url_params, escape_params)
# Append the URL parameters to the URL.
if parameter_list:
if uri.find('?') != -1:
# If there are already URL parameters in the uri string, add the
# parameters after a new & character.
full_uri = '&'.join([uri] + parameter_list)
else:
# The uri string did not have any URL parameters (no ? character)
# so put a ? between the uri and URL parameters.
full_uri = '%s%s' % (uri, '?%s' % ('&'.join([] + parameter_list)))
else:
full_uri = uri
return full_uri
def HttpRequest(service, operation, data, uri, extra_headers=None,
url_params=None, escape_params=True, content_type='application/atom+xml; charset=UTF-8'):
"""Performs an HTTP call to the server, supports GET, POST, PUT, and DELETE.
This method is deprecated, use atom.http.HttpClient.request instead.
Usage example, perform and HTTP GET on http://www.google.com/:
import atom.service
client = atom.service.AtomService()
http_response = client.Get('http://www.google.com/')
or you could set the client.server to 'www.google.com' and use the
following:
client.server = 'www.google.com'
http_response = client.Get('/')
Args:
service: atom.AtomService object which contains some of the parameters
needed to make the request. The following members are used to
construct the HTTP call: server (str), additional_headers (dict),
port (int), and ssl (bool).
operation: str The HTTP operation to be performed. This is usually one of
'GET', 'POST', 'PUT', or 'DELETE'
data: ElementTree, filestream, list of parts, or other object which can be
converted to a string.
Should be set to None when performing a GET or PUT.
If data is a file-like object which can be read, this method will read
a chunk of 100K bytes at a time and send them.
If the data is a list of parts to be sent, each part will be evaluated
and sent.
uri: The beginning of the URL to which the request should be sent.
Examples: '/', '/base/feeds/snippets',
'/m8/feeds/contacts/default/base'
extra_headers: dict of strings. HTTP headers which should be sent
in the request. These headers are in addition to those stored in
service.additional_headers.
url_params: dict of strings. Key value pairs to be added to the URL as
URL parameters. For example {'foo':'bar', 'test':'param'} will
become ?foo=bar&test=param.
escape_params: bool default True. If true, the keys and values in
url_params will be URL escaped when the form is constructed
(Special characters converted to %XX form.)
content_type: str The MIME type for the data being sent. Defaults to
'application/atom+xml', this is only used if data is set.
"""
deprecation('call to deprecated function HttpRequest')
full_uri = BuildUri(uri, url_params, escape_params)
(connection, full_uri) = PrepareConnection(service, full_uri)
if extra_headers is None:
extra_headers = {}
# Turn on debug mode if the debug member is set.
if service.debug:
connection.debuglevel = 1
connection.putrequest(operation, full_uri)
# If the list of headers does not include a Content-Length, attempt to
# calculate it based on the data object.
if (data and not service.additional_headers.has_key('Content-Length') and
not extra_headers.has_key('Content-Length')):
content_length = CalculateDataLength(data)
if content_length:
extra_headers['Content-Length'] = str(content_length)
if content_type:
extra_headers['Content-Type'] = content_type
# Send the HTTP headers.
if isinstance(service.additional_headers, dict):
for header in service.additional_headers:
connection.putheader(header, service.additional_headers[header])
if isinstance(extra_headers, dict):
for header in extra_headers:
connection.putheader(header, extra_headers[header])
connection.endheaders()
# If there is data, send it in the request.
if data:
if isinstance(data, list):
for data_part in data:
__SendDataPart(data_part, connection)
else:
__SendDataPart(data, connection)
# Return the HTTP Response from the server.
return connection.getresponse()
def __SendDataPart(data, connection):
"""This method is deprecated, use atom.http._send_data_part"""
deprecated('call to deprecated function __SendDataPart')
if isinstance(data, str):
#TODO add handling for unicode.
connection.send(data)
return
elif ElementTree.iselement(data):
connection.send(ElementTree.tostring(data))
return
# Check to see if data is a file-like object that has a read method.
elif hasattr(data, 'read'):
# Read the file and send it a chunk at a time.
while 1:
binarydata = data.read(100000)
if binarydata == '': break
connection.send(binarydata)
return
else:
# The data object was not a file.
# Try to convert to a string and send the data.
connection.send(str(data))
return
def CalculateDataLength(data):
"""Attempts to determine the length of the data to send.
This method will respond with a length only if the data is a string or
and ElementTree element.
Args:
data: object If this is not a string or ElementTree element this funtion
will return None.
"""
if isinstance(data, str):
return len(data)
elif isinstance(data, list):
return None
elif ElementTree.iselement(data):
return len(ElementTree.tostring(data))
elif hasattr(data, 'read'):
# If this is a file-like object, don't try to guess the length.
return None
else:
return len(str(data))
def deprecation(message):
warnings.warn(message, DeprecationWarning, stacklevel=2)

117
atom/token_store.py Normal file
View File

@ -0,0 +1,117 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""This module provides a TokenStore class which is designed to manage
auth tokens required for different services.
Each token is valid for a set of scopes which is the start of a URL. An HTTP
client will use a token store to find a valid Authorization header to send
in requests to the specified URL. If the HTTP client determines that a token
has expired or been revoked, it can remove the token from the store so that
it will not be used in future requests.
"""
__author__ = 'api.jscudder (Jeff Scudder)'
import atom.http_interface
import atom.url
SCOPE_ALL = 'http'
class TokenStore(object):
"""Manages Authorization tokens which will be sent in HTTP headers."""
def __init__(self, scoped_tokens=None):
self._tokens = scoped_tokens or {}
def add_token(self, token):
"""Adds a new token to the store (replaces tokens with the same scope).
Args:
token: A subclass of http_interface.GenericToken. The token object is
responsible for adding the Authorization header to the HTTP request.
The scopes defined in the token are used to determine if the token
is valid for a requested scope when find_token is called.
Returns:
True if the token was added, False if the token was not added becase
no scopes were provided.
"""
if not hasattr(token, 'scopes') or not token.scopes:
return False
for scope in token.scopes:
self._tokens[str(scope)] = token
return True
def find_token(self, url):
"""Selects an Authorization header token which can be used for the URL.
Args:
url: str or atom.url.Url or a list containing the same.
The URL which is going to be requested. All
tokens are examined to see if any scopes begin match the beginning
of the URL. The first match found is returned.
Returns:
The token object which should execute the HTTP request. If there was
no token for the url (the url did not begin with any of the token
scopes available), then the atom.http_interface.GenericToken will be
returned because the GenericToken calls through to the http client
without adding an Authorization header.
"""
if url is None:
return None
if isinstance(url, (str, unicode)):
url = atom.url.parse_url(url)
if url in self._tokens:
token = self._tokens[url]
if token.valid_for_scope(url):
return token
else:
del self._tokens[url]
for scope, token in self._tokens.iteritems():
if token.valid_for_scope(url):
return token
return atom.http_interface.GenericToken()
def remove_token(self, token):
"""Removes the token from the token_store.
This method is used when a token is determined to be invalid. If the
token was found by find_token, but resulted in a 401 or 403 error stating
that the token was invlid, then the token should be removed to prevent
future use.
Returns:
True if a token was found and then removed from the token
store. False if the token was not in the TokenStore.
"""
token_found = False
scopes_to_delete = []
for scope, stored_token in self._tokens.iteritems():
if stored_token == token:
scopes_to_delete.append(scope)
token_found = True
for scope in scopes_to_delete:
del self._tokens[scope]
return token_found
def remove_all_tokens(self):
self._tokens = {}

139
atom/url.py Normal file
View File

@ -0,0 +1,139 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
__author__ = 'api.jscudder (Jeff Scudder)'
import urlparse
import urllib
DEFAULT_PROTOCOL = 'http'
DEFAULT_PORT = 80
def parse_url(url_string):
"""Creates a Url object which corresponds to the URL string.
This method can accept partial URLs, but it will leave missing
members of the Url unset.
"""
parts = urlparse.urlparse(url_string)
url = Url()
if parts[0]:
url.protocol = parts[0]
if parts[1]:
host_parts = parts[1].split(':')
if host_parts[0]:
url.host = host_parts[0]
if len(host_parts) > 1:
url.port = host_parts[1]
if parts[2]:
url.path = parts[2]
if parts[4]:
param_pairs = parts[4].split('&')
for pair in param_pairs:
pair_parts = pair.split('=')
if len(pair_parts) > 1:
url.params[urllib.unquote_plus(pair_parts[0])] = (
urllib.unquote_plus(pair_parts[1]))
elif len(pair_parts) == 1:
url.params[urllib.unquote_plus(pair_parts[0])] = None
return url
class Url(object):
"""Represents a URL and implements comparison logic.
URL strings which are not identical can still be equivalent, so this object
provides a better interface for comparing and manipulating URLs than
strings. URL parameters are represented as a dictionary of strings, and
defaults are used for the protocol (http) and port (80) if not provided.
"""
def __init__(self, protocol=None, host=None, port=None, path=None,
params=None):
self.protocol = protocol
self.host = host
self.port = port
self.path = path
self.params = params or {}
def to_string(self):
url_parts = ['', '', '', '', '', '']
if self.protocol:
url_parts[0] = self.protocol
if self.host:
if self.port:
url_parts[1] = ':'.join((self.host, str(self.port)))
else:
url_parts[1] = self.host
if self.path:
url_parts[2] = self.path
if self.params:
url_parts[4] = self.get_param_string()
return urlparse.urlunparse(url_parts)
def get_param_string(self):
param_pairs = []
for key, value in self.params.iteritems():
param_pairs.append('='.join((urllib.quote_plus(key),
urllib.quote_plus(str(value)))))
return '&'.join(param_pairs)
def get_request_uri(self):
"""Returns the path with the parameters escaped and appended."""
param_string = self.get_param_string()
if param_string:
return '?'.join([self.path, param_string])
else:
return self.path
def __cmp__(self, other):
if not isinstance(other, Url):
return cmp(self.to_string(), str(other))
difference = 0
# Compare the protocol
if self.protocol and other.protocol:
difference = cmp(self.protocol, other.protocol)
elif self.protocol and not other.protocol:
difference = cmp(self.protocol, DEFAULT_PROTOCOL)
elif not self.protocol and other.protocol:
difference = cmp(DEFAULT_PROTOCOL, other.protocol)
if difference != 0:
return difference
# Compare the host
difference = cmp(self.host, other.host)
if difference != 0:
return difference
# Compare the port
if self.port and other.port:
difference = cmp(self.port, other.port)
elif self.port and not other.port:
difference = cmp(self.port, DEFAULT_PORT)
elif not self.port and other.port:
difference = cmp(DEFAULT_PORT, other.port)
if difference != 0:
return difference
# Compare the path
difference = cmp(self.path, other.path)
if difference != 0:
return difference
# Compare the parameters
return cmp(self.params, other.params)
def __str__(self):
return self.to_string()

3338
cacert.pem Normal file
View File

File diff suppressed because it is too large Load Diff

7401
gam.py Normal file
View File

File diff suppressed because it is too large Load Diff

33
gdata/Crypto/Cipher/__init__.py Normal file
View File

@ -0,0 +1,33 @@
"""Secret-key encryption algorithms.
Secret-key encryption algorithms transform plaintext in some way that
is dependent on a key, producing ciphertext. This transformation can
easily be reversed, if (and, hopefully, only if) one knows the key.
The encryption modules here all support the interface described in PEP
272, "API for Block Encryption Algorithms".
If you don't know which algorithm to choose, use AES because it's
standard and has undergone a fair bit of examination.
Crypto.Cipher.AES Advanced Encryption Standard
Crypto.Cipher.ARC2 Alleged RC2
Crypto.Cipher.ARC4 Alleged RC4
Crypto.Cipher.Blowfish
Crypto.Cipher.CAST
Crypto.Cipher.DES The Data Encryption Standard. Very commonly used
in the past, but today its 56-bit keys are too small.
Crypto.Cipher.DES3 Triple DES.
Crypto.Cipher.IDEA
Crypto.Cipher.RC5
Crypto.Cipher.XOR The simple XOR cipher.
"""
__all__ = ['AES', 'ARC2', 'ARC4',
'Blowfish', 'CAST', 'DES', 'DES3', 'IDEA', 'RC5',
'XOR'
]
__revision__ = "$Id: __init__.py,v 1.7 2003/02/28 15:28:35 akuchling Exp $"

108
gdata/Crypto/Hash/HMAC.py Normal file
View File

@ -0,0 +1,108 @@
"""HMAC (Keyed-Hashing for Message Authentication) Python module.
Implements the HMAC algorithm as described by RFC 2104.
This is just a copy of the Python 2.2 HMAC module, modified to work when
used on versions of Python before 2.2.
"""
__revision__ = "$Id: HMAC.py,v 1.5 2002/07/25 17:19:02 z3p Exp $"
import string
def _strxor(s1, s2):
"""Utility method. XOR the two strings s1 and s2 (must have same length).
"""
return "".join(map(lambda x, y: chr(ord(x) ^ ord(y)), s1, s2))
# The size of the digests returned by HMAC depends on the underlying
# hashing module used.
digest_size = None
class HMAC:
"""RFC2104 HMAC class.
This supports the API for Cryptographic Hash Functions (PEP 247).
"""
def __init__(self, key, msg = None, digestmod = None):
"""Create a new HMAC object.
key: key for the keyed hash object.
msg: Initial input for the hash, if provided.
digestmod: A module supporting PEP 247. Defaults to the md5 module.
"""
if digestmod == None:
import md5
digestmod = md5
self.digestmod = digestmod
self.outer = digestmod.new()
self.inner = digestmod.new()
try:
self.digest_size = digestmod.digest_size
except AttributeError:
self.digest_size = len(self.outer.digest())
blocksize = 64
ipad = "\x36" * blocksize
opad = "\x5C" * blocksize
if len(key) > blocksize:
key = digestmod.new(key).digest()
key = key + chr(0) * (blocksize - len(key))
self.outer.update(_strxor(key, opad))
self.inner.update(_strxor(key, ipad))
if (msg):
self.update(msg)
## def clear(self):
## raise NotImplementedError, "clear() method not available in HMAC."
def update(self, msg):
"""Update this hashing object with the string msg.
"""
self.inner.update(msg)
def copy(self):
"""Return a separate copy of this hashing object.
An update to this copy won't affect the original object.
"""
other = HMAC("")
other.digestmod = self.digestmod
other.inner = self.inner.copy()
other.outer = self.outer.copy()
return other
def digest(self):
"""Return the hash value of this hashing object.
This returns a string containing 8-bit data. The object is
not altered in any way by this function; you can continue
updating the object after calling this function.
"""
h = self.outer.copy()
h.update(self.inner.digest())
return h.digest()
def hexdigest(self):
"""Like digest(), but returns a string of hexadecimal digits instead.
"""
return "".join([string.zfill(hex(ord(x))[2:], 2)
for x in tuple(self.digest())])
def new(key, msg = None, digestmod = None):
"""Create a new hashing object and return it.
key: The starting key for the hash.
msg: if available, will immediately be hashed into the object's starting
state.
You can now feed arbitrary strings into the object using its update()
method, and can ask for the hash value at any time by calling its digest()
method.
"""
return HMAC(key, msg, digestmod)

13
gdata/Crypto/Hash/MD5.py Normal file
View File

@ -0,0 +1,13 @@
# Just use the MD5 module from the Python standard library
__revision__ = "$Id: MD5.py,v 1.4 2002/07/11 14:31:19 akuchling Exp $"
from md5 import *
import md5
if hasattr(md5, 'digestsize'):
digest_size = digestsize
del digestsize
del md5

11
gdata/Crypto/Hash/SHA.py Normal file
View File

@ -0,0 +1,11 @@
# Just use the SHA module from the Python standard library
__revision__ = "$Id: SHA.py,v 1.4 2002/07/11 14:31:19 akuchling Exp $"
from sha import *
import sha
if hasattr(sha, 'digestsize'):
digest_size = digestsize
del digestsize
del sha

24
gdata/Crypto/Hash/__init__.py Normal file
View File

@ -0,0 +1,24 @@
"""Hashing algorithms
Hash functions take arbitrary strings as input, and produce an output
of fixed size that is dependent on the input; it should never be
possible to derive the input data given only the hash function's
output. Hash functions can be used simply as a checksum, or, in
association with a public-key algorithm, can be used to implement
digital signatures.
The hashing modules here all support the interface described in PEP
247, "API for Cryptographic Hash Functions".
Submodules:
Crypto.Hash.HMAC RFC 2104: Keyed-Hashing for Message Authentication
Crypto.Hash.MD2
Crypto.Hash.MD4
Crypto.Hash.MD5
Crypto.Hash.RIPEMD
Crypto.Hash.SHA
"""
__all__ = ['HMAC', 'MD2', 'MD4', 'MD5', 'RIPEMD', 'SHA', 'SHA256']
__revision__ = "$Id: __init__.py,v 1.6 2003/12/19 14:24:25 akuchling Exp $"

295
gdata/Crypto/Protocol/AllOrNothing.py Normal file
View File

@ -0,0 +1,295 @@
"""This file implements all-or-nothing package transformations.
An all-or-nothing package transformation is one in which some text is
transformed into message blocks, such that all blocks must be obtained before
the reverse transformation can be applied. Thus, if any blocks are corrupted
or lost, the original message cannot be reproduced.
An all-or-nothing package transformation is not encryption, although a block
cipher algorithm is used. The encryption key is randomly generated and is
extractable from the message blocks.
This class implements the All-Or-Nothing package transformation algorithm
described in:
Ronald L. Rivest. "All-Or-Nothing Encryption and The Package Transform"
http://theory.lcs.mit.edu/~rivest/fusion.pdf
"""
__revision__ = "$Id: AllOrNothing.py,v 1.8 2003/02/28 15:23:20 akuchling Exp $"
import operator
import string
from Crypto.Util.number import bytes_to_long, long_to_bytes
class AllOrNothing:
"""Class implementing the All-or-Nothing package transform.
Methods for subclassing:
_inventkey(key_size):
Returns a randomly generated key. Subclasses can use this to
implement better random key generating algorithms. The default
algorithm is probably not very cryptographically secure.
"""
def __init__(self, ciphermodule, mode=None, IV=None):
"""AllOrNothing(ciphermodule, mode=None, IV=None)
ciphermodule is a module implementing the cipher algorithm to
use. It must provide the PEP272 interface.
Note that the encryption key is randomly generated
automatically when needed. Optional arguments mode and IV are
passed directly through to the ciphermodule.new() method; they
are the feedback mode and initialization vector to use. All
three arguments must be the same for the object used to create
the digest, and to undigest'ify the message blocks.
"""
self.__ciphermodule = ciphermodule
self.__mode = mode
self.__IV = IV
self.__key_size = ciphermodule.key_size
if self.__key_size == 0:
self.__key_size = 16
__K0digit = chr(0x69)
def digest(self, text):
"""digest(text:string) : [string]
Perform the All-or-Nothing package transform on the given
string. Output is a list of message blocks describing the
transformed text, where each block is a string of bit length equal
to the ciphermodule's block_size.
"""
# generate a random session key and K0, the key used to encrypt the
# hash blocks. Rivest calls this a fixed, publically-known encryption
# key, but says nothing about the security implications of this key or
# how to choose it.
key = self._inventkey(self.__key_size)
K0 = self.__K0digit * self.__key_size
# we need two cipher objects here, one that is used to encrypt the
# message blocks and one that is used to encrypt the hashes. The
# former uses the randomly generated key, while the latter uses the
# well-known key.
mcipher = self.__newcipher(key)
hcipher = self.__newcipher(K0)
# Pad the text so that its length is a multiple of the cipher's
# block_size. Pad with trailing spaces, which will be eliminated in
# the undigest() step.
block_size = self.__ciphermodule.block_size
padbytes = block_size - (len(text) % block_size)
text = text + ' ' * padbytes
# Run through the algorithm:
# s: number of message blocks (size of text / block_size)
# input sequence: m1, m2, ... ms
# random key K' (`key' in the code)
# Compute output sequence: m'1, m'2, ... m's' for s' = s + 1
# Let m'i = mi ^ E(K', i) for i = 1, 2, 3, ..., s
# Let m's' = K' ^ h1 ^ h2 ^ ... hs
# where hi = E(K0, m'i ^ i) for i = 1, 2, ... s
#
# The one complication I add is that the last message block is hard
# coded to the number of padbytes added, so that these can be stripped
# during the undigest() step
s = len(text) / block_size
blocks = []
hashes = []
for i in range(1, s+1):
start = (i-1) * block_size
end = start + block_size
mi = text[start:end]
assert len(mi) == block_size
cipherblock = mcipher.encrypt(long_to_bytes(i, block_size))
mticki = bytes_to_long(mi) ^ bytes_to_long(cipherblock)
blocks.append(mticki)
# calculate the hash block for this block
hi = hcipher.encrypt(long_to_bytes(mticki ^ i, block_size))
hashes.append(bytes_to_long(hi))
# Add the padbytes length as a message block
i = i + 1
cipherblock = mcipher.encrypt(long_to_bytes(i, block_size))
mticki = padbytes ^ bytes_to_long(cipherblock)
blocks.append(mticki)
# calculate this block's hash
hi = hcipher.encrypt(long_to_bytes(mticki ^ i, block_size))
hashes.append(bytes_to_long(hi))
# Now calculate the last message block of the sequence 1..s'. This
# will contain the random session key XOR'd with all the hash blocks,
# so that for undigest(), once all the hash blocks are calculated, the
# session key can be trivially extracted. Calculating all the hash
# blocks requires that all the message blocks be received, thus the
# All-or-Nothing algorithm succeeds.
mtick_stick = bytes_to_long(key) ^ reduce(operator.xor, hashes)
blocks.append(mtick_stick)
# we convert the blocks to strings since in Python, byte sequences are
# always represented as strings. This is more consistent with the
# model that encryption and hash algorithms always operate on strings.
return map(long_to_bytes, blocks)
def undigest(self, blocks):
"""undigest(blocks : [string]) : string
Perform the reverse package transformation on a list of message
blocks. Note that the ciphermodule used for both transformations
must be the same. blocks is a list of strings of bit length
equal to the ciphermodule's block_size.
"""
# better have at least 2 blocks, for the padbytes package and the hash
# block accumulator
if len(blocks) < 2:
raise ValueError, "List must be at least length 2."
# blocks is a list of strings. We need to deal with them as long
# integers
blocks = map(bytes_to_long, blocks)
# Calculate the well-known key, to which the hash blocks are
# encrypted, and create the hash cipher.
K0 = self.__K0digit * self.__key_size
hcipher = self.__newcipher(K0)
# Since we have all the blocks (or this method would have been called
# prematurely), we can calcualte all the hash blocks.
hashes = []
for i in range(1, len(blocks)):
mticki = blocks[i-1] ^ i
hi = hcipher.encrypt(long_to_bytes(mticki))
hashes.append(bytes_to_long(hi))
# now we can calculate K' (key). remember the last block contains
# m's' which we don't include here
key = blocks[-1] ^ reduce(operator.xor, hashes)
# and now we can create the cipher object
mcipher = self.__newcipher(long_to_bytes(key))
block_size = self.__ciphermodule.block_size
# And we can now decode the original message blocks
parts = []
for i in range(1, len(blocks)):
cipherblock = mcipher.encrypt(long_to_bytes(i, block_size))
mi = blocks[i-1] ^ bytes_to_long(cipherblock)
parts.append(mi)
# The last message block contains the number of pad bytes appended to
# the original text string, such that its length was an even multiple
# of the cipher's block_size. This number should be small enough that
# the conversion from long integer to integer should never overflow
padbytes = int(parts[-1])
text = string.join(map(long_to_bytes, parts[:-1]), '')
return text[:-padbytes]
def _inventkey(self, key_size):
# TBD: Not a very secure algorithm. Eventually, I'd like to use JHy's
# kernelrand module
import time
from Crypto.Util import randpool
# TBD: key_size * 2 to work around possible bug in RandomPool?
pool = randpool.RandomPool(key_size * 2)
while key_size > pool.entropy:
pool.add_event()
# we now have enough entropy in the pool to get a key_size'd key
return pool.get_bytes(key_size)
def __newcipher(self, key):
if self.__mode is None and self.__IV is None:
return self.__ciphermodule.new(key)
elif self.__IV is None:
return self.__ciphermodule.new(key, self.__mode)
else:
return self.__ciphermodule.new(key, self.__mode, self.__IV)
if __name__ == '__main__':
import sys
import getopt
import base64
usagemsg = '''\
Test module usage: %(program)s [-c cipher] [-l] [-h]
Where:
--cipher module
-c module
Cipher module to use. Default: %(ciphermodule)s
--aslong
-l
Print the encoded message blocks as long integers instead of base64
encoded strings
--help
-h
Print this help message
'''
ciphermodule = 'AES'
aslong = 0
def usage(code, msg=None):
if msg:
print msg
print usagemsg % {'program': sys.argv[0],
'ciphermodule': ciphermodule}
sys.exit(code)
try:
opts, args = getopt.getopt(sys.argv[1:],
'c:l', ['cipher=', 'aslong'])
except getopt.error, msg:
usage(1, msg)
if args:
usage(1, 'Too many arguments')
for opt, arg in opts:
if opt in ('-h', '--help'):
usage(0)
elif opt in ('-c', '--cipher'):
ciphermodule = arg
elif opt in ('-l', '--aslong'):
aslong = 1
# ugly hack to force __import__ to give us the end-path module
module = __import__('Crypto.Cipher.'+ciphermodule, None, None, ['new'])
a = AllOrNothing(module)
print 'Original text:\n=========='
print __doc__
print '=========='
msgblocks = a.digest(__doc__)
print 'message blocks:'
for i, blk in map(None, range(len(msgblocks)), msgblocks):
# base64 adds a trailing newline
print ' %3d' % i,
if aslong:
print bytes_to_long(blk)
else:
print base64.encodestring(blk)[:-1]
#
# get a new undigest-only object so there's no leakage
b = AllOrNothing(module)
text = b.undigest(msgblocks)
if text == __doc__:
print 'They match!'
else:
print 'They differ!'

229
gdata/Crypto/Protocol/Chaffing.py Normal file
View File

@ -0,0 +1,229 @@
"""This file implements the chaffing algorithm.
Winnowing and chaffing is a technique for enhancing privacy without requiring
strong encryption. In short, the technique takes a set of authenticated
message blocks (the wheat) and adds a number of chaff blocks which have
randomly chosen data and MAC fields. This means that to an adversary, the
chaff blocks look as valid as the wheat blocks, and so the authentication
would have to be performed on every block. By tailoring the number of chaff
blocks added to the message, the sender can make breaking the message
computationally infeasible. There are many other interesting properties of
the winnow/chaff technique.
For example, say Alice is sending a message to Bob. She packetizes the
message and performs an all-or-nothing transformation on the packets. Then
she authenticates each packet with a message authentication code (MAC). The
MAC is a hash of the data packet, and there is a secret key which she must
share with Bob (key distribution is an exercise left to the reader). She then
adds a serial number to each packet, and sends the packets to Bob.
Bob receives the packets, and using the shared secret authentication key,
authenticates the MACs for each packet. Those packets that have bad MACs are
simply discarded. The remainder are sorted by serial number, and passed
through the reverse all-or-nothing transform. The transform means that an
eavesdropper (say Eve) must acquire all the packets before any of the data can
be read. If even one packet is missing, the data is useless.
There's one twist: by adding chaff packets, Alice and Bob can make Eve's job
much harder, since Eve now has to break the shared secret key, or try every
combination of wheat and chaff packet to read any of the message. The cool
thing is that Bob doesn't need to add any additional code; the chaff packets
are already filtered out because their MACs don't match (in all likelihood --
since the data and MACs for the chaff packets are randomly chosen it is
possible, but very unlikely that a chaff MAC will match the chaff data). And
Alice need not even be the party adding the chaff! She could be completely
unaware that a third party, say Charles, is adding chaff packets to her
messages as they are transmitted.
For more information on winnowing and chaffing see this paper:
Ronald L. Rivest, "Chaffing and Winnowing: Confidentiality without Encryption"
http://theory.lcs.mit.edu/~rivest/chaffing.txt
"""
__revision__ = "$Id: Chaffing.py,v 1.7 2003/02/28 15:23:21 akuchling Exp $"
from Crypto.Util.number import bytes_to_long
class Chaff:
"""Class implementing the chaff adding algorithm.
Methods for subclasses:
_randnum(size):
Returns a randomly generated number with a byte-length equal
to size. Subclasses can use this to implement better random
data and MAC generating algorithms. The default algorithm is
probably not very cryptographically secure. It is most
important that the chaff data does not contain any patterns
that can be used to discern it from wheat data without running
the MAC.
"""
def __init__(self, factor=1.0, blocksper=1):
"""Chaff(factor:float, blocksper:int)
factor is the number of message blocks to add chaff to,
expressed as a percentage between 0.0 and 1.0. blocksper is
the number of chaff blocks to include for each block being
chaffed. Thus the defaults add one chaff block to every
message block. By changing the defaults, you can adjust how
computationally difficult it could be for an adversary to
brute-force crack the message. The difficulty is expressed
as:
pow(blocksper, int(factor * number-of-blocks))
For ease of implementation, when factor < 1.0, only the first
int(factor*number-of-blocks) message blocks are chaffed.
"""
if not (0.0<=factor<=1.0):
raise ValueError, "'factor' must be between 0.0 and 1.0"
if blocksper < 0:
raise ValueError, "'blocksper' must be zero or more"
self.__factor = factor
self.__blocksper = blocksper
def chaff(self, blocks):
"""chaff( [(serial-number:int, data:string, MAC:string)] )
: [(int, string, string)]
Add chaff to message blocks. blocks is a list of 3-tuples of the
form (serial-number, data, MAC).
Chaff is created by choosing a random number of the same
byte-length as data, and another random number of the same
byte-length as MAC. The message block's serial number is
placed on the chaff block and all the packet's chaff blocks
are randomly interspersed with the single wheat block. This
method then returns a list of 3-tuples of the same form.
Chaffed blocks will contain multiple instances of 3-tuples
with the same serial number, but the only way to figure out
which blocks are wheat and which are chaff is to perform the
MAC hash and compare values.
"""
chaffedblocks = []
# count is the number of blocks to add chaff to. blocksper is the
# number of chaff blocks to add per message block that is being
# chaffed.
count = len(blocks) * self.__factor
blocksper = range(self.__blocksper)
for i, wheat in map(None, range(len(blocks)), blocks):
# it shouldn't matter which of the n blocks we add chaff to, so for
# ease of implementation, we'll just add them to the first count
# blocks
if i < count:
serial, data, mac = wheat
datasize = len(data)
macsize = len(mac)
addwheat = 1
# add chaff to this block
for j in blocksper:
import sys
chaffdata = self._randnum(datasize)
chaffmac = self._randnum(macsize)
chaff = (serial, chaffdata, chaffmac)
# mix up the order, if the 5th bit is on then put the
# wheat on the list
if addwheat and bytes_to_long(self._randnum(16)) & 0x40:
chaffedblocks.append(wheat)
addwheat = 0
chaffedblocks.append(chaff)
if addwheat:
chaffedblocks.append(wheat)
else:
# just add the wheat
chaffedblocks.append(wheat)
return chaffedblocks
def _randnum(self, size):
# TBD: Not a very secure algorithm.
# TBD: size * 2 to work around possible bug in RandomPool
from Crypto.Util import randpool
import time
pool = randpool.RandomPool(size * 2)
while size > pool.entropy:
pass
# we now have enough entropy in the pool to get size bytes of random
# data... well, probably
return pool.get_bytes(size)
if __name__ == '__main__':
text = """\
We hold these truths to be self-evident, that all men are created equal, that
they are endowed by their Creator with certain unalienable Rights, that among
these are Life, Liberty, and the pursuit of Happiness. That to secure these
rights, Governments are instituted among Men, deriving their just powers from
the consent of the governed. That whenever any Form of Government becomes
destructive of these ends, it is the Right of the People to alter or to
abolish it, and to institute new Government, laying its foundation on such
principles and organizing its powers in such form, as to them shall seem most
likely to effect their Safety and Happiness.
"""
print 'Original text:\n=========='
print text
print '=========='
# first transform the text into packets
blocks = [] ; size = 40
for i in range(0, len(text), size):
blocks.append( text[i:i+size] )
# now get MACs for all the text blocks. The key is obvious...
print 'Calculating MACs...'
from Crypto.Hash import HMAC, SHA
key = 'Jefferson'
macs = [HMAC.new(key, block, digestmod=SHA).digest()
for block in blocks]
assert len(blocks) == len(macs)
# put these into a form acceptable as input to the chaffing procedure
source = []
m = map(None, range(len(blocks)), blocks, macs)
print m
for i, data, mac in m:
source.append((i, data, mac))
# now chaff these
print 'Adding chaff...'
c = Chaff(factor=0.5, blocksper=2)
chaffed = c.chaff(source)
from base64 import encodestring
# print the chaffed message blocks. meanwhile, separate the wheat from
# the chaff
wheat = []
print 'chaffed message blocks:'
for i, data, mac in chaffed:
# do the authentication
h = HMAC.new(key, data, digestmod=SHA)
pmac = h.digest()
if pmac == mac:
tag = '-->'
wheat.append(data)
else:
tag = ' '
# base64 adds a trailing newline
print tag, '%3d' % i, \
repr(data), encodestring(mac)[:-1]
# now decode the message packets and check it against the original text
print 'Undigesting wheat...'
newtext = "".join(wheat)
if newtext == text:
print 'They match!'
else:
print 'They differ!'

17
gdata/Crypto/Protocol/__init__.py Normal file
View File

@ -0,0 +1,17 @@
"""Cryptographic protocols
Implements various cryptographic protocols. (Don't expect to find
network protocols here.)
Crypto.Protocol.AllOrNothing Transforms a message into a set of message
blocks, such that the blocks can be
recombined to get the message back.
Crypto.Protocol.Chaffing Takes a set of authenticated message blocks
(the wheat) and adds a number of
randomly generated blocks (the chaff).
"""
__all__ = ['AllOrNothing', 'Chaffing']
__revision__ = "$Id: __init__.py,v 1.4 2003/02/28 15:23:21 akuchling Exp $"

238
gdata/Crypto/PublicKey/DSA.py Normal file
View File

@ -0,0 +1,238 @@
#
# DSA.py : Digital Signature Algorithm
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: DSA.py,v 1.16 2004/05/06 12:52:54 akuchling Exp $"
from Crypto.PublicKey.pubkey import *
from Crypto.Util import number
from Crypto.Util.number import bytes_to_long, long_to_bytes
from Crypto.Hash import SHA
try:
from Crypto.PublicKey import _fastmath
except ImportError:
_fastmath = None
class error (Exception):
pass
def generateQ(randfunc):
S=randfunc(20)
hash1=SHA.new(S).digest()
hash2=SHA.new(long_to_bytes(bytes_to_long(S)+1)).digest()
q = bignum(0)
for i in range(0,20):
c=ord(hash1[i])^ord(hash2[i])
if i==0:
c=c | 128
if i==19:
c= c | 1
q=q*256+c
while (not isPrime(q)):
q=q+2
if pow(2,159L) < q < pow(2,160L):
return S, q
raise error, 'Bad q value generated'
def generate(bits, randfunc, progress_func=None):
"""generate(bits:int, randfunc:callable, progress_func:callable)
Generate a DSA key of length 'bits', using 'randfunc' to get
random data and 'progress_func', if present, to display
the progress of the key generation.
"""
if bits<160:
raise error, 'Key length <160 bits'
obj=DSAobj()
# Generate string S and prime q
if progress_func:
progress_func('p,q\n')
while (1):
S, obj.q = generateQ(randfunc)
n=(bits-1)/160
C, N, V = 0, 2, {}
b=(obj.q >> 5) & 15
powb=pow(bignum(2), b)
powL1=pow(bignum(2), bits-1)
while C<4096:
for k in range(0, n+1):
V[k]=bytes_to_long(SHA.new(S+str(N)+str(k)).digest())
W=V[n] % powb
for k in range(n-1, -1, -1):
W=(W<<160L)+V[k]
X=W+powL1
p=X-(X%(2*obj.q)-1)
if powL1<=p and isPrime(p):
break
C, N = C+1, N+n+1
if C<4096:
break
if progress_func:
progress_func('4096 multiples failed\n')
obj.p = p
power=(p-1)/obj.q
if progress_func:
progress_func('h,g\n')
while (1):
h=bytes_to_long(randfunc(bits)) % (p-1)
g=pow(h, power, p)
if 1<h<p-1 and g>1:
break
obj.g=g
if progress_func:
progress_func('x,y\n')
while (1):
x=bytes_to_long(randfunc(20))
if 0 < x < obj.q:
break
obj.x, obj.y = x, pow(g, x, p)
return obj
def construct(tuple):
"""construct(tuple:(long,long,long,long)|(long,long,long,long,long)):DSAobj
Construct a DSA object from a 4- or 5-tuple of numbers.
"""
obj=DSAobj()
if len(tuple) not in [4,5]:
raise error, 'argument for construct() wrong length'
for i in range(len(tuple)):
field = obj.keydata[i]
setattr(obj, field, tuple[i])
return obj
class DSAobj(pubkey):
keydata=['y', 'g', 'p', 'q', 'x']
def _encrypt(self, s, Kstr):
raise error, 'DSA algorithm cannot encrypt data'
def _decrypt(self, s):
raise error, 'DSA algorithm cannot decrypt data'
def _sign(self, M, K):
if (K<2 or self.q<=K):
raise error, 'K is not between 2 and q'
r=pow(self.g, K, self.p) % self.q
s=(inverse(K, self.q)*(M+self.x*r)) % self.q
return (r,s)
def _verify(self, M, sig):
r, s = sig
if r<=0 or r>=self.q or s<=0 or s>=self.q:
return 0
w=inverse(s, self.q)
u1, u2 = (M*w) % self.q, (r*w) % self.q
v1 = pow(self.g, u1, self.p)
v2 = pow(self.y, u2, self.p)
v = ((v1*v2) % self.p)
v = v % self.q
if v==r:
return 1
return 0
def size(self):
"Return the maximum number of bits that can be handled by this key."
return number.size(self.p) - 1
def has_private(self):
"""Return a Boolean denoting whether the object contains
private components."""
if hasattr(self, 'x'):
return 1
else:
return 0
def can_sign(self):
"""Return a Boolean value recording whether this algorithm can generate signatures."""
return 1
def can_encrypt(self):
"""Return a Boolean value recording whether this algorithm can encrypt data."""
return 0
def publickey(self):
"""Return a new key object containing only the public information."""
return construct((self.y, self.g, self.p, self.q))
object=DSAobj
generate_py = generate
construct_py = construct
class DSAobj_c(pubkey):
keydata = ['y', 'g', 'p', 'q', 'x']
def __init__(self, key):
self.key = key
def __getattr__(self, attr):
if attr in self.keydata:
return getattr(self.key, attr)
else:
if self.__dict__.has_key(attr):
self.__dict__[attr]
else:
raise AttributeError, '%s instance has no attribute %s' % (self.__class__, attr)
def __getstate__(self):
d = {}
for k in self.keydata:
if hasattr(self.key, k):
d[k]=getattr(self.key, k)
return d
def __setstate__(self, state):
y,g,p,q = state['y'], state['g'], state['p'], state['q']
if not state.has_key('x'):
self.key = _fastmath.dsa_construct(y,g,p,q)
else:
x = state['x']
self.key = _fastmath.dsa_construct(y,g,p,q,x)
def _sign(self, M, K):
return self.key._sign(M, K)
def _verify(self, M, (r, s)):
return self.key._verify(M, r, s)
def size(self):
return self.key.size()
def has_private(self):
return self.key.has_private()
def publickey(self):
return construct_c((self.key.y, self.key.g, self.key.p, self.key.q))
def can_sign(self):
return 1
def can_encrypt(self):
return 0
def generate_c(bits, randfunc, progress_func=None):
obj = generate_py(bits, randfunc, progress_func)
y,g,p,q,x = obj.y, obj.g, obj.p, obj.q, obj.x
return construct_c((y,g,p,q,x))
def construct_c(tuple):
key = apply(_fastmath.dsa_construct, tuple)
return DSAobj_c(key)
if _fastmath:
#print "using C version of DSA"
generate = generate_c
construct = construct_c
error = _fastmath.error

132
gdata/Crypto/PublicKey/ElGamal.py Normal file
View File

@ -0,0 +1,132 @@
#
# ElGamal.py : ElGamal encryption/decryption and signatures
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: ElGamal.py,v 1.9 2003/04/04 19:44:26 akuchling Exp $"
from Crypto.PublicKey.pubkey import *
from Crypto.Util import number
class error (Exception):
pass
# Generate an ElGamal key with N bits
def generate(bits, randfunc, progress_func=None):
"""generate(bits:int, randfunc:callable, progress_func:callable)
Generate an ElGamal key of length 'bits', using 'randfunc' to get
random data and 'progress_func', if present, to display
the progress of the key generation.
"""
obj=ElGamalobj()
# Generate prime p
if progress_func:
progress_func('p\n')
obj.p=bignum(getPrime(bits, randfunc))
# Generate random number g
if progress_func:
progress_func('g\n')
size=bits-1-(ord(randfunc(1)) & 63) # g will be from 1--64 bits smaller than p
if size<1:
size=bits-1
while (1):
obj.g=bignum(getPrime(size, randfunc))
if obj.g < obj.p:
break
size=(size+1) % bits
if size==0:
size=4
# Generate random number x
if progress_func:
progress_func('x\n')
while (1):
size=bits-1-ord(randfunc(1)) # x will be from 1 to 256 bits smaller than p
if size>2:
break
while (1):
obj.x=bignum(getPrime(size, randfunc))
if obj.x < obj.p:
break
size = (size+1) % bits
if size==0:
size=4
if progress_func:
progress_func('y\n')
obj.y = pow(obj.g, obj.x, obj.p)
return obj
def construct(tuple):
"""construct(tuple:(long,long,long,long)|(long,long,long,long,long)))
: ElGamalobj
Construct an ElGamal key from a 3- or 4-tuple of numbers.
"""
obj=ElGamalobj()
if len(tuple) not in [3,4]:
raise error, 'argument for construct() wrong length'
for i in range(len(tuple)):
field = obj.keydata[i]
setattr(obj, field, tuple[i])
return obj
class ElGamalobj(pubkey):
keydata=['p', 'g', 'y', 'x']
def _encrypt(self, M, K):
a=pow(self.g, K, self.p)
b=( M*pow(self.y, K, self.p) ) % self.p
return ( a,b )
def _decrypt(self, M):
if (not hasattr(self, 'x')):
raise error, 'Private key not available in this object'
ax=pow(M[0], self.x, self.p)
plaintext=(M[1] * inverse(ax, self.p ) ) % self.p
return plaintext
def _sign(self, M, K):
if (not hasattr(self, 'x')):
raise error, 'Private key not available in this object'
p1=self.p-1
if (GCD(K, p1)!=1):
raise error, 'Bad K value: GCD(K,p-1)!=1'
a=pow(self.g, K, self.p)
t=(M-self.x*a) % p1
while t<0: t=t+p1
b=(t*inverse(K, p1)) % p1
return (a, b)
def _verify(self, M, sig):
v1=pow(self.y, sig[0], self.p)
v1=(v1*pow(sig[0], sig[1], self.p)) % self.p
v2=pow(self.g, M, self.p)
if v1==v2:
return 1
return 0
def size(self):
"Return the maximum number of bits that can be handled by this key."
return number.size(self.p) - 1
def has_private(self):
"""Return a Boolean denoting whether the object contains
private components."""
if hasattr(self, 'x'):
return 1
else:
return 0
def publickey(self):
"""Return a new key object containing only the public information."""
return construct((self.p, self.g, self.y))
object=ElGamalobj

256
gdata/Crypto/PublicKey/RSA.py Normal file
View File

@ -0,0 +1,256 @@
#
# RSA.py : RSA encryption/decryption
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: RSA.py,v 1.20 2004/05/06 12:52:54 akuchling Exp $"
from Crypto.PublicKey import pubkey
from Crypto.Util import number
try:
from Crypto.PublicKey import _fastmath
except ImportError:
_fastmath = None
class error (Exception):
pass
def generate(bits, randfunc, progress_func=None):
"""generate(bits:int, randfunc:callable, progress_func:callable)
Generate an RSA key of length 'bits', using 'randfunc' to get
random data and 'progress_func', if present, to display
the progress of the key generation.
"""
obj=RSAobj()
# Generate the prime factors of n
if progress_func:
progress_func('p,q\n')
p = q = 1L
while number.size(p*q) < bits:
p = pubkey.getPrime(bits/2, randfunc)
q = pubkey.getPrime(bits/2, randfunc)
# p shall be smaller than q (for calc of u)
if p > q:
(p, q)=(q, p)
obj.p = p
obj.q = q
if progress_func:
progress_func('u\n')
obj.u = pubkey.inverse(obj.p, obj.q)
obj.n = obj.p*obj.q
obj.e = 65537L
if progress_func:
progress_func('d\n')
obj.d=pubkey.inverse(obj.e, (obj.p-1)*(obj.q-1))
assert bits <= 1+obj.size(), "Generated key is too small"
return obj
def construct(tuple):
"""construct(tuple:(long,) : RSAobj
Construct an RSA object from a 2-, 3-, 5-, or 6-tuple of numbers.
"""
obj=RSAobj()
if len(tuple) not in [2,3,5,6]:
raise error, 'argument for construct() wrong length'
for i in range(len(tuple)):
field = obj.keydata[i]
setattr(obj, field, tuple[i])
if len(tuple) >= 5:
# Ensure p is smaller than q
if obj.p>obj.q:
(obj.p, obj.q)=(obj.q, obj.p)
if len(tuple) == 5:
# u not supplied, so we're going to have to compute it.
obj.u=pubkey.inverse(obj.p, obj.q)
return obj
class RSAobj(pubkey.pubkey):
keydata = ['n', 'e', 'd', 'p', 'q', 'u']
def _encrypt(self, plaintext, K=''):
if self.n<=plaintext:
raise error, 'Plaintext too large'
return (pow(plaintext, self.e, self.n),)
def _decrypt(self, ciphertext):
if (not hasattr(self, 'd')):
raise error, 'Private key not available in this object'
if self.n<=ciphertext[0]:
raise error, 'Ciphertext too large'
return pow(ciphertext[0], self.d, self.n)
def _sign(self, M, K=''):
return (self._decrypt((M,)),)
def _verify(self, M, sig):
m2=self._encrypt(sig[0])
if m2[0]==M:
return 1
else: return 0
def _blind(self, M, B):
tmp = pow(B, self.e, self.n)
return (M * tmp) % self.n
def _unblind(self, M, B):
tmp = pubkey.inverse(B, self.n)
return (M * tmp) % self.n
def can_blind (self):
"""can_blind() : bool
Return a Boolean value recording whether this algorithm can
blind data. (This does not imply that this
particular key object has the private information required to
to blind a message.)
"""
return 1
def size(self):
"""size() : int
Return the maximum number of bits that can be handled by this key.
"""
return number.size(self.n) - 1
def has_private(self):
"""has_private() : bool
Return a Boolean denoting whether the object contains
private components.
"""
if hasattr(self, 'd'):
return 1
else: return 0
def publickey(self):
"""publickey(): RSAobj
Return a new key object containing only the public key information.
"""
return construct((self.n, self.e))
class RSAobj_c(pubkey.pubkey):
keydata = ['n', 'e', 'd', 'p', 'q', 'u']
def __init__(self, key):
self.key = key
def __getattr__(self, attr):
if attr in self.keydata:
return getattr(self.key, attr)
else:
if self.__dict__.has_key(attr):
self.__dict__[attr]
else:
raise AttributeError, '%s instance has no attribute %s' % (self.__class__, attr)
def __getstate__(self):
d = {}
for k in self.keydata:
if hasattr(self.key, k):
d[k]=getattr(self.key, k)
return d
def __setstate__(self, state):
n,e = state['n'], state['e']
if not state.has_key('d'):
self.key = _fastmath.rsa_construct(n,e)
else:
d = state['d']
if not state.has_key('q'):
self.key = _fastmath.rsa_construct(n,e,d)
else:
p, q, u = state['p'], state['q'], state['u']
self.key = _fastmath.rsa_construct(n,e,d,p,q,u)
def _encrypt(self, plain, K):
return (self.key._encrypt(plain),)
def _decrypt(self, cipher):
return self.key._decrypt(cipher[0])
def _sign(self, M, K):
return (self.key._sign(M),)
def _verify(self, M, sig):
return self.key._verify(M, sig[0])
def _blind(self, M, B):
return self.key._blind(M, B)
def _unblind(self, M, B):
return self.key._unblind(M, B)
def can_blind (self):
return 1
def size(self):
return self.key.size()
def has_private(self):
return self.key.has_private()
def publickey(self):
return construct_c((self.key.n, self.key.e))
def generate_c(bits, randfunc, progress_func = None):
# Generate the prime factors of n
if progress_func:
progress_func('p,q\n')
p = q = 1L
while number.size(p*q) < bits:
p = pubkey.getPrime(bits/2, randfunc)
q = pubkey.getPrime(bits/2, randfunc)
# p shall be smaller than q (for calc of u)
if p > q:
(p, q)=(q, p)
if progress_func:
progress_func('u\n')
u=pubkey.inverse(p, q)
n=p*q
e = 65537L
if progress_func:
progress_func('d\n')
d=pubkey.inverse(e, (p-1)*(q-1))
key = _fastmath.rsa_construct(n,e,d,p,q,u)
obj = RSAobj_c(key)
## print p
## print q
## print number.size(p), number.size(q), number.size(q*p),
## print obj.size(), bits
assert bits <= 1+obj.size(), "Generated key is too small"
return obj
def construct_c(tuple):
key = apply(_fastmath.rsa_construct, tuple)
return RSAobj_c(key)
object = RSAobj
generate_py = generate
construct_py = construct
if _fastmath:
#print "using C version of RSA"
generate = generate_c
construct = construct_c
error = _fastmath.error

17
gdata/Crypto/PublicKey/__init__.py Normal file
View File

@ -0,0 +1,17 @@
"""Public-key encryption and signature algorithms.
Public-key encryption uses two different keys, one for encryption and
one for decryption. The encryption key can be made public, and the
decryption key is kept private. Many public-key algorithms can also
be used to sign messages, and some can *only* be used for signatures.
Crypto.PublicKey.DSA Digital Signature Algorithm. (Signature only)
Crypto.PublicKey.ElGamal (Signing and encryption)
Crypto.PublicKey.RSA (Signing, encryption, and blinding)
Crypto.PublicKey.qNEW (Signature only)
"""
__all__ = ['RSA', 'DSA', 'ElGamal', 'qNEW']
__revision__ = "$Id: __init__.py,v 1.4 2003/04/03 20:27:13 akuchling Exp $"

172
gdata/Crypto/PublicKey/pubkey.py Normal file
View File

@ -0,0 +1,172 @@
#
# pubkey.py : Internal functions for public key operations
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: pubkey.py,v 1.11 2003/04/03 20:36:14 akuchling Exp $"
import types, warnings
from Crypto.Util.number import *
# Basic public key class
class pubkey:
def __init__(self):
pass
def __getstate__(self):
"""To keep key objects platform-independent, the key data is
converted to standard Python long integers before being
written out. It will then be reconverted as necessary on
restoration."""
d=self.__dict__
for key in self.keydata:
if d.has_key(key): d[key]=long(d[key])
return d
def __setstate__(self, d):
"""On unpickling a key object, the key data is converted to the big
number representation being used, whether that is Python long
integers, MPZ objects, or whatever."""
for key in self.keydata:
if d.has_key(key): self.__dict__[key]=bignum(d[key])
def encrypt(self, plaintext, K):
"""encrypt(plaintext:string|long, K:string|long) : tuple
Encrypt the string or integer plaintext. K is a random
parameter required by some algorithms.
"""
wasString=0
if isinstance(plaintext, types.StringType):
plaintext=bytes_to_long(plaintext) ; wasString=1
if isinstance(K, types.StringType):
K=bytes_to_long(K)
ciphertext=self._encrypt(plaintext, K)
if wasString: return tuple(map(long_to_bytes, ciphertext))
else: return ciphertext
def decrypt(self, ciphertext):
"""decrypt(ciphertext:tuple|string|long): string
Decrypt 'ciphertext' using this key.
"""
wasString=0
if not isinstance(ciphertext, types.TupleType):
ciphertext=(ciphertext,)
if isinstance(ciphertext[0], types.StringType):
ciphertext=tuple(map(bytes_to_long, ciphertext)) ; wasString=1
plaintext=self._decrypt(ciphertext)
if wasString: return long_to_bytes(plaintext)
else: return plaintext
def sign(self, M, K):
"""sign(M : string|long, K:string|long) : tuple
Return a tuple containing the signature for the message M.
K is a random parameter required by some algorithms.
"""
if (not self.has_private()):
raise error, 'Private key not available in this object'
if isinstance(M, types.StringType): M=bytes_to_long(M)
if isinstance(K, types.StringType): K=bytes_to_long(K)
return self._sign(M, K)
def verify (self, M, signature):
"""verify(M:string|long, signature:tuple) : bool
Verify that the signature is valid for the message M;
returns true if the signature checks out.
"""
if isinstance(M, types.StringType): M=bytes_to_long(M)
return self._verify(M, signature)
# alias to compensate for the old validate() name
def validate (self, M, signature):
warnings.warn("validate() method name is obsolete; use verify()",
DeprecationWarning)
def blind(self, M, B):
"""blind(M : string|long, B : string|long) : string|long
Blind message M using blinding factor B.
"""
wasString=0
if isinstance(M, types.StringType):
M=bytes_to_long(M) ; wasString=1
if isinstance(B, types.StringType): B=bytes_to_long(B)
blindedmessage=self._blind(M, B)
if wasString: return long_to_bytes(blindedmessage)
else: return blindedmessage
def unblind(self, M, B):
"""unblind(M : string|long, B : string|long) : string|long
Unblind message M using blinding factor B.
"""
wasString=0
if isinstance(M, types.StringType):
M=bytes_to_long(M) ; wasString=1
if isinstance(B, types.StringType): B=bytes_to_long(B)
unblindedmessage=self._unblind(M, B)
if wasString: return long_to_bytes(unblindedmessage)
else: return unblindedmessage
# The following methods will usually be left alone, except for
# signature-only algorithms. They both return Boolean values
# recording whether this key's algorithm can sign and encrypt.
def can_sign (self):
"""can_sign() : bool
Return a Boolean value recording whether this algorithm can
generate signatures. (This does not imply that this
particular key object has the private information required to
to generate a signature.)
"""
return 1
def can_encrypt (self):
"""can_encrypt() : bool
Return a Boolean value recording whether this algorithm can
encrypt data. (This does not imply that this
particular key object has the private information required to
to decrypt a message.)
"""
return 1
def can_blind (self):
"""can_blind() : bool
Return a Boolean value recording whether this algorithm can
blind data. (This does not imply that this
particular key object has the private information required to
to blind a message.)
"""
return 0
# The following methods will certainly be overridden by
# subclasses.
def size (self):
"""size() : int
Return the maximum number of bits that can be handled by this key.
"""
return 0
def has_private (self):
"""has_private() : bool
Return a Boolean denoting whether the object contains
private components.
"""
return 0
def publickey (self):
"""publickey(): object
Return a new key object containing only the public information.
"""
return self
def __eq__ (self, other):
"""__eq__(other): 0, 1
Compare us to other for equality.
"""
return self.__getstate__() == other.__getstate__()

170
gdata/Crypto/PublicKey/qNEW.py Normal file
View File

@ -0,0 +1,170 @@
#
# qNEW.py : The q-NEW signature algorithm.
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: qNEW.py,v 1.8 2003/04/04 15:13:35 akuchling Exp $"
from Crypto.PublicKey import pubkey
from Crypto.Util.number import *
from Crypto.Hash import SHA
class error (Exception):
pass
HASHBITS = 160 # Size of SHA digests
def generate(bits, randfunc, progress_func=None):
"""generate(bits:int, randfunc:callable, progress_func:callable)
Generate a qNEW key of length 'bits', using 'randfunc' to get
random data and 'progress_func', if present, to display
the progress of the key generation.
"""
obj=qNEWobj()
# Generate prime numbers p and q. q is a 160-bit prime
# number. p is another prime number (the modulus) whose bit
# size is chosen by the caller, and is generated so that p-1
# is a multiple of q.
#
# Note that only a single seed is used to
# generate p and q; if someone generates a key for you, you can
# use the seed to duplicate the key generation. This can
# protect you from someone generating values of p,q that have
# some special form that's easy to break.
if progress_func:
progress_func('p,q\n')
while (1):
obj.q = getPrime(160, randfunc)
# assert pow(2, 159L)<obj.q<pow(2, 160L)
obj.seed = S = long_to_bytes(obj.q)
C, N, V = 0, 2, {}
# Compute b and n such that bits-1 = b + n*HASHBITS
n= (bits-1) / HASHBITS
b= (bits-1) % HASHBITS ; powb=2L << b
powL1=pow(long(2), bits-1)
while C<4096:
# The V array will contain (bits-1) bits of random
# data, that are assembled to produce a candidate
# value for p.
for k in range(0, n+1):
V[k]=bytes_to_long(SHA.new(S+str(N)+str(k)).digest())
p = V[n] % powb
for k in range(n-1, -1, -1):
p= (p << long(HASHBITS) )+V[k]
p = p+powL1 # Ensure the high bit is set
# Ensure that p-1 is a multiple of q
p = p - (p % (2*obj.q)-1)
# If p is still the right size, and it's prime, we're done!
if powL1<=p and isPrime(p):
break
# Otherwise, increment the counter and try again
C, N = C+1, N+n+1
if C<4096:
break # Ended early, so exit the while loop
if progress_func:
progress_func('4096 values of p tried\n')
obj.p = p
power=(p-1)/obj.q
# Next parameter: g = h**((p-1)/q) mod p, such that h is any
# number <p-1, and g>1. g is kept; h can be discarded.
if progress_func:
progress_func('h,g\n')
while (1):
h=bytes_to_long(randfunc(bits)) % (p-1)
g=pow(h, power, p)
if 1<h<p-1 and g>1:
break
obj.g=g
# x is the private key information, and is
# just a random number between 0 and q.
# y=g**x mod p, and is part of the public information.
if progress_func:
progress_func('x,y\n')
while (1):
x=bytes_to_long(randfunc(20))
if 0 < x < obj.q:
break
obj.x, obj.y=x, pow(g, x, p)
return obj
# Construct a qNEW object
def construct(tuple):
"""construct(tuple:(long,long,long,long)|(long,long,long,long,long)
Construct a qNEW object from a 4- or 5-tuple of numbers.
"""
obj=qNEWobj()
if len(tuple) not in [4,5]:
raise error, 'argument for construct() wrong length'
for i in range(len(tuple)):
field = obj.keydata[i]
setattr(obj, field, tuple[i])
return obj
class qNEWobj(pubkey.pubkey):
keydata=['p', 'q', 'g', 'y', 'x']
def _sign(self, M, K=''):
if (self.q<=K):
raise error, 'K is greater than q'
if M<0:
raise error, 'Illegal value of M (<0)'
if M>=pow(2,161L):
raise error, 'Illegal value of M (too large)'
r=pow(self.g, K, self.p) % self.q
s=(K- (r*M*self.x % self.q)) % self.q
return (r,s)
def _verify(self, M, sig):
r, s = sig
if r<=0 or r>=self.q or s<=0 or s>=self.q:
return 0
if M<0:
raise error, 'Illegal value of M (<0)'
if M<=0 or M>=pow(2,161L):
return 0
v1 = pow(self.g, s, self.p)
v2 = pow(self.y, M*r, self.p)
v = ((v1*v2) % self.p)
v = v % self.q
if v==r:
return 1
return 0
def size(self):
"Return the maximum number of bits that can be handled by this key."
return 160
def has_private(self):
"""Return a Boolean denoting whether the object contains
private components."""
return hasattr(self, 'x')
def can_sign(self):
"""Return a Boolean value recording whether this algorithm can generate signatures."""
return 1
def can_encrypt(self):
"""Return a Boolean value recording whether this algorithm can encrypt data."""
return 0
def publickey(self):
"""Return a new key object containing only the public information."""
return construct((self.p, self.q, self.g, self.y))
object = qNEWobj

342
gdata/Crypto/Util/RFC1751.py Normal file
View File

@ -0,0 +1,342 @@
#!/usr/local/bin/python
# rfc1751.py : Converts between 128-bit strings and a human-readable
# sequence of words, as defined in RFC1751: "A Convention for
# Human-Readable 128-bit Keys", by Daniel L. McDonald.
__revision__ = "$Id: RFC1751.py,v 1.6 2003/04/04 15:15:10 akuchling Exp $"
import string, binascii
binary={0:'0000', 1:'0001', 2:'0010', 3:'0011', 4:'0100', 5:'0101',
6:'0110', 7:'0111', 8:'1000', 9:'1001', 10:'1010', 11:'1011',
12:'1100', 13:'1101', 14:'1110', 15:'1111'}
def _key2bin(s):
"Convert a key into a string of binary digits"
kl=map(lambda x: ord(x), s)
kl=map(lambda x: binary[x/16]+binary[x&15], kl)
return ''.join(kl)
def _extract(key, start, length):
"""Extract a bitstring from a string of binary digits, and return its
numeric value."""
k=key[start:start+length]
return reduce(lambda x,y: x*2+ord(y)-48, k, 0)
def key_to_english (key):
"""key_to_english(key:string) : string
Transform an arbitrary key into a string containing English words.
The key length must be a multiple of 8.
"""
english=''
for index in range(0, len(key), 8): # Loop over 8-byte subkeys
subkey=key[index:index+8]
# Compute the parity of the key
skbin=_key2bin(subkey) ; p=0
for i in range(0, 64, 2): p=p+_extract(skbin, i, 2)
# Append parity bits to the subkey
skbin=_key2bin(subkey+chr((p<<6) & 255))
for i in range(0, 64, 11):
english=english+wordlist[_extract(skbin, i, 11)]+' '
return english[:-1] # Remove the trailing space
def english_to_key (str):
"""english_to_key(string):string
Transform a string into a corresponding key.
The string must contain words separated by whitespace; the number
of words must be a multiple of 6.
"""
L=string.split(string.upper(str)) ; key=''
for index in range(0, len(L), 6):
sublist=L[index:index+6] ; char=9*[0] ; bits=0
for i in sublist:
index = wordlist.index(i)
shift = (8-(bits+11)%8) %8
y = index << shift
cl, cc, cr = (y>>16), (y>>8)&0xff, y & 0xff
if (shift>5):
char[bits/8] = char[bits/8] | cl
char[bits/8+1] = char[bits/8+1] | cc
char[bits/8+2] = char[bits/8+2] | cr
elif shift>-3:
char[bits/8] = char[bits/8] | cc
char[bits/8+1] = char[bits/8+1] | cr
else: char[bits/8] = char[bits/8] | cr
bits=bits+11
subkey=reduce(lambda x,y:x+chr(y), char, '')
# Check the parity of the resulting key
skbin=_key2bin(subkey)
p=0
for i in range(0, 64, 2): p=p+_extract(skbin, i, 2)
if (p&3) != _extract(skbin, 64, 2):
raise ValueError, "Parity error in resulting key"
key=key+subkey[0:8]
return key
wordlist=[ "A", "ABE", "ACE", "ACT", "AD", "ADA", "ADD",
"AGO", "AID", "AIM", "AIR", "ALL", "ALP", "AM", "AMY", "AN", "ANA",
"AND", "ANN", "ANT", "ANY", "APE", "APS", "APT", "ARC", "ARE", "ARK",
"ARM", "ART", "AS", "ASH", "ASK", "AT", "ATE", "AUG", "AUK", "AVE",
"AWE", "AWK", "AWL", "AWN", "AX", "AYE", "BAD", "BAG", "BAH", "BAM",
"BAN", "BAR", "BAT", "BAY", "BE", "BED", "BEE", "BEG", "BEN", "BET",
"BEY", "BIB", "BID", "BIG", "BIN", "BIT", "BOB", "BOG", "BON", "BOO",
"BOP", "BOW", "BOY", "BUB", "BUD", "BUG", "BUM", "BUN", "BUS", "BUT",
"BUY", "BY", "BYE", "CAB", "CAL", "CAM", "CAN", "CAP", "CAR", "CAT",
"CAW", "COD", "COG", "COL", "CON", "COO", "COP", "COT", "COW", "COY",
"CRY", "CUB", "CUE", "CUP", "CUR", "CUT", "DAB", "DAD", "DAM", "DAN",
"DAR", "DAY", "DEE", "DEL", "DEN", "DES", "DEW", "DID", "DIE", "DIG",
"DIN", "DIP", "DO", "DOE", "DOG", "DON", "DOT", "DOW", "DRY", "DUB",
"DUD", "DUE", "DUG", "DUN", "EAR", "EAT", "ED", "EEL", "EGG", "EGO",
"ELI", "ELK", "ELM", "ELY", "EM", "END", "EST", "ETC", "EVA", "EVE",
"EWE", "EYE", "FAD", "FAN", "FAR", "FAT", "FAY", "FED", "FEE", "FEW",
"FIB", "FIG", "FIN", "FIR", "FIT", "FLO", "FLY", "FOE", "FOG", "FOR",
"FRY", "FUM", "FUN", "FUR", "GAB", "GAD", "GAG", "GAL", "GAM", "GAP",
"GAS", "GAY", "GEE", "GEL", "GEM", "GET", "GIG", "GIL", "GIN", "GO",
"GOT", "GUM", "GUN", "GUS", "GUT", "GUY", "GYM", "GYP", "HA", "HAD",
"HAL", "HAM", "HAN", "HAP", "HAS", "HAT", "HAW", "HAY", "HE", "HEM",
"HEN", "HER", "HEW", "HEY", "HI", "HID", "HIM", "HIP", "HIS", "HIT",
"HO", "HOB", "HOC", "HOE", "HOG", "HOP", "HOT", "HOW", "HUB", "HUE",
"HUG", "HUH", "HUM", "HUT", "I", "ICY", "IDA", "IF", "IKE", "ILL",
"INK", "INN", "IO", "ION", "IQ", "IRA", "IRE", "IRK", "IS", "IT",
"ITS", "IVY", "JAB", "JAG", "JAM", "JAN", "JAR", "JAW", "JAY", "JET",
"JIG", "JIM", "JO", "JOB", "JOE", "JOG", "JOT", "JOY", "JUG", "JUT",
"KAY", "KEG", "KEN", "KEY", "KID", "KIM", "KIN", "KIT", "LA", "LAB",
"LAC", "LAD", "LAG", "LAM", "LAP", "LAW", "LAY", "LEA", "LED", "LEE",
"LEG", "LEN", "LEO", "LET", "LEW", "LID", "LIE", "LIN", "LIP", "LIT",
"LO", "LOB", "LOG", "LOP", "LOS", "LOT", "LOU", "LOW", "LOY", "LUG",
"LYE", "MA", "MAC", "MAD", "MAE", "MAN", "MAO", "MAP", "MAT", "MAW",
"MAY", "ME", "MEG", "MEL", "MEN", "MET", "MEW", "MID", "MIN", "MIT",
"MOB", "MOD", "MOE", "MOO", "MOP", "MOS", "MOT", "MOW", "MUD", "MUG",
"MUM", "MY", "NAB", "NAG", "NAN", "NAP", "NAT", "NAY", "NE", "NED",
"NEE", "NET", "NEW", "NIB", "NIL", "NIP", "NIT", "NO", "NOB", "NOD",
"NON", "NOR", "NOT", "NOV", "NOW", "NU", "NUN", "NUT", "O", "OAF",
"OAK", "OAR", "OAT", "ODD", "ODE", "OF", "OFF", "OFT", "OH", "OIL",
"OK", "OLD", "ON", "ONE", "OR", "ORB", "ORE", "ORR", "OS", "OTT",
"OUR", "OUT", "OVA", "OW", "OWE", "OWL", "OWN", "OX", "PA", "PAD",
"PAL", "PAM", "PAN", "PAP", "PAR", "PAT", "PAW", "PAY", "PEA", "PEG",
"PEN", "PEP", "PER", "PET", "PEW", "PHI", "PI", "PIE", "PIN", "PIT",
"PLY", "PO", "POD", "POE", "POP", "POT", "POW", "PRO", "PRY", "PUB",
"PUG", "PUN", "PUP", "PUT", "QUO", "RAG", "RAM", "RAN", "RAP", "RAT",
"RAW", "RAY", "REB", "RED", "REP", "RET", "RIB", "RID", "RIG", "RIM",
"RIO", "RIP", "ROB", "ROD", "ROE", "RON", "ROT", "ROW", "ROY", "RUB",
"RUE", "RUG", "RUM", "RUN", "RYE", "SAC", "SAD", "SAG", "SAL", "SAM",
"SAN", "SAP", "SAT", "SAW", "SAY", "SEA", "SEC", "SEE", "SEN", "SET",
"SEW", "SHE", "SHY", "SIN", "SIP", "SIR", "SIS", "SIT", "SKI", "SKY",
"SLY", "SO", "SOB", "SOD", "SON", "SOP", "SOW", "SOY", "SPA", "SPY",
"SUB", "SUD", "SUE", "SUM", "SUN", "SUP", "TAB", "TAD", "TAG", "TAN",
"TAP", "TAR", "TEA", "TED", "TEE", "TEN", "THE", "THY", "TIC", "TIE",
"TIM", "TIN", "TIP", "TO", "TOE", "TOG", "TOM", "TON", "TOO", "TOP",
"TOW", "TOY", "TRY", "TUB", "TUG", "TUM", "TUN", "TWO", "UN", "UP",
"US", "USE", "VAN", "VAT", "VET", "VIE", "WAD", "WAG", "WAR", "WAS",
"WAY", "WE", "WEB", "WED", "WEE", "WET", "WHO", "WHY", "WIN", "WIT",
"WOK", "WON", "WOO", "WOW", "WRY", "WU", "YAM", "YAP", "YAW", "YE",
"YEA", "YES", "YET", "YOU", "ABED", "ABEL", "ABET", "ABLE", "ABUT",
"ACHE", "ACID", "ACME", "ACRE", "ACTA", "ACTS", "ADAM", "ADDS",
"ADEN", "AFAR", "AFRO", "AGEE", "AHEM", "AHOY", "AIDA", "AIDE",
"AIDS", "AIRY", "AJAR", "AKIN", "ALAN", "ALEC", "ALGA", "ALIA",
"ALLY", "ALMA", "ALOE", "ALSO", "ALTO", "ALUM", "ALVA", "AMEN",
"AMES", "AMID", "AMMO", "AMOK", "AMOS", "AMRA", "ANDY", "ANEW",
"ANNA", "ANNE", "ANTE", "ANTI", "AQUA", "ARAB", "ARCH", "AREA",
"ARGO", "ARID", "ARMY", "ARTS", "ARTY", "ASIA", "ASKS", "ATOM",
"AUNT", "AURA", "AUTO", "AVER", "AVID", "AVIS", "AVON", "AVOW",
"AWAY", "AWRY", "BABE", "BABY", "BACH", "BACK", "BADE", "BAIL",
"BAIT", "BAKE", "BALD", "BALE", "BALI", "BALK", "BALL", "BALM",
"BAND", "BANE", "BANG", "BANK", "BARB", "BARD", "BARE", "BARK",
"BARN", "BARR", "BASE", "BASH", "BASK", "BASS", "BATE", "BATH",
"BAWD", "BAWL", "BEAD", "BEAK", "BEAM", "BEAN", "BEAR", "BEAT",
"BEAU", "BECK", "BEEF", "BEEN", "BEER",
"BEET", "BELA", "BELL", "BELT", "BEND", "BENT", "BERG", "BERN",
"BERT", "BESS", "BEST", "BETA", "BETH", "BHOY", "BIAS", "BIDE",
"BIEN", "BILE", "BILK", "BILL", "BIND", "BING", "BIRD", "BITE",
"BITS", "BLAB", "BLAT", "BLED", "BLEW", "BLOB", "BLOC", "BLOT",
"BLOW", "BLUE", "BLUM", "BLUR", "BOAR", "BOAT", "BOCA", "BOCK",
"BODE", "BODY", "BOGY", "BOHR", "BOIL", "BOLD", "BOLO", "BOLT",
"BOMB", "BONA", "BOND", "BONE", "BONG", "BONN", "BONY", "BOOK",
"BOOM", "BOON", "BOOT", "BORE", "BORG", "BORN", "BOSE", "BOSS",
"BOTH", "BOUT", "BOWL", "BOYD", "BRAD", "BRAE", "BRAG", "BRAN",
"BRAY", "BRED", "BREW", "BRIG", "BRIM", "BROW", "BUCK", "BUDD",
"BUFF", "BULB", "BULK", "BULL", "BUNK", "BUNT", "BUOY", "BURG",
"BURL", "BURN", "BURR", "BURT", "BURY", "BUSH", "BUSS", "BUST",
"BUSY", "BYTE", "CADY", "CAFE", "CAGE", "CAIN", "CAKE", "CALF",
"CALL", "CALM", "CAME", "CANE", "CANT", "CARD", "CARE", "CARL",
"CARR", "CART", "CASE", "CASH", "CASK", "CAST", "CAVE", "CEIL",
"CELL", "CENT", "CERN", "CHAD", "CHAR", "CHAT", "CHAW", "CHEF",
"CHEN", "CHEW", "CHIC", "CHIN", "CHOU", "CHOW", "CHUB", "CHUG",
"CHUM", "CITE", "CITY", "CLAD", "CLAM", "CLAN", "CLAW", "CLAY",
"CLOD", "CLOG", "CLOT", "CLUB", "CLUE", "COAL", "COAT", "COCA",
"COCK", "COCO", "CODA", "CODE", "CODY", "COED", "COIL", "COIN",
"COKE", "COLA", "COLD", "COLT", "COMA", "COMB", "COME", "COOK",
"COOL", "COON", "COOT", "CORD", "CORE", "CORK", "CORN", "COST",
"COVE", "COWL", "CRAB", "CRAG", "CRAM", "CRAY", "CREW", "CRIB",
"CROW", "CRUD", "CUBA", "CUBE", "CUFF", "CULL", "CULT", "CUNY",
"CURB", "CURD", "CURE", "CURL", "CURT", "CUTS", "DADE", "DALE",
"DAME", "DANA", "DANE", "DANG", "DANK", "DARE", "DARK", "DARN",
"DART", "DASH", "DATA", "DATE", "DAVE", "DAVY", "DAWN", "DAYS",
"DEAD", "DEAF", "DEAL", "DEAN", "DEAR", "DEBT", "DECK", "DEED",
"DEEM", "DEER", "DEFT", "DEFY", "DELL", "DENT", "DENY", "DESK",
"DIAL", "DICE", "DIED", "DIET", "DIME", "DINE", "DING", "DINT",
"DIRE", "DIRT", "DISC", "DISH", "DISK", "DIVE", "DOCK", "DOES",
"DOLE", "DOLL", "DOLT", "DOME", "DONE", "DOOM", "DOOR", "DORA",
"DOSE", "DOTE", "DOUG", "DOUR", "DOVE", "DOWN", "DRAB", "DRAG",
"DRAM", "DRAW", "DREW", "DRUB", "DRUG", "DRUM", "DUAL", "DUCK",
"DUCT", "DUEL", "DUET", "DUKE", "DULL", "DUMB", "DUNE", "DUNK",
"DUSK", "DUST", "DUTY", "EACH", "EARL", "EARN", "EASE", "EAST",
"EASY", "EBEN", "ECHO", "EDDY", "EDEN", "EDGE", "EDGY", "EDIT",
"EDNA", "EGAN", "ELAN", "ELBA", "ELLA", "ELSE", "EMIL", "EMIT",
"EMMA", "ENDS", "ERIC", "EROS", "EVEN", "EVER", "EVIL", "EYED",
"FACE", "FACT", "FADE", "FAIL", "FAIN", "FAIR", "FAKE", "FALL",
"FAME", "FANG", "FARM", "FAST", "FATE", "FAWN", "FEAR", "FEAT",
"FEED", "FEEL", "FEET", "FELL", "FELT", "FEND", "FERN", "FEST",
"FEUD", "FIEF", "FIGS", "FILE", "FILL", "FILM", "FIND", "FINE",
"FINK", "FIRE", "FIRM", "FISH", "FISK", "FIST", "FITS", "FIVE",
"FLAG", "FLAK", "FLAM", "FLAT", "FLAW", "FLEA", "FLED", "FLEW",
"FLIT", "FLOC", "FLOG", "FLOW", "FLUB", "FLUE", "FOAL", "FOAM",
"FOGY", "FOIL", "FOLD", "FOLK", "FOND", "FONT", "FOOD", "FOOL",
"FOOT", "FORD", "FORE", "FORK", "FORM", "FORT", "FOSS", "FOUL",
"FOUR", "FOWL", "FRAU", "FRAY", "FRED", "FREE", "FRET", "FREY",
"FROG", "FROM", "FUEL", "FULL", "FUME", "FUND", "FUNK", "FURY",
"FUSE", "FUSS", "GAFF", "GAGE", "GAIL", "GAIN", "GAIT", "GALA",
"GALE", "GALL", "GALT", "GAME", "GANG", "GARB", "GARY", "GASH",
"GATE", "GAUL", "GAUR", "GAVE", "GAWK", "GEAR", "GELD", "GENE",
"GENT", "GERM", "GETS", "GIBE", "GIFT", "GILD", "GILL", "GILT",
"GINA", "GIRD", "GIRL", "GIST", "GIVE", "GLAD", "GLEE", "GLEN",
"GLIB", "GLOB", "GLOM", "GLOW", "GLUE", "GLUM", "GLUT", "GOAD",
"GOAL", "GOAT", "GOER", "GOES", "GOLD", "GOLF", "GONE", "GONG",
"GOOD", "GOOF", "GORE", "GORY", "GOSH", "GOUT", "GOWN", "GRAB",
"GRAD", "GRAY", "GREG", "GREW", "GREY", "GRID", "GRIM", "GRIN",
"GRIT", "GROW", "GRUB", "GULF", "GULL", "GUNK", "GURU", "GUSH",
"GUST", "GWEN", "GWYN", "HAAG", "HAAS", "HACK", "HAIL", "HAIR",
"HALE", "HALF", "HALL", "HALO", "HALT", "HAND", "HANG", "HANK",
"HANS", "HARD", "HARK", "HARM", "HART", "HASH", "HAST", "HATE",
"HATH", "HAUL", "HAVE", "HAWK", "HAYS", "HEAD", "HEAL", "HEAR",
"HEAT", "HEBE", "HECK", "HEED", "HEEL", "HEFT", "HELD", "HELL",
"HELM", "HERB", "HERD", "HERE", "HERO", "HERS", "HESS", "HEWN",
"HICK", "HIDE", "HIGH", "HIKE", "HILL", "HILT", "HIND", "HINT",
"HIRE", "HISS", "HIVE", "HOBO", "HOCK", "HOFF", "HOLD", "HOLE",
"HOLM", "HOLT", "HOME", "HONE", "HONK", "HOOD", "HOOF", "HOOK",
"HOOT", "HORN", "HOSE", "HOST", "HOUR", "HOVE", "HOWE", "HOWL",
"HOYT", "HUCK", "HUED", "HUFF", "HUGE", "HUGH", "HUGO", "HULK",
"HULL", "HUNK", "HUNT", "HURD", "HURL", "HURT", "HUSH", "HYDE",
"HYMN", "IBIS", "ICON", "IDEA", "IDLE", "IFFY", "INCA", "INCH",
"INTO", "IONS", "IOTA", "IOWA", "IRIS", "IRMA", "IRON", "ISLE",
"ITCH", "ITEM", "IVAN", "JACK", "JADE", "JAIL", "JAKE", "JANE",
"JAVA", "JEAN", "JEFF", "JERK", "JESS", "JEST", "JIBE", "JILL",
"JILT", "JIVE", "JOAN", "JOBS", "JOCK", "JOEL", "JOEY", "JOHN",
"JOIN", "JOKE", "JOLT", "JOVE", "JUDD", "JUDE", "JUDO", "JUDY",
"JUJU", "JUKE", "JULY", "JUNE", "JUNK", "JUNO", "JURY", "JUST",
"JUTE", "KAHN", "KALE", "KANE", "KANT", "KARL", "KATE", "KEEL",
"KEEN", "KENO", "KENT", "KERN", "KERR", "KEYS", "KICK", "KILL",
"KIND", "KING", "KIRK", "KISS", "KITE", "KLAN", "KNEE", "KNEW",
"KNIT", "KNOB", "KNOT", "KNOW", "KOCH", "KONG", "KUDO", "KURD",
"KURT", "KYLE", "LACE", "LACK", "LACY", "LADY", "LAID", "LAIN",
"LAIR", "LAKE", "LAMB", "LAME", "LAND", "LANE", "LANG", "LARD",
"LARK", "LASS", "LAST", "LATE", "LAUD", "LAVA", "LAWN", "LAWS",
"LAYS", "LEAD", "LEAF", "LEAK", "LEAN", "LEAR", "LEEK", "LEER",
"LEFT", "LEND", "LENS", "LENT", "LEON", "LESK", "LESS", "LEST",
"LETS", "LIAR", "LICE", "LICK", "LIED", "LIEN", "LIES", "LIEU",
"LIFE", "LIFT", "LIKE", "LILA", "LILT", "LILY", "LIMA", "LIMB",
"LIME", "LIND", "LINE", "LINK", "LINT", "LION", "LISA", "LIST",
"LIVE", "LOAD", "LOAF", "LOAM", "LOAN", "LOCK", "LOFT", "LOGE",
"LOIS", "LOLA", "LONE", "LONG", "LOOK", "LOON", "LOOT", "LORD",
"LORE", "LOSE", "LOSS", "LOST", "LOUD", "LOVE", "LOWE", "LUCK",
"LUCY", "LUGE", "LUKE", "LULU", "LUND", "LUNG", "LURA", "LURE",
"LURK", "LUSH", "LUST", "LYLE", "LYNN", "LYON", "LYRA", "MACE",
"MADE", "MAGI", "MAID", "MAIL", "MAIN", "MAKE", "MALE", "MALI",
"MALL", "MALT", "MANA", "MANN", "MANY", "MARC", "MARE", "MARK",
"MARS", "MART", "MARY", "MASH", "MASK", "MASS", "MAST", "MATE",
"MATH", "MAUL", "MAYO", "MEAD", "MEAL", "MEAN", "MEAT", "MEEK",
"MEET", "MELD", "MELT", "MEMO", "MEND", "MENU", "MERT", "MESH",
"MESS", "MICE", "MIKE", "MILD", "MILE", "MILK", "MILL", "MILT",
"MIMI", "MIND", "MINE", "MINI", "MINK", "MINT", "MIRE", "MISS",
"MIST", "MITE", "MITT", "MOAN", "MOAT", "MOCK", "MODE", "MOLD",
"MOLE", "MOLL", "MOLT", "MONA", "MONK", "MONT", "MOOD", "MOON",
"MOOR", "MOOT", "MORE", "MORN", "MORT", "MOSS", "MOST", "MOTH",
"MOVE", "MUCH", "MUCK", "MUDD", "MUFF", "MULE", "MULL", "MURK",
"MUSH", "MUST", "MUTE", "MUTT", "MYRA", "MYTH", "NAGY", "NAIL",
"NAIR", "NAME", "NARY", "NASH", "NAVE", "NAVY", "NEAL", "NEAR",
"NEAT", "NECK", "NEED", "NEIL", "NELL", "NEON", "NERO", "NESS",
"NEST", "NEWS", "NEWT", "NIBS", "NICE", "NICK", "NILE", "NINA",
"NINE", "NOAH", "NODE", "NOEL", "NOLL", "NONE", "NOOK", "NOON",
"NORM", "NOSE", "NOTE", "NOUN", "NOVA", "NUDE", "NULL", "NUMB",
"OATH", "OBEY", "OBOE", "ODIN", "OHIO", "OILY", "OINT", "OKAY",
"OLAF", "OLDY", "OLGA", "OLIN", "OMAN", "OMEN", "OMIT", "ONCE",
"ONES", "ONLY", "ONTO", "ONUS", "ORAL", "ORGY", "OSLO", "OTIS",
"OTTO", "OUCH", "OUST", "OUTS", "OVAL", "OVEN", "OVER", "OWLY",
"OWNS", "QUAD", "QUIT", "QUOD", "RACE", "RACK", "RACY", "RAFT",
"RAGE", "RAID", "RAIL", "RAIN", "RAKE", "RANK", "RANT", "RARE",
"RASH", "RATE", "RAVE", "RAYS", "READ", "REAL", "REAM", "REAR",
"RECK", "REED", "REEF", "REEK", "REEL", "REID", "REIN", "RENA",
"REND", "RENT", "REST", "RICE", "RICH", "RICK", "RIDE", "RIFT",
"RILL", "RIME", "RING", "RINK", "RISE", "RISK", "RITE", "ROAD",
"ROAM", "ROAR", "ROBE", "ROCK", "RODE", "ROIL", "ROLL", "ROME",
"ROOD", "ROOF", "ROOK", "ROOM", "ROOT", "ROSA", "ROSE", "ROSS",
"ROSY", "ROTH", "ROUT", "ROVE", "ROWE", "ROWS", "RUBE", "RUBY",
"RUDE", "RUDY", "RUIN", "RULE", "RUNG", "RUNS", "RUNT", "RUSE",
"RUSH", "RUSK", "RUSS", "RUST", "RUTH", "SACK", "SAFE", "SAGE",
"SAID", "SAIL", "SALE", "SALK", "SALT", "SAME", "SAND", "SANE",
"SANG", "SANK", "SARA", "SAUL", "SAVE", "SAYS", "SCAN", "SCAR",
"SCAT", "SCOT", "SEAL", "SEAM", "SEAR", "SEAT", "SEED", "SEEK",
"SEEM", "SEEN", "SEES", "SELF", "SELL", "SEND", "SENT", "SETS",
"SEWN", "SHAG", "SHAM", "SHAW", "SHAY", "SHED", "SHIM", "SHIN",
"SHOD", "SHOE", "SHOT", "SHOW", "SHUN", "SHUT", "SICK", "SIDE",
"SIFT", "SIGH", "SIGN", "SILK", "SILL", "SILO", "SILT", "SINE",
"SING", "SINK", "SIRE", "SITE", "SITS", "SITU", "SKAT", "SKEW",
"SKID", "SKIM", "SKIN", "SKIT", "SLAB", "SLAM", "SLAT", "SLAY",
"SLED", "SLEW", "SLID", "SLIM", "SLIT", "SLOB", "SLOG", "SLOT",
"SLOW", "SLUG", "SLUM", "SLUR", "SMOG", "SMUG", "SNAG", "SNOB",
"SNOW", "SNUB", "SNUG", "SOAK", "SOAR", "SOCK", "SODA", "SOFA",
"SOFT", "SOIL", "SOLD", "SOME", "SONG", "SOON", "SOOT", "SORE",
"SORT", "SOUL", "SOUR", "SOWN", "STAB", "STAG", "STAN", "STAR",
"STAY", "STEM", "STEW", "STIR", "STOW", "STUB", "STUN", "SUCH",
"SUDS", "SUIT", "SULK", "SUMS", "SUNG", "SUNK", "SURE", "SURF",
"SWAB", "SWAG", "SWAM", "SWAN", "SWAT", "SWAY", "SWIM", "SWUM",
"TACK", "TACT", "TAIL", "TAKE", "TALE", "TALK", "TALL", "TANK",
"TASK", "TATE", "TAUT", "TEAL", "TEAM", "TEAR", "TECH", "TEEM",
"TEEN", "TEET", "TELL", "TEND", "TENT", "TERM", "TERN", "TESS",
"TEST", "THAN", "THAT", "THEE", "THEM", "THEN", "THEY", "THIN",
"THIS", "THUD", "THUG", "TICK", "TIDE", "TIDY", "TIED", "TIER",
"TILE", "TILL", "TILT", "TIME", "TINA", "TINE", "TINT", "TINY",
"TIRE", "TOAD", "TOGO", "TOIL", "TOLD", "TOLL", "TONE", "TONG",
"TONY", "TOOK", "TOOL", "TOOT", "TORE", "TORN", "TOTE", "TOUR",
"TOUT", "TOWN", "TRAG", "TRAM", "TRAY", "TREE", "TREK", "TRIG",
"TRIM", "TRIO", "TROD", "TROT", "TROY", "TRUE", "TUBA", "TUBE",
"TUCK", "TUFT", "TUNA", "TUNE", "TUNG", "TURF", "TURN", "TUSK",
"TWIG", "TWIN", "TWIT", "ULAN", "UNIT", "URGE", "USED", "USER",
"USES", "UTAH", "VAIL", "VAIN", "VALE", "VARY", "VASE", "VAST",
"VEAL", "VEDA", "VEIL", "VEIN", "VEND", "VENT", "VERB", "VERY",
"VETO", "VICE", "VIEW", "VINE", "VISE", "VOID", "VOLT", "VOTE",
"WACK", "WADE", "WAGE", "WAIL", "WAIT", "WAKE", "WALE", "WALK",
"WALL", "WALT", "WAND", "WANE", "WANG", "WANT", "WARD", "WARM",
"WARN", "WART", "WASH", "WAST", "WATS", "WATT", "WAVE", "WAVY",
"WAYS", "WEAK", "WEAL", "WEAN", "WEAR", "WEED", "WEEK", "WEIR",
"WELD", "WELL", "WELT", "WENT", "WERE", "WERT", "WEST", "WHAM",
"WHAT", "WHEE", "WHEN", "WHET", "WHOA", "WHOM", "WICK", "WIFE",
"WILD", "WILL", "WIND", "WINE", "WING", "WINK", "WINO", "WIRE",
"WISE", "WISH", "WITH", "WOLF", "WONT", "WOOD", "WOOL", "WORD",
"WORE", "WORK", "WORM", "WORN", "WOVE", "WRIT", "WYNN", "YALE",
"YANG", "YANK", "YARD", "YARN", "YAWL", "YAWN", "YEAH", "YEAR",
"YELL", "YOGA", "YOKE" ]
if __name__=='__main__':
data = [('EB33F77EE73D4053', 'TIDE ITCH SLOW REIN RULE MOT'),
('CCAC2AED591056BE4F90FD441C534766',
'RASH BUSH MILK LOOK BAD BRIM AVID GAFF BAIT ROT POD LOVE'),
('EFF81F9BFBC65350920CDD7416DE8009',
'TROD MUTE TAIL WARM CHAR KONG HAAG CITY BORE O TEAL AWL')
]
for key, words in data:
print 'Trying key', key
key=binascii.a2b_hex(key)
w2=key_to_english(key)
if w2!=words:
print 'key_to_english fails on key', repr(key), ', producing', str(w2)
k2=english_to_key(words)
if k2!=key:
print 'english_to_key fails on key', repr(key), ', producing', repr(k2)

16
gdata/Crypto/Util/__init__.py Normal file
View File

@ -0,0 +1,16 @@
"""Miscellaneous modules
Contains useful modules that don't belong into any of the
other Crypto.* subpackages.
Crypto.Util.number Number-theoretic functions (primality testing, etc.)
Crypto.Util.randpool Random number generation
Crypto.Util.RFC1751 Converts between 128-bit keys and human-readable
strings of words.
"""
__all__ = ['randpool', 'RFC1751', 'number']
__revision__ = "$Id: __init__.py,v 1.4 2003/02/28 15:26:00 akuchling Exp $"

201
gdata/Crypto/Util/number.py Normal file
View File

@ -0,0 +1,201 @@
#
# number.py : Number-theoretic functions
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: number.py,v 1.13 2003/04/04 18:21:07 akuchling Exp $"
bignum = long
try:
from Crypto.PublicKey import _fastmath
except ImportError:
_fastmath = None
# Commented out and replaced with faster versions below
## def long2str(n):
## s=''
## while n>0:
## s=chr(n & 255)+s
## n=n>>8
## return s
## import types
## def str2long(s):
## if type(s)!=types.StringType: return s # Integers will be left alone
## return reduce(lambda x,y : x*256+ord(y), s, 0L)
def size (N):
"""size(N:long) : int
Returns the size of the number N in bits.
"""
bits, power = 0,1L
while N >= power:
bits += 1
power = power << 1
return bits
def getRandomNumber(N, randfunc):
"""getRandomNumber(N:int, randfunc:callable):long
Return an N-bit random number."""
S = randfunc(N/8)
odd_bits = N % 8
if odd_bits != 0:
char = ord(randfunc(1)) >> (8-odd_bits)
S = chr(char) + S
value = bytes_to_long(S)
value |= 2L ** (N-1) # Ensure high bit is set
assert size(value) >= N
return value
def GCD(x,y):
"""GCD(x:long, y:long): long
Return the GCD of x and y.
"""
x = abs(x) ; y = abs(y)
while x > 0:
x, y = y % x, x
return y
def inverse(u, v):
"""inverse(u:long, u:long):long
Return the inverse of u mod v.
"""
u3, v3 = long(u), long(v)
u1, v1 = 1L, 0L
while v3 > 0:
q=u3 / v3
u1, v1 = v1, u1 - v1*q
u3, v3 = v3, u3 - v3*q
while u1<0:
u1 = u1 + v
return u1
# Given a number of bits to generate and a random generation function,
# find a prime number of the appropriate size.
def getPrime(N, randfunc):
"""getPrime(N:int, randfunc:callable):long
Return a random N-bit prime number.
"""
number=getRandomNumber(N, randfunc) | 1
while (not isPrime(number)):
number=number+2
return number
def isPrime(N):
"""isPrime(N:long):bool
Return true if N is prime.
"""
if N == 1:
return 0
if N in sieve:
return 1
for i in sieve:
if (N % i)==0:
return 0
# Use the accelerator if available
if _fastmath is not None:
return _fastmath.isPrime(N)
# Compute the highest bit that's set in N
N1 = N - 1L
n = 1L
while (n<N):
n=n<<1L
n = n >> 1L
# Rabin-Miller test
for c in sieve[:7]:
a=long(c) ; d=1L ; t=n
while (t): # Iterate over the bits in N1
x=(d*d) % N
if x==1L and d!=1L and d!=N1:
return 0 # Square root of 1 found
if N1 & t:
d=(x*a) % N
else:
d=x
t = t >> 1L
if d!=1L:
return 0
return 1
# Small primes used for checking primality; these are all the primes
# less than 256. This should be enough to eliminate most of the odd
# numbers before needing to do a Rabin-Miller test at all.
sieve=[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59,
61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113, 127,
131, 137, 139, 149, 151, 157, 163, 167, 173, 179, 181, 191, 193,
197, 199, 211, 223, 227, 229, 233, 239, 241, 251]
# Improved conversion functions contributed by Barry Warsaw, after
# careful benchmarking
import struct
def long_to_bytes(n, blocksize=0):
"""long_to_bytes(n:long, blocksize:int) : string
Convert a long integer to a byte string.
If optional blocksize is given and greater than zero, pad the front of the
byte string with binary zeros so that the length is a multiple of
blocksize.
"""
# after much testing, this algorithm was deemed to be the fastest
s = ''
n = long(n)
pack = struct.pack
while n > 0:
s = pack('>I', n & 0xffffffffL) + s
n = n >> 32
# strip off leading zeros
for i in range(len(s)):
if s[i] != '\000':
break
else:
# only happens when n == 0
s = '\000'
i = 0
s = s[i:]
# add back some pad bytes. this could be done more efficiently w.r.t. the
# de-padding being done above, but sigh...
if blocksize > 0 and len(s) % blocksize:
s = (blocksize - len(s) % blocksize) * '\000' + s
return s
def bytes_to_long(s):
"""bytes_to_long(string) : long
Convert a byte string to a long integer.
This is (essentially) the inverse of long_to_bytes().
"""
acc = 0L
unpack = struct.unpack
length = len(s)
if length % 4:
extra = (4 - length % 4)
s = '\000' * extra + s
length = length + extra
for i in range(0, length, 4):
acc = (acc << 32) + unpack('>I', s[i:i+4])[0]
return acc
# For backwards compatibility...
import warnings
def long2str(n, blocksize=0):
warnings.warn("long2str() has been replaced by long_to_bytes()")
return long_to_bytes(n, blocksize)
def str2long(s):
warnings.warn("str2long() has been replaced by bytes_to_long()")
return bytes_to_long(s)

421
gdata/Crypto/Util/randpool.py Normal file
View File

@ -0,0 +1,421 @@
#
# randpool.py : Cryptographically strong random number generation
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: randpool.py,v 1.14 2004/05/06 12:56:54 akuchling Exp $"
import time, array, types, warnings, os.path
from Crypto.Util.number import long_to_bytes
try:
import Crypto.Util.winrandom as winrandom
except:
winrandom = None
STIRNUM = 3
class RandomPool:
"""randpool.py : Cryptographically strong random number generation.
The implementation here is similar to the one in PGP. To be
cryptographically strong, it must be difficult to determine the RNG's
output, whether in the future or the past. This is done by using
a cryptographic hash function to "stir" the random data.
Entropy is gathered in the same fashion as PGP; the highest-resolution
clock around is read and the data is added to the random number pool.
A conservative estimate of the entropy is then kept.
If a cryptographically secure random source is available (/dev/urandom
on many Unixes, Windows CryptGenRandom on most Windows), then use
it.
Instance Attributes:
bits : int
Maximum size of pool in bits
bytes : int
Maximum size of pool in bytes
entropy : int
Number of bits of entropy in this pool.
Methods:
add_event([s]) : add some entropy to the pool
get_bytes(int) : get N bytes of random data
randomize([N]) : get N bytes of randomness from external source
"""
def __init__(self, numbytes = 160, cipher=None, hash=None):
if hash is None:
from Crypto.Hash import SHA as hash
# The cipher argument is vestigial; it was removed from
# version 1.1 so RandomPool would work even in the limited
# exportable subset of the code
if cipher is not None:
warnings.warn("'cipher' parameter is no longer used")
if isinstance(hash, types.StringType):
# ugly hack to force __import__ to give us the end-path module
hash = __import__('Crypto.Hash.'+hash,
None, None, ['new'])
warnings.warn("'hash' parameter should now be a hashing module")
self.bytes = numbytes
self.bits = self.bytes*8
self.entropy = 0
self._hash = hash
# Construct an array to hold the random pool,
# initializing it to 0.
self._randpool = array.array('B', [0]*self.bytes)
self._event1 = self._event2 = 0
self._addPos = 0
self._getPos = hash.digest_size
self._lastcounter=time.time()
self.__counter = 0
self._measureTickSize() # Estimate timer resolution
self._randomize()
def _updateEntropyEstimate(self, nbits):
self.entropy += nbits
if self.entropy < 0:
self.entropy = 0
elif self.entropy > self.bits:
self.entropy = self.bits
def _randomize(self, N = 0, devname = '/dev/urandom'):
"""_randomize(N, DEVNAME:device-filepath)
collects N bits of randomness from some entropy source (e.g.,
/dev/urandom on Unixes that have it, Windows CryptoAPI
CryptGenRandom, etc)
DEVNAME is optional, defaults to /dev/urandom. You can change it
to /dev/random if you want to block till you get enough
entropy.
"""
data = ''
if N <= 0:
nbytes = int((self.bits - self.entropy)/8+0.5)
else:
nbytes = int(N/8+0.5)
if winrandom:
# Windows CryptGenRandom provides random data.
data = winrandom.new().get_bytes(nbytes)
elif os.path.exists(devname):
# Many OSes support a /dev/urandom device
try:
f=open(devname)
data=f.read(nbytes)
f.close()
except IOError, (num, msg):
if num!=2: raise IOError, (num, msg)
# If the file wasn't found, ignore the error
if data:
self._addBytes(data)
# Entropy estimate: The number of bits of
# data obtained from the random source.
self._updateEntropyEstimate(8*len(data))
self.stir_n() # Wash the random pool
def randomize(self, N=0):
"""randomize(N:int)
use the class entropy source to get some entropy data.
This is overridden by KeyboardRandomize().
"""
return self._randomize(N)
def stir_n(self, N = STIRNUM):
"""stir_n(N)
stirs the random pool N times
"""
for i in xrange(N):
self.stir()
def stir (self, s = ''):
"""stir(s:string)
Mix up the randomness pool. This will call add_event() twice,
but out of paranoia the entropy attribute will not be
increased. The optional 's' parameter is a string that will
be hashed with the randomness pool.
"""
entropy=self.entropy # Save inital entropy value
self.add_event()
# Loop over the randomness pool: hash its contents
# along with a counter, and add the resulting digest
# back into the pool.
for i in range(self.bytes / self._hash.digest_size):
h = self._hash.new(self._randpool)
h.update(str(self.__counter) + str(i) + str(self._addPos) + s)
self._addBytes( h.digest() )
self.__counter = (self.__counter + 1) & 0xFFFFffffL
self._addPos, self._getPos = 0, self._hash.digest_size
self.add_event()
# Restore the old value of the entropy.
self.entropy=entropy
def get_bytes (self, N):
"""get_bytes(N:int) : string
Return N bytes of random data.
"""
s=''
i, pool = self._getPos, self._randpool
h=self._hash.new()
dsize = self._hash.digest_size
num = N
while num > 0:
h.update( self._randpool[i:i+dsize] )
s = s + h.digest()
num = num - dsize
i = (i + dsize) % self.bytes
if i<dsize:
self.stir()
i=self._getPos
self._getPos = i
self._updateEntropyEstimate(- 8*N)
return s[:N]
def add_event(self, s=''):
"""add_event(s:string)
Add an event to the random pool. The current time is stored
between calls and used to estimate the entropy. The optional
's' parameter is a string that will also be XORed into the pool.
Returns the estimated number of additional bits of entropy gain.
"""
event = time.time()*1000
delta = self._noise()
s = (s + long_to_bytes(event) +
4*chr(0xaa) + long_to_bytes(delta) )
self._addBytes(s)
if event==self._event1 and event==self._event2:
# If events are coming too closely together, assume there's
# no effective entropy being added.
bits=0
else:
# Count the number of bits in delta, and assume that's the entropy.
bits=0
while delta:
delta, bits = delta>>1, bits+1
if bits>8: bits=8
self._event1, self._event2 = event, self._event1
self._updateEntropyEstimate(bits)
return bits
# Private functions
def _noise(self):
# Adds a bit of noise to the random pool, by adding in the
# current time and CPU usage of this process.
# The difference from the previous call to _noise() is taken
# in an effort to estimate the entropy.
t=time.time()
delta = (t - self._lastcounter)/self._ticksize*1e6
self._lastcounter = t
self._addBytes(long_to_bytes(long(1000*time.time())))
self._addBytes(long_to_bytes(long(1000*time.clock())))
self._addBytes(long_to_bytes(long(1000*time.time())))
self._addBytes(long_to_bytes(long(delta)))
# Reduce delta to a maximum of 8 bits so we don't add too much
# entropy as a result of this call.
delta=delta % 0xff
return int(delta)
def _measureTickSize(self):
# _measureTickSize() tries to estimate a rough average of the
# resolution of time that you can see from Python. It does
# this by measuring the time 100 times, computing the delay
# between measurements, and taking the median of the resulting
# list. (We also hash all the times and add them to the pool)
interval = [None] * 100
h = self._hash.new(`(id(self),id(interval))`)
# Compute 100 differences
t=time.time()
h.update(`t`)
i = 0
j = 0
while i < 100:
t2=time.time()
h.update(`(i,j,t2)`)
j += 1
delta=int((t2-t)*1e6)
if delta:
interval[i] = delta
i += 1
t=t2
# Take the median of the array of intervals
interval.sort()
self._ticksize=interval[len(interval)/2]
h.update(`(interval,self._ticksize)`)
# mix in the measurement times and wash the random pool
self.stir(h.digest())
def _addBytes(self, s):
"XOR the contents of the string S into the random pool"
i, pool = self._addPos, self._randpool
for j in range(0, len(s)):
pool[i]=pool[i] ^ ord(s[j])
i=(i+1) % self.bytes
self._addPos = i
# Deprecated method names: remove in PCT 2.1 or later.
def getBytes(self, N):
warnings.warn("getBytes() method replaced by get_bytes()",
DeprecationWarning)
return self.get_bytes(N)
def addEvent (self, event, s=""):
warnings.warn("addEvent() method replaced by add_event()",
DeprecationWarning)
return self.add_event(s + str(event))
class PersistentRandomPool (RandomPool):
def __init__ (self, filename=None, *args, **kwargs):
RandomPool.__init__(self, *args, **kwargs)
self.filename = filename
if filename:
try:
# the time taken to open and read the file might have
# a little disk variability, modulo disk/kernel caching...
f=open(filename, 'rb')
self.add_event()
data = f.read()
self.add_event()
# mix in the data from the file and wash the random pool
self.stir(data)
f.close()
except IOError:
# Oh, well; the file doesn't exist or is unreadable, so
# we'll just ignore it.
pass
def save(self):
if self.filename == "":
raise ValueError, "No filename set for this object"
# wash the random pool before save, provides some forward secrecy for
# old values of the pool.
self.stir_n()
f=open(self.filename, 'wb')
self.add_event()
f.write(self._randpool.tostring())
f.close()
self.add_event()
# wash the pool again, provide some protection for future values
self.stir()
# non-echoing Windows keyboard entry
_kb = 0
if not _kb:
try:
import msvcrt
class KeyboardEntry:
def getch(self):
c = msvcrt.getch()
if c in ('\000', '\xe0'):
# function key
c += msvcrt.getch()
return c
def close(self, delay = 0):
if delay:
time.sleep(delay)
while msvcrt.kbhit():
msvcrt.getch()
_kb = 1
except:
pass
# non-echoing Posix keyboard entry
if not _kb:
try:
import termios
class KeyboardEntry:
def __init__(self, fd = 0):
self._fd = fd
self._old = termios.tcgetattr(fd)
new = termios.tcgetattr(fd)
new[3]=new[3] & ~termios.ICANON & ~termios.ECHO
termios.tcsetattr(fd, termios.TCSANOW, new)
def getch(self):
termios.tcflush(0, termios.TCIFLUSH) # XXX Leave this in?
return os.read(self._fd, 1)
def close(self, delay = 0):
if delay:
time.sleep(delay)
termios.tcflush(self._fd, termios.TCIFLUSH)
termios.tcsetattr(self._fd, termios.TCSAFLUSH, self._old)
_kb = 1
except:
pass
class KeyboardRandomPool (PersistentRandomPool):
def __init__(self, *args, **kwargs):
PersistentRandomPool.__init__(self, *args, **kwargs)
def randomize(self, N = 0):
"Adds N bits of entropy to random pool. If N is 0, fill up pool."
import os, string, time
if N <= 0:
bits = self.bits - self.entropy
else:
bits = N*8
if bits == 0:
return
print bits,'bits of entropy are now required. Please type on the keyboard'
print 'until enough randomness has been accumulated.'
kb = KeyboardEntry()
s='' # We'll save the characters typed and add them to the pool.
hash = self._hash
e = 0
try:
while e < bits:
temp=str(bits-e).rjust(6)
os.write(1, temp)
s=s+kb.getch()
e += self.add_event(s)
os.write(1, 6*chr(8))
self.add_event(s+hash.new(s).digest() )
finally:
kb.close()
print '\n\007 Enough. Please wait a moment.\n'
self.stir_n() # wash the random pool.
kb.close(4)
if __name__ == '__main__':
pool = RandomPool()
print 'random pool entropy', pool.entropy, 'bits'
pool.add_event('something')
print `pool.get_bytes(100)`
import tempfile, os
fname = tempfile.mktemp()
pool = KeyboardRandomPool(filename=fname)
print 'keyboard random pool entropy', pool.entropy, 'bits'
pool.randomize()
print 'keyboard random pool entropy', pool.entropy, 'bits'
pool.randomize(128)
pool.save()
saved = open(fname, 'rb').read()
print 'saved', `saved`
print 'pool ', `pool._randpool.tostring()`
newpool = PersistentRandomPool(fname)
print 'persistent random pool entropy', pool.entropy, 'bits'
os.remove(fname)

453
gdata/Crypto/Util/test.py Normal file
View File

@ -0,0 +1,453 @@
#
# test.py : Functions used for testing the modules
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: test.py,v 1.16 2004/08/13 22:24:18 akuchling Exp $"
import binascii
import string
import testdata
from Crypto.Cipher import *
def die(string):
import sys
print '***ERROR: ', string
# sys.exit(0) # Will default to continuing onward...
def print_timing (size, delta, verbose):
if verbose:
if delta == 0:
print 'Unable to measure time -- elapsed time too small'
else:
print '%.2f K/sec' % (size/delta)
def exerciseBlockCipher(cipher, verbose):
import string, time
try:
ciph = eval(cipher)
except NameError:
print cipher, 'module not available'
return None
print cipher+ ':'
str='1' # Build 128K of test data
for i in xrange(0, 17):
str=str+str
if ciph.key_size==0: ciph.key_size=16
password = 'password12345678Extra text for password'[0:ciph.key_size]
IV = 'Test IV Test IV Test IV Test'[0:ciph.block_size]
if verbose: print ' ECB mode:',
obj=ciph.new(password, ciph.MODE_ECB)
if obj.block_size != ciph.block_size:
die("Module and cipher object block_size don't match")
text='1234567812345678'[0:ciph.block_size]
c=obj.encrypt(text)
if (obj.decrypt(c)!=text): die('Error encrypting "'+text+'"')
text='KuchlingKuchling'[0:ciph.block_size]
c=obj.encrypt(text)
if (obj.decrypt(c)!=text): die('Error encrypting "'+text+'"')
text='NotTodayNotEver!'[0:ciph.block_size]
c=obj.encrypt(text)
if (obj.decrypt(c)!=text): die('Error encrypting "'+text+'"')
start=time.time()
s=obj.encrypt(str)
s2=obj.decrypt(s)
end=time.time()
if (str!=s2):
die('Error in resulting plaintext from ECB mode')
print_timing(256, end-start, verbose)
del obj
if verbose: print ' CFB mode:',
obj1=ciph.new(password, ciph.MODE_CFB, IV)
obj2=ciph.new(password, ciph.MODE_CFB, IV)
start=time.time()
ciphertext=obj1.encrypt(str[0:65536])
plaintext=obj2.decrypt(ciphertext)
end=time.time()
if (plaintext!=str[0:65536]):
die('Error in resulting plaintext from CFB mode')
print_timing(64, end-start, verbose)
del obj1, obj2
if verbose: print ' CBC mode:',
obj1=ciph.new(password, ciph.MODE_CBC, IV)
obj2=ciph.new(password, ciph.MODE_CBC, IV)
start=time.time()
ciphertext=obj1.encrypt(str)
plaintext=obj2.decrypt(ciphertext)
end=time.time()
if (plaintext!=str):
die('Error in resulting plaintext from CBC mode')
print_timing(256, end-start, verbose)
del obj1, obj2
if verbose: print ' PGP mode:',
obj1=ciph.new(password, ciph.MODE_PGP, IV)
obj2=ciph.new(password, ciph.MODE_PGP, IV)
start=time.time()
ciphertext=obj1.encrypt(str)
plaintext=obj2.decrypt(ciphertext)
end=time.time()
if (plaintext!=str):
die('Error in resulting plaintext from PGP mode')
print_timing(256, end-start, verbose)
del obj1, obj2
if verbose: print ' OFB mode:',
obj1=ciph.new(password, ciph.MODE_OFB, IV)
obj2=ciph.new(password, ciph.MODE_OFB, IV)
start=time.time()
ciphertext=obj1.encrypt(str)
plaintext=obj2.decrypt(ciphertext)
end=time.time()
if (plaintext!=str):
die('Error in resulting plaintext from OFB mode')
print_timing(256, end-start, verbose)
del obj1, obj2
def counter(length=ciph.block_size):
return length * 'a'
if verbose: print ' CTR mode:',
obj1=ciph.new(password, ciph.MODE_CTR, counter=counter)
obj2=ciph.new(password, ciph.MODE_CTR, counter=counter)
start=time.time()
ciphertext=obj1.encrypt(str)
plaintext=obj2.decrypt(ciphertext)
end=time.time()
if (plaintext!=str):
die('Error in resulting plaintext from CTR mode')
print_timing(256, end-start, verbose)
del obj1, obj2
# Test the IV handling
if verbose: print ' Testing IV handling'
obj1=ciph.new(password, ciph.MODE_CBC, IV)
plaintext='Test'*(ciph.block_size/4)*3
ciphertext1=obj1.encrypt(plaintext)
obj1.IV=IV
ciphertext2=obj1.encrypt(plaintext)
if ciphertext1!=ciphertext2:
die('Error in setting IV')
# Test keyword arguments
obj1=ciph.new(key=password)
obj1=ciph.new(password, mode=ciph.MODE_CBC)
obj1=ciph.new(mode=ciph.MODE_CBC, key=password)
obj1=ciph.new(IV=IV, mode=ciph.MODE_CBC, key=password)
return ciph
def exerciseStreamCipher(cipher, verbose):
import string, time
try:
ciph = eval(cipher)
except (NameError):
print cipher, 'module not available'
return None
print cipher + ':',
str='1' # Build 128K of test data
for i in xrange(0, 17):
str=str+str
key_size = ciph.key_size or 16
password = 'password12345678Extra text for password'[0:key_size]
obj1=ciph.new(password)
obj2=ciph.new(password)
if obj1.block_size != ciph.block_size:
die("Module and cipher object block_size don't match")
if obj1.key_size != ciph.key_size:
die("Module and cipher object key_size don't match")
text='1234567812345678Python'
c=obj1.encrypt(text)
if (obj2.decrypt(c)!=text): die('Error encrypting "'+text+'"')
text='B1FF I2 A R3A11Y |<00L D00D!!!!!'
c=obj1.encrypt(text)
if (obj2.decrypt(c)!=text): die('Error encrypting "'+text+'"')
text='SpamSpamSpamSpamSpamSpamSpamSpamSpam'
c=obj1.encrypt(text)
if (obj2.decrypt(c)!=text): die('Error encrypting "'+text+'"')
start=time.time()
s=obj1.encrypt(str)
str=obj2.decrypt(s)
end=time.time()
print_timing(256, end-start, verbose)
del obj1, obj2
return ciph
def TestStreamModules(args=['arc4', 'XOR'], verbose=1):
import sys, string
args=map(string.lower, args)
if 'arc4' in args:
# Test ARC4 stream cipher
arc4=exerciseStreamCipher('ARC4', verbose)
if (arc4!=None):
for entry in testdata.arc4:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=arc4.new(key)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('ARC4 failed on entry '+`entry`)
if 'xor' in args:
# Test XOR stream cipher
XOR=exerciseStreamCipher('XOR', verbose)
if (XOR!=None):
for entry in testdata.xor:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=XOR.new(key)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('XOR failed on entry '+`entry`)
def TestBlockModules(args=['aes', 'arc2', 'des', 'blowfish', 'cast', 'des3',
'idea', 'rc5'],
verbose=1):
import string
args=map(string.lower, args)
if 'aes' in args:
ciph=exerciseBlockCipher('AES', verbose) # AES
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.aes:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('AES failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print
for entry in testdata.aes_modes:
mode, key, plain, cipher, kw = entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, mode, **kw)
obj2=ciph.new(key, mode, **kw)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('AES encrypt failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print
plain2=obj2.decrypt(ciphertext)
if plain2!=plain:
die('AES decrypt failed on entry '+`entry`)
for i in plain2:
if verbose: print hex(ord(i)),
if verbose: print
if 'arc2' in args:
ciph=exerciseBlockCipher('ARC2', verbose) # Alleged RC2
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.arc2:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('ARC2 failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
print
if 'blowfish' in args:
ciph=exerciseBlockCipher('Blowfish',verbose)# Bruce Schneier's Blowfish cipher
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.blowfish:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('Blowfish failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print
if 'cast' in args:
ciph=exerciseBlockCipher('CAST', verbose) # CAST-128
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.cast:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('CAST failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print
if 0:
# The full-maintenance test; it requires 4 million encryptions,
# and correspondingly is quite time-consuming. I've disabled
# it; it's faster to compile block/cast.c with -DTEST and run
# the resulting program.
a = b = '\x01\x23\x45\x67\x12\x34\x56\x78\x23\x45\x67\x89\x34\x56\x78\x9A'
for i in range(0, 1000000):
obj = cast.new(b, cast.MODE_ECB)
a = obj.encrypt(a[:8]) + obj.encrypt(a[-8:])
obj = cast.new(a, cast.MODE_ECB)
b = obj.encrypt(b[:8]) + obj.encrypt(b[-8:])
if a!="\xEE\xA9\xD0\xA2\x49\xFD\x3B\xA6\xB3\x43\x6F\xB8\x9D\x6D\xCA\x92":
if verbose: print 'CAST test failed: value of "a" doesn\'t match'
if b!="\xB2\xC9\x5E\xB0\x0C\x31\xAD\x71\x80\xAC\x05\xB8\xE8\x3D\x69\x6E":
if verbose: print 'CAST test failed: value of "b" doesn\'t match'
if 'des' in args:
# Test/benchmark DES block cipher
des=exerciseBlockCipher('DES', verbose)
if (des!=None):
# Various tests taken from the DES library packaged with Kerberos V4
obj=des.new(binascii.a2b_hex('0123456789abcdef'), des.MODE_ECB)
s=obj.encrypt('Now is t')
if (s!=binascii.a2b_hex('3fa40e8a984d4815')):
die('DES fails test 1')
obj=des.new(binascii.a2b_hex('08192a3b4c5d6e7f'), des.MODE_ECB)
s=obj.encrypt('\000\000\000\000\000\000\000\000')
if (s!=binascii.a2b_hex('25ddac3e96176467')):
die('DES fails test 2')
obj=des.new(binascii.a2b_hex('0123456789abcdef'), des.MODE_CBC,
binascii.a2b_hex('1234567890abcdef'))
s=obj.encrypt("Now is the time for all ")
if (s!=binascii.a2b_hex('e5c7cdde872bf27c43e934008c389c0f683788499a7c05f6')):
die('DES fails test 3')
obj=des.new(binascii.a2b_hex('0123456789abcdef'), des.MODE_CBC,
binascii.a2b_hex('fedcba9876543210'))
s=obj.encrypt("7654321 Now is the time for \000\000\000\000")
if (s!=binascii.a2b_hex("ccd173ffab2039f4acd8aefddfd8a1eb468e91157888ba681d269397f7fe62b4")):
die('DES fails test 4')
del obj,s
# R. Rivest's test: see http://theory.lcs.mit.edu/~rivest/destest.txt
x=binascii.a2b_hex('9474B8E8C73BCA7D')
for i in range(0, 16):
obj=des.new(x, des.MODE_ECB)
if (i & 1): x=obj.decrypt(x)
else: x=obj.encrypt(x)
if x!=binascii.a2b_hex('1B1A2DDB4C642438'):
die("DES fails Rivest's test")
if verbose: print ' Verifying against test suite...'
for entry in testdata.des:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=des.new(key, des.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('DES failed on entry '+`entry`)
for entry in testdata.des_cbc:
key, iv, plain, cipher=entry
key, iv, cipher=binascii.a2b_hex(key),binascii.a2b_hex(iv),binascii.a2b_hex(cipher)
obj1=des.new(key, des.MODE_CBC, iv)
obj2=des.new(key, des.MODE_CBC, iv)
ciphertext=obj1.encrypt(plain)
if (ciphertext!=cipher):
die('DES CBC mode failed on entry '+`entry`)
if 'des3' in args:
ciph=exerciseBlockCipher('DES3', verbose) # Triple DES
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.des3:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('DES3 failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print
for entry in testdata.des3_cbc:
key, iv, plain, cipher=entry
key, iv, cipher=binascii.a2b_hex(key),binascii.a2b_hex(iv),binascii.a2b_hex(cipher)
obj1=ciph.new(key, ciph.MODE_CBC, iv)
obj2=ciph.new(key, ciph.MODE_CBC, iv)
ciphertext=obj1.encrypt(plain)
if (ciphertext!=cipher):
die('DES3 CBC mode failed on entry '+`entry`)
if 'idea' in args:
ciph=exerciseBlockCipher('IDEA', verbose) # IDEA block cipher
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.idea:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('IDEA failed on entry '+`entry`)
if 'rc5' in args:
# Ronald Rivest's RC5 algorithm
ciph=exerciseBlockCipher('RC5', verbose)
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.rc5:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key[4:], ciph.MODE_ECB,
version =ord(key[0]),
word_size=ord(key[1]),
rounds =ord(key[2]) )
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('RC5 failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print

25
gdata/Crypto/__init__.py Normal file
View File

@ -0,0 +1,25 @@
"""Python Cryptography Toolkit
A collection of cryptographic modules implementing various algorithms
and protocols.
Subpackages:
Crypto.Cipher Secret-key encryption algorithms (AES, DES, ARC4)
Crypto.Hash Hashing algorithms (MD5, SHA, HMAC)
Crypto.Protocol Cryptographic protocols (Chaffing, all-or-nothing
transform). This package does not contain any
network protocols.
Crypto.PublicKey Public-key encryption and signature algorithms
(RSA, DSA)
Crypto.Util Various useful modules and functions (long-to-string
conversion, random number generation, number
theoretic functions)
"""
__all__ = ['Cipher', 'Hash', 'Protocol', 'PublicKey', 'Util']
__version__ = '2.0.1'
__revision__ = "$Id: __init__.py,v 1.12 2005/06/14 01:20:22 akuchling Exp $"

38
gdata/Crypto/test.py Normal file
View File

@ -0,0 +1,38 @@
#
# Test script for the Python Cryptography Toolkit.
#
__revision__ = "$Id: test.py,v 1.7 2002/07/11 14:31:19 akuchling Exp $"
import os, sys
# Add the build directory to the front of sys.path
from distutils.util import get_platform
s = "build/lib.%s-%.3s" % (get_platform(), sys.version)
s = os.path.join(os.getcwd(), s)
sys.path.insert(0, s)
s = os.path.join(os.getcwd(), 'test')
sys.path.insert(0, s)
from Crypto.Util import test
args = sys.argv[1:]
quiet = "--quiet" in args
if quiet: args.remove('--quiet')
if not quiet:
print '\nStream Ciphers:'
print '==============='
if args: test.TestStreamModules(args, verbose= not quiet)
else: test.TestStreamModules(verbose= not quiet)
if not quiet:
print '\nBlock Ciphers:'
print '=============='
if args: test.TestBlockModules(args, verbose= not quiet)
else: test.TestBlockModules(verbose= not quiet)

835
gdata/__init__.py Normal file
View File

@ -0,0 +1,835 @@
#!/usr/bin/python
#
# Copyright (C) 2006 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Contains classes representing Google Data elements.
Extends Atom classes to add Google Data specific elements.
"""
__author__ = 'j.s@google.com (Jeffrey Scudder)'
import os
import atom
try:
from xml.etree import cElementTree as ElementTree
except ImportError:
try:
import cElementTree as ElementTree
except ImportError:
try:
from xml.etree import ElementTree
except ImportError:
from elementtree import ElementTree
# XML namespaces which are often used in GData entities.
GDATA_NAMESPACE = 'http://schemas.google.com/g/2005'
GDATA_TEMPLATE = '{http://schemas.google.com/g/2005}%s'
OPENSEARCH_NAMESPACE = 'http://a9.com/-/spec/opensearchrss/1.0/'
OPENSEARCH_TEMPLATE = '{http://a9.com/-/spec/opensearchrss/1.0/}%s'
BATCH_NAMESPACE = 'http://schemas.google.com/gdata/batch'
GACL_NAMESPACE = 'http://schemas.google.com/acl/2007'
GACL_TEMPLATE = '{http://schemas.google.com/acl/2007}%s'
# Labels used in batch request entries to specify the desired CRUD operation.
BATCH_INSERT = 'insert'
BATCH_UPDATE = 'update'
BATCH_DELETE = 'delete'
BATCH_QUERY = 'query'
class Error(Exception):
pass
class MissingRequiredParameters(Error):
pass
class MediaSource(object):
"""GData Entries can refer to media sources, so this class provides a
place to store references to these objects along with some metadata.
"""
def __init__(self, file_handle=None, content_type=None, content_length=None,
file_path=None, file_name=None):
"""Creates an object of type MediaSource.
Args:
file_handle: A file handle pointing to the file to be encapsulated in the
MediaSource
content_type: string The MIME type of the file. Required if a file_handle
is given.
content_length: int The size of the file. Required if a file_handle is
given.
file_path: string (optional) A full path name to the file. Used in
place of a file_handle.
file_name: string The name of the file without any path information.
Required if a file_handle is given.
"""
self.file_handle = file_handle
self.content_type = content_type
self.content_length = content_length
self.file_name = file_name
if (file_handle is None and content_type is not None and
file_path is not None):
self.setFile(file_path, content_type)
def setFile(self, file_name, content_type):
"""A helper function which can create a file handle from a given filename
and set the content type and length all at once.
Args:
file_name: string The path and file name to the file containing the media
content_type: string A MIME type representing the type of the media
"""
self.file_handle = open(file_name, 'rb')
self.content_type = content_type
self.content_length = os.path.getsize(file_name)
self.file_name = os.path.basename(file_name)
class LinkFinder(atom.LinkFinder):
"""An "interface" providing methods to find link elements
GData Entry elements often contain multiple links which differ in the rel
attribute or content type. Often, developers are interested in a specific
type of link so this class provides methods to find specific classes of
links.
This class is used as a mixin in GData entries.
"""
def GetSelfLink(self):
"""Find the first link with rel set to 'self'
Returns:
An atom.Link or none if none of the links had rel equal to 'self'
"""
for a_link in self.link:
if a_link.rel == 'self':
return a_link
return None
def GetEditLink(self):
for a_link in self.link:
if a_link.rel == 'edit':
return a_link
return None
def GetEditMediaLink(self):
"""The Picasa API mistakenly returns media-edit rather than edit-media, but
this may change soon.
"""
for a_link in self.link:
if a_link.rel == 'edit-media':
return a_link
if a_link.rel == 'media-edit':
return a_link
return None
def GetHtmlLink(self):
"""Find the first link with rel of alternate and type of text/html
Returns:
An atom.Link or None if no links matched
"""
for a_link in self.link:
if a_link.rel == 'alternate' and a_link.type == 'text/html':
return a_link
return None
def GetPostLink(self):
"""Get a link containing the POST target URL.
The POST target URL is used to insert new entries.
Returns:
A link object with a rel matching the POST type.
"""
for a_link in self.link:
if a_link.rel == 'http://schemas.google.com/g/2005#post':
return a_link
return None
def GetAclLink(self):
for a_link in self.link:
if a_link.rel == 'http://schemas.google.com/acl/2007#accessControlList':
return a_link
return None
def GetFeedLink(self):
for a_link in self.link:
if a_link.rel == 'http://schemas.google.com/g/2005#feed':
return a_link
return None
def GetNextLink(self):
for a_link in self.link:
if a_link.rel == 'next':
return a_link
return None
def GetPrevLink(self):
for a_link in self.link:
if a_link.rel == 'previous':
return a_link
return None
class TotalResults(atom.AtomBase):
"""opensearch:TotalResults for a GData feed"""
_tag = 'totalResults'
_namespace = OPENSEARCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
def __init__(self, extension_elements=None,
extension_attributes=None, text=None):
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def TotalResultsFromString(xml_string):
return atom.CreateClassFromXMLString(TotalResults, xml_string)
class StartIndex(atom.AtomBase):
"""The opensearch:startIndex element in GData feed"""
_tag = 'startIndex'
_namespace = OPENSEARCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
def __init__(self, extension_elements=None,
extension_attributes=None, text=None):
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def StartIndexFromString(xml_string):
return atom.CreateClassFromXMLString(StartIndex, xml_string)
class ItemsPerPage(atom.AtomBase):
"""The opensearch:itemsPerPage element in GData feed"""
_tag = 'itemsPerPage'
_namespace = OPENSEARCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
def __init__(self, extension_elements=None,
extension_attributes=None, text=None):
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def ItemsPerPageFromString(xml_string):
return atom.CreateClassFromXMLString(ItemsPerPage, xml_string)
class ExtendedProperty(atom.AtomBase):
"""The Google Data extendedProperty element.
Used to store arbitrary key-value information specific to your
application. The value can either be a text string stored as an XML
attribute (.value), or an XML node (XmlBlob) as a child element.
This element is used in the Google Calendar data API and the Google
Contacts data API.
"""
_tag = 'extendedProperty'
_namespace = GDATA_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['name'] = 'name'
_attributes['value'] = 'value'
def __init__(self, name=None, value=None, extension_elements=None,
extension_attributes=None, text=None):
self.name = name
self.value = value
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def GetXmlBlobExtensionElement(self):
"""Returns the XML blob as an atom.ExtensionElement.
Returns:
An atom.ExtensionElement representing the blob's XML, or None if no
blob was set.
"""
if len(self.extension_elements) < 1:
return None
else:
return self.extension_elements[0]
def GetXmlBlobString(self):
"""Returns the XML blob as a string.
Returns:
A string containing the blob's XML, or None if no blob was set.
"""
blob = self.GetXmlBlobExtensionElement()
if blob:
return blob.ToString()
return None
def SetXmlBlob(self, blob):
"""Sets the contents of the extendedProperty to XML as a child node.
Since the extendedProperty is only allowed one child element as an XML
blob, setting the XML blob will erase any preexisting extension elements
in this object.
Args:
blob: str, ElementTree Element or atom.ExtensionElement representing
the XML blob stored in the extendedProperty.
"""
# Erase any existing extension_elements, clears the child nodes from the
# extendedProperty.
self.extension_elements = []
if isinstance(blob, atom.ExtensionElement):
self.extension_elements.append(blob)
elif ElementTree.iselement(blob):
self.extension_elements.append(atom._ExtensionElementFromElementTree(
blob))
else:
self.extension_elements.append(atom.ExtensionElementFromString(blob))
def ExtendedPropertyFromString(xml_string):
return atom.CreateClassFromXMLString(ExtendedProperty, xml_string)
class GDataEntry(atom.Entry, LinkFinder):
"""Extends Atom Entry to provide data processing"""
_tag = atom.Entry._tag
_namespace = atom.Entry._namespace
_children = atom.Entry._children.copy()
_attributes = atom.Entry._attributes.copy()
def __GetId(self):
return self.__id
# This method was created to strip the unwanted whitespace from the id's
# text node.
def __SetId(self, id):
self.__id = id
if id is not None and id.text is not None:
self.__id.text = id.text.strip()
id = property(__GetId, __SetId)
def IsMedia(self):
"""Determines whether or not an entry is a GData Media entry.
"""
if (self.GetEditMediaLink()):
return True
else:
return False
def GetMediaURL(self):
"""Returns the URL to the media content, if the entry is a media entry.
Otherwise returns None.
"""
if not self.IsMedia():
return None
else:
return self.content.src
def GDataEntryFromString(xml_string):
"""Creates a new GDataEntry instance given a string of XML."""
return atom.CreateClassFromXMLString(GDataEntry, xml_string)
class GDataFeed(atom.Feed, LinkFinder):
"""A Feed from a GData service"""
_tag = 'feed'
_namespace = atom.ATOM_NAMESPACE
_children = atom.Feed._children.copy()
_attributes = atom.Feed._attributes.copy()
_children['{%s}totalResults' % OPENSEARCH_NAMESPACE] = ('total_results',
TotalResults)
_children['{%s}startIndex' % OPENSEARCH_NAMESPACE] = ('start_index',
StartIndex)
_children['{%s}itemsPerPage' % OPENSEARCH_NAMESPACE] = ('items_per_page',
ItemsPerPage)
# Add a conversion rule for atom:entry to make it into a GData
# Entry.
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', [GDataEntry])
def __GetId(self):
return self.__id
def __SetId(self, id):
self.__id = id
if id is not None and id.text is not None:
self.__id.text = id.text.strip()
id = property(__GetId, __SetId)
def __GetGenerator(self):
return self.__generator
def __SetGenerator(self, generator):
self.__generator = generator
if generator is not None:
self.__generator.text = generator.text.strip()
generator = property(__GetGenerator, __SetGenerator)
def __init__(self, author=None, category=None, contributor=None,
generator=None, icon=None, atom_id=None, link=None, logo=None,
rights=None, subtitle=None, title=None, updated=None, entry=None,
total_results=None, start_index=None, items_per_page=None,
extension_elements=None, extension_attributes=None, text=None):
"""Constructor for Source
Args:
author: list (optional) A list of Author instances which belong to this
class.
category: list (optional) A list of Category instances
contributor: list (optional) A list on Contributor instances
generator: Generator (optional)
icon: Icon (optional)
id: Id (optional) The entry's Id element
link: list (optional) A list of Link instances
logo: Logo (optional)
rights: Rights (optional) The entry's Rights element
subtitle: Subtitle (optional) The entry's subtitle element
title: Title (optional) the entry's title element
updated: Updated (optional) the entry's updated element
entry: list (optional) A list of the Entry instances contained in the
feed.
text: String (optional) The text contents of the element. This is the
contents of the Entry's XML text node.
(Example: <foo>This is the text</foo>)
extension_elements: list (optional) A list of ExtensionElement instances
which are children of this element.
extension_attributes: dict (optional) A dictionary of strings which are
the values for additional XML attributes of this element.
"""
self.author = author or []
self.category = category or []
self.contributor = contributor or []
self.generator = generator
self.icon = icon
self.id = atom_id
self.link = link or []
self.logo = logo
self.rights = rights
self.subtitle = subtitle
self.title = title
self.updated = updated
self.entry = entry or []
self.total_results = total_results
self.start_index = start_index
self.items_per_page = items_per_page
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def GDataFeedFromString(xml_string):
return atom.CreateClassFromXMLString(GDataFeed, xml_string)
class BatchId(atom.AtomBase):
_tag = 'id'
_namespace = BATCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
def BatchIdFromString(xml_string):
return atom.CreateClassFromXMLString(BatchId, xml_string)
class BatchOperation(atom.AtomBase):
_tag = 'operation'
_namespace = BATCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['type'] = 'type'
def __init__(self, op_type=None, extension_elements=None,
extension_attributes=None,
text=None):
self.type = op_type
atom.AtomBase.__init__(self,
extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def BatchOperationFromString(xml_string):
return atom.CreateClassFromXMLString(BatchOperation, xml_string)
class BatchStatus(atom.AtomBase):
"""The batch:status element present in a batch response entry.
A status element contains the code (HTTP response code) and
reason as elements. In a single request these fields would
be part of the HTTP response, but in a batch request each
Entry operation has a corresponding Entry in the response
feed which includes status information.
See http://code.google.com/apis/gdata/batch.html#Handling_Errors
"""
_tag = 'status'
_namespace = BATCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['code'] = 'code'
_attributes['reason'] = 'reason'
_attributes['content-type'] = 'content_type'
def __init__(self, code=None, reason=None, content_type=None,
extension_elements=None, extension_attributes=None, text=None):
self.code = code
self.reason = reason
self.content_type = content_type
atom.AtomBase.__init__(self, extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def BatchStatusFromString(xml_string):
return atom.CreateClassFromXMLString(BatchStatus, xml_string)
class BatchEntry(GDataEntry):
"""An atom:entry for use in batch requests.
The BatchEntry contains additional members to specify the operation to be
performed on this entry and a batch ID so that the server can reference
individual operations in the response feed. For more information, see:
http://code.google.com/apis/gdata/batch.html
"""
_tag = GDataEntry._tag
_namespace = GDataEntry._namespace
_children = GDataEntry._children.copy()
_children['{%s}operation' % BATCH_NAMESPACE] = ('batch_operation', BatchOperation)
_children['{%s}id' % BATCH_NAMESPACE] = ('batch_id', BatchId)
_children['{%s}status' % BATCH_NAMESPACE] = ('batch_status', BatchStatus)
_attributes = GDataEntry._attributes.copy()
def __init__(self, author=None, category=None, content=None,
contributor=None, atom_id=None, link=None, published=None, rights=None,
source=None, summary=None, control=None, title=None, updated=None,
batch_operation=None, batch_id=None, batch_status=None,
extension_elements=None, extension_attributes=None, text=None):
self.batch_operation = batch_operation
self.batch_id = batch_id
self.batch_status = batch_status
GDataEntry.__init__(self, author=author, category=category,
content=content, contributor=contributor, atom_id=atom_id, link=link,
published=published, rights=rights, source=source, summary=summary,
control=control, title=title, updated=updated,
extension_elements=extension_elements,
extension_attributes=extension_attributes, text=text)
def BatchEntryFromString(xml_string):
return atom.CreateClassFromXMLString(BatchEntry, xml_string)
class BatchInterrupted(atom.AtomBase):
"""The batch:interrupted element sent if batch request was interrupted.
Only appears in a feed if some of the batch entries could not be processed.
See: http://code.google.com/apis/gdata/batch.html#Handling_Errors
"""
_tag = 'interrupted'
_namespace = BATCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['reason'] = 'reason'
_attributes['success'] = 'success'
_attributes['failures'] = 'failures'
_attributes['parsed'] = 'parsed'
def __init__(self, reason=None, success=None, failures=None, parsed=None,
extension_elements=None, extension_attributes=None, text=None):
self.reason = reason
self.success = success
self.failures = failures
self.parsed = parsed
atom.AtomBase.__init__(self, extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def BatchInterruptedFromString(xml_string):
return atom.CreateClassFromXMLString(BatchInterrupted, xml_string)
class BatchFeed(GDataFeed):
"""A feed containing a list of batch request entries."""
_tag = GDataFeed._tag
_namespace = GDataFeed._namespace
_children = GDataFeed._children.copy()
_attributes = GDataFeed._attributes.copy()
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', [BatchEntry])
_children['{%s}interrupted' % BATCH_NAMESPACE] = ('interrupted', BatchInterrupted)
def __init__(self, author=None, category=None, contributor=None,
generator=None, icon=None, atom_id=None, link=None, logo=None,
rights=None, subtitle=None, title=None, updated=None, entry=None,
total_results=None, start_index=None, items_per_page=None,
interrupted=None,
extension_elements=None, extension_attributes=None, text=None):
self.interrupted = interrupted
GDataFeed.__init__(self, author=author, category=category,
contributor=contributor, generator=generator,
icon=icon, atom_id=atom_id, link=link,
logo=logo, rights=rights, subtitle=subtitle,
title=title, updated=updated, entry=entry,
total_results=total_results, start_index=start_index,
items_per_page=items_per_page,
extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def AddBatchEntry(self, entry=None, id_url_string=None,
batch_id_string=None, operation_string=None):
"""Logic for populating members of a BatchEntry and adding to the feed.
If the entry is not a BatchEntry, it is converted to a BatchEntry so
that the batch specific members will be present.
The id_url_string can be used in place of an entry if the batch operation
applies to a URL. For example query and delete operations require just
the URL of an entry, no body is sent in the HTTP request. If an
id_url_string is sent instead of an entry, a BatchEntry is created and
added to the feed.
This method also assigns the desired batch id to the entry so that it
can be referenced in the server's response. If the batch_id_string is
None, this method will assign a batch_id to be the index at which this
entry will be in the feed's entry list.
Args:
entry: BatchEntry, atom.Entry, or another Entry flavor (optional) The
entry which will be sent to the server as part of the batch request.
The item must have a valid atom id so that the server knows which
entry this request references.
id_url_string: str (optional) The URL of the entry to be acted on. You
can find this URL in the text member of the atom id for an entry.
If an entry is not sent, this id will be used to construct a new
BatchEntry which will be added to the request feed.
batch_id_string: str (optional) The batch ID to be used to reference
this batch operation in the results feed. If this parameter is None,
the current length of the feed's entry array will be used as a
count. Note that batch_ids should either always be specified or
never, mixing could potentially result in duplicate batch ids.
operation_string: str (optional) The desired batch operation which will
set the batch_operation.type member of the entry. Options are
'insert', 'update', 'delete', and 'query'
Raises:
MissingRequiredParameters: Raised if neither an id_ url_string nor an
entry are provided in the request.
Returns:
The added entry.
"""
if entry is None and id_url_string is None:
raise MissingRequiredParameters('supply either an entry or URL string')
if entry is None and id_url_string is not None:
entry = BatchEntry(atom_id=atom.Id(text=id_url_string))
# TODO: handle cases in which the entry lacks batch_... members.
#if not isinstance(entry, BatchEntry):
# Convert the entry to a batch entry.
if batch_id_string is not None:
entry.batch_id = BatchId(text=batch_id_string)
elif entry.batch_id is None or entry.batch_id.text is None:
entry.batch_id = BatchId(text=str(len(self.entry)))
if operation_string is not None:
entry.batch_operation = BatchOperation(op_type=operation_string)
self.entry.append(entry)
return entry
def AddInsert(self, entry, batch_id_string=None):
"""Add an insert request to the operations in this batch request feed.
If the entry doesn't yet have an operation or a batch id, these will
be set to the insert operation and a batch_id specified as a parameter.
Args:
entry: BatchEntry The entry which will be sent in the batch feed as an
insert request.
batch_id_string: str (optional) The batch ID to be used to reference
this batch operation in the results feed. If this parameter is None,
the current length of the feed's entry array will be used as a
count. Note that batch_ids should either always be specified or
never, mixing could potentially result in duplicate batch ids.
"""
entry = self.AddBatchEntry(entry=entry, batch_id_string=batch_id_string,
operation_string=BATCH_INSERT)
def AddUpdate(self, entry, batch_id_string=None):
"""Add an update request to the list of batch operations in this feed.
Sets the operation type of the entry to insert if it is not already set
and assigns the desired batch id to the entry so that it can be
referenced in the server's response.
Args:
entry: BatchEntry The entry which will be sent to the server as an
update (HTTP PUT) request. The item must have a valid atom id
so that the server knows which entry to replace.
batch_id_string: str (optional) The batch ID to be used to reference
this batch operation in the results feed. If this parameter is None,
the current length of the feed's entry array will be used as a
count. See also comments for AddInsert.
"""
entry = self.AddBatchEntry(entry=entry, batch_id_string=batch_id_string,
operation_string=BATCH_UPDATE)
def AddDelete(self, url_string=None, entry=None, batch_id_string=None):
"""Adds a delete request to the batch request feed.
This method takes either the url_string which is the atom id of the item
to be deleted, or the entry itself. The atom id of the entry must be
present so that the server knows which entry should be deleted.
Args:
url_string: str (optional) The URL of the entry to be deleted. You can
find this URL in the text member of the atom id for an entry.
entry: BatchEntry (optional) The entry to be deleted.
batch_id_string: str (optional)
Raises:
MissingRequiredParameters: Raised if neither a url_string nor an entry
are provided in the request.
"""
entry = self.AddBatchEntry(entry=entry, id_url_string=url_string,
batch_id_string=batch_id_string,
operation_string=BATCH_DELETE)
def AddQuery(self, url_string=None, entry=None, batch_id_string=None):
"""Adds a query request to the batch request feed.
This method takes either the url_string which is the query URL
whose results will be added to the result feed. The query URL will
be encapsulated in a BatchEntry, and you may pass in the BatchEntry
with a query URL instead of sending a url_string.
Args:
url_string: str (optional)
entry: BatchEntry (optional)
batch_id_string: str (optional)
Raises:
MissingRequiredParameters
"""
entry = self.AddBatchEntry(entry=entry, id_url_string=url_string,
batch_id_string=batch_id_string,
operation_string=BATCH_QUERY)
def GetBatchLink(self):
for link in self.link:
if link.rel == 'http://schemas.google.com/g/2005#batch':
return link
return None
def BatchFeedFromString(xml_string):
return atom.CreateClassFromXMLString(BatchFeed, xml_string)
class EntryLink(atom.AtomBase):
"""The gd:entryLink element"""
_tag = 'entryLink'
_namespace = GDATA_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
# The entry used to be an atom.Entry, now it is a GDataEntry.
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', GDataEntry)
_attributes['rel'] = 'rel'
_attributes['readOnly'] = 'read_only'
_attributes['href'] = 'href'
def __init__(self, href=None, read_only=None, rel=None,
entry=None, extension_elements=None,
extension_attributes=None, text=None):
self.href = href
self.read_only = read_only
self.rel = rel
self.entry = entry
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def EntryLinkFromString(xml_string):
return atom.CreateClassFromXMLString(EntryLink, xml_string)
class FeedLink(atom.AtomBase):
"""The gd:feedLink element"""
_tag = 'feedLink'
_namespace = GDATA_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_children['{%s}feed' % atom.ATOM_NAMESPACE] = ('feed', GDataFeed)
_attributes['rel'] = 'rel'
_attributes['readOnly'] = 'read_only'
_attributes['countHint'] = 'count_hint'
_attributes['href'] = 'href'
def __init__(self, count_hint=None, href=None, read_only=None, rel=None,
feed=None, extension_elements=None, extension_attributes=None,
text=None):
self.count_hint = count_hint
self.href = href
self.read_only = read_only
self.rel = rel
self.feed = feed
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def FeedLinkFromString(xml_string):
return atom.CreateClassFromXMLString(FeedLink, xml_string)

15
gdata/acl/__init__.py Normal file
View File

@ -0,0 +1,15 @@
#!/usr/bin/python
#
# Copyright (C) 2009 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

55
gdata/acl/data.py Normal file
View File

@ -0,0 +1,55 @@
#!/usr/bin/python
#
# Copyright (C) 2009 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Contains the data classes of the Google Access Control List (ACL) Extension"""
__author__ = 'j.s@google.com (Jeff Scudder)'
import atom.core
import atom.data
import gdata.data
import gdata.opensearch.data
GACL_TEMPLATE = '{http://schemas.google.com/acl/2007}%s'
class AclRole(atom.core.XmlElement):
"""Describes the role of an entry in an access control list."""
_qname = GACL_TEMPLATE % 'role'
value = 'value'
class AclScope(atom.core.XmlElement):
"""Describes the scope of an entry in an access control list."""
_qname = GACL_TEMPLATE % 'scope'
type = 'type'
value = 'value'
class AclEntry(gdata.data.GDEntry):
"""Describes an entry in a feed of an access control list (ACL)."""
scope = AclScope
role = AclRole
class AclFeed(gdata.data.GDFeed):
"""Describes a feed of an access control list (ACL)."""
entry = [AclEntry]

20
gdata/alt/__init__.py Normal file
View File

@ -0,0 +1,20 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""This package's modules adapt the gdata library to run in other environments
The first example is the appengine module which contains functions and
classes which modify a GDataService object to run on Google App Engine.
"""

101
gdata/alt/app_engine.py Normal file
View File

@ -0,0 +1,101 @@
#!/usr/bin/python
#
# Copyright (C) 2009 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Provides functions to persist serialized auth tokens in the datastore.
The get_token and set_token functions should be used in conjunction with
gdata.gauth's token_from_blob and token_to_blob to allow auth token objects
to be reused across requests. It is up to your own code to ensure that the
token key's are unique.
"""
__author__ = 'j.s@google.com (Jeff Scudder)'
from google.appengine.ext import db
from google.appengine.api import memcache
class Token(db.Model):
"""Datastore Model which stores a serialized auth token."""
t = db.BlobProperty()
def get_token(unique_key):
"""Searches for a stored token with the desired key.
Checks memcache and then the datastore if required.
Args:
unique_key: str which uniquely identifies the desired auth token.
Returns:
A string encoding the auth token data. Use gdata.gauth.token_from_blob to
convert back into a usable token object. None if the token was not found
in memcache or the datastore.
"""
token_string = memcache.get(unique_key)
if token_string is None:
# The token wasn't in memcache, so look in the datastore.
token = Token.get_by_key_name(unique_key)
if token is None:
return None
return token.t
return token_string
def set_token(unique_key, token_str):
"""Saves the serialized auth token in the datastore.
The token is also stored in memcache to speed up retrieval on a cache hit.
Args:
unique_key: The unique name for this token as a string. It is up to your
code to ensure that this token value is unique in your application.
Previous values will be silently overwitten.
token_str: A serialized auth token as a string. I expect that this string
will be generated by gdata.gauth.token_to_blob.
Returns:
True if the token was stored sucessfully, False if the token could not be
safely cached (if an old value could not be cleared). If the token was
set in memcache, but not in the datastore, this function will return None.
However, in that situation an exception will likely be raised.
Raises:
Datastore exceptions may be raised from the App Engine SDK in the event of
failure.
"""
# First try to save in memcache.
result = memcache.set(unique_key, token_str)
# If memcache fails to save the value, clear the cached value.
if not result:
result = memcache.delete(unique_key)
# If we could not clear the cached value for this token, refuse to save.
if result == 0:
return False
# Save to the datastore.
if Token(key_name=unique_key, t=token_str).put():
return True
return None
def delete_token(unique_key):
# Clear from memcache.
memcache.delete(unique_key)
# Clear from the datastore.
Token(key_name=unique_key).delete()

321
gdata/alt/appengine.py Normal file
View File

@ -0,0 +1,321 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Provides HTTP functions for gdata.service to use on Google App Engine
AppEngineHttpClient: Provides an HTTP request method which uses App Engine's
urlfetch API. Set the http_client member of a GDataService object to an
instance of an AppEngineHttpClient to allow the gdata library to run on
Google App Engine.
run_on_appengine: Function which will modify an existing GDataService object
to allow it to run on App Engine. It works by creating a new instance of
the AppEngineHttpClient and replacing the GDataService object's
http_client.
"""
__author__ = 'api.jscudder (Jeff Scudder)'
import StringIO
import pickle
import atom.http_interface
import atom.token_store
from google.appengine.api import urlfetch
from google.appengine.ext import db
from google.appengine.api import users
from google.appengine.api import memcache
def run_on_appengine(gdata_service, store_tokens=True,
single_user_mode=False, deadline=None):
"""Modifies a GDataService object to allow it to run on App Engine.
Args:
gdata_service: An instance of AtomService, GDataService, or any
of their subclasses which has an http_client member and a
token_store member.
store_tokens: Boolean, defaults to True. If True, the gdata_service
will attempt to add each token to it's token_store when
SetClientLoginToken or SetAuthSubToken is called. If False
the tokens will not automatically be added to the
token_store.
single_user_mode: Boolean, defaults to False. If True, the current_token
member of gdata_service will be set when
SetClientLoginToken or SetAuthTubToken is called. If set
to True, the current_token is set in the gdata_service
and anyone who accesses the object will use the same
token.
Note: If store_tokens is set to False and
single_user_mode is set to False, all tokens will be
ignored, since the library assumes: the tokens should not
be stored in the datastore and they should not be stored
in the gdata_service object. This will make it
impossible to make requests which require authorization.
deadline: int (optional) The number of seconds to wait for a response
before timing out on the HTTP request. If no deadline is
specified, the deafault deadline for HTTP requests from App
Engine is used. The maximum is currently 10 (for 10 seconds).
The default deadline for App Engine is 5 seconds.
"""
gdata_service.http_client = AppEngineHttpClient(deadline=deadline)
gdata_service.token_store = AppEngineTokenStore()
gdata_service.auto_store_tokens = store_tokens
gdata_service.auto_set_current_token = single_user_mode
return gdata_service
class AppEngineHttpClient(atom.http_interface.GenericHttpClient):
def __init__(self, headers=None, deadline=None):
self.debug = False
self.headers = headers or {}
self.deadline = deadline
def request(self, operation, url, data=None, headers=None):
"""Performs an HTTP call to the server, supports GET, POST, PUT, and
DELETE.
Usage example, perform and HTTP GET on http://www.google.com/:
import atom.http
client = atom.http.HttpClient()
http_response = client.request('GET', 'http://www.google.com/')
Args:
operation: str The HTTP operation to be performed. This is usually one
of 'GET', 'POST', 'PUT', or 'DELETE'
data: filestream, list of parts, or other object which can be converted
to a string. Should be set to None when performing a GET or DELETE.
If data is a file-like object which can be read, this method will
read a chunk of 100K bytes at a time and send them.
If the data is a list of parts to be sent, each part will be
evaluated and sent.
url: The full URL to which the request should be sent. Can be a string
or atom.url.Url.
headers: dict of strings. HTTP headers which should be sent
in the request.
"""
all_headers = self.headers.copy()
if headers:
all_headers.update(headers)
# Construct the full payload.
# Assume that data is None or a string.
data_str = data
if data:
if isinstance(data, list):
# If data is a list of different objects, convert them all to strings
# and join them together.
converted_parts = [_convert_data_part(x) for x in data]
data_str = ''.join(converted_parts)
else:
data_str = _convert_data_part(data)
# If the list of headers does not include a Content-Length, attempt to
# calculate it based on the data object.
if data and 'Content-Length' not in all_headers:
all_headers['Content-Length'] = str(len(data_str))
# Set the content type to the default value if none was set.
if 'Content-Type' not in all_headers:
all_headers['Content-Type'] = 'application/atom+xml'
# Lookup the urlfetch operation which corresponds to the desired HTTP verb.
if operation == 'GET':
method = urlfetch.GET
elif operation == 'POST':
method = urlfetch.POST
elif operation == 'PUT':
method = urlfetch.PUT
elif operation == 'DELETE':
method = urlfetch.DELETE
else:
method = None
if self.deadline is None:
return HttpResponse(urlfetch.Fetch(url=str(url), payload=data_str,
method=method, headers=all_headers, follow_redirects=False))
return HttpResponse(urlfetch.Fetch(url=str(url), payload=data_str,
method=method, headers=all_headers, follow_redirects=False,
deadline=self.deadline))
def _convert_data_part(data):
if not data or isinstance(data, str):
return data
elif hasattr(data, 'read'):
# data is a file like object, so read it completely.
return data.read()
# The data object was not a file.
# Try to convert to a string and send the data.
return str(data)
class HttpResponse(object):
"""Translates a urlfetch resoinse to look like an hhtplib resoinse.
Used to allow the resoinse from HttpRequest to be usable by gdata.service
methods.
"""
def __init__(self, urlfetch_response):
self.body = StringIO.StringIO(urlfetch_response.content)
self.headers = urlfetch_response.headers
self.status = urlfetch_response.status_code
self.reason = ''
def read(self, length=None):
if not length:
return self.body.read()
else:
return self.body.read(length)
def getheader(self, name):
if not self.headers.has_key(name):
return self.headers[name.lower()]
return self.headers[name]
class TokenCollection(db.Model):
"""Datastore Model which associates auth tokens with the current user."""
user = db.UserProperty()
pickled_tokens = db.BlobProperty()
class AppEngineTokenStore(atom.token_store.TokenStore):
"""Stores the user's auth tokens in the App Engine datastore.
Tokens are only written to the datastore if a user is signed in (if
users.get_current_user() returns a user object).
"""
def __init__(self):
self.user = None
def add_token(self, token):
"""Associates the token with the current user and stores it.
If there is no current user, the token will not be stored.
Returns:
False if the token was not stored.
"""
tokens = load_auth_tokens(self.user)
if not hasattr(token, 'scopes') or not token.scopes:
return False
for scope in token.scopes:
tokens[str(scope)] = token
key = save_auth_tokens(tokens, self.user)
if key:
return True
return False
def find_token(self, url):
"""Searches the current user's collection of token for a token which can
be used for a request to the url.
Returns:
The stored token which belongs to the current user and is valid for the
desired URL. If there is no current user, or there is no valid user
token in the datastore, a atom.http_interface.GenericToken is returned.
"""
if url is None:
return None
if isinstance(url, (str, unicode)):
url = atom.url.parse_url(url)
tokens = load_auth_tokens(self.user)
if url in tokens:
token = tokens[url]
if token.valid_for_scope(url):
return token
else:
del tokens[url]
save_auth_tokens(tokens, self.user)
for scope, token in tokens.iteritems():
if token.valid_for_scope(url):
return token
return atom.http_interface.GenericToken()
def remove_token(self, token):
"""Removes the token from the current user's collection in the datastore.
Returns:
False if the token was not removed, this could be because the token was
not in the datastore, or because there is no current user.
"""
token_found = False
scopes_to_delete = []
tokens = load_auth_tokens(self.user)
for scope, stored_token in tokens.iteritems():
if stored_token == token:
scopes_to_delete.append(scope)
token_found = True
for scope in scopes_to_delete:
del tokens[scope]
if token_found:
save_auth_tokens(tokens, self.user)
return token_found
def remove_all_tokens(self):
"""Removes all of the current user's tokens from the datastore."""
save_auth_tokens({}, self.user)
def save_auth_tokens(token_dict, user=None):
"""Associates the tokens with the current user and writes to the datastore.
If there us no current user, the tokens are not written and this function
returns None.
Returns:
The key of the datastore entity containing the user's tokens, or None if
there was no current user.
"""
if user is None:
user = users.get_current_user()
if user is None:
return None
memcache.set('gdata_pickled_tokens:%s' % user, pickle.dumps(token_dict))
user_tokens = TokenCollection.all().filter('user =', user).get()
if user_tokens:
user_tokens.pickled_tokens = pickle.dumps(token_dict)
return user_tokens.put()
else:
user_tokens = TokenCollection(
user=user,
pickled_tokens=pickle.dumps(token_dict))
return user_tokens.put()
def load_auth_tokens(user=None):
"""Reads a dictionary of the current user's tokens from the datastore.
If there is no current user (a user is not signed in to the app) or the user
does not have any tokens, an empty dictionary is returned.
"""
if user is None:
user = users.get_current_user()
if user is None:
return {}
pickled_tokens = memcache.get('gdata_pickled_tokens:%s' % user)
if pickled_tokens:
return pickle.loads(pickled_tokens)
user_tokens = TokenCollection.all().filter('user =', user).get()
if user_tokens:
memcache.set('gdata_pickled_tokens:%s' % user, user_tokens.pickled_tokens)
return pickle.loads(user_tokens.pickled_tokens)
return {}

33
gdata/analytics/Crypto/Cipher/__init__.py Normal file
View File

@ -0,0 +1,33 @@
"""Secret-key encryption algorithms.
Secret-key encryption algorithms transform plaintext in some way that
is dependent on a key, producing ciphertext. This transformation can
easily be reversed, if (and, hopefully, only if) one knows the key.
The encryption modules here all support the interface described in PEP
272, "API for Block Encryption Algorithms".
If you don't know which algorithm to choose, use AES because it's
standard and has undergone a fair bit of examination.
Crypto.Cipher.AES Advanced Encryption Standard
Crypto.Cipher.ARC2 Alleged RC2
Crypto.Cipher.ARC4 Alleged RC4
Crypto.Cipher.Blowfish
Crypto.Cipher.CAST
Crypto.Cipher.DES The Data Encryption Standard. Very commonly used
in the past, but today its 56-bit keys are too small.
Crypto.Cipher.DES3 Triple DES.
Crypto.Cipher.IDEA
Crypto.Cipher.RC5
Crypto.Cipher.XOR The simple XOR cipher.
"""
__all__ = ['AES', 'ARC2', 'ARC4',
'Blowfish', 'CAST', 'DES', 'DES3', 'IDEA', 'RC5',
'XOR'
]
__revision__ = "$Id: __init__.py,v 1.7 2003/02/28 15:28:35 akuchling Exp $"

108
gdata/analytics/Crypto/Hash/HMAC.py Normal file
View File

@ -0,0 +1,108 @@
"""HMAC (Keyed-Hashing for Message Authentication) Python module.
Implements the HMAC algorithm as described by RFC 2104.
This is just a copy of the Python 2.2 HMAC module, modified to work when
used on versions of Python before 2.2.
"""
__revision__ = "$Id: HMAC.py,v 1.5 2002/07/25 17:19:02 z3p Exp $"
import string
def _strxor(s1, s2):
"""Utility method. XOR the two strings s1 and s2 (must have same length).
"""
return "".join(map(lambda x, y: chr(ord(x) ^ ord(y)), s1, s2))
# The size of the digests returned by HMAC depends on the underlying
# hashing module used.
digest_size = None
class HMAC:
"""RFC2104 HMAC class.
This supports the API for Cryptographic Hash Functions (PEP 247).
"""
def __init__(self, key, msg = None, digestmod = None):
"""Create a new HMAC object.
key: key for the keyed hash object.
msg: Initial input for the hash, if provided.
digestmod: A module supporting PEP 247. Defaults to the md5 module.
"""
if digestmod == None:
import md5
digestmod = md5
self.digestmod = digestmod
self.outer = digestmod.new()
self.inner = digestmod.new()
try:
self.digest_size = digestmod.digest_size
except AttributeError:
self.digest_size = len(self.outer.digest())
blocksize = 64
ipad = "\x36" * blocksize
opad = "\x5C" * blocksize
if len(key) > blocksize:
key = digestmod.new(key).digest()
key = key + chr(0) * (blocksize - len(key))
self.outer.update(_strxor(key, opad))
self.inner.update(_strxor(key, ipad))
if (msg):
self.update(msg)
## def clear(self):
## raise NotImplementedError, "clear() method not available in HMAC."
def update(self, msg):
"""Update this hashing object with the string msg.
"""
self.inner.update(msg)
def copy(self):
"""Return a separate copy of this hashing object.
An update to this copy won't affect the original object.
"""
other = HMAC("")
other.digestmod = self.digestmod
other.inner = self.inner.copy()
other.outer = self.outer.copy()
return other
def digest(self):
"""Return the hash value of this hashing object.
This returns a string containing 8-bit data. The object is
not altered in any way by this function; you can continue
updating the object after calling this function.
"""
h = self.outer.copy()
h.update(self.inner.digest())
return h.digest()
def hexdigest(self):
"""Like digest(), but returns a string of hexadecimal digits instead.
"""
return "".join([string.zfill(hex(ord(x))[2:], 2)
for x in tuple(self.digest())])
def new(key, msg = None, digestmod = None):
"""Create a new hashing object and return it.
key: The starting key for the hash.
msg: if available, will immediately be hashed into the object's starting
state.
You can now feed arbitrary strings into the object using its update()
method, and can ask for the hash value at any time by calling its digest()
method.
"""
return HMAC(key, msg, digestmod)

13
gdata/analytics/Crypto/Hash/MD5.py Normal file
View File

@ -0,0 +1,13 @@
# Just use the MD5 module from the Python standard library
__revision__ = "$Id: MD5.py,v 1.4 2002/07/11 14:31:19 akuchling Exp $"
from md5 import *
import md5
if hasattr(md5, 'digestsize'):
digest_size = digestsize
del digestsize
del md5

11
gdata/analytics/Crypto/Hash/SHA.py Normal file
View File

@ -0,0 +1,11 @@
# Just use the SHA module from the Python standard library
__revision__ = "$Id: SHA.py,v 1.4 2002/07/11 14:31:19 akuchling Exp $"
from sha import *
import sha
if hasattr(sha, 'digestsize'):
digest_size = digestsize
del digestsize
del sha

24
gdata/analytics/Crypto/Hash/__init__.py Normal file
View File

@ -0,0 +1,24 @@
"""Hashing algorithms
Hash functions take arbitrary strings as input, and produce an output
of fixed size that is dependent on the input; it should never be
possible to derive the input data given only the hash function's
output. Hash functions can be used simply as a checksum, or, in
association with a public-key algorithm, can be used to implement
digital signatures.
The hashing modules here all support the interface described in PEP
247, "API for Cryptographic Hash Functions".
Submodules:
Crypto.Hash.HMAC RFC 2104: Keyed-Hashing for Message Authentication
Crypto.Hash.MD2
Crypto.Hash.MD4
Crypto.Hash.MD5
Crypto.Hash.RIPEMD
Crypto.Hash.SHA
"""
__all__ = ['HMAC', 'MD2', 'MD4', 'MD5', 'RIPEMD', 'SHA', 'SHA256']
__revision__ = "$Id: __init__.py,v 1.6 2003/12/19 14:24:25 akuchling Exp $"

295
gdata/analytics/Crypto/Protocol/AllOrNothing.py Normal file
View File

@ -0,0 +1,295 @@
"""This file implements all-or-nothing package transformations.
An all-or-nothing package transformation is one in which some text is
transformed into message blocks, such that all blocks must be obtained before
the reverse transformation can be applied. Thus, if any blocks are corrupted
or lost, the original message cannot be reproduced.
An all-or-nothing package transformation is not encryption, although a block
cipher algorithm is used. The encryption key is randomly generated and is
extractable from the message blocks.
This class implements the All-Or-Nothing package transformation algorithm
described in:
Ronald L. Rivest. "All-Or-Nothing Encryption and The Package Transform"
http://theory.lcs.mit.edu/~rivest/fusion.pdf
"""
__revision__ = "$Id: AllOrNothing.py,v 1.8 2003/02/28 15:23:20 akuchling Exp $"
import operator
import string
from Crypto.Util.number import bytes_to_long, long_to_bytes
class AllOrNothing:
"""Class implementing the All-or-Nothing package transform.
Methods for subclassing:
_inventkey(key_size):
Returns a randomly generated key. Subclasses can use this to
implement better random key generating algorithms. The default
algorithm is probably not very cryptographically secure.
"""
def __init__(self, ciphermodule, mode=None, IV=None):
"""AllOrNothing(ciphermodule, mode=None, IV=None)
ciphermodule is a module implementing the cipher algorithm to
use. It must provide the PEP272 interface.
Note that the encryption key is randomly generated
automatically when needed. Optional arguments mode and IV are
passed directly through to the ciphermodule.new() method; they
are the feedback mode and initialization vector to use. All
three arguments must be the same for the object used to create
the digest, and to undigest'ify the message blocks.
"""
self.__ciphermodule = ciphermodule
self.__mode = mode
self.__IV = IV
self.__key_size = ciphermodule.key_size
if self.__key_size == 0:
self.__key_size = 16
__K0digit = chr(0x69)
def digest(self, text):
"""digest(text:string) : [string]
Perform the All-or-Nothing package transform on the given
string. Output is a list of message blocks describing the
transformed text, where each block is a string of bit length equal
to the ciphermodule's block_size.
"""
# generate a random session key and K0, the key used to encrypt the
# hash blocks. Rivest calls this a fixed, publically-known encryption
# key, but says nothing about the security implications of this key or
# how to choose it.
key = self._inventkey(self.__key_size)
K0 = self.__K0digit * self.__key_size
# we need two cipher objects here, one that is used to encrypt the
# message blocks and one that is used to encrypt the hashes. The
# former uses the randomly generated key, while the latter uses the
# well-known key.
mcipher = self.__newcipher(key)
hcipher = self.__newcipher(K0)
# Pad the text so that its length is a multiple of the cipher's
# block_size. Pad with trailing spaces, which will be eliminated in
# the undigest() step.
block_size = self.__ciphermodule.block_size
padbytes = block_size - (len(text) % block_size)
text = text + ' ' * padbytes
# Run through the algorithm:
# s: number of message blocks (size of text / block_size)
# input sequence: m1, m2, ... ms
# random key K' (`key' in the code)
# Compute output sequence: m'1, m'2, ... m's' for s' = s + 1
# Let m'i = mi ^ E(K', i) for i = 1, 2, 3, ..., s
# Let m's' = K' ^ h1 ^ h2 ^ ... hs
# where hi = E(K0, m'i ^ i) for i = 1, 2, ... s
#
# The one complication I add is that the last message block is hard
# coded to the number of padbytes added, so that these can be stripped
# during the undigest() step
s = len(text) / block_size
blocks = []
hashes = []
for i in range(1, s+1):
start = (i-1) * block_size
end = start + block_size
mi = text[start:end]
assert len(mi) == block_size
cipherblock = mcipher.encrypt(long_to_bytes(i, block_size))
mticki = bytes_to_long(mi) ^ bytes_to_long(cipherblock)
blocks.append(mticki)
# calculate the hash block for this block
hi = hcipher.encrypt(long_to_bytes(mticki ^ i, block_size))
hashes.append(bytes_to_long(hi))
# Add the padbytes length as a message block
i = i + 1
cipherblock = mcipher.encrypt(long_to_bytes(i, block_size))
mticki = padbytes ^ bytes_to_long(cipherblock)
blocks.append(mticki)
# calculate this block's hash
hi = hcipher.encrypt(long_to_bytes(mticki ^ i, block_size))
hashes.append(bytes_to_long(hi))
# Now calculate the last message block of the sequence 1..s'. This
# will contain the random session key XOR'd with all the hash blocks,
# so that for undigest(), once all the hash blocks are calculated, the
# session key can be trivially extracted. Calculating all the hash
# blocks requires that all the message blocks be received, thus the
# All-or-Nothing algorithm succeeds.
mtick_stick = bytes_to_long(key) ^ reduce(operator.xor, hashes)
blocks.append(mtick_stick)
# we convert the blocks to strings since in Python, byte sequences are
# always represented as strings. This is more consistent with the
# model that encryption and hash algorithms always operate on strings.
return map(long_to_bytes, blocks)
def undigest(self, blocks):
"""undigest(blocks : [string]) : string
Perform the reverse package transformation on a list of message
blocks. Note that the ciphermodule used for both transformations
must be the same. blocks is a list of strings of bit length
equal to the ciphermodule's block_size.
"""
# better have at least 2 blocks, for the padbytes package and the hash
# block accumulator
if len(blocks) < 2:
raise ValueError, "List must be at least length 2."
# blocks is a list of strings. We need to deal with them as long
# integers
blocks = map(bytes_to_long, blocks)
# Calculate the well-known key, to which the hash blocks are
# encrypted, and create the hash cipher.
K0 = self.__K0digit * self.__key_size
hcipher = self.__newcipher(K0)
# Since we have all the blocks (or this method would have been called
# prematurely), we can calcualte all the hash blocks.
hashes = []
for i in range(1, len(blocks)):
mticki = blocks[i-1] ^ i
hi = hcipher.encrypt(long_to_bytes(mticki))
hashes.append(bytes_to_long(hi))
# now we can calculate K' (key). remember the last block contains
# m's' which we don't include here
key = blocks[-1] ^ reduce(operator.xor, hashes)
# and now we can create the cipher object
mcipher = self.__newcipher(long_to_bytes(key))
block_size = self.__ciphermodule.block_size
# And we can now decode the original message blocks
parts = []
for i in range(1, len(blocks)):
cipherblock = mcipher.encrypt(long_to_bytes(i, block_size))
mi = blocks[i-1] ^ bytes_to_long(cipherblock)
parts.append(mi)
# The last message block contains the number of pad bytes appended to
# the original text string, such that its length was an even multiple
# of the cipher's block_size. This number should be small enough that
# the conversion from long integer to integer should never overflow
padbytes = int(parts[-1])
text = string.join(map(long_to_bytes, parts[:-1]), '')
return text[:-padbytes]
def _inventkey(self, key_size):
# TBD: Not a very secure algorithm. Eventually, I'd like to use JHy's
# kernelrand module
import time
from Crypto.Util import randpool
# TBD: key_size * 2 to work around possible bug in RandomPool?
pool = randpool.RandomPool(key_size * 2)
while key_size > pool.entropy:
pool.add_event()
# we now have enough entropy in the pool to get a key_size'd key
return pool.get_bytes(key_size)
def __newcipher(self, key):
if self.__mode is None and self.__IV is None:
return self.__ciphermodule.new(key)
elif self.__IV is None:
return self.__ciphermodule.new(key, self.__mode)
else:
return self.__ciphermodule.new(key, self.__mode, self.__IV)
if __name__ == '__main__':
import sys
import getopt
import base64
usagemsg = '''\
Test module usage: %(program)s [-c cipher] [-l] [-h]
Where:
--cipher module
-c module
Cipher module to use. Default: %(ciphermodule)s
--aslong
-l
Print the encoded message blocks as long integers instead of base64
encoded strings
--help
-h
Print this help message
'''
ciphermodule = 'AES'
aslong = 0
def usage(code, msg=None):
if msg:
print msg
print usagemsg % {'program': sys.argv[0],
'ciphermodule': ciphermodule}
sys.exit(code)
try:
opts, args = getopt.getopt(sys.argv[1:],
'c:l', ['cipher=', 'aslong'])
except getopt.error, msg:
usage(1, msg)
if args:
usage(1, 'Too many arguments')
for opt, arg in opts:
if opt in ('-h', '--help'):
usage(0)
elif opt in ('-c', '--cipher'):
ciphermodule = arg
elif opt in ('-l', '--aslong'):
aslong = 1
# ugly hack to force __import__ to give us the end-path module
module = __import__('Crypto.Cipher.'+ciphermodule, None, None, ['new'])
a = AllOrNothing(module)
print 'Original text:\n=========='
print __doc__
print '=========='
msgblocks = a.digest(__doc__)
print 'message blocks:'
for i, blk in map(None, range(len(msgblocks)), msgblocks):
# base64 adds a trailing newline
print ' %3d' % i,
if aslong:
print bytes_to_long(blk)
else:
print base64.encodestring(blk)[:-1]
#
# get a new undigest-only object so there's no leakage
b = AllOrNothing(module)
text = b.undigest(msgblocks)
if text == __doc__:
print 'They match!'
else:
print 'They differ!'

229
gdata/analytics/Crypto/Protocol/Chaffing.py Normal file
View File

@ -0,0 +1,229 @@
"""This file implements the chaffing algorithm.
Winnowing and chaffing is a technique for enhancing privacy without requiring
strong encryption. In short, the technique takes a set of authenticated
message blocks (the wheat) and adds a number of chaff blocks which have
randomly chosen data and MAC fields. This means that to an adversary, the
chaff blocks look as valid as the wheat blocks, and so the authentication
would have to be performed on every block. By tailoring the number of chaff
blocks added to the message, the sender can make breaking the message
computationally infeasible. There are many other interesting properties of
the winnow/chaff technique.
For example, say Alice is sending a message to Bob. She packetizes the
message and performs an all-or-nothing transformation on the packets. Then
she authenticates each packet with a message authentication code (MAC). The
MAC is a hash of the data packet, and there is a secret key which she must
share with Bob (key distribution is an exercise left to the reader). She then
adds a serial number to each packet, and sends the packets to Bob.
Bob receives the packets, and using the shared secret authentication key,
authenticates the MACs for each packet. Those packets that have bad MACs are
simply discarded. The remainder are sorted by serial number, and passed
through the reverse all-or-nothing transform. The transform means that an
eavesdropper (say Eve) must acquire all the packets before any of the data can
be read. If even one packet is missing, the data is useless.
There's one twist: by adding chaff packets, Alice and Bob can make Eve's job
much harder, since Eve now has to break the shared secret key, or try every
combination of wheat and chaff packet to read any of the message. The cool
thing is that Bob doesn't need to add any additional code; the chaff packets
are already filtered out because their MACs don't match (in all likelihood --
since the data and MACs for the chaff packets are randomly chosen it is
possible, but very unlikely that a chaff MAC will match the chaff data). And
Alice need not even be the party adding the chaff! She could be completely
unaware that a third party, say Charles, is adding chaff packets to her
messages as they are transmitted.
For more information on winnowing and chaffing see this paper:
Ronald L. Rivest, "Chaffing and Winnowing: Confidentiality without Encryption"
http://theory.lcs.mit.edu/~rivest/chaffing.txt
"""
__revision__ = "$Id: Chaffing.py,v 1.7 2003/02/28 15:23:21 akuchling Exp $"
from Crypto.Util.number import bytes_to_long
class Chaff:
"""Class implementing the chaff adding algorithm.
Methods for subclasses:
_randnum(size):
Returns a randomly generated number with a byte-length equal
to size. Subclasses can use this to implement better random
data and MAC generating algorithms. The default algorithm is
probably not very cryptographically secure. It is most
important that the chaff data does not contain any patterns
that can be used to discern it from wheat data without running
the MAC.
"""
def __init__(self, factor=1.0, blocksper=1):
"""Chaff(factor:float, blocksper:int)
factor is the number of message blocks to add chaff to,
expressed as a percentage between 0.0 and 1.0. blocksper is
the number of chaff blocks to include for each block being
chaffed. Thus the defaults add one chaff block to every
message block. By changing the defaults, you can adjust how
computationally difficult it could be for an adversary to
brute-force crack the message. The difficulty is expressed
as:
pow(blocksper, int(factor * number-of-blocks))
For ease of implementation, when factor < 1.0, only the first
int(factor*number-of-blocks) message blocks are chaffed.
"""
if not (0.0<=factor<=1.0):
raise ValueError, "'factor' must be between 0.0 and 1.0"
if blocksper < 0:
raise ValueError, "'blocksper' must be zero or more"
self.__factor = factor
self.__blocksper = blocksper
def chaff(self, blocks):
"""chaff( [(serial-number:int, data:string, MAC:string)] )
: [(int, string, string)]
Add chaff to message blocks. blocks is a list of 3-tuples of the
form (serial-number, data, MAC).
Chaff is created by choosing a random number of the same
byte-length as data, and another random number of the same
byte-length as MAC. The message block's serial number is
placed on the chaff block and all the packet's chaff blocks
are randomly interspersed with the single wheat block. This
method then returns a list of 3-tuples of the same form.
Chaffed blocks will contain multiple instances of 3-tuples
with the same serial number, but the only way to figure out
which blocks are wheat and which are chaff is to perform the
MAC hash and compare values.
"""
chaffedblocks = []
# count is the number of blocks to add chaff to. blocksper is the
# number of chaff blocks to add per message block that is being
# chaffed.
count = len(blocks) * self.__factor
blocksper = range(self.__blocksper)
for i, wheat in map(None, range(len(blocks)), blocks):
# it shouldn't matter which of the n blocks we add chaff to, so for
# ease of implementation, we'll just add them to the first count
# blocks
if i < count:
serial, data, mac = wheat
datasize = len(data)
macsize = len(mac)
addwheat = 1
# add chaff to this block
for j in blocksper:
import sys
chaffdata = self._randnum(datasize)
chaffmac = self._randnum(macsize)
chaff = (serial, chaffdata, chaffmac)
# mix up the order, if the 5th bit is on then put the
# wheat on the list
if addwheat and bytes_to_long(self._randnum(16)) & 0x40:
chaffedblocks.append(wheat)
addwheat = 0
chaffedblocks.append(chaff)
if addwheat:
chaffedblocks.append(wheat)
else:
# just add the wheat
chaffedblocks.append(wheat)
return chaffedblocks
def _randnum(self, size):
# TBD: Not a very secure algorithm.
# TBD: size * 2 to work around possible bug in RandomPool
from Crypto.Util import randpool
import time
pool = randpool.RandomPool(size * 2)
while size > pool.entropy:
pass
# we now have enough entropy in the pool to get size bytes of random
# data... well, probably
return pool.get_bytes(size)
if __name__ == '__main__':
text = """\
We hold these truths to be self-evident, that all men are created equal, that
they are endowed by their Creator with certain unalienable Rights, that among
these are Life, Liberty, and the pursuit of Happiness. That to secure these
rights, Governments are instituted among Men, deriving their just powers from
the consent of the governed. That whenever any Form of Government becomes
destructive of these ends, it is the Right of the People to alter or to
abolish it, and to institute new Government, laying its foundation on such
principles and organizing its powers in such form, as to them shall seem most
likely to effect their Safety and Happiness.
"""
print 'Original text:\n=========='
print text
print '=========='
# first transform the text into packets
blocks = [] ; size = 40
for i in range(0, len(text), size):
blocks.append( text[i:i+size] )
# now get MACs for all the text blocks. The key is obvious...
print 'Calculating MACs...'
from Crypto.Hash import HMAC, SHA
key = 'Jefferson'
macs = [HMAC.new(key, block, digestmod=SHA).digest()
for block in blocks]
assert len(blocks) == len(macs)
# put these into a form acceptable as input to the chaffing procedure
source = []
m = map(None, range(len(blocks)), blocks, macs)
print m
for i, data, mac in m:
source.append((i, data, mac))
# now chaff these
print 'Adding chaff...'
c = Chaff(factor=0.5, blocksper=2)
chaffed = c.chaff(source)
from base64 import encodestring
# print the chaffed message blocks. meanwhile, separate the wheat from
# the chaff
wheat = []
print 'chaffed message blocks:'
for i, data, mac in chaffed:
# do the authentication
h = HMAC.new(key, data, digestmod=SHA)
pmac = h.digest()
if pmac == mac:
tag = '-->'
wheat.append(data)
else:
tag = ' '
# base64 adds a trailing newline
print tag, '%3d' % i, \
repr(data), encodestring(mac)[:-1]
# now decode the message packets and check it against the original text
print 'Undigesting wheat...'
newtext = "".join(wheat)
if newtext == text:
print 'They match!'
else:
print 'They differ!'

17
gdata/analytics/Crypto/Protocol/__init__.py Normal file
View File

@ -0,0 +1,17 @@
"""Cryptographic protocols
Implements various cryptographic protocols. (Don't expect to find
network protocols here.)
Crypto.Protocol.AllOrNothing Transforms a message into a set of message
blocks, such that the blocks can be
recombined to get the message back.
Crypto.Protocol.Chaffing Takes a set of authenticated message blocks
(the wheat) and adds a number of
randomly generated blocks (the chaff).
"""
__all__ = ['AllOrNothing', 'Chaffing']
__revision__ = "$Id: __init__.py,v 1.4 2003/02/28 15:23:21 akuchling Exp $"

238
gdata/analytics/Crypto/PublicKey/DSA.py Normal file
View File

@ -0,0 +1,238 @@
#
# DSA.py : Digital Signature Algorithm
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: DSA.py,v 1.16 2004/05/06 12:52:54 akuchling Exp $"
from Crypto.PublicKey.pubkey import *
from Crypto.Util import number
from Crypto.Util.number import bytes_to_long, long_to_bytes
from Crypto.Hash import SHA
try:
from Crypto.PublicKey import _fastmath
except ImportError:
_fastmath = None
class error (Exception):
pass
def generateQ(randfunc):
S=randfunc(20)
hash1=SHA.new(S).digest()
hash2=SHA.new(long_to_bytes(bytes_to_long(S)+1)).digest()
q = bignum(0)
for i in range(0,20):
c=ord(hash1[i])^ord(hash2[i])
if i==0:
c=c | 128
if i==19:
c= c | 1
q=q*256+c
while (not isPrime(q)):
q=q+2
if pow(2,159L) < q < pow(2,160L):
return S, q
raise error, 'Bad q value generated'
def generate(bits, randfunc, progress_func=None):
"""generate(bits:int, randfunc:callable, progress_func:callable)
Generate a DSA key of length 'bits', using 'randfunc' to get
random data and 'progress_func', if present, to display
the progress of the key generation.
"""
if bits<160:
raise error, 'Key length <160 bits'
obj=DSAobj()
# Generate string S and prime q
if progress_func:
progress_func('p,q\n')
while (1):
S, obj.q = generateQ(randfunc)
n=(bits-1)/160
C, N, V = 0, 2, {}
b=(obj.q >> 5) & 15
powb=pow(bignum(2), b)
powL1=pow(bignum(2), bits-1)
while C<4096:
for k in range(0, n+1):
V[k]=bytes_to_long(SHA.new(S+str(N)+str(k)).digest())
W=V[n] % powb
for k in range(n-1, -1, -1):
W=(W<<160L)+V[k]
X=W+powL1
p=X-(X%(2*obj.q)-1)
if powL1<=p and isPrime(p):
break
C, N = C+1, N+n+1
if C<4096:
break
if progress_func:
progress_func('4096 multiples failed\n')
obj.p = p
power=(p-1)/obj.q
if progress_func:
progress_func('h,g\n')
while (1):
h=bytes_to_long(randfunc(bits)) % (p-1)
g=pow(h, power, p)
if 1<h<p-1 and g>1:
break
obj.g=g
if progress_func:
progress_func('x,y\n')
while (1):
x=bytes_to_long(randfunc(20))
if 0 < x < obj.q:
break
obj.x, obj.y = x, pow(g, x, p)
return obj
def construct(tuple):
"""construct(tuple:(long,long,long,long)|(long,long,long,long,long)):DSAobj
Construct a DSA object from a 4- or 5-tuple of numbers.
"""
obj=DSAobj()
if len(tuple) not in [4,5]:
raise error, 'argument for construct() wrong length'
for i in range(len(tuple)):
field = obj.keydata[i]
setattr(obj, field, tuple[i])
return obj
class DSAobj(pubkey):
keydata=['y', 'g', 'p', 'q', 'x']
def _encrypt(self, s, Kstr):
raise error, 'DSA algorithm cannot encrypt data'
def _decrypt(self, s):
raise error, 'DSA algorithm cannot decrypt data'
def _sign(self, M, K):
if (K<2 or self.q<=K):
raise error, 'K is not between 2 and q'
r=pow(self.g, K, self.p) % self.q
s=(inverse(K, self.q)*(M+self.x*r)) % self.q
return (r,s)
def _verify(self, M, sig):
r, s = sig
if r<=0 or r>=self.q or s<=0 or s>=self.q:
return 0
w=inverse(s, self.q)
u1, u2 = (M*w) % self.q, (r*w) % self.q
v1 = pow(self.g, u1, self.p)
v2 = pow(self.y, u2, self.p)
v = ((v1*v2) % self.p)
v = v % self.q
if v==r:
return 1
return 0
def size(self):
"Return the maximum number of bits that can be handled by this key."
return number.size(self.p) - 1
def has_private(self):
"""Return a Boolean denoting whether the object contains
private components."""
if hasattr(self, 'x'):
return 1
else:
return 0
def can_sign(self):
"""Return a Boolean value recording whether this algorithm can generate signatures."""
return 1
def can_encrypt(self):
"""Return a Boolean value recording whether this algorithm can encrypt data."""
return 0
def publickey(self):
"""Return a new key object containing only the public information."""
return construct((self.y, self.g, self.p, self.q))
object=DSAobj
generate_py = generate
construct_py = construct
class DSAobj_c(pubkey):
keydata = ['y', 'g', 'p', 'q', 'x']
def __init__(self, key):
self.key = key
def __getattr__(self, attr):
if attr in self.keydata:
return getattr(self.key, attr)
else:
if self.__dict__.has_key(attr):
self.__dict__[attr]
else:
raise AttributeError, '%s instance has no attribute %s' % (self.__class__, attr)
def __getstate__(self):
d = {}
for k in self.keydata:
if hasattr(self.key, k):
d[k]=getattr(self.key, k)
return d
def __setstate__(self, state):
y,g,p,q = state['y'], state['g'], state['p'], state['q']
if not state.has_key('x'):
self.key = _fastmath.dsa_construct(y,g,p,q)
else:
x = state['x']
self.key = _fastmath.dsa_construct(y,g,p,q,x)
def _sign(self, M, K):
return self.key._sign(M, K)
def _verify(self, M, (r, s)):
return self.key._verify(M, r, s)
def size(self):
return self.key.size()
def has_private(self):
return self.key.has_private()
def publickey(self):
return construct_c((self.key.y, self.key.g, self.key.p, self.key.q))
def can_sign(self):
return 1
def can_encrypt(self):
return 0
def generate_c(bits, randfunc, progress_func=None):
obj = generate_py(bits, randfunc, progress_func)
y,g,p,q,x = obj.y, obj.g, obj.p, obj.q, obj.x
return construct_c((y,g,p,q,x))
def construct_c(tuple):
key = apply(_fastmath.dsa_construct, tuple)
return DSAobj_c(key)
if _fastmath:
#print "using C version of DSA"
generate = generate_c
construct = construct_c
error = _fastmath.error

132
gdata/analytics/Crypto/PublicKey/ElGamal.py Normal file
View File

@ -0,0 +1,132 @@
#
# ElGamal.py : ElGamal encryption/decryption and signatures
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: ElGamal.py,v 1.9 2003/04/04 19:44:26 akuchling Exp $"
from Crypto.PublicKey.pubkey import *
from Crypto.Util import number
class error (Exception):
pass
# Generate an ElGamal key with N bits
def generate(bits, randfunc, progress_func=None):
"""generate(bits:int, randfunc:callable, progress_func:callable)
Generate an ElGamal key of length 'bits', using 'randfunc' to get
random data and 'progress_func', if present, to display
the progress of the key generation.
"""
obj=ElGamalobj()
# Generate prime p
if progress_func:
progress_func('p\n')
obj.p=bignum(getPrime(bits, randfunc))
# Generate random number g
if progress_func:
progress_func('g\n')
size=bits-1-(ord(randfunc(1)) & 63) # g will be from 1--64 bits smaller than p
if size<1:
size=bits-1
while (1):
obj.g=bignum(getPrime(size, randfunc))
if obj.g < obj.p:
break
size=(size+1) % bits
if size==0:
size=4
# Generate random number x
if progress_func:
progress_func('x\n')
while (1):
size=bits-1-ord(randfunc(1)) # x will be from 1 to 256 bits smaller than p
if size>2:
break
while (1):
obj.x=bignum(getPrime(size, randfunc))
if obj.x < obj.p:
break
size = (size+1) % bits
if size==0:
size=4
if progress_func:
progress_func('y\n')
obj.y = pow(obj.g, obj.x, obj.p)
return obj
def construct(tuple):
"""construct(tuple:(long,long,long,long)|(long,long,long,long,long)))
: ElGamalobj
Construct an ElGamal key from a 3- or 4-tuple of numbers.
"""
obj=ElGamalobj()
if len(tuple) not in [3,4]:
raise error, 'argument for construct() wrong length'
for i in range(len(tuple)):
field = obj.keydata[i]
setattr(obj, field, tuple[i])
return obj
class ElGamalobj(pubkey):
keydata=['p', 'g', 'y', 'x']
def _encrypt(self, M, K):
a=pow(self.g, K, self.p)
b=( M*pow(self.y, K, self.p) ) % self.p
return ( a,b )
def _decrypt(self, M):
if (not hasattr(self, 'x')):
raise error, 'Private key not available in this object'
ax=pow(M[0], self.x, self.p)
plaintext=(M[1] * inverse(ax, self.p ) ) % self.p
return plaintext
def _sign(self, M, K):
if (not hasattr(self, 'x')):
raise error, 'Private key not available in this object'
p1=self.p-1
if (GCD(K, p1)!=1):
raise error, 'Bad K value: GCD(K,p-1)!=1'
a=pow(self.g, K, self.p)
t=(M-self.x*a) % p1
while t<0: t=t+p1
b=(t*inverse(K, p1)) % p1
return (a, b)
def _verify(self, M, sig):
v1=pow(self.y, sig[0], self.p)
v1=(v1*pow(sig[0], sig[1], self.p)) % self.p
v2=pow(self.g, M, self.p)
if v1==v2:
return 1
return 0
def size(self):
"Return the maximum number of bits that can be handled by this key."
return number.size(self.p) - 1
def has_private(self):
"""Return a Boolean denoting whether the object contains
private components."""
if hasattr(self, 'x'):
return 1
else:
return 0
def publickey(self):
"""Return a new key object containing only the public information."""
return construct((self.p, self.g, self.y))
object=ElGamalobj

256
gdata/analytics/Crypto/PublicKey/RSA.py Normal file
View File

@ -0,0 +1,256 @@
#
# RSA.py : RSA encryption/decryption
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: RSA.py,v 1.20 2004/05/06 12:52:54 akuchling Exp $"
from Crypto.PublicKey import pubkey
from Crypto.Util import number
try:
from Crypto.PublicKey import _fastmath
except ImportError:
_fastmath = None
class error (Exception):
pass
def generate(bits, randfunc, progress_func=None):
"""generate(bits:int, randfunc:callable, progress_func:callable)
Generate an RSA key of length 'bits', using 'randfunc' to get
random data and 'progress_func', if present, to display
the progress of the key generation.
"""
obj=RSAobj()
# Generate the prime factors of n
if progress_func:
progress_func('p,q\n')
p = q = 1L
while number.size(p*q) < bits:
p = pubkey.getPrime(bits/2, randfunc)
q = pubkey.getPrime(bits/2, randfunc)
# p shall be smaller than q (for calc of u)
if p > q:
(p, q)=(q, p)
obj.p = p
obj.q = q
if progress_func:
progress_func('u\n')
obj.u = pubkey.inverse(obj.p, obj.q)
obj.n = obj.p*obj.q
obj.e = 65537L
if progress_func:
progress_func('d\n')
obj.d=pubkey.inverse(obj.e, (obj.p-1)*(obj.q-1))
assert bits <= 1+obj.size(), "Generated key is too small"
return obj
def construct(tuple):
"""construct(tuple:(long,) : RSAobj
Construct an RSA object from a 2-, 3-, 5-, or 6-tuple of numbers.
"""
obj=RSAobj()
if len(tuple) not in [2,3,5,6]:
raise error, 'argument for construct() wrong length'
for i in range(len(tuple)):
field = obj.keydata[i]
setattr(obj, field, tuple[i])
if len(tuple) >= 5:
# Ensure p is smaller than q
if obj.p>obj.q:
(obj.p, obj.q)=(obj.q, obj.p)
if len(tuple) == 5:
# u not supplied, so we're going to have to compute it.
obj.u=pubkey.inverse(obj.p, obj.q)
return obj
class RSAobj(pubkey.pubkey):
keydata = ['n', 'e', 'd', 'p', 'q', 'u']
def _encrypt(self, plaintext, K=''):
if self.n<=plaintext:
raise error, 'Plaintext too large'
return (pow(plaintext, self.e, self.n),)
def _decrypt(self, ciphertext):
if (not hasattr(self, 'd')):
raise error, 'Private key not available in this object'
if self.n<=ciphertext[0]:
raise error, 'Ciphertext too large'
return pow(ciphertext[0], self.d, self.n)
def _sign(self, M, K=''):
return (self._decrypt((M,)),)
def _verify(self, M, sig):
m2=self._encrypt(sig[0])
if m2[0]==M:
return 1
else: return 0
def _blind(self, M, B):
tmp = pow(B, self.e, self.n)
return (M * tmp) % self.n
def _unblind(self, M, B):
tmp = pubkey.inverse(B, self.n)
return (M * tmp) % self.n
def can_blind (self):
"""can_blind() : bool
Return a Boolean value recording whether this algorithm can
blind data. (This does not imply that this
particular key object has the private information required to
to blind a message.)
"""
return 1
def size(self):
"""size() : int
Return the maximum number of bits that can be handled by this key.
"""
return number.size(self.n) - 1
def has_private(self):
"""has_private() : bool
Return a Boolean denoting whether the object contains
private components.
"""
if hasattr(self, 'd'):
return 1
else: return 0
def publickey(self):
"""publickey(): RSAobj
Return a new key object containing only the public key information.
"""
return construct((self.n, self.e))
class RSAobj_c(pubkey.pubkey):
keydata = ['n', 'e', 'd', 'p', 'q', 'u']
def __init__(self, key):
self.key = key
def __getattr__(self, attr):
if attr in self.keydata:
return getattr(self.key, attr)
else:
if self.__dict__.has_key(attr):
self.__dict__[attr]
else:
raise AttributeError, '%s instance has no attribute %s' % (self.__class__, attr)
def __getstate__(self):
d = {}
for k in self.keydata:
if hasattr(self.key, k):
d[k]=getattr(self.key, k)
return d
def __setstate__(self, state):
n,e = state['n'], state['e']
if not state.has_key('d'):
self.key = _fastmath.rsa_construct(n,e)
else:
d = state['d']
if not state.has_key('q'):
self.key = _fastmath.rsa_construct(n,e,d)
else:
p, q, u = state['p'], state['q'], state['u']
self.key = _fastmath.rsa_construct(n,e,d,p,q,u)
def _encrypt(self, plain, K):
return (self.key._encrypt(plain),)
def _decrypt(self, cipher):
return self.key._decrypt(cipher[0])
def _sign(self, M, K):
return (self.key._sign(M),)
def _verify(self, M, sig):
return self.key._verify(M, sig[0])
def _blind(self, M, B):
return self.key._blind(M, B)
def _unblind(self, M, B):
return self.key._unblind(M, B)
def can_blind (self):
return 1
def size(self):
return self.key.size()
def has_private(self):
return self.key.has_private()
def publickey(self):
return construct_c((self.key.n, self.key.e))
def generate_c(bits, randfunc, progress_func = None):
# Generate the prime factors of n
if progress_func:
progress_func('p,q\n')
p = q = 1L
while number.size(p*q) < bits:
p = pubkey.getPrime(bits/2, randfunc)
q = pubkey.getPrime(bits/2, randfunc)
# p shall be smaller than q (for calc of u)
if p > q:
(p, q)=(q, p)
if progress_func:
progress_func('u\n')
u=pubkey.inverse(p, q)
n=p*q
e = 65537L
if progress_func:
progress_func('d\n')
d=pubkey.inverse(e, (p-1)*(q-1))
key = _fastmath.rsa_construct(n,e,d,p,q,u)
obj = RSAobj_c(key)
## print p
## print q
## print number.size(p), number.size(q), number.size(q*p),
## print obj.size(), bits
assert bits <= 1+obj.size(), "Generated key is too small"
return obj
def construct_c(tuple):
key = apply(_fastmath.rsa_construct, tuple)
return RSAobj_c(key)
object = RSAobj
generate_py = generate
construct_py = construct
if _fastmath:
#print "using C version of RSA"
generate = generate_c
construct = construct_c
error = _fastmath.error

17
gdata/analytics/Crypto/PublicKey/__init__.py Normal file
View File

@ -0,0 +1,17 @@
"""Public-key encryption and signature algorithms.
Public-key encryption uses two different keys, one for encryption and
one for decryption. The encryption key can be made public, and the
decryption key is kept private. Many public-key algorithms can also
be used to sign messages, and some can *only* be used for signatures.
Crypto.PublicKey.DSA Digital Signature Algorithm. (Signature only)
Crypto.PublicKey.ElGamal (Signing and encryption)
Crypto.PublicKey.RSA (Signing, encryption, and blinding)
Crypto.PublicKey.qNEW (Signature only)
"""
__all__ = ['RSA', 'DSA', 'ElGamal', 'qNEW']
__revision__ = "$Id: __init__.py,v 1.4 2003/04/03 20:27:13 akuchling Exp $"

172
gdata/analytics/Crypto/PublicKey/pubkey.py Normal file
View File

@ -0,0 +1,172 @@
#
# pubkey.py : Internal functions for public key operations
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: pubkey.py,v 1.11 2003/04/03 20:36:14 akuchling Exp $"
import types, warnings
from Crypto.Util.number import *
# Basic public key class
class pubkey:
def __init__(self):
pass
def __getstate__(self):
"""To keep key objects platform-independent, the key data is
converted to standard Python long integers before being
written out. It will then be reconverted as necessary on
restoration."""
d=self.__dict__
for key in self.keydata:
if d.has_key(key): d[key]=long(d[key])
return d
def __setstate__(self, d):
"""On unpickling a key object, the key data is converted to the big
number representation being used, whether that is Python long
integers, MPZ objects, or whatever."""
for key in self.keydata:
if d.has_key(key): self.__dict__[key]=bignum(d[key])
def encrypt(self, plaintext, K):
"""encrypt(plaintext:string|long, K:string|long) : tuple
Encrypt the string or integer plaintext. K is a random
parameter required by some algorithms.
"""
wasString=0
if isinstance(plaintext, types.StringType):
plaintext=bytes_to_long(plaintext) ; wasString=1
if isinstance(K, types.StringType):
K=bytes_to_long(K)
ciphertext=self._encrypt(plaintext, K)
if wasString: return tuple(map(long_to_bytes, ciphertext))
else: return ciphertext
def decrypt(self, ciphertext):
"""decrypt(ciphertext:tuple|string|long): string
Decrypt 'ciphertext' using this key.
"""
wasString=0
if not isinstance(ciphertext, types.TupleType):
ciphertext=(ciphertext,)
if isinstance(ciphertext[0], types.StringType):
ciphertext=tuple(map(bytes_to_long, ciphertext)) ; wasString=1
plaintext=self._decrypt(ciphertext)
if wasString: return long_to_bytes(plaintext)
else: return plaintext
def sign(self, M, K):
"""sign(M : string|long, K:string|long) : tuple
Return a tuple containing the signature for the message M.
K is a random parameter required by some algorithms.
"""
if (not self.has_private()):
raise error, 'Private key not available in this object'
if isinstance(M, types.StringType): M=bytes_to_long(M)
if isinstance(K, types.StringType): K=bytes_to_long(K)
return self._sign(M, K)
def verify (self, M, signature):
"""verify(M:string|long, signature:tuple) : bool
Verify that the signature is valid for the message M;
returns true if the signature checks out.
"""
if isinstance(M, types.StringType): M=bytes_to_long(M)
return self._verify(M, signature)
# alias to compensate for the old validate() name
def validate (self, M, signature):
warnings.warn("validate() method name is obsolete; use verify()",
DeprecationWarning)
def blind(self, M, B):
"""blind(M : string|long, B : string|long) : string|long
Blind message M using blinding factor B.
"""
wasString=0
if isinstance(M, types.StringType):
M=bytes_to_long(M) ; wasString=1
if isinstance(B, types.StringType): B=bytes_to_long(B)
blindedmessage=self._blind(M, B)
if wasString: return long_to_bytes(blindedmessage)
else: return blindedmessage
def unblind(self, M, B):
"""unblind(M : string|long, B : string|long) : string|long
Unblind message M using blinding factor B.
"""
wasString=0
if isinstance(M, types.StringType):
M=bytes_to_long(M) ; wasString=1
if isinstance(B, types.StringType): B=bytes_to_long(B)
unblindedmessage=self._unblind(M, B)
if wasString: return long_to_bytes(unblindedmessage)
else: return unblindedmessage
# The following methods will usually be left alone, except for
# signature-only algorithms. They both return Boolean values
# recording whether this key's algorithm can sign and encrypt.
def can_sign (self):
"""can_sign() : bool
Return a Boolean value recording whether this algorithm can
generate signatures. (This does not imply that this
particular key object has the private information required to
to generate a signature.)
"""
return 1
def can_encrypt (self):
"""can_encrypt() : bool
Return a Boolean value recording whether this algorithm can
encrypt data. (This does not imply that this
particular key object has the private information required to
to decrypt a message.)
"""
return 1
def can_blind (self):
"""can_blind() : bool
Return a Boolean value recording whether this algorithm can
blind data. (This does not imply that this
particular key object has the private information required to
to blind a message.)
"""
return 0
# The following methods will certainly be overridden by
# subclasses.
def size (self):
"""size() : int
Return the maximum number of bits that can be handled by this key.
"""
return 0
def has_private (self):
"""has_private() : bool
Return a Boolean denoting whether the object contains
private components.
"""
return 0
def publickey (self):
"""publickey(): object
Return a new key object containing only the public information.
"""
return self
def __eq__ (self, other):
"""__eq__(other): 0, 1
Compare us to other for equality.
"""
return self.__getstate__() == other.__getstate__()

170
gdata/analytics/Crypto/PublicKey/qNEW.py Normal file
View File

@ -0,0 +1,170 @@
#
# qNEW.py : The q-NEW signature algorithm.
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: qNEW.py,v 1.8 2003/04/04 15:13:35 akuchling Exp $"
from Crypto.PublicKey import pubkey
from Crypto.Util.number import *
from Crypto.Hash import SHA
class error (Exception):
pass
HASHBITS = 160 # Size of SHA digests
def generate(bits, randfunc, progress_func=None):
"""generate(bits:int, randfunc:callable, progress_func:callable)
Generate a qNEW key of length 'bits', using 'randfunc' to get
random data and 'progress_func', if present, to display
the progress of the key generation.
"""
obj=qNEWobj()
# Generate prime numbers p and q. q is a 160-bit prime
# number. p is another prime number (the modulus) whose bit
# size is chosen by the caller, and is generated so that p-1
# is a multiple of q.
#
# Note that only a single seed is used to
# generate p and q; if someone generates a key for you, you can
# use the seed to duplicate the key generation. This can
# protect you from someone generating values of p,q that have
# some special form that's easy to break.
if progress_func:
progress_func('p,q\n')
while (1):
obj.q = getPrime(160, randfunc)
# assert pow(2, 159L)<obj.q<pow(2, 160L)
obj.seed = S = long_to_bytes(obj.q)
C, N, V = 0, 2, {}
# Compute b and n such that bits-1 = b + n*HASHBITS
n= (bits-1) / HASHBITS
b= (bits-1) % HASHBITS ; powb=2L << b
powL1=pow(long(2), bits-1)
while C<4096:
# The V array will contain (bits-1) bits of random
# data, that are assembled to produce a candidate
# value for p.
for k in range(0, n+1):
V[k]=bytes_to_long(SHA.new(S+str(N)+str(k)).digest())
p = V[n] % powb
for k in range(n-1, -1, -1):
p= (p << long(HASHBITS) )+V[k]
p = p+powL1 # Ensure the high bit is set
# Ensure that p-1 is a multiple of q
p = p - (p % (2*obj.q)-1)
# If p is still the right size, and it's prime, we're done!
if powL1<=p and isPrime(p):
break
# Otherwise, increment the counter and try again
C, N = C+1, N+n+1
if C<4096:
break # Ended early, so exit the while loop
if progress_func:
progress_func('4096 values of p tried\n')
obj.p = p
power=(p-1)/obj.q
# Next parameter: g = h**((p-1)/q) mod p, such that h is any
# number <p-1, and g>1. g is kept; h can be discarded.
if progress_func:
progress_func('h,g\n')
while (1):
h=bytes_to_long(randfunc(bits)) % (p-1)
g=pow(h, power, p)
if 1<h<p-1 and g>1:
break
obj.g=g
# x is the private key information, and is
# just a random number between 0 and q.
# y=g**x mod p, and is part of the public information.
if progress_func:
progress_func('x,y\n')
while (1):
x=bytes_to_long(randfunc(20))
if 0 < x < obj.q:
break
obj.x, obj.y=x, pow(g, x, p)
return obj
# Construct a qNEW object
def construct(tuple):
"""construct(tuple:(long,long,long,long)|(long,long,long,long,long)
Construct a qNEW object from a 4- or 5-tuple of numbers.
"""
obj=qNEWobj()
if len(tuple) not in [4,5]:
raise error, 'argument for construct() wrong length'
for i in range(len(tuple)):
field = obj.keydata[i]
setattr(obj, field, tuple[i])
return obj
class qNEWobj(pubkey.pubkey):
keydata=['p', 'q', 'g', 'y', 'x']
def _sign(self, M, K=''):
if (self.q<=K):
raise error, 'K is greater than q'
if M<0:
raise error, 'Illegal value of M (<0)'
if M>=pow(2,161L):
raise error, 'Illegal value of M (too large)'
r=pow(self.g, K, self.p) % self.q
s=(K- (r*M*self.x % self.q)) % self.q
return (r,s)
def _verify(self, M, sig):
r, s = sig
if r<=0 or r>=self.q or s<=0 or s>=self.q:
return 0
if M<0:
raise error, 'Illegal value of M (<0)'
if M<=0 or M>=pow(2,161L):
return 0
v1 = pow(self.g, s, self.p)
v2 = pow(self.y, M*r, self.p)
v = ((v1*v2) % self.p)
v = v % self.q
if v==r:
return 1
return 0
def size(self):
"Return the maximum number of bits that can be handled by this key."
return 160
def has_private(self):
"""Return a Boolean denoting whether the object contains
private components."""
return hasattr(self, 'x')
def can_sign(self):
"""Return a Boolean value recording whether this algorithm can generate signatures."""
return 1
def can_encrypt(self):
"""Return a Boolean value recording whether this algorithm can encrypt data."""
return 0
def publickey(self):
"""Return a new key object containing only the public information."""
return construct((self.p, self.q, self.g, self.y))
object = qNEWobj

342
gdata/analytics/Crypto/Util/RFC1751.py Normal file
View File

@ -0,0 +1,342 @@
#!/usr/local/bin/python
# rfc1751.py : Converts between 128-bit strings and a human-readable
# sequence of words, as defined in RFC1751: "A Convention for
# Human-Readable 128-bit Keys", by Daniel L. McDonald.
__revision__ = "$Id: RFC1751.py,v 1.6 2003/04/04 15:15:10 akuchling Exp $"
import string, binascii
binary={0:'0000', 1:'0001', 2:'0010', 3:'0011', 4:'0100', 5:'0101',
6:'0110', 7:'0111', 8:'1000', 9:'1001', 10:'1010', 11:'1011',
12:'1100', 13:'1101', 14:'1110', 15:'1111'}
def _key2bin(s):
"Convert a key into a string of binary digits"
kl=map(lambda x: ord(x), s)
kl=map(lambda x: binary[x/16]+binary[x&15], kl)
return ''.join(kl)
def _extract(key, start, length):
"""Extract a bitstring from a string of binary digits, and return its
numeric value."""
k=key[start:start+length]
return reduce(lambda x,y: x*2+ord(y)-48, k, 0)
def key_to_english (key):
"""key_to_english(key:string) : string
Transform an arbitrary key into a string containing English words.
The key length must be a multiple of 8.
"""
english=''
for index in range(0, len(key), 8): # Loop over 8-byte subkeys
subkey=key[index:index+8]
# Compute the parity of the key
skbin=_key2bin(subkey) ; p=0
for i in range(0, 64, 2): p=p+_extract(skbin, i, 2)
# Append parity bits to the subkey
skbin=_key2bin(subkey+chr((p<<6) & 255))
for i in range(0, 64, 11):
english=english+wordlist[_extract(skbin, i, 11)]+' '
return english[:-1] # Remove the trailing space
def english_to_key (str):
"""english_to_key(string):string
Transform a string into a corresponding key.
The string must contain words separated by whitespace; the number
of words must be a multiple of 6.
"""
L=string.split(string.upper(str)) ; key=''
for index in range(0, len(L), 6):
sublist=L[index:index+6] ; char=9*[0] ; bits=0
for i in sublist:
index = wordlist.index(i)
shift = (8-(bits+11)%8) %8
y = index << shift
cl, cc, cr = (y>>16), (y>>8)&0xff, y & 0xff
if (shift>5):
char[bits/8] = char[bits/8] | cl
char[bits/8+1] = char[bits/8+1] | cc
char[bits/8+2] = char[bits/8+2] | cr
elif shift>-3:
char[bits/8] = char[bits/8] | cc
char[bits/8+1] = char[bits/8+1] | cr
else: char[bits/8] = char[bits/8] | cr
bits=bits+11
subkey=reduce(lambda x,y:x+chr(y), char, '')
# Check the parity of the resulting key
skbin=_key2bin(subkey)
p=0
for i in range(0, 64, 2): p=p+_extract(skbin, i, 2)
if (p&3) != _extract(skbin, 64, 2):
raise ValueError, "Parity error in resulting key"
key=key+subkey[0:8]
return key
wordlist=[ "A", "ABE", "ACE", "ACT", "AD", "ADA", "ADD",
"AGO", "AID", "AIM", "AIR", "ALL", "ALP", "AM", "AMY", "AN", "ANA",
"AND", "ANN", "ANT", "ANY", "APE", "APS", "APT", "ARC", "ARE", "ARK",
"ARM", "ART", "AS", "ASH", "ASK", "AT", "ATE", "AUG", "AUK", "AVE",
"AWE", "AWK", "AWL", "AWN", "AX", "AYE", "BAD", "BAG", "BAH", "BAM",
"BAN", "BAR", "BAT", "BAY", "BE", "BED", "BEE", "BEG", "BEN", "BET",
"BEY", "BIB", "BID", "BIG", "BIN", "BIT", "BOB", "BOG", "BON", "BOO",
"BOP", "BOW", "BOY", "BUB", "BUD", "BUG", "BUM", "BUN", "BUS", "BUT",
"BUY", "BY", "BYE", "CAB", "CAL", "CAM", "CAN", "CAP", "CAR", "CAT",
"CAW", "COD", "COG", "COL", "CON", "COO", "COP", "COT", "COW", "COY",
"CRY", "CUB", "CUE", "CUP", "CUR", "CUT", "DAB", "DAD", "DAM", "DAN",
"DAR", "DAY", "DEE", "DEL", "DEN", "DES", "DEW", "DID", "DIE", "DIG",
"DIN", "DIP", "DO", "DOE", "DOG", "DON", "DOT", "DOW", "DRY", "DUB",
"DUD", "DUE", "DUG", "DUN", "EAR", "EAT", "ED", "EEL", "EGG", "EGO",
"ELI", "ELK", "ELM", "ELY", "EM", "END", "EST", "ETC", "EVA", "EVE",
"EWE", "EYE", "FAD", "FAN", "FAR", "FAT", "FAY", "FED", "FEE", "FEW",
"FIB", "FIG", "FIN", "FIR", "FIT", "FLO", "FLY", "FOE", "FOG", "FOR",
"FRY", "FUM", "FUN", "FUR", "GAB", "GAD", "GAG", "GAL", "GAM", "GAP",
"GAS", "GAY", "GEE", "GEL", "GEM", "GET", "GIG", "GIL", "GIN", "GO",
"GOT", "GUM", "GUN", "GUS", "GUT", "GUY", "GYM", "GYP", "HA", "HAD",
"HAL", "HAM", "HAN", "HAP", "HAS", "HAT", "HAW", "HAY", "HE", "HEM",
"HEN", "HER", "HEW", "HEY", "HI", "HID", "HIM", "HIP", "HIS", "HIT",
"HO", "HOB", "HOC", "HOE", "HOG", "HOP", "HOT", "HOW", "HUB", "HUE",
"HUG", "HUH", "HUM", "HUT", "I", "ICY", "IDA", "IF", "IKE", "ILL",
"INK", "INN", "IO", "ION", "IQ", "IRA", "IRE", "IRK", "IS", "IT",
"ITS", "IVY", "JAB", "JAG", "JAM", "JAN", "JAR", "JAW", "JAY", "JET",
"JIG", "JIM", "JO", "JOB", "JOE", "JOG", "JOT", "JOY", "JUG", "JUT",
"KAY", "KEG", "KEN", "KEY", "KID", "KIM", "KIN", "KIT", "LA", "LAB",
"LAC", "LAD", "LAG", "LAM", "LAP", "LAW", "LAY", "LEA", "LED", "LEE",
"LEG", "LEN", "LEO", "LET", "LEW", "LID", "LIE", "LIN", "LIP", "LIT",
"LO", "LOB", "LOG", "LOP", "LOS", "LOT", "LOU", "LOW", "LOY", "LUG",
"LYE", "MA", "MAC", "MAD", "MAE", "MAN", "MAO", "MAP", "MAT", "MAW",
"MAY", "ME", "MEG", "MEL", "MEN", "MET", "MEW", "MID", "MIN", "MIT",
"MOB", "MOD", "MOE", "MOO", "MOP", "MOS", "MOT", "MOW", "MUD", "MUG",
"MUM", "MY", "NAB", "NAG", "NAN", "NAP", "NAT", "NAY", "NE", "NED",
"NEE", "NET", "NEW", "NIB", "NIL", "NIP", "NIT", "NO", "NOB", "NOD",
"NON", "NOR", "NOT", "NOV", "NOW", "NU", "NUN", "NUT", "O", "OAF",
"OAK", "OAR", "OAT", "ODD", "ODE", "OF", "OFF", "OFT", "OH", "OIL",
"OK", "OLD", "ON", "ONE", "OR", "ORB", "ORE", "ORR", "OS", "OTT",
"OUR", "OUT", "OVA", "OW", "OWE", "OWL", "OWN", "OX", "PA", "PAD",
"PAL", "PAM", "PAN", "PAP", "PAR", "PAT", "PAW", "PAY", "PEA", "PEG",
"PEN", "PEP", "PER", "PET", "PEW", "PHI", "PI", "PIE", "PIN", "PIT",
"PLY", "PO", "POD", "POE", "POP", "POT", "POW", "PRO", "PRY", "PUB",
"PUG", "PUN", "PUP", "PUT", "QUO", "RAG", "RAM", "RAN", "RAP", "RAT",
"RAW", "RAY", "REB", "RED", "REP", "RET", "RIB", "RID", "RIG", "RIM",
"RIO", "RIP", "ROB", "ROD", "ROE", "RON", "ROT", "ROW", "ROY", "RUB",
"RUE", "RUG", "RUM", "RUN", "RYE", "SAC", "SAD", "SAG", "SAL", "SAM",
"SAN", "SAP", "SAT", "SAW", "SAY", "SEA", "SEC", "SEE", "SEN", "SET",
"SEW", "SHE", "SHY", "SIN", "SIP", "SIR", "SIS", "SIT", "SKI", "SKY",
"SLY", "SO", "SOB", "SOD", "SON", "SOP", "SOW", "SOY", "SPA", "SPY",
"SUB", "SUD", "SUE", "SUM", "SUN", "SUP", "TAB", "TAD", "TAG", "TAN",
"TAP", "TAR", "TEA", "TED", "TEE", "TEN", "THE", "THY", "TIC", "TIE",
"TIM", "TIN", "TIP", "TO", "TOE", "TOG", "TOM", "TON", "TOO", "TOP",
"TOW", "TOY", "TRY", "TUB", "TUG", "TUM", "TUN", "TWO", "UN", "UP",
"US", "USE", "VAN", "VAT", "VET", "VIE", "WAD", "WAG", "WAR", "WAS",
"WAY", "WE", "WEB", "WED", "WEE", "WET", "WHO", "WHY", "WIN", "WIT",
"WOK", "WON", "WOO", "WOW", "WRY", "WU", "YAM", "YAP", "YAW", "YE",
"YEA", "YES", "YET", "YOU", "ABED", "ABEL", "ABET", "ABLE", "ABUT",
"ACHE", "ACID", "ACME", "ACRE", "ACTA", "ACTS", "ADAM", "ADDS",
"ADEN", "AFAR", "AFRO", "AGEE", "AHEM", "AHOY", "AIDA", "AIDE",
"AIDS", "AIRY", "AJAR", "AKIN", "ALAN", "ALEC", "ALGA", "ALIA",
"ALLY", "ALMA", "ALOE", "ALSO", "ALTO", "ALUM", "ALVA", "AMEN",
"AMES", "AMID", "AMMO", "AMOK", "AMOS", "AMRA", "ANDY", "ANEW",
"ANNA", "ANNE", "ANTE", "ANTI", "AQUA", "ARAB", "ARCH", "AREA",
"ARGO", "ARID", "ARMY", "ARTS", "ARTY", "ASIA", "ASKS", "ATOM",
"AUNT", "AURA", "AUTO", "AVER", "AVID", "AVIS", "AVON", "AVOW",
"AWAY", "AWRY", "BABE", "BABY", "BACH", "BACK", "BADE", "BAIL",
"BAIT", "BAKE", "BALD", "BALE", "BALI", "BALK", "BALL", "BALM",
"BAND", "BANE", "BANG", "BANK", "BARB", "BARD", "BARE", "BARK",
"BARN", "BARR", "BASE", "BASH", "BASK", "BASS", "BATE", "BATH",
"BAWD", "BAWL", "BEAD", "BEAK", "BEAM", "BEAN", "BEAR", "BEAT",
"BEAU", "BECK", "BEEF", "BEEN", "BEER",
"BEET", "BELA", "BELL", "BELT", "BEND", "BENT", "BERG", "BERN",
"BERT", "BESS", "BEST", "BETA", "BETH", "BHOY", "BIAS", "BIDE",
"BIEN", "BILE", "BILK", "BILL", "BIND", "BING", "BIRD", "BITE",
"BITS", "BLAB", "BLAT", "BLED", "BLEW", "BLOB", "BLOC", "BLOT",
"BLOW", "BLUE", "BLUM", "BLUR", "BOAR", "BOAT", "BOCA", "BOCK",
"BODE", "BODY", "BOGY", "BOHR", "BOIL", "BOLD", "BOLO", "BOLT",
"BOMB", "BONA", "BOND", "BONE", "BONG", "BONN", "BONY", "BOOK",
"BOOM", "BOON", "BOOT", "BORE", "BORG", "BORN", "BOSE", "BOSS",
"BOTH", "BOUT", "BOWL", "BOYD", "BRAD", "BRAE", "BRAG", "BRAN",
"BRAY", "BRED", "BREW", "BRIG", "BRIM", "BROW", "BUCK", "BUDD",
"BUFF", "BULB", "BULK", "BULL", "BUNK", "BUNT", "BUOY", "BURG",
"BURL", "BURN", "BURR", "BURT", "BURY", "BUSH", "BUSS", "BUST",
"BUSY", "BYTE", "CADY", "CAFE", "CAGE", "CAIN", "CAKE", "CALF",
"CALL", "CALM", "CAME", "CANE", "CANT", "CARD", "CARE", "CARL",
"CARR", "CART", "CASE", "CASH", "CASK", "CAST", "CAVE", "CEIL",
"CELL", "CENT", "CERN", "CHAD", "CHAR", "CHAT", "CHAW", "CHEF",
"CHEN", "CHEW", "CHIC", "CHIN", "CHOU", "CHOW", "CHUB", "CHUG",
"CHUM", "CITE", "CITY", "CLAD", "CLAM", "CLAN", "CLAW", "CLAY",
"CLOD", "CLOG", "CLOT", "CLUB", "CLUE", "COAL", "COAT", "COCA",
"COCK", "COCO", "CODA", "CODE", "CODY", "COED", "COIL", "COIN",
"COKE", "COLA", "COLD", "COLT", "COMA", "COMB", "COME", "COOK",
"COOL", "COON", "COOT", "CORD", "CORE", "CORK", "CORN", "COST",
"COVE", "COWL", "CRAB", "CRAG", "CRAM", "CRAY", "CREW", "CRIB",
"CROW", "CRUD", "CUBA", "CUBE", "CUFF", "CULL", "CULT", "CUNY",
"CURB", "CURD", "CURE", "CURL", "CURT", "CUTS", "DADE", "DALE",
"DAME", "DANA", "DANE", "DANG", "DANK", "DARE", "DARK", "DARN",
"DART", "DASH", "DATA", "DATE", "DAVE", "DAVY", "DAWN", "DAYS",
"DEAD", "DEAF", "DEAL", "DEAN", "DEAR", "DEBT", "DECK", "DEED",
"DEEM", "DEER", "DEFT", "DEFY", "DELL", "DENT", "DENY", "DESK",
"DIAL", "DICE", "DIED", "DIET", "DIME", "DINE", "DING", "DINT",
"DIRE", "DIRT", "DISC", "DISH", "DISK", "DIVE", "DOCK", "DOES",
"DOLE", "DOLL", "DOLT", "DOME", "DONE", "DOOM", "DOOR", "DORA",
"DOSE", "DOTE", "DOUG", "DOUR", "DOVE", "DOWN", "DRAB", "DRAG",
"DRAM", "DRAW", "DREW", "DRUB", "DRUG", "DRUM", "DUAL", "DUCK",
"DUCT", "DUEL", "DUET", "DUKE", "DULL", "DUMB", "DUNE", "DUNK",
"DUSK", "DUST", "DUTY", "EACH", "EARL", "EARN", "EASE", "EAST",
"EASY", "EBEN", "ECHO", "EDDY", "EDEN", "EDGE", "EDGY", "EDIT",
"EDNA", "EGAN", "ELAN", "ELBA", "ELLA", "ELSE", "EMIL", "EMIT",
"EMMA", "ENDS", "ERIC", "EROS", "EVEN", "EVER", "EVIL", "EYED",
"FACE", "FACT", "FADE", "FAIL", "FAIN", "FAIR", "FAKE", "FALL",
"FAME", "FANG", "FARM", "FAST", "FATE", "FAWN", "FEAR", "FEAT",
"FEED", "FEEL", "FEET", "FELL", "FELT", "FEND", "FERN", "FEST",
"FEUD", "FIEF", "FIGS", "FILE", "FILL", "FILM", "FIND", "FINE",
"FINK", "FIRE", "FIRM", "FISH", "FISK", "FIST", "FITS", "FIVE",
"FLAG", "FLAK", "FLAM", "FLAT", "FLAW", "FLEA", "FLED", "FLEW",
"FLIT", "FLOC", "FLOG", "FLOW", "FLUB", "FLUE", "FOAL", "FOAM",
"FOGY", "FOIL", "FOLD", "FOLK", "FOND", "FONT", "FOOD", "FOOL",
"FOOT", "FORD", "FORE", "FORK", "FORM", "FORT", "FOSS", "FOUL",
"FOUR", "FOWL", "FRAU", "FRAY", "FRED", "FREE", "FRET", "FREY",
"FROG", "FROM", "FUEL", "FULL", "FUME", "FUND", "FUNK", "FURY",
"FUSE", "FUSS", "GAFF", "GAGE", "GAIL", "GAIN", "GAIT", "GALA",
"GALE", "GALL", "GALT", "GAME", "GANG", "GARB", "GARY", "GASH",
"GATE", "GAUL", "GAUR", "GAVE", "GAWK", "GEAR", "GELD", "GENE",
"GENT", "GERM", "GETS", "GIBE", "GIFT", "GILD", "GILL", "GILT",
"GINA", "GIRD", "GIRL", "GIST", "GIVE", "GLAD", "GLEE", "GLEN",
"GLIB", "GLOB", "GLOM", "GLOW", "GLUE", "GLUM", "GLUT", "GOAD",
"GOAL", "GOAT", "GOER", "GOES", "GOLD", "GOLF", "GONE", "GONG",
"GOOD", "GOOF", "GORE", "GORY", "GOSH", "GOUT", "GOWN", "GRAB",
"GRAD", "GRAY", "GREG", "GREW", "GREY", "GRID", "GRIM", "GRIN",
"GRIT", "GROW", "GRUB", "GULF", "GULL", "GUNK", "GURU", "GUSH",
"GUST", "GWEN", "GWYN", "HAAG", "HAAS", "HACK", "HAIL", "HAIR",
"HALE", "HALF", "HALL", "HALO", "HALT", "HAND", "HANG", "HANK",
"HANS", "HARD", "HARK", "HARM", "HART", "HASH", "HAST", "HATE",
"HATH", "HAUL", "HAVE", "HAWK", "HAYS", "HEAD", "HEAL", "HEAR",
"HEAT", "HEBE", "HECK", "HEED", "HEEL", "HEFT", "HELD", "HELL",
"HELM", "HERB", "HERD", "HERE", "HERO", "HERS", "HESS", "HEWN",
"HICK", "HIDE", "HIGH", "HIKE", "HILL", "HILT", "HIND", "HINT",
"HIRE", "HISS", "HIVE", "HOBO", "HOCK", "HOFF", "HOLD", "HOLE",
"HOLM", "HOLT", "HOME", "HONE", "HONK", "HOOD", "HOOF", "HOOK",
"HOOT", "HORN", "HOSE", "HOST", "HOUR", "HOVE", "HOWE", "HOWL",
"HOYT", "HUCK", "HUED", "HUFF", "HUGE", "HUGH", "HUGO", "HULK",
"HULL", "HUNK", "HUNT", "HURD", "HURL", "HURT", "HUSH", "HYDE",
"HYMN", "IBIS", "ICON", "IDEA", "IDLE", "IFFY", "INCA", "INCH",
"INTO", "IONS", "IOTA", "IOWA", "IRIS", "IRMA", "IRON", "ISLE",
"ITCH", "ITEM", "IVAN", "JACK", "JADE", "JAIL", "JAKE", "JANE",
"JAVA", "JEAN", "JEFF", "JERK", "JESS", "JEST", "JIBE", "JILL",
"JILT", "JIVE", "JOAN", "JOBS", "JOCK", "JOEL", "JOEY", "JOHN",
"JOIN", "JOKE", "JOLT", "JOVE", "JUDD", "JUDE", "JUDO", "JUDY",
"JUJU", "JUKE", "JULY", "JUNE", "JUNK", "JUNO", "JURY", "JUST",
"JUTE", "KAHN", "KALE", "KANE", "KANT", "KARL", "KATE", "KEEL",
"KEEN", "KENO", "KENT", "KERN", "KERR", "KEYS", "KICK", "KILL",
"KIND", "KING", "KIRK", "KISS", "KITE", "KLAN", "KNEE", "KNEW",
"KNIT", "KNOB", "KNOT", "KNOW", "KOCH", "KONG", "KUDO", "KURD",
"KURT", "KYLE", "LACE", "LACK", "LACY", "LADY", "LAID", "LAIN",
"LAIR", "LAKE", "LAMB", "LAME", "LAND", "LANE", "LANG", "LARD",
"LARK", "LASS", "LAST", "LATE", "LAUD", "LAVA", "LAWN", "LAWS",
"LAYS", "LEAD", "LEAF", "LEAK", "LEAN", "LEAR", "LEEK", "LEER",
"LEFT", "LEND", "LENS", "LENT", "LEON", "LESK", "LESS", "LEST",
"LETS", "LIAR", "LICE", "LICK", "LIED", "LIEN", "LIES", "LIEU",
"LIFE", "LIFT", "LIKE", "LILA", "LILT", "LILY", "LIMA", "LIMB",
"LIME", "LIND", "LINE", "LINK", "LINT", "LION", "LISA", "LIST",
"LIVE", "LOAD", "LOAF", "LOAM", "LOAN", "LOCK", "LOFT", "LOGE",
"LOIS", "LOLA", "LONE", "LONG", "LOOK", "LOON", "LOOT", "LORD",
"LORE", "LOSE", "LOSS", "LOST", "LOUD", "LOVE", "LOWE", "LUCK",
"LUCY", "LUGE", "LUKE", "LULU", "LUND", "LUNG", "LURA", "LURE",
"LURK", "LUSH", "LUST", "LYLE", "LYNN", "LYON", "LYRA", "MACE",
"MADE", "MAGI", "MAID", "MAIL", "MAIN", "MAKE", "MALE", "MALI",
"MALL", "MALT", "MANA", "MANN", "MANY", "MARC", "MARE", "MARK",
"MARS", "MART", "MARY", "MASH", "MASK", "MASS", "MAST", "MATE",
"MATH", "MAUL", "MAYO", "MEAD", "MEAL", "MEAN", "MEAT", "MEEK",
"MEET", "MELD", "MELT", "MEMO", "MEND", "MENU", "MERT", "MESH",
"MESS", "MICE", "MIKE", "MILD", "MILE", "MILK", "MILL", "MILT",
"MIMI", "MIND", "MINE", "MINI", "MINK", "MINT", "MIRE", "MISS",
"MIST", "MITE", "MITT", "MOAN", "MOAT", "MOCK", "MODE", "MOLD",
"MOLE", "MOLL", "MOLT", "MONA", "MONK", "MONT", "MOOD", "MOON",
"MOOR", "MOOT", "MORE", "MORN", "MORT", "MOSS", "MOST", "MOTH",
"MOVE", "MUCH", "MUCK", "MUDD", "MUFF", "MULE", "MULL", "MURK",
"MUSH", "MUST", "MUTE", "MUTT", "MYRA", "MYTH", "NAGY", "NAIL",
"NAIR", "NAME", "NARY", "NASH", "NAVE", "NAVY", "NEAL", "NEAR",
"NEAT", "NECK", "NEED", "NEIL", "NELL", "NEON", "NERO", "NESS",
"NEST", "NEWS", "NEWT", "NIBS", "NICE", "NICK", "NILE", "NINA",
"NINE", "NOAH", "NODE", "NOEL", "NOLL", "NONE", "NOOK", "NOON",
"NORM", "NOSE", "NOTE", "NOUN", "NOVA", "NUDE", "NULL", "NUMB",
"OATH", "OBEY", "OBOE", "ODIN", "OHIO", "OILY", "OINT", "OKAY",
"OLAF", "OLDY", "OLGA", "OLIN", "OMAN", "OMEN", "OMIT", "ONCE",
"ONES", "ONLY", "ONTO", "ONUS", "ORAL", "ORGY", "OSLO", "OTIS",
"OTTO", "OUCH", "OUST", "OUTS", "OVAL", "OVEN", "OVER", "OWLY",
"OWNS", "QUAD", "QUIT", "QUOD", "RACE", "RACK", "RACY", "RAFT",
"RAGE", "RAID", "RAIL", "RAIN", "RAKE", "RANK", "RANT", "RARE",
"RASH", "RATE", "RAVE", "RAYS", "READ", "REAL", "REAM", "REAR",
"RECK", "REED", "REEF", "REEK", "REEL", "REID", "REIN", "RENA",
"REND", "RENT", "REST", "RICE", "RICH", "RICK", "RIDE", "RIFT",
"RILL", "RIME", "RING", "RINK", "RISE", "RISK", "RITE", "ROAD",
"ROAM", "ROAR", "ROBE", "ROCK", "RODE", "ROIL", "ROLL", "ROME",
"ROOD", "ROOF", "ROOK", "ROOM", "ROOT", "ROSA", "ROSE", "ROSS",
"ROSY", "ROTH", "ROUT", "ROVE", "ROWE", "ROWS", "RUBE", "RUBY",
"RUDE", "RUDY", "RUIN", "RULE", "RUNG", "RUNS", "RUNT", "RUSE",
"RUSH", "RUSK", "RUSS", "RUST", "RUTH", "SACK", "SAFE", "SAGE",
"SAID", "SAIL", "SALE", "SALK", "SALT", "SAME", "SAND", "SANE",
"SANG", "SANK", "SARA", "SAUL", "SAVE", "SAYS", "SCAN", "SCAR",
"SCAT", "SCOT", "SEAL", "SEAM", "SEAR", "SEAT", "SEED", "SEEK",
"SEEM", "SEEN", "SEES", "SELF", "SELL", "SEND", "SENT", "SETS",
"SEWN", "SHAG", "SHAM", "SHAW", "SHAY", "SHED", "SHIM", "SHIN",
"SHOD", "SHOE", "SHOT", "SHOW", "SHUN", "SHUT", "SICK", "SIDE",
"SIFT", "SIGH", "SIGN", "SILK", "SILL", "SILO", "SILT", "SINE",
"SING", "SINK", "SIRE", "SITE", "SITS", "SITU", "SKAT", "SKEW",
"SKID", "SKIM", "SKIN", "SKIT", "SLAB", "SLAM", "SLAT", "SLAY",
"SLED", "SLEW", "SLID", "SLIM", "SLIT", "SLOB", "SLOG", "SLOT",
"SLOW", "SLUG", "SLUM", "SLUR", "SMOG", "SMUG", "SNAG", "SNOB",
"SNOW", "SNUB", "SNUG", "SOAK", "SOAR", "SOCK", "SODA", "SOFA",
"SOFT", "SOIL", "SOLD", "SOME", "SONG", "SOON", "SOOT", "SORE",
"SORT", "SOUL", "SOUR", "SOWN", "STAB", "STAG", "STAN", "STAR",
"STAY", "STEM", "STEW", "STIR", "STOW", "STUB", "STUN", "SUCH",
"SUDS", "SUIT", "SULK", "SUMS", "SUNG", "SUNK", "SURE", "SURF",
"SWAB", "SWAG", "SWAM", "SWAN", "SWAT", "SWAY", "SWIM", "SWUM",
"TACK", "TACT", "TAIL", "TAKE", "TALE", "TALK", "TALL", "TANK",
"TASK", "TATE", "TAUT", "TEAL", "TEAM", "TEAR", "TECH", "TEEM",
"TEEN", "TEET", "TELL", "TEND", "TENT", "TERM", "TERN", "TESS",
"TEST", "THAN", "THAT", "THEE", "THEM", "THEN", "THEY", "THIN",
"THIS", "THUD", "THUG", "TICK", "TIDE", "TIDY", "TIED", "TIER",
"TILE", "TILL", "TILT", "TIME", "TINA", "TINE", "TINT", "TINY",
"TIRE", "TOAD", "TOGO", "TOIL", "TOLD", "TOLL", "TONE", "TONG",
"TONY", "TOOK", "TOOL", "TOOT", "TORE", "TORN", "TOTE", "TOUR",
"TOUT", "TOWN", "TRAG", "TRAM", "TRAY", "TREE", "TREK", "TRIG",
"TRIM", "TRIO", "TROD", "TROT", "TROY", "TRUE", "TUBA", "TUBE",
"TUCK", "TUFT", "TUNA", "TUNE", "TUNG", "TURF", "TURN", "TUSK",
"TWIG", "TWIN", "TWIT", "ULAN", "UNIT", "URGE", "USED", "USER",
"USES", "UTAH", "VAIL", "VAIN", "VALE", "VARY", "VASE", "VAST",
"VEAL", "VEDA", "VEIL", "VEIN", "VEND", "VENT", "VERB", "VERY",
"VETO", "VICE", "VIEW", "VINE", "VISE", "VOID", "VOLT", "VOTE",
"WACK", "WADE", "WAGE", "WAIL", "WAIT", "WAKE", "WALE", "WALK",
"WALL", "WALT", "WAND", "WANE", "WANG", "WANT", "WARD", "WARM",
"WARN", "WART", "WASH", "WAST", "WATS", "WATT", "WAVE", "WAVY",
"WAYS", "WEAK", "WEAL", "WEAN", "WEAR", "WEED", "WEEK", "WEIR",
"WELD", "WELL", "WELT", "WENT", "WERE", "WERT", "WEST", "WHAM",
"WHAT", "WHEE", "WHEN", "WHET", "WHOA", "WHOM", "WICK", "WIFE",
"WILD", "WILL", "WIND", "WINE", "WING", "WINK", "WINO", "WIRE",
"WISE", "WISH", "WITH", "WOLF", "WONT", "WOOD", "WOOL", "WORD",
"WORE", "WORK", "WORM", "WORN", "WOVE", "WRIT", "WYNN", "YALE",
"YANG", "YANK", "YARD", "YARN", "YAWL", "YAWN", "YEAH", "YEAR",
"YELL", "YOGA", "YOKE" ]
if __name__=='__main__':
data = [('EB33F77EE73D4053', 'TIDE ITCH SLOW REIN RULE MOT'),
('CCAC2AED591056BE4F90FD441C534766',
'RASH BUSH MILK LOOK BAD BRIM AVID GAFF BAIT ROT POD LOVE'),
('EFF81F9BFBC65350920CDD7416DE8009',
'TROD MUTE TAIL WARM CHAR KONG HAAG CITY BORE O TEAL AWL')
]
for key, words in data:
print 'Trying key', key
key=binascii.a2b_hex(key)
w2=key_to_english(key)
if w2!=words:
print 'key_to_english fails on key', repr(key), ', producing', str(w2)
k2=english_to_key(words)
if k2!=key:
print 'english_to_key fails on key', repr(key), ', producing', repr(k2)

16
gdata/analytics/Crypto/Util/__init__.py Normal file
View File

@ -0,0 +1,16 @@
"""Miscellaneous modules
Contains useful modules that don't belong into any of the
other Crypto.* subpackages.
Crypto.Util.number Number-theoretic functions (primality testing, etc.)
Crypto.Util.randpool Random number generation
Crypto.Util.RFC1751 Converts between 128-bit keys and human-readable
strings of words.
"""
__all__ = ['randpool', 'RFC1751', 'number']
__revision__ = "$Id: __init__.py,v 1.4 2003/02/28 15:26:00 akuchling Exp $"

201
gdata/analytics/Crypto/Util/number.py Normal file
View File

@ -0,0 +1,201 @@
#
# number.py : Number-theoretic functions
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: number.py,v 1.13 2003/04/04 18:21:07 akuchling Exp $"
bignum = long
try:
from Crypto.PublicKey import _fastmath
except ImportError:
_fastmath = None
# Commented out and replaced with faster versions below
## def long2str(n):
## s=''
## while n>0:
## s=chr(n & 255)+s
## n=n>>8
## return s
## import types
## def str2long(s):
## if type(s)!=types.StringType: return s # Integers will be left alone
## return reduce(lambda x,y : x*256+ord(y), s, 0L)
def size (N):
"""size(N:long) : int
Returns the size of the number N in bits.
"""
bits, power = 0,1L
while N >= power:
bits += 1
power = power << 1
return bits
def getRandomNumber(N, randfunc):
"""getRandomNumber(N:int, randfunc:callable):long
Return an N-bit random number."""
S = randfunc(N/8)
odd_bits = N % 8
if odd_bits != 0:
char = ord(randfunc(1)) >> (8-odd_bits)
S = chr(char) + S
value = bytes_to_long(S)
value |= 2L ** (N-1) # Ensure high bit is set
assert size(value) >= N
return value
def GCD(x,y):
"""GCD(x:long, y:long): long
Return the GCD of x and y.
"""
x = abs(x) ; y = abs(y)
while x > 0:
x, y = y % x, x
return y
def inverse(u, v):
"""inverse(u:long, u:long):long
Return the inverse of u mod v.
"""
u3, v3 = long(u), long(v)
u1, v1 = 1L, 0L
while v3 > 0:
q=u3 / v3
u1, v1 = v1, u1 - v1*q
u3, v3 = v3, u3 - v3*q
while u1<0:
u1 = u1 + v
return u1
# Given a number of bits to generate and a random generation function,
# find a prime number of the appropriate size.
def getPrime(N, randfunc):
"""getPrime(N:int, randfunc:callable):long
Return a random N-bit prime number.
"""
number=getRandomNumber(N, randfunc) | 1
while (not isPrime(number)):
number=number+2
return number
def isPrime(N):
"""isPrime(N:long):bool
Return true if N is prime.
"""
if N == 1:
return 0
if N in sieve:
return 1
for i in sieve:
if (N % i)==0:
return 0
# Use the accelerator if available
if _fastmath is not None:
return _fastmath.isPrime(N)
# Compute the highest bit that's set in N
N1 = N - 1L
n = 1L
while (n<N):
n=n<<1L
n = n >> 1L
# Rabin-Miller test
for c in sieve[:7]:
a=long(c) ; d=1L ; t=n
while (t): # Iterate over the bits in N1
x=(d*d) % N
if x==1L and d!=1L and d!=N1:
return 0 # Square root of 1 found
if N1 & t:
d=(x*a) % N
else:
d=x
t = t >> 1L
if d!=1L:
return 0
return 1
# Small primes used for checking primality; these are all the primes
# less than 256. This should be enough to eliminate most of the odd
# numbers before needing to do a Rabin-Miller test at all.
sieve=[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59,
61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113, 127,
131, 137, 139, 149, 151, 157, 163, 167, 173, 179, 181, 191, 193,
197, 199, 211, 223, 227, 229, 233, 239, 241, 251]
# Improved conversion functions contributed by Barry Warsaw, after
# careful benchmarking
import struct
def long_to_bytes(n, blocksize=0):
"""long_to_bytes(n:long, blocksize:int) : string
Convert a long integer to a byte string.
If optional blocksize is given and greater than zero, pad the front of the
byte string with binary zeros so that the length is a multiple of
blocksize.
"""
# after much testing, this algorithm was deemed to be the fastest
s = ''
n = long(n)
pack = struct.pack
while n > 0:
s = pack('>I', n & 0xffffffffL) + s
n = n >> 32
# strip off leading zeros
for i in range(len(s)):
if s[i] != '\000':
break
else:
# only happens when n == 0
s = '\000'
i = 0
s = s[i:]
# add back some pad bytes. this could be done more efficiently w.r.t. the
# de-padding being done above, but sigh...
if blocksize > 0 and len(s) % blocksize:
s = (blocksize - len(s) % blocksize) * '\000' + s
return s
def bytes_to_long(s):
"""bytes_to_long(string) : long
Convert a byte string to a long integer.
This is (essentially) the inverse of long_to_bytes().
"""
acc = 0L
unpack = struct.unpack
length = len(s)
if length % 4:
extra = (4 - length % 4)
s = '\000' * extra + s
length = length + extra
for i in range(0, length, 4):
acc = (acc << 32) + unpack('>I', s[i:i+4])[0]
return acc
# For backwards compatibility...
import warnings
def long2str(n, blocksize=0):
warnings.warn("long2str() has been replaced by long_to_bytes()")
return long_to_bytes(n, blocksize)
def str2long(s):
warnings.warn("str2long() has been replaced by bytes_to_long()")
return bytes_to_long(s)

421
gdata/analytics/Crypto/Util/randpool.py Normal file
View File

@ -0,0 +1,421 @@
#
# randpool.py : Cryptographically strong random number generation
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: randpool.py,v 1.14 2004/05/06 12:56:54 akuchling Exp $"
import time, array, types, warnings, os.path
from Crypto.Util.number import long_to_bytes
try:
import Crypto.Util.winrandom as winrandom
except:
winrandom = None
STIRNUM = 3
class RandomPool:
"""randpool.py : Cryptographically strong random number generation.
The implementation here is similar to the one in PGP. To be
cryptographically strong, it must be difficult to determine the RNG's
output, whether in the future or the past. This is done by using
a cryptographic hash function to "stir" the random data.
Entropy is gathered in the same fashion as PGP; the highest-resolution
clock around is read and the data is added to the random number pool.
A conservative estimate of the entropy is then kept.
If a cryptographically secure random source is available (/dev/urandom
on many Unixes, Windows CryptGenRandom on most Windows), then use
it.
Instance Attributes:
bits : int
Maximum size of pool in bits
bytes : int
Maximum size of pool in bytes
entropy : int
Number of bits of entropy in this pool.
Methods:
add_event([s]) : add some entropy to the pool
get_bytes(int) : get N bytes of random data
randomize([N]) : get N bytes of randomness from external source
"""
def __init__(self, numbytes = 160, cipher=None, hash=None):
if hash is None:
from Crypto.Hash import SHA as hash
# The cipher argument is vestigial; it was removed from
# version 1.1 so RandomPool would work even in the limited
# exportable subset of the code
if cipher is not None:
warnings.warn("'cipher' parameter is no longer used")
if isinstance(hash, types.StringType):
# ugly hack to force __import__ to give us the end-path module
hash = __import__('Crypto.Hash.'+hash,
None, None, ['new'])
warnings.warn("'hash' parameter should now be a hashing module")
self.bytes = numbytes
self.bits = self.bytes*8
self.entropy = 0
self._hash = hash
# Construct an array to hold the random pool,
# initializing it to 0.
self._randpool = array.array('B', [0]*self.bytes)
self._event1 = self._event2 = 0
self._addPos = 0
self._getPos = hash.digest_size
self._lastcounter=time.time()
self.__counter = 0
self._measureTickSize() # Estimate timer resolution
self._randomize()
def _updateEntropyEstimate(self, nbits):
self.entropy += nbits
if self.entropy < 0:
self.entropy = 0
elif self.entropy > self.bits:
self.entropy = self.bits
def _randomize(self, N = 0, devname = '/dev/urandom'):
"""_randomize(N, DEVNAME:device-filepath)
collects N bits of randomness from some entropy source (e.g.,
/dev/urandom on Unixes that have it, Windows CryptoAPI
CryptGenRandom, etc)
DEVNAME is optional, defaults to /dev/urandom. You can change it
to /dev/random if you want to block till you get enough
entropy.
"""
data = ''
if N <= 0:
nbytes = int((self.bits - self.entropy)/8+0.5)
else:
nbytes = int(N/8+0.5)
if winrandom:
# Windows CryptGenRandom provides random data.
data = winrandom.new().get_bytes(nbytes)
elif os.path.exists(devname):
# Many OSes support a /dev/urandom device
try:
f=open(devname)
data=f.read(nbytes)
f.close()
except IOError, (num, msg):
if num!=2: raise IOError, (num, msg)
# If the file wasn't found, ignore the error
if data:
self._addBytes(data)
# Entropy estimate: The number of bits of
# data obtained from the random source.
self._updateEntropyEstimate(8*len(data))
self.stir_n() # Wash the random pool
def randomize(self, N=0):
"""randomize(N:int)
use the class entropy source to get some entropy data.
This is overridden by KeyboardRandomize().
"""
return self._randomize(N)
def stir_n(self, N = STIRNUM):
"""stir_n(N)
stirs the random pool N times
"""
for i in xrange(N):
self.stir()
def stir (self, s = ''):
"""stir(s:string)
Mix up the randomness pool. This will call add_event() twice,
but out of paranoia the entropy attribute will not be
increased. The optional 's' parameter is a string that will
be hashed with the randomness pool.
"""
entropy=self.entropy # Save inital entropy value
self.add_event()
# Loop over the randomness pool: hash its contents
# along with a counter, and add the resulting digest
# back into the pool.
for i in range(self.bytes / self._hash.digest_size):
h = self._hash.new(self._randpool)
h.update(str(self.__counter) + str(i) + str(self._addPos) + s)
self._addBytes( h.digest() )
self.__counter = (self.__counter + 1) & 0xFFFFffffL
self._addPos, self._getPos = 0, self._hash.digest_size
self.add_event()
# Restore the old value of the entropy.
self.entropy=entropy
def get_bytes (self, N):
"""get_bytes(N:int) : string
Return N bytes of random data.
"""
s=''
i, pool = self._getPos, self._randpool
h=self._hash.new()
dsize = self._hash.digest_size
num = N
while num > 0:
h.update( self._randpool[i:i+dsize] )
s = s + h.digest()
num = num - dsize
i = (i + dsize) % self.bytes
if i<dsize:
self.stir()
i=self._getPos
self._getPos = i
self._updateEntropyEstimate(- 8*N)
return s[:N]
def add_event(self, s=''):
"""add_event(s:string)
Add an event to the random pool. The current time is stored
between calls and used to estimate the entropy. The optional
's' parameter is a string that will also be XORed into the pool.
Returns the estimated number of additional bits of entropy gain.
"""
event = time.time()*1000
delta = self._noise()
s = (s + long_to_bytes(event) +
4*chr(0xaa) + long_to_bytes(delta) )
self._addBytes(s)
if event==self._event1 and event==self._event2:
# If events are coming too closely together, assume there's
# no effective entropy being added.
bits=0
else:
# Count the number of bits in delta, and assume that's the entropy.
bits=0
while delta:
delta, bits = delta>>1, bits+1
if bits>8: bits=8
self._event1, self._event2 = event, self._event1
self._updateEntropyEstimate(bits)
return bits
# Private functions
def _noise(self):
# Adds a bit of noise to the random pool, by adding in the
# current time and CPU usage of this process.
# The difference from the previous call to _noise() is taken
# in an effort to estimate the entropy.
t=time.time()
delta = (t - self._lastcounter)/self._ticksize*1e6
self._lastcounter = t
self._addBytes(long_to_bytes(long(1000*time.time())))
self._addBytes(long_to_bytes(long(1000*time.clock())))
self._addBytes(long_to_bytes(long(1000*time.time())))
self._addBytes(long_to_bytes(long(delta)))
# Reduce delta to a maximum of 8 bits so we don't add too much
# entropy as a result of this call.
delta=delta % 0xff
return int(delta)
def _measureTickSize(self):
# _measureTickSize() tries to estimate a rough average of the
# resolution of time that you can see from Python. It does
# this by measuring the time 100 times, computing the delay
# between measurements, and taking the median of the resulting
# list. (We also hash all the times and add them to the pool)
interval = [None] * 100
h = self._hash.new(`(id(self),id(interval))`)
# Compute 100 differences
t=time.time()
h.update(`t`)
i = 0
j = 0
while i < 100:
t2=time.time()
h.update(`(i,j,t2)`)
j += 1
delta=int((t2-t)*1e6)
if delta:
interval[i] = delta
i += 1
t=t2
# Take the median of the array of intervals
interval.sort()
self._ticksize=interval[len(interval)/2]
h.update(`(interval,self._ticksize)`)
# mix in the measurement times and wash the random pool
self.stir(h.digest())
def _addBytes(self, s):
"XOR the contents of the string S into the random pool"
i, pool = self._addPos, self._randpool
for j in range(0, len(s)):
pool[i]=pool[i] ^ ord(s[j])
i=(i+1) % self.bytes
self._addPos = i
# Deprecated method names: remove in PCT 2.1 or later.
def getBytes(self, N):
warnings.warn("getBytes() method replaced by get_bytes()",
DeprecationWarning)
return self.get_bytes(N)
def addEvent (self, event, s=""):
warnings.warn("addEvent() method replaced by add_event()",
DeprecationWarning)
return self.add_event(s + str(event))
class PersistentRandomPool (RandomPool):
def __init__ (self, filename=None, *args, **kwargs):
RandomPool.__init__(self, *args, **kwargs)
self.filename = filename
if filename:
try:
# the time taken to open and read the file might have
# a little disk variability, modulo disk/kernel caching...
f=open(filename, 'rb')
self.add_event()
data = f.read()
self.add_event()
# mix in the data from the file and wash the random pool
self.stir(data)
f.close()
except IOError:
# Oh, well; the file doesn't exist or is unreadable, so
# we'll just ignore it.
pass
def save(self):
if self.filename == "":
raise ValueError, "No filename set for this object"
# wash the random pool before save, provides some forward secrecy for
# old values of the pool.
self.stir_n()
f=open(self.filename, 'wb')
self.add_event()
f.write(self._randpool.tostring())
f.close()
self.add_event()
# wash the pool again, provide some protection for future values
self.stir()
# non-echoing Windows keyboard entry
_kb = 0
if not _kb:
try:
import msvcrt
class KeyboardEntry:
def getch(self):
c = msvcrt.getch()
if c in ('\000', '\xe0'):
# function key
c += msvcrt.getch()
return c
def close(self, delay = 0):
if delay:
time.sleep(delay)
while msvcrt.kbhit():
msvcrt.getch()
_kb = 1
except:
pass
# non-echoing Posix keyboard entry
if not _kb:
try:
import termios
class KeyboardEntry:
def __init__(self, fd = 0):
self._fd = fd
self._old = termios.tcgetattr(fd)
new = termios.tcgetattr(fd)
new[3]=new[3] & ~termios.ICANON & ~termios.ECHO
termios.tcsetattr(fd, termios.TCSANOW, new)
def getch(self):
termios.tcflush(0, termios.TCIFLUSH) # XXX Leave this in?
return os.read(self._fd, 1)
def close(self, delay = 0):
if delay:
time.sleep(delay)
termios.tcflush(self._fd, termios.TCIFLUSH)
termios.tcsetattr(self._fd, termios.TCSAFLUSH, self._old)
_kb = 1
except:
pass
class KeyboardRandomPool (PersistentRandomPool):
def __init__(self, *args, **kwargs):
PersistentRandomPool.__init__(self, *args, **kwargs)
def randomize(self, N = 0):
"Adds N bits of entropy to random pool. If N is 0, fill up pool."
import os, string, time
if N <= 0:
bits = self.bits - self.entropy
else:
bits = N*8
if bits == 0:
return
print bits,'bits of entropy are now required. Please type on the keyboard'
print 'until enough randomness has been accumulated.'
kb = KeyboardEntry()
s='' # We'll save the characters typed and add them to the pool.
hash = self._hash
e = 0
try:
while e < bits:
temp=str(bits-e).rjust(6)
os.write(1, temp)
s=s+kb.getch()
e += self.add_event(s)
os.write(1, 6*chr(8))
self.add_event(s+hash.new(s).digest() )
finally:
kb.close()
print '\n\007 Enough. Please wait a moment.\n'
self.stir_n() # wash the random pool.
kb.close(4)
if __name__ == '__main__':
pool = RandomPool()
print 'random pool entropy', pool.entropy, 'bits'
pool.add_event('something')
print `pool.get_bytes(100)`
import tempfile, os
fname = tempfile.mktemp()
pool = KeyboardRandomPool(filename=fname)
print 'keyboard random pool entropy', pool.entropy, 'bits'
pool.randomize()
print 'keyboard random pool entropy', pool.entropy, 'bits'
pool.randomize(128)
pool.save()
saved = open(fname, 'rb').read()
print 'saved', `saved`
print 'pool ', `pool._randpool.tostring()`
newpool = PersistentRandomPool(fname)
print 'persistent random pool entropy', pool.entropy, 'bits'
os.remove(fname)

453
gdata/analytics/Crypto/Util/test.py Normal file
View File

@ -0,0 +1,453 @@
#
# test.py : Functions used for testing the modules
#
# Part of the Python Cryptography Toolkit
#
# Distribute and use freely; there are no restrictions on further
# dissemination and usage except those imposed by the laws of your
# country of residence. This software is provided "as is" without
# warranty of fitness for use or suitability for any purpose, express
# or implied. Use at your own risk or not at all.
#
__revision__ = "$Id: test.py,v 1.16 2004/08/13 22:24:18 akuchling Exp $"
import binascii
import string
import testdata
from Crypto.Cipher import *
def die(string):
import sys
print '***ERROR: ', string
# sys.exit(0) # Will default to continuing onward...
def print_timing (size, delta, verbose):
if verbose:
if delta == 0:
print 'Unable to measure time -- elapsed time too small'
else:
print '%.2f K/sec' % (size/delta)
def exerciseBlockCipher(cipher, verbose):
import string, time
try:
ciph = eval(cipher)
except NameError:
print cipher, 'module not available'
return None
print cipher+ ':'
str='1' # Build 128K of test data
for i in xrange(0, 17):
str=str+str
if ciph.key_size==0: ciph.key_size=16
password = 'password12345678Extra text for password'[0:ciph.key_size]
IV = 'Test IV Test IV Test IV Test'[0:ciph.block_size]
if verbose: print ' ECB mode:',
obj=ciph.new(password, ciph.MODE_ECB)
if obj.block_size != ciph.block_size:
die("Module and cipher object block_size don't match")
text='1234567812345678'[0:ciph.block_size]
c=obj.encrypt(text)
if (obj.decrypt(c)!=text): die('Error encrypting "'+text+'"')
text='KuchlingKuchling'[0:ciph.block_size]
c=obj.encrypt(text)
if (obj.decrypt(c)!=text): die('Error encrypting "'+text+'"')
text='NotTodayNotEver!'[0:ciph.block_size]
c=obj.encrypt(text)
if (obj.decrypt(c)!=text): die('Error encrypting "'+text+'"')
start=time.time()
s=obj.encrypt(str)
s2=obj.decrypt(s)
end=time.time()
if (str!=s2):
die('Error in resulting plaintext from ECB mode')
print_timing(256, end-start, verbose)
del obj
if verbose: print ' CFB mode:',
obj1=ciph.new(password, ciph.MODE_CFB, IV)
obj2=ciph.new(password, ciph.MODE_CFB, IV)
start=time.time()
ciphertext=obj1.encrypt(str[0:65536])
plaintext=obj2.decrypt(ciphertext)
end=time.time()
if (plaintext!=str[0:65536]):
die('Error in resulting plaintext from CFB mode')
print_timing(64, end-start, verbose)
del obj1, obj2
if verbose: print ' CBC mode:',
obj1=ciph.new(password, ciph.MODE_CBC, IV)
obj2=ciph.new(password, ciph.MODE_CBC, IV)
start=time.time()
ciphertext=obj1.encrypt(str)
plaintext=obj2.decrypt(ciphertext)
end=time.time()
if (plaintext!=str):
die('Error in resulting plaintext from CBC mode')
print_timing(256, end-start, verbose)
del obj1, obj2
if verbose: print ' PGP mode:',
obj1=ciph.new(password, ciph.MODE_PGP, IV)
obj2=ciph.new(password, ciph.MODE_PGP, IV)
start=time.time()
ciphertext=obj1.encrypt(str)
plaintext=obj2.decrypt(ciphertext)
end=time.time()
if (plaintext!=str):
die('Error in resulting plaintext from PGP mode')
print_timing(256, end-start, verbose)
del obj1, obj2
if verbose: print ' OFB mode:',
obj1=ciph.new(password, ciph.MODE_OFB, IV)
obj2=ciph.new(password, ciph.MODE_OFB, IV)
start=time.time()
ciphertext=obj1.encrypt(str)
plaintext=obj2.decrypt(ciphertext)
end=time.time()
if (plaintext!=str):
die('Error in resulting plaintext from OFB mode')
print_timing(256, end-start, verbose)
del obj1, obj2
def counter(length=ciph.block_size):
return length * 'a'
if verbose: print ' CTR mode:',
obj1=ciph.new(password, ciph.MODE_CTR, counter=counter)
obj2=ciph.new(password, ciph.MODE_CTR, counter=counter)
start=time.time()
ciphertext=obj1.encrypt(str)
plaintext=obj2.decrypt(ciphertext)
end=time.time()
if (plaintext!=str):
die('Error in resulting plaintext from CTR mode')
print_timing(256, end-start, verbose)
del obj1, obj2
# Test the IV handling
if verbose: print ' Testing IV handling'
obj1=ciph.new(password, ciph.MODE_CBC, IV)
plaintext='Test'*(ciph.block_size/4)*3
ciphertext1=obj1.encrypt(plaintext)
obj1.IV=IV
ciphertext2=obj1.encrypt(plaintext)
if ciphertext1!=ciphertext2:
die('Error in setting IV')
# Test keyword arguments
obj1=ciph.new(key=password)
obj1=ciph.new(password, mode=ciph.MODE_CBC)
obj1=ciph.new(mode=ciph.MODE_CBC, key=password)
obj1=ciph.new(IV=IV, mode=ciph.MODE_CBC, key=password)
return ciph
def exerciseStreamCipher(cipher, verbose):
import string, time
try:
ciph = eval(cipher)
except (NameError):
print cipher, 'module not available'
return None
print cipher + ':',
str='1' # Build 128K of test data
for i in xrange(0, 17):
str=str+str
key_size = ciph.key_size or 16
password = 'password12345678Extra text for password'[0:key_size]
obj1=ciph.new(password)
obj2=ciph.new(password)
if obj1.block_size != ciph.block_size:
die("Module and cipher object block_size don't match")
if obj1.key_size != ciph.key_size:
die("Module and cipher object key_size don't match")
text='1234567812345678Python'
c=obj1.encrypt(text)
if (obj2.decrypt(c)!=text): die('Error encrypting "'+text+'"')
text='B1FF I2 A R3A11Y |<00L D00D!!!!!'
c=obj1.encrypt(text)
if (obj2.decrypt(c)!=text): die('Error encrypting "'+text+'"')
text='SpamSpamSpamSpamSpamSpamSpamSpamSpam'
c=obj1.encrypt(text)
if (obj2.decrypt(c)!=text): die('Error encrypting "'+text+'"')
start=time.time()
s=obj1.encrypt(str)
str=obj2.decrypt(s)
end=time.time()
print_timing(256, end-start, verbose)
del obj1, obj2
return ciph
def TestStreamModules(args=['arc4', 'XOR'], verbose=1):
import sys, string
args=map(string.lower, args)
if 'arc4' in args:
# Test ARC4 stream cipher
arc4=exerciseStreamCipher('ARC4', verbose)
if (arc4!=None):
for entry in testdata.arc4:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=arc4.new(key)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('ARC4 failed on entry '+`entry`)
if 'xor' in args:
# Test XOR stream cipher
XOR=exerciseStreamCipher('XOR', verbose)
if (XOR!=None):
for entry in testdata.xor:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=XOR.new(key)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('XOR failed on entry '+`entry`)
def TestBlockModules(args=['aes', 'arc2', 'des', 'blowfish', 'cast', 'des3',
'idea', 'rc5'],
verbose=1):
import string
args=map(string.lower, args)
if 'aes' in args:
ciph=exerciseBlockCipher('AES', verbose) # AES
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.aes:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('AES failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print
for entry in testdata.aes_modes:
mode, key, plain, cipher, kw = entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, mode, **kw)
obj2=ciph.new(key, mode, **kw)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('AES encrypt failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print
plain2=obj2.decrypt(ciphertext)
if plain2!=plain:
die('AES decrypt failed on entry '+`entry`)
for i in plain2:
if verbose: print hex(ord(i)),
if verbose: print
if 'arc2' in args:
ciph=exerciseBlockCipher('ARC2', verbose) # Alleged RC2
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.arc2:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('ARC2 failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
print
if 'blowfish' in args:
ciph=exerciseBlockCipher('Blowfish',verbose)# Bruce Schneier's Blowfish cipher
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.blowfish:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('Blowfish failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print
if 'cast' in args:
ciph=exerciseBlockCipher('CAST', verbose) # CAST-128
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.cast:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('CAST failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print
if 0:
# The full-maintenance test; it requires 4 million encryptions,
# and correspondingly is quite time-consuming. I've disabled
# it; it's faster to compile block/cast.c with -DTEST and run
# the resulting program.
a = b = '\x01\x23\x45\x67\x12\x34\x56\x78\x23\x45\x67\x89\x34\x56\x78\x9A'
for i in range(0, 1000000):
obj = cast.new(b, cast.MODE_ECB)
a = obj.encrypt(a[:8]) + obj.encrypt(a[-8:])
obj = cast.new(a, cast.MODE_ECB)
b = obj.encrypt(b[:8]) + obj.encrypt(b[-8:])
if a!="\xEE\xA9\xD0\xA2\x49\xFD\x3B\xA6\xB3\x43\x6F\xB8\x9D\x6D\xCA\x92":
if verbose: print 'CAST test failed: value of "a" doesn\'t match'
if b!="\xB2\xC9\x5E\xB0\x0C\x31\xAD\x71\x80\xAC\x05\xB8\xE8\x3D\x69\x6E":
if verbose: print 'CAST test failed: value of "b" doesn\'t match'
if 'des' in args:
# Test/benchmark DES block cipher
des=exerciseBlockCipher('DES', verbose)
if (des!=None):
# Various tests taken from the DES library packaged with Kerberos V4
obj=des.new(binascii.a2b_hex('0123456789abcdef'), des.MODE_ECB)
s=obj.encrypt('Now is t')
if (s!=binascii.a2b_hex('3fa40e8a984d4815')):
die('DES fails test 1')
obj=des.new(binascii.a2b_hex('08192a3b4c5d6e7f'), des.MODE_ECB)
s=obj.encrypt('\000\000\000\000\000\000\000\000')
if (s!=binascii.a2b_hex('25ddac3e96176467')):
die('DES fails test 2')
obj=des.new(binascii.a2b_hex('0123456789abcdef'), des.MODE_CBC,
binascii.a2b_hex('1234567890abcdef'))
s=obj.encrypt("Now is the time for all ")
if (s!=binascii.a2b_hex('e5c7cdde872bf27c43e934008c389c0f683788499a7c05f6')):
die('DES fails test 3')
obj=des.new(binascii.a2b_hex('0123456789abcdef'), des.MODE_CBC,
binascii.a2b_hex('fedcba9876543210'))
s=obj.encrypt("7654321 Now is the time for \000\000\000\000")
if (s!=binascii.a2b_hex("ccd173ffab2039f4acd8aefddfd8a1eb468e91157888ba681d269397f7fe62b4")):
die('DES fails test 4')
del obj,s
# R. Rivest's test: see http://theory.lcs.mit.edu/~rivest/destest.txt
x=binascii.a2b_hex('9474B8E8C73BCA7D')
for i in range(0, 16):
obj=des.new(x, des.MODE_ECB)
if (i & 1): x=obj.decrypt(x)
else: x=obj.encrypt(x)
if x!=binascii.a2b_hex('1B1A2DDB4C642438'):
die("DES fails Rivest's test")
if verbose: print ' Verifying against test suite...'
for entry in testdata.des:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=des.new(key, des.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('DES failed on entry '+`entry`)
for entry in testdata.des_cbc:
key, iv, plain, cipher=entry
key, iv, cipher=binascii.a2b_hex(key),binascii.a2b_hex(iv),binascii.a2b_hex(cipher)
obj1=des.new(key, des.MODE_CBC, iv)
obj2=des.new(key, des.MODE_CBC, iv)
ciphertext=obj1.encrypt(plain)
if (ciphertext!=cipher):
die('DES CBC mode failed on entry '+`entry`)
if 'des3' in args:
ciph=exerciseBlockCipher('DES3', verbose) # Triple DES
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.des3:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('DES3 failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print
for entry in testdata.des3_cbc:
key, iv, plain, cipher=entry
key, iv, cipher=binascii.a2b_hex(key),binascii.a2b_hex(iv),binascii.a2b_hex(cipher)
obj1=ciph.new(key, ciph.MODE_CBC, iv)
obj2=ciph.new(key, ciph.MODE_CBC, iv)
ciphertext=obj1.encrypt(plain)
if (ciphertext!=cipher):
die('DES3 CBC mode failed on entry '+`entry`)
if 'idea' in args:
ciph=exerciseBlockCipher('IDEA', verbose) # IDEA block cipher
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.idea:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key, ciph.MODE_ECB)
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('IDEA failed on entry '+`entry`)
if 'rc5' in args:
# Ronald Rivest's RC5 algorithm
ciph=exerciseBlockCipher('RC5', verbose)
if (ciph!=None):
if verbose: print ' Verifying against test suite...'
for entry in testdata.rc5:
key,plain,cipher=entry
key=binascii.a2b_hex(key)
plain=binascii.a2b_hex(plain)
cipher=binascii.a2b_hex(cipher)
obj=ciph.new(key[4:], ciph.MODE_ECB,
version =ord(key[0]),
word_size=ord(key[1]),
rounds =ord(key[2]) )
ciphertext=obj.encrypt(plain)
if (ciphertext!=cipher):
die('RC5 failed on entry '+`entry`)
for i in ciphertext:
if verbose: print hex(ord(i)),
if verbose: print

25
gdata/analytics/Crypto/__init__.py Normal file
View File

@ -0,0 +1,25 @@
"""Python Cryptography Toolkit
A collection of cryptographic modules implementing various algorithms
and protocols.
Subpackages:
Crypto.Cipher Secret-key encryption algorithms (AES, DES, ARC4)
Crypto.Hash Hashing algorithms (MD5, SHA, HMAC)
Crypto.Protocol Cryptographic protocols (Chaffing, all-or-nothing
transform). This package does not contain any
network protocols.
Crypto.PublicKey Public-key encryption and signature algorithms
(RSA, DSA)
Crypto.Util Various useful modules and functions (long-to-string
conversion, random number generation, number
theoretic functions)
"""
__all__ = ['Cipher', 'Hash', 'Protocol', 'PublicKey', 'Util']
__version__ = '2.0.1'
__revision__ = "$Id: __init__.py,v 1.12 2005/06/14 01:20:22 akuchling Exp $"

38
gdata/analytics/Crypto/test.py Normal file
View File

@ -0,0 +1,38 @@
#
# Test script for the Python Cryptography Toolkit.
#
__revision__ = "$Id: test.py,v 1.7 2002/07/11 14:31:19 akuchling Exp $"
import os, sys
# Add the build directory to the front of sys.path
from distutils.util import get_platform
s = "build/lib.%s-%.3s" % (get_platform(), sys.version)
s = os.path.join(os.getcwd(), s)
sys.path.insert(0, s)
s = os.path.join(os.getcwd(), 'test')
sys.path.insert(0, s)
from Crypto.Util import test
args = sys.argv[1:]
quiet = "--quiet" in args
if quiet: args.remove('--quiet')
if not quiet:
print '\nStream Ciphers:'
print '==============='
if args: test.TestStreamModules(args, verbose= not quiet)
else: test.TestStreamModules(verbose= not quiet)
if not quiet:
print '\nBlock Ciphers:'
print '=============='
if args: test.TestBlockModules(args, verbose= not quiet)
else: test.TestBlockModules(verbose= not quiet)

835
gdata/analytics/__init__.py Normal file
View File

@ -0,0 +1,835 @@
#!/usr/bin/python
#
# Copyright (C) 2006 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Contains classes representing Google Data elements.
Extends Atom classes to add Google Data specific elements.
"""
__author__ = 'j.s@google.com (Jeffrey Scudder)'
import os
import atom
try:
from xml.etree import cElementTree as ElementTree
except ImportError:
try:
import cElementTree as ElementTree
except ImportError:
try:
from xml.etree import ElementTree
except ImportError:
from elementtree import ElementTree
# XML namespaces which are often used in GData entities.
GDATA_NAMESPACE = 'http://schemas.google.com/g/2005'
GDATA_TEMPLATE = '{http://schemas.google.com/g/2005}%s'
OPENSEARCH_NAMESPACE = 'http://a9.com/-/spec/opensearchrss/1.0/'
OPENSEARCH_TEMPLATE = '{http://a9.com/-/spec/opensearchrss/1.0/}%s'
BATCH_NAMESPACE = 'http://schemas.google.com/gdata/batch'
GACL_NAMESPACE = 'http://schemas.google.com/acl/2007'
GACL_TEMPLATE = '{http://schemas.google.com/acl/2007}%s'
# Labels used in batch request entries to specify the desired CRUD operation.
BATCH_INSERT = 'insert'
BATCH_UPDATE = 'update'
BATCH_DELETE = 'delete'
BATCH_QUERY = 'query'
class Error(Exception):
pass
class MissingRequiredParameters(Error):
pass
class MediaSource(object):
"""GData Entries can refer to media sources, so this class provides a
place to store references to these objects along with some metadata.
"""
def __init__(self, file_handle=None, content_type=None, content_length=None,
file_path=None, file_name=None):
"""Creates an object of type MediaSource.
Args:
file_handle: A file handle pointing to the file to be encapsulated in the
MediaSource
content_type: string The MIME type of the file. Required if a file_handle
is given.
content_length: int The size of the file. Required if a file_handle is
given.
file_path: string (optional) A full path name to the file. Used in
place of a file_handle.
file_name: string The name of the file without any path information.
Required if a file_handle is given.
"""
self.file_handle = file_handle
self.content_type = content_type
self.content_length = content_length
self.file_name = file_name
if (file_handle is None and content_type is not None and
file_path is not None):
self.setFile(file_path, content_type)
def setFile(self, file_name, content_type):
"""A helper function which can create a file handle from a given filename
and set the content type and length all at once.
Args:
file_name: string The path and file name to the file containing the media
content_type: string A MIME type representing the type of the media
"""
self.file_handle = open(file_name, 'rb')
self.content_type = content_type
self.content_length = os.path.getsize(file_name)
self.file_name = os.path.basename(file_name)
class LinkFinder(atom.LinkFinder):
"""An "interface" providing methods to find link elements
GData Entry elements often contain multiple links which differ in the rel
attribute or content type. Often, developers are interested in a specific
type of link so this class provides methods to find specific classes of
links.
This class is used as a mixin in GData entries.
"""
def GetSelfLink(self):
"""Find the first link with rel set to 'self'
Returns:
An atom.Link or none if none of the links had rel equal to 'self'
"""
for a_link in self.link:
if a_link.rel == 'self':
return a_link
return None
def GetEditLink(self):
for a_link in self.link:
if a_link.rel == 'edit':
return a_link
return None
def GetEditMediaLink(self):
"""The Picasa API mistakenly returns media-edit rather than edit-media, but
this may change soon.
"""
for a_link in self.link:
if a_link.rel == 'edit-media':
return a_link
if a_link.rel == 'media-edit':
return a_link
return None
def GetHtmlLink(self):
"""Find the first link with rel of alternate and type of text/html
Returns:
An atom.Link or None if no links matched
"""
for a_link in self.link:
if a_link.rel == 'alternate' and a_link.type == 'text/html':
return a_link
return None
def GetPostLink(self):
"""Get a link containing the POST target URL.
The POST target URL is used to insert new entries.
Returns:
A link object with a rel matching the POST type.
"""
for a_link in self.link:
if a_link.rel == 'http://schemas.google.com/g/2005#post':
return a_link
return None
def GetAclLink(self):
for a_link in self.link:
if a_link.rel == 'http://schemas.google.com/acl/2007#accessControlList':
return a_link
return None
def GetFeedLink(self):
for a_link in self.link:
if a_link.rel == 'http://schemas.google.com/g/2005#feed':
return a_link
return None
def GetNextLink(self):
for a_link in self.link:
if a_link.rel == 'next':
return a_link
return None
def GetPrevLink(self):
for a_link in self.link:
if a_link.rel == 'previous':
return a_link
return None
class TotalResults(atom.AtomBase):
"""opensearch:TotalResults for a GData feed"""
_tag = 'totalResults'
_namespace = OPENSEARCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
def __init__(self, extension_elements=None,
extension_attributes=None, text=None):
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def TotalResultsFromString(xml_string):
return atom.CreateClassFromXMLString(TotalResults, xml_string)
class StartIndex(atom.AtomBase):
"""The opensearch:startIndex element in GData feed"""
_tag = 'startIndex'
_namespace = OPENSEARCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
def __init__(self, extension_elements=None,
extension_attributes=None, text=None):
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def StartIndexFromString(xml_string):
return atom.CreateClassFromXMLString(StartIndex, xml_string)
class ItemsPerPage(atom.AtomBase):
"""The opensearch:itemsPerPage element in GData feed"""
_tag = 'itemsPerPage'
_namespace = OPENSEARCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
def __init__(self, extension_elements=None,
extension_attributes=None, text=None):
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def ItemsPerPageFromString(xml_string):
return atom.CreateClassFromXMLString(ItemsPerPage, xml_string)
class ExtendedProperty(atom.AtomBase):
"""The Google Data extendedProperty element.
Used to store arbitrary key-value information specific to your
application. The value can either be a text string stored as an XML
attribute (.value), or an XML node (XmlBlob) as a child element.
This element is used in the Google Calendar data API and the Google
Contacts data API.
"""
_tag = 'extendedProperty'
_namespace = GDATA_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['name'] = 'name'
_attributes['value'] = 'value'
def __init__(self, name=None, value=None, extension_elements=None,
extension_attributes=None, text=None):
self.name = name
self.value = value
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def GetXmlBlobExtensionElement(self):
"""Returns the XML blob as an atom.ExtensionElement.
Returns:
An atom.ExtensionElement representing the blob's XML, or None if no
blob was set.
"""
if len(self.extension_elements) < 1:
return None
else:
return self.extension_elements[0]
def GetXmlBlobString(self):
"""Returns the XML blob as a string.
Returns:
A string containing the blob's XML, or None if no blob was set.
"""
blob = self.GetXmlBlobExtensionElement()
if blob:
return blob.ToString()
return None
def SetXmlBlob(self, blob):
"""Sets the contents of the extendedProperty to XML as a child node.
Since the extendedProperty is only allowed one child element as an XML
blob, setting the XML blob will erase any preexisting extension elements
in this object.
Args:
blob: str, ElementTree Element or atom.ExtensionElement representing
the XML blob stored in the extendedProperty.
"""
# Erase any existing extension_elements, clears the child nodes from the
# extendedProperty.
self.extension_elements = []
if isinstance(blob, atom.ExtensionElement):
self.extension_elements.append(blob)
elif ElementTree.iselement(blob):
self.extension_elements.append(atom._ExtensionElementFromElementTree(
blob))
else:
self.extension_elements.append(atom.ExtensionElementFromString(blob))
def ExtendedPropertyFromString(xml_string):
return atom.CreateClassFromXMLString(ExtendedProperty, xml_string)
class GDataEntry(atom.Entry, LinkFinder):
"""Extends Atom Entry to provide data processing"""
_tag = atom.Entry._tag
_namespace = atom.Entry._namespace
_children = atom.Entry._children.copy()
_attributes = atom.Entry._attributes.copy()
def __GetId(self):
return self.__id
# This method was created to strip the unwanted whitespace from the id's
# text node.
def __SetId(self, id):
self.__id = id
if id is not None and id.text is not None:
self.__id.text = id.text.strip()
id = property(__GetId, __SetId)
def IsMedia(self):
"""Determines whether or not an entry is a GData Media entry.
"""
if (self.GetEditMediaLink()):
return True
else:
return False
def GetMediaURL(self):
"""Returns the URL to the media content, if the entry is a media entry.
Otherwise returns None.
"""
if not self.IsMedia():
return None
else:
return self.content.src
def GDataEntryFromString(xml_string):
"""Creates a new GDataEntry instance given a string of XML."""
return atom.CreateClassFromXMLString(GDataEntry, xml_string)
class GDataFeed(atom.Feed, LinkFinder):
"""A Feed from a GData service"""
_tag = 'feed'
_namespace = atom.ATOM_NAMESPACE
_children = atom.Feed._children.copy()
_attributes = atom.Feed._attributes.copy()
_children['{%s}totalResults' % OPENSEARCH_NAMESPACE] = ('total_results',
TotalResults)
_children['{%s}startIndex' % OPENSEARCH_NAMESPACE] = ('start_index',
StartIndex)
_children['{%s}itemsPerPage' % OPENSEARCH_NAMESPACE] = ('items_per_page',
ItemsPerPage)
# Add a conversion rule for atom:entry to make it into a GData
# Entry.
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', [GDataEntry])
def __GetId(self):
return self.__id
def __SetId(self, id):
self.__id = id
if id is not None and id.text is not None:
self.__id.text = id.text.strip()
id = property(__GetId, __SetId)
def __GetGenerator(self):
return self.__generator
def __SetGenerator(self, generator):
self.__generator = generator
if generator is not None:
self.__generator.text = generator.text.strip()
generator = property(__GetGenerator, __SetGenerator)
def __init__(self, author=None, category=None, contributor=None,
generator=None, icon=None, atom_id=None, link=None, logo=None,
rights=None, subtitle=None, title=None, updated=None, entry=None,
total_results=None, start_index=None, items_per_page=None,
extension_elements=None, extension_attributes=None, text=None):
"""Constructor for Source
Args:
author: list (optional) A list of Author instances which belong to this
class.
category: list (optional) A list of Category instances
contributor: list (optional) A list on Contributor instances
generator: Generator (optional)
icon: Icon (optional)
id: Id (optional) The entry's Id element
link: list (optional) A list of Link instances
logo: Logo (optional)
rights: Rights (optional) The entry's Rights element
subtitle: Subtitle (optional) The entry's subtitle element
title: Title (optional) the entry's title element
updated: Updated (optional) the entry's updated element
entry: list (optional) A list of the Entry instances contained in the
feed.
text: String (optional) The text contents of the element. This is the
contents of the Entry's XML text node.
(Example: <foo>This is the text</foo>)
extension_elements: list (optional) A list of ExtensionElement instances
which are children of this element.
extension_attributes: dict (optional) A dictionary of strings which are
the values for additional XML attributes of this element.
"""
self.author = author or []
self.category = category or []
self.contributor = contributor or []
self.generator = generator
self.icon = icon
self.id = atom_id
self.link = link or []
self.logo = logo
self.rights = rights
self.subtitle = subtitle
self.title = title
self.updated = updated
self.entry = entry or []
self.total_results = total_results
self.start_index = start_index
self.items_per_page = items_per_page
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def GDataFeedFromString(xml_string):
return atom.CreateClassFromXMLString(GDataFeed, xml_string)
class BatchId(atom.AtomBase):
_tag = 'id'
_namespace = BATCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
def BatchIdFromString(xml_string):
return atom.CreateClassFromXMLString(BatchId, xml_string)
class BatchOperation(atom.AtomBase):
_tag = 'operation'
_namespace = BATCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['type'] = 'type'
def __init__(self, op_type=None, extension_elements=None,
extension_attributes=None,
text=None):
self.type = op_type
atom.AtomBase.__init__(self,
extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def BatchOperationFromString(xml_string):
return atom.CreateClassFromXMLString(BatchOperation, xml_string)
class BatchStatus(atom.AtomBase):
"""The batch:status element present in a batch response entry.
A status element contains the code (HTTP response code) and
reason as elements. In a single request these fields would
be part of the HTTP response, but in a batch request each
Entry operation has a corresponding Entry in the response
feed which includes status information.
See http://code.google.com/apis/gdata/batch.html#Handling_Errors
"""
_tag = 'status'
_namespace = BATCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['code'] = 'code'
_attributes['reason'] = 'reason'
_attributes['content-type'] = 'content_type'
def __init__(self, code=None, reason=None, content_type=None,
extension_elements=None, extension_attributes=None, text=None):
self.code = code
self.reason = reason
self.content_type = content_type
atom.AtomBase.__init__(self, extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def BatchStatusFromString(xml_string):
return atom.CreateClassFromXMLString(BatchStatus, xml_string)
class BatchEntry(GDataEntry):
"""An atom:entry for use in batch requests.
The BatchEntry contains additional members to specify the operation to be
performed on this entry and a batch ID so that the server can reference
individual operations in the response feed. For more information, see:
http://code.google.com/apis/gdata/batch.html
"""
_tag = GDataEntry._tag
_namespace = GDataEntry._namespace
_children = GDataEntry._children.copy()
_children['{%s}operation' % BATCH_NAMESPACE] = ('batch_operation', BatchOperation)
_children['{%s}id' % BATCH_NAMESPACE] = ('batch_id', BatchId)
_children['{%s}status' % BATCH_NAMESPACE] = ('batch_status', BatchStatus)
_attributes = GDataEntry._attributes.copy()
def __init__(self, author=None, category=None, content=None,
contributor=None, atom_id=None, link=None, published=None, rights=None,
source=None, summary=None, control=None, title=None, updated=None,
batch_operation=None, batch_id=None, batch_status=None,
extension_elements=None, extension_attributes=None, text=None):
self.batch_operation = batch_operation
self.batch_id = batch_id
self.batch_status = batch_status
GDataEntry.__init__(self, author=author, category=category,
content=content, contributor=contributor, atom_id=atom_id, link=link,
published=published, rights=rights, source=source, summary=summary,
control=control, title=title, updated=updated,
extension_elements=extension_elements,
extension_attributes=extension_attributes, text=text)
def BatchEntryFromString(xml_string):
return atom.CreateClassFromXMLString(BatchEntry, xml_string)
class BatchInterrupted(atom.AtomBase):
"""The batch:interrupted element sent if batch request was interrupted.
Only appears in a feed if some of the batch entries could not be processed.
See: http://code.google.com/apis/gdata/batch.html#Handling_Errors
"""
_tag = 'interrupted'
_namespace = BATCH_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['reason'] = 'reason'
_attributes['success'] = 'success'
_attributes['failures'] = 'failures'
_attributes['parsed'] = 'parsed'
def __init__(self, reason=None, success=None, failures=None, parsed=None,
extension_elements=None, extension_attributes=None, text=None):
self.reason = reason
self.success = success
self.failures = failures
self.parsed = parsed
atom.AtomBase.__init__(self, extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def BatchInterruptedFromString(xml_string):
return atom.CreateClassFromXMLString(BatchInterrupted, xml_string)
class BatchFeed(GDataFeed):
"""A feed containing a list of batch request entries."""
_tag = GDataFeed._tag
_namespace = GDataFeed._namespace
_children = GDataFeed._children.copy()
_attributes = GDataFeed._attributes.copy()
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', [BatchEntry])
_children['{%s}interrupted' % BATCH_NAMESPACE] = ('interrupted', BatchInterrupted)
def __init__(self, author=None, category=None, contributor=None,
generator=None, icon=None, atom_id=None, link=None, logo=None,
rights=None, subtitle=None, title=None, updated=None, entry=None,
total_results=None, start_index=None, items_per_page=None,
interrupted=None,
extension_elements=None, extension_attributes=None, text=None):
self.interrupted = interrupted
GDataFeed.__init__(self, author=author, category=category,
contributor=contributor, generator=generator,
icon=icon, atom_id=atom_id, link=link,
logo=logo, rights=rights, subtitle=subtitle,
title=title, updated=updated, entry=entry,
total_results=total_results, start_index=start_index,
items_per_page=items_per_page,
extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def AddBatchEntry(self, entry=None, id_url_string=None,
batch_id_string=None, operation_string=None):
"""Logic for populating members of a BatchEntry and adding to the feed.
If the entry is not a BatchEntry, it is converted to a BatchEntry so
that the batch specific members will be present.
The id_url_string can be used in place of an entry if the batch operation
applies to a URL. For example query and delete operations require just
the URL of an entry, no body is sent in the HTTP request. If an
id_url_string is sent instead of an entry, a BatchEntry is created and
added to the feed.
This method also assigns the desired batch id to the entry so that it
can be referenced in the server's response. If the batch_id_string is
None, this method will assign a batch_id to be the index at which this
entry will be in the feed's entry list.
Args:
entry: BatchEntry, atom.Entry, or another Entry flavor (optional) The
entry which will be sent to the server as part of the batch request.
The item must have a valid atom id so that the server knows which
entry this request references.
id_url_string: str (optional) The URL of the entry to be acted on. You
can find this URL in the text member of the atom id for an entry.
If an entry is not sent, this id will be used to construct a new
BatchEntry which will be added to the request feed.
batch_id_string: str (optional) The batch ID to be used to reference
this batch operation in the results feed. If this parameter is None,
the current length of the feed's entry array will be used as a
count. Note that batch_ids should either always be specified or
never, mixing could potentially result in duplicate batch ids.
operation_string: str (optional) The desired batch operation which will
set the batch_operation.type member of the entry. Options are
'insert', 'update', 'delete', and 'query'
Raises:
MissingRequiredParameters: Raised if neither an id_ url_string nor an
entry are provided in the request.
Returns:
The added entry.
"""
if entry is None and id_url_string is None:
raise MissingRequiredParameters('supply either an entry or URL string')
if entry is None and id_url_string is not None:
entry = BatchEntry(atom_id=atom.Id(text=id_url_string))
# TODO: handle cases in which the entry lacks batch_... members.
#if not isinstance(entry, BatchEntry):
# Convert the entry to a batch entry.
if batch_id_string is not None:
entry.batch_id = BatchId(text=batch_id_string)
elif entry.batch_id is None or entry.batch_id.text is None:
entry.batch_id = BatchId(text=str(len(self.entry)))
if operation_string is not None:
entry.batch_operation = BatchOperation(op_type=operation_string)
self.entry.append(entry)
return entry
def AddInsert(self, entry, batch_id_string=None):
"""Add an insert request to the operations in this batch request feed.
If the entry doesn't yet have an operation or a batch id, these will
be set to the insert operation and a batch_id specified as a parameter.
Args:
entry: BatchEntry The entry which will be sent in the batch feed as an
insert request.
batch_id_string: str (optional) The batch ID to be used to reference
this batch operation in the results feed. If this parameter is None,
the current length of the feed's entry array will be used as a
count. Note that batch_ids should either always be specified or
never, mixing could potentially result in duplicate batch ids.
"""
entry = self.AddBatchEntry(entry=entry, batch_id_string=batch_id_string,
operation_string=BATCH_INSERT)
def AddUpdate(self, entry, batch_id_string=None):
"""Add an update request to the list of batch operations in this feed.
Sets the operation type of the entry to insert if it is not already set
and assigns the desired batch id to the entry so that it can be
referenced in the server's response.
Args:
entry: BatchEntry The entry which will be sent to the server as an
update (HTTP PUT) request. The item must have a valid atom id
so that the server knows which entry to replace.
batch_id_string: str (optional) The batch ID to be used to reference
this batch operation in the results feed. If this parameter is None,
the current length of the feed's entry array will be used as a
count. See also comments for AddInsert.
"""
entry = self.AddBatchEntry(entry=entry, batch_id_string=batch_id_string,
operation_string=BATCH_UPDATE)
def AddDelete(self, url_string=None, entry=None, batch_id_string=None):
"""Adds a delete request to the batch request feed.
This method takes either the url_string which is the atom id of the item
to be deleted, or the entry itself. The atom id of the entry must be
present so that the server knows which entry should be deleted.
Args:
url_string: str (optional) The URL of the entry to be deleted. You can
find this URL in the text member of the atom id for an entry.
entry: BatchEntry (optional) The entry to be deleted.
batch_id_string: str (optional)
Raises:
MissingRequiredParameters: Raised if neither a url_string nor an entry
are provided in the request.
"""
entry = self.AddBatchEntry(entry=entry, id_url_string=url_string,
batch_id_string=batch_id_string,
operation_string=BATCH_DELETE)
def AddQuery(self, url_string=None, entry=None, batch_id_string=None):
"""Adds a query request to the batch request feed.
This method takes either the url_string which is the query URL
whose results will be added to the result feed. The query URL will
be encapsulated in a BatchEntry, and you may pass in the BatchEntry
with a query URL instead of sending a url_string.
Args:
url_string: str (optional)
entry: BatchEntry (optional)
batch_id_string: str (optional)
Raises:
MissingRequiredParameters
"""
entry = self.AddBatchEntry(entry=entry, id_url_string=url_string,
batch_id_string=batch_id_string,
operation_string=BATCH_QUERY)
def GetBatchLink(self):
for link in self.link:
if link.rel == 'http://schemas.google.com/g/2005#batch':
return link
return None
def BatchFeedFromString(xml_string):
return atom.CreateClassFromXMLString(BatchFeed, xml_string)
class EntryLink(atom.AtomBase):
"""The gd:entryLink element"""
_tag = 'entryLink'
_namespace = GDATA_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
# The entry used to be an atom.Entry, now it is a GDataEntry.
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', GDataEntry)
_attributes['rel'] = 'rel'
_attributes['readOnly'] = 'read_only'
_attributes['href'] = 'href'
def __init__(self, href=None, read_only=None, rel=None,
entry=None, extension_elements=None,
extension_attributes=None, text=None):
self.href = href
self.read_only = read_only
self.rel = rel
self.entry = entry
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def EntryLinkFromString(xml_string):
return atom.CreateClassFromXMLString(EntryLink, xml_string)
class FeedLink(atom.AtomBase):
"""The gd:feedLink element"""
_tag = 'feedLink'
_namespace = GDATA_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_children['{%s}feed' % atom.ATOM_NAMESPACE] = ('feed', GDataFeed)
_attributes['rel'] = 'rel'
_attributes['readOnly'] = 'read_only'
_attributes['countHint'] = 'count_hint'
_attributes['href'] = 'href'
def __init__(self, count_hint=None, href=None, read_only=None, rel=None,
feed=None, extension_elements=None, extension_attributes=None,
text=None):
self.count_hint = count_hint
self.href = href
self.read_only = read_only
self.rel = rel
self.feed = feed
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def FeedLinkFromString(xml_string):
return atom.CreateClassFromXMLString(FeedLink, xml_string)

15
gdata/analytics/acl/__init__.py Normal file
View File

@ -0,0 +1,15 @@
#!/usr/bin/python
#
# Copyright (C) 2009 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

63
gdata/analytics/acl/data.py Normal file
View File

@ -0,0 +1,63 @@
#!/usr/bin/python
#
# Copyright (C) 2009 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Contains the data classes of the Google Access Control List (ACL) Extension"""
__author__ = 'j.s@google.com (Jeff Scudder)'
import atom.core
import atom.data
import gdata.data
import gdata.opensearch.data
GACL_TEMPLATE = '{http://schemas.google.com/acl/2007}%s'
class AclRole(atom.core.XmlElement):
"""Describes the role of an entry in an access control list."""
_qname = GACL_TEMPLATE % 'role'
value = 'value'
class AclScope(atom.core.XmlElement):
"""Describes the scope of an entry in an access control list."""
_qname = GACL_TEMPLATE % 'scope'
type = 'type'
value = 'value'
class AclWithKey(atom.core.XmlElement):
"""Describes a key that can be used to access a document."""
_qname = GACL_TEMPLATE % 'withKey'
key = 'key'
role = AclRole
class AclEntry(gdata.data.GDEntry):
"""Describes an entry in a feed of an access control list (ACL)."""
scope = AclScope
role = AclRole
with_key = AclWithKey
class AclFeed(gdata.data.GDFeed):
"""Describes a feed of an access control list (ACL)."""
entry = [AclEntry]

20
gdata/analytics/alt/__init__.py Normal file
View File

@ -0,0 +1,20 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""This package's modules adapt the gdata library to run in other environments
The first example is the appengine module which contains functions and
classes which modify a GDataService object to run on Google App Engine.
"""

101
gdata/analytics/alt/app_engine.py Normal file
View File

@ -0,0 +1,101 @@
#!/usr/bin/python
#
# Copyright (C) 2009 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Provides functions to persist serialized auth tokens in the datastore.
The get_token and set_token functions should be used in conjunction with
gdata.gauth's token_from_blob and token_to_blob to allow auth token objects
to be reused across requests. It is up to your own code to ensure that the
token key's are unique.
"""
__author__ = 'j.s@google.com (Jeff Scudder)'
from google.appengine.ext import db
from google.appengine.api import memcache
class Token(db.Model):
"""Datastore Model which stores a serialized auth token."""
t = db.BlobProperty()
def get_token(unique_key):
"""Searches for a stored token with the desired key.
Checks memcache and then the datastore if required.
Args:
unique_key: str which uniquely identifies the desired auth token.
Returns:
A string encoding the auth token data. Use gdata.gauth.token_from_blob to
convert back into a usable token object. None if the token was not found
in memcache or the datastore.
"""
token_string = memcache.get(unique_key)
if token_string is None:
# The token wasn't in memcache, so look in the datastore.
token = Token.get_by_key_name(unique_key)
if token is None:
return None
return token.t
return token_string
def set_token(unique_key, token_str):
"""Saves the serialized auth token in the datastore.
The token is also stored in memcache to speed up retrieval on a cache hit.
Args:
unique_key: The unique name for this token as a string. It is up to your
code to ensure that this token value is unique in your application.
Previous values will be silently overwitten.
token_str: A serialized auth token as a string. I expect that this string
will be generated by gdata.gauth.token_to_blob.
Returns:
True if the token was stored sucessfully, False if the token could not be
safely cached (if an old value could not be cleared). If the token was
set in memcache, but not in the datastore, this function will return None.
However, in that situation an exception will likely be raised.
Raises:
Datastore exceptions may be raised from the App Engine SDK in the event of
failure.
"""
# First try to save in memcache.
result = memcache.set(unique_key, token_str)
# If memcache fails to save the value, clear the cached value.
if not result:
result = memcache.delete(unique_key)
# If we could not clear the cached value for this token, refuse to save.
if result == 0:
return False
# Save to the datastore.
if Token(key_name=unique_key, t=token_str).put():
return True
return None
def delete_token(unique_key):
# Clear from memcache.
memcache.delete(unique_key)
# Clear from the datastore.
Token(key_name=unique_key).delete()

321
gdata/analytics/alt/appengine.py Normal file
View File

@ -0,0 +1,321 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Provides HTTP functions for gdata.service to use on Google App Engine
AppEngineHttpClient: Provides an HTTP request method which uses App Engine's
urlfetch API. Set the http_client member of a GDataService object to an
instance of an AppEngineHttpClient to allow the gdata library to run on
Google App Engine.
run_on_appengine: Function which will modify an existing GDataService object
to allow it to run on App Engine. It works by creating a new instance of
the AppEngineHttpClient and replacing the GDataService object's
http_client.
"""
__author__ = 'api.jscudder (Jeff Scudder)'
import StringIO
import pickle
import atom.http_interface
import atom.token_store
from google.appengine.api import urlfetch
from google.appengine.ext import db
from google.appengine.api import users
from google.appengine.api import memcache
def run_on_appengine(gdata_service, store_tokens=True,
single_user_mode=False, deadline=None):
"""Modifies a GDataService object to allow it to run on App Engine.
Args:
gdata_service: An instance of AtomService, GDataService, or any
of their subclasses which has an http_client member and a
token_store member.
store_tokens: Boolean, defaults to True. If True, the gdata_service
will attempt to add each token to it's token_store when
SetClientLoginToken or SetAuthSubToken is called. If False
the tokens will not automatically be added to the
token_store.
single_user_mode: Boolean, defaults to False. If True, the current_token
member of gdata_service will be set when
SetClientLoginToken or SetAuthTubToken is called. If set
to True, the current_token is set in the gdata_service
and anyone who accesses the object will use the same
token.
Note: If store_tokens is set to False and
single_user_mode is set to False, all tokens will be
ignored, since the library assumes: the tokens should not
be stored in the datastore and they should not be stored
in the gdata_service object. This will make it
impossible to make requests which require authorization.
deadline: int (optional) The number of seconds to wait for a response
before timing out on the HTTP request. If no deadline is
specified, the deafault deadline for HTTP requests from App
Engine is used. The maximum is currently 10 (for 10 seconds).
The default deadline for App Engine is 5 seconds.
"""
gdata_service.http_client = AppEngineHttpClient(deadline=deadline)
gdata_service.token_store = AppEngineTokenStore()
gdata_service.auto_store_tokens = store_tokens
gdata_service.auto_set_current_token = single_user_mode
return gdata_service
class AppEngineHttpClient(atom.http_interface.GenericHttpClient):
def __init__(self, headers=None, deadline=None):
self.debug = False
self.headers = headers or {}
self.deadline = deadline
def request(self, operation, url, data=None, headers=None):
"""Performs an HTTP call to the server, supports GET, POST, PUT, and
DELETE.
Usage example, perform and HTTP GET on http://www.google.com/:
import atom.http
client = atom.http.HttpClient()
http_response = client.request('GET', 'http://www.google.com/')
Args:
operation: str The HTTP operation to be performed. This is usually one
of 'GET', 'POST', 'PUT', or 'DELETE'
data: filestream, list of parts, or other object which can be converted
to a string. Should be set to None when performing a GET or DELETE.
If data is a file-like object which can be read, this method will
read a chunk of 100K bytes at a time and send them.
If the data is a list of parts to be sent, each part will be
evaluated and sent.
url: The full URL to which the request should be sent. Can be a string
or atom.url.Url.
headers: dict of strings. HTTP headers which should be sent
in the request.
"""
all_headers = self.headers.copy()
if headers:
all_headers.update(headers)
# Construct the full payload.
# Assume that data is None or a string.
data_str = data
if data:
if isinstance(data, list):
# If data is a list of different objects, convert them all to strings
# and join them together.
converted_parts = [_convert_data_part(x) for x in data]
data_str = ''.join(converted_parts)
else:
data_str = _convert_data_part(data)
# If the list of headers does not include a Content-Length, attempt to
# calculate it based on the data object.
if data and 'Content-Length' not in all_headers:
all_headers['Content-Length'] = str(len(data_str))
# Set the content type to the default value if none was set.
if 'Content-Type' not in all_headers:
all_headers['Content-Type'] = 'application/atom+xml'
# Lookup the urlfetch operation which corresponds to the desired HTTP verb.
if operation == 'GET':
method = urlfetch.GET
elif operation == 'POST':
method = urlfetch.POST
elif operation == 'PUT':
method = urlfetch.PUT
elif operation == 'DELETE':
method = urlfetch.DELETE
else:
method = None
if self.deadline is None:
return HttpResponse(urlfetch.Fetch(url=str(url), payload=data_str,
method=method, headers=all_headers, follow_redirects=False))
return HttpResponse(urlfetch.Fetch(url=str(url), payload=data_str,
method=method, headers=all_headers, follow_redirects=False,
deadline=self.deadline))
def _convert_data_part(data):
if not data or isinstance(data, str):
return data
elif hasattr(data, 'read'):
# data is a file like object, so read it completely.
return data.read()
# The data object was not a file.
# Try to convert to a string and send the data.
return str(data)
class HttpResponse(object):
"""Translates a urlfetch resoinse to look like an hhtplib resoinse.
Used to allow the resoinse from HttpRequest to be usable by gdata.service
methods.
"""
def __init__(self, urlfetch_response):
self.body = StringIO.StringIO(urlfetch_response.content)
self.headers = urlfetch_response.headers
self.status = urlfetch_response.status_code
self.reason = ''
def read(self, length=None):
if not length:
return self.body.read()
else:
return self.body.read(length)
def getheader(self, name):
if not self.headers.has_key(name):
return self.headers[name.lower()]
return self.headers[name]
class TokenCollection(db.Model):
"""Datastore Model which associates auth tokens with the current user."""
user = db.UserProperty()
pickled_tokens = db.BlobProperty()
class AppEngineTokenStore(atom.token_store.TokenStore):
"""Stores the user's auth tokens in the App Engine datastore.
Tokens are only written to the datastore if a user is signed in (if
users.get_current_user() returns a user object).
"""
def __init__(self):
self.user = None
def add_token(self, token):
"""Associates the token with the current user and stores it.
If there is no current user, the token will not be stored.
Returns:
False if the token was not stored.
"""
tokens = load_auth_tokens(self.user)
if not hasattr(token, 'scopes') or not token.scopes:
return False
for scope in token.scopes:
tokens[str(scope)] = token
key = save_auth_tokens(tokens, self.user)
if key:
return True
return False
def find_token(self, url):
"""Searches the current user's collection of token for a token which can
be used for a request to the url.
Returns:
The stored token which belongs to the current user and is valid for the
desired URL. If there is no current user, or there is no valid user
token in the datastore, a atom.http_interface.GenericToken is returned.
"""
if url is None:
return None
if isinstance(url, (str, unicode)):
url = atom.url.parse_url(url)
tokens = load_auth_tokens(self.user)
if url in tokens:
token = tokens[url]
if token.valid_for_scope(url):
return token
else:
del tokens[url]
save_auth_tokens(tokens, self.user)
for scope, token in tokens.iteritems():
if token.valid_for_scope(url):
return token
return atom.http_interface.GenericToken()
def remove_token(self, token):
"""Removes the token from the current user's collection in the datastore.
Returns:
False if the token was not removed, this could be because the token was
not in the datastore, or because there is no current user.
"""
token_found = False
scopes_to_delete = []
tokens = load_auth_tokens(self.user)
for scope, stored_token in tokens.iteritems():
if stored_token == token:
scopes_to_delete.append(scope)
token_found = True
for scope in scopes_to_delete:
del tokens[scope]
if token_found:
save_auth_tokens(tokens, self.user)
return token_found
def remove_all_tokens(self):
"""Removes all of the current user's tokens from the datastore."""
save_auth_tokens({}, self.user)
def save_auth_tokens(token_dict, user=None):
"""Associates the tokens with the current user and writes to the datastore.
If there us no current user, the tokens are not written and this function
returns None.
Returns:
The key of the datastore entity containing the user's tokens, or None if
there was no current user.
"""
if user is None:
user = users.get_current_user()
if user is None:
return None
memcache.set('gdata_pickled_tokens:%s' % user, pickle.dumps(token_dict))
user_tokens = TokenCollection.all().filter('user =', user).get()
if user_tokens:
user_tokens.pickled_tokens = pickle.dumps(token_dict)
return user_tokens.put()
else:
user_tokens = TokenCollection(
user=user,
pickled_tokens=pickle.dumps(token_dict))
return user_tokens.put()
def load_auth_tokens(user=None):
"""Reads a dictionary of the current user's tokens from the datastore.
If there is no current user (a user is not signed in to the app) or the user
does not have any tokens, an empty dictionary is returned.
"""
if user is None:
user = users.get_current_user()
if user is None:
return {}
pickled_tokens = memcache.get('gdata_pickled_tokens:%s' % user)
if pickled_tokens:
return pickle.loads(pickled_tokens)
user_tokens = TokenCollection.all().filter('user =', user).get()
if user_tokens:
memcache.set('gdata_pickled_tokens:%s' % user, user_tokens.pickled_tokens)
return pickle.loads(user_tokens.pickled_tokens)
return {}

223
gdata/analytics/analytics/__init__.py Normal file
View File

@ -0,0 +1,223 @@
#!/usr/bin/python
#
# Original Copyright (C) 2006 Google Inc.
# Refactored in 2009 to work for Google Analytics by Sal Uryasev at Juice Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Note that this module will not function without specifically adding
# 'analytics': [ #Google Analytics
# 'https://www.google.com/analytics/feeds/'],
# to CLIENT_LOGIN_SCOPES in the gdata/service.py file
"""Contains extensions to Atom objects used with Google Analytics."""
__author__ = 'api.suryasev (Sal Uryasev)'
import atom
import gdata
GAN_NAMESPACE = 'http://schemas.google.com/analytics/2009'
class TableId(gdata.GDataEntry):
"""tableId element."""
_tag = 'tableId'
_namespace = GAN_NAMESPACE
class Property(gdata.GDataEntry):
_tag = 'property'
_namespace = GAN_NAMESPACE
_children = gdata.GDataEntry._children.copy()
_attributes = gdata.GDataEntry._attributes.copy()
_attributes['name'] = 'name'
_attributes['value'] = 'value'
def __init__(self, name=None, value=None, *args, **kwargs):
self.name = name
self.value = value
super(Property, self).__init__(*args, **kwargs)
def __str__(self):
return self.value
def __repr__(self):
return self.value
class AccountListEntry(gdata.GDataEntry):
"""The Google Documents version of an Atom Entry"""
_tag = 'entry'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataEntry._children.copy()
_attributes = gdata.GDataEntry._attributes.copy()
_children['{%s}tableId' % GAN_NAMESPACE] = ('tableId',
[TableId])
_children['{%s}property' % GAN_NAMESPACE] = ('property',
[Property])
def __init__(self, tableId=None, property=None,
*args, **kwargs):
self.tableId = tableId
self.property = property
super(AccountListEntry, self).__init__(*args, **kwargs)
def AccountListEntryFromString(xml_string):
"""Converts an XML string into an AccountListEntry object.
Args:
xml_string: string The XML describing a Document List feed entry.
Returns:
A AccountListEntry object corresponding to the given XML.
"""
return atom.CreateClassFromXMLString(AccountListEntry, xml_string)
class AccountListFeed(gdata.GDataFeed):
"""A feed containing a list of Google Documents Items"""
_tag = 'feed'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataFeed._children.copy()
_attributes = gdata.GDataFeed._attributes.copy()
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry',
[AccountListEntry])
def AccountListFeedFromString(xml_string):
"""Converts an XML string into an AccountListFeed object.
Args:
xml_string: string The XML describing an AccountList feed.
Returns:
An AccountListFeed object corresponding to the given XML.
All properties are also linked to with a direct reference
from each entry object for convenience. (e.g. entry.AccountName)
"""
feed = atom.CreateClassFromXMLString(AccountListFeed, xml_string)
for entry in feed.entry:
for pro in entry.property:
entry.__dict__[pro.name.replace('ga:','')] = pro
for td in entry.tableId:
td.__dict__['value'] = td.text
return feed
class Dimension(gdata.GDataEntry):
_tag = 'dimension'
_namespace = GAN_NAMESPACE
_children = gdata.GDataEntry._children.copy()
_attributes = gdata.GDataEntry._attributes.copy()
_attributes['name'] = 'name'
_attributes['value'] = 'value'
_attributes['type'] = 'type'
_attributes['confidenceInterval'] = 'confidence_interval'
def __init__(self, name=None, value=None, type=None,
confidence_interval = None, *args, **kwargs):
self.name = name
self.value = value
self.type = type
self.confidence_interval = confidence_interval
super(Dimension, self).__init__(*args, **kwargs)
def __str__(self):
return self.value
def __repr__(self):
return self.value
class Metric(gdata.GDataEntry):
_tag = 'metric'
_namespace = GAN_NAMESPACE
_children = gdata.GDataEntry._children.copy()
_attributes = gdata.GDataEntry._attributes.copy()
_attributes['name'] = 'name'
_attributes['value'] = 'value'
_attributes['type'] = 'type'
_attributes['confidenceInterval'] = 'confidence_interval'
def __init__(self, name=None, value=None, type=None,
confidence_interval = None, *args, **kwargs):
self.name = name
self.value = value
self.type = type
self.confidence_interval = confidence_interval
super(Metric, self).__init__(*args, **kwargs)
def __str__(self):
return self.value
def __repr__(self):
return self.value
class AnalyticsDataEntry(gdata.GDataEntry):
"""The Google Analytics version of an Atom Entry"""
_tag = 'entry'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataEntry._children.copy()
_attributes = gdata.GDataEntry._attributes.copy()
_children['{%s}dimension' % GAN_NAMESPACE] = ('dimension',
[Dimension])
_children['{%s}metric' % GAN_NAMESPACE] = ('metric',
[Metric])
def __init__(self, dimension=None, metric=None, *args, **kwargs):
self.dimension = dimension
self.metric = metric
super(AnalyticsDataEntry, self).__init__(*args, **kwargs)
class AnalyticsDataFeed(gdata.GDataFeed):
"""A feed containing a list of Google Analytics Data Feed"""
_tag = 'feed'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataFeed._children.copy()
_attributes = gdata.GDataFeed._attributes.copy()
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry',
[AnalyticsDataEntry])
"""
Data Feed
"""
def AnalyticsDataFeedFromString(xml_string):
"""Converts an XML string into an AccountListFeed object.
Args:
xml_string: string The XML describing an AccountList feed.
Returns:
An AccountListFeed object corresponding to the given XML.
Each metric and dimension is also referenced directly from
the entry for easier access. (e.g. entry.keyword.value)
"""
feed = atom.CreateClassFromXMLString(AnalyticsDataFeed, xml_string)
if feed.entry:
for entry in feed.entry:
for met in entry.metric:
entry.__dict__[met.name.replace('ga:','')] = met
if entry.dimension is not None:
for dim in entry.dimension:
entry.__dict__[dim.name.replace('ga:','')] = dim
return feed

313
gdata/analytics/analytics/client.py Normal file
View File

@ -0,0 +1,313 @@
#!/usr/bin/python
#
# Copyright 2010 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Streamlines requests to the Google Analytics APIs."""
__author__ = 'api.nickm@google.com (Nick Mihailovski)'
import atom.data
import gdata.client
import gdata.analytics.data
import gdata.gauth
class AnalyticsClient(gdata.client.GDClient):
"""Client extension for the Google Analytics API service."""
api_version = '2'
auth_service = 'analytics'
auth_scopes = gdata.gauth.AUTH_SCOPES['analytics']
account_type = 'GOOGLE'
ssl = True
def __init__(self, auth_token=None, **kwargs):
"""Initializes a new client for the Google Analytics Data Export API.
Args:
auth_token: gdata.gauth.ClientLoginToken, AuthSubToken, or
OAuthToken (optional) Authorizes this client to edit the user's data.
kwargs: The other parameters to pass to gdata.client.GDClient
constructor.
"""
gdata.client.GDClient.__init__(self, auth_token=auth_token, **kwargs)
def get_account_feed(self, feed_uri, auth_token=None, **kwargs):
"""Makes a request to the Analytics API Account Feed.
Args:
feed_uri: str or gdata.analytics.AccountFeedQuery The Analytics Account
Feed uri to define what data to retrieve from the API. Can also be
used with a gdata.analytics.AccountFeedQuery object.
"""
return self.get_feed(feed_uri,
desired_class=gdata.analytics.data.AccountFeed,
auth_token=auth_token,
**kwargs)
GetAccountFeed = get_account_feed
def get_data_feed(self, feed_uri, auth_token=None, **kwargs):
"""Makes a request to the Analytics API Data Feed.
Args:
feed_uri: str or gdata.analytics.AccountFeedQuery The Analytics Data
Feed uri to define what data to retrieve from the API. Can also be
used with a gdata.analytics.AccountFeedQuery object.
"""
return self.get_feed(feed_uri,
desired_class=gdata.analytics.data.DataFeed,
auth_token=auth_token,
**kwargs)
GetDataFeed = get_data_feed
def get_management_feed(self, feed_uri, auth_token=None, **kwargs):
"""Makes a request to the Google Analytics Management API.
The Management API provides read-only access to configuration data for
Google Analytics and supercedes the Data Export API Account Feed.
The Management API supports 5 feeds: account, web property, profile,
goal, advanced segment.
You can access each feed through the respective management query class
below. All requests return the same data object.
Args:
feed_uri: str or AccountQuery, WebPropertyQuery,
ProfileQuery, GoalQuery, MgmtAdvSegFeedQuery
The Management API Feed uri to define which feed to retrieve.
Either use a string or one of the wrapper classes.
"""
return self.get_feed(feed_uri,
desired_class=gdata.analytics.data.ManagementFeed,
auth_token=auth_token,
**kwargs)
GetMgmtFeed = GetManagementFeed = get_management_feed
class AnalyticsBaseQuery(gdata.client.GDQuery):
"""Abstracts common configuration across all query objects.
Attributes:
scheme: string The default scheme. Should always be https.
host: string The default host.
"""
scheme = 'https'
host = 'www.google.com'
class AccountFeedQuery(AnalyticsBaseQuery):
"""Account Feed query class to simplify constructing Account Feed Urls.
To use this class, you can either pass a dict in the constructor that has
all the data feed query parameters as keys:
queryUrl = AccountFeedQuery({'max-results': '10000'})
Alternatively you can add new parameters directly to the query object:
queryUrl = AccountFeedQuery()
queryUrl.query['max-results'] = '10000'
Args:
query: dict (optional) Contains all the GA Data Feed query parameters
as keys.
"""
path = '/analytics/feeds/accounts/default'
def __init__(self, query={}, **kwargs):
self.query = query
gdata.client.GDQuery(self, **kwargs)
class DataFeedQuery(AnalyticsBaseQuery):
"""Data Feed query class to simplify constructing Data Feed Urls.
To use this class, you can either pass a dict in the constructor that has
all the data feed query parameters as keys:
queryUrl = DataFeedQuery({'start-date': '2008-10-01'})
Alternatively you can add new parameters directly to the query object:
queryUrl = DataFeedQuery()
queryUrl.query['start-date'] = '2008-10-01'
Args:
query: dict (optional) Contains all the GA Data Feed query parameters
as keys.
"""
path = '/analytics/feeds/data'
def __init__(self, query={}, **kwargs):
self.query = query
gdata.client.GDQuery(self, **kwargs)
class AccountQuery(AnalyticsBaseQuery):
"""Management API Account Feed query class.
Example Usage:
queryUrl = AccountQuery()
queryUrl = AccountQuery({'max-results': 100})
queryUrl2 = AccountQuery()
queryUrl2.query['max-results'] = 100
Args:
query: dict (optional) A dictionary of query parameters.
"""
path = '/analytics/feeds/datasources/ga/accounts'
def __init__(self, query={}, **kwargs):
self.query = query
gdata.client.GDQuery(self, **kwargs)
class WebPropertyQuery(AnalyticsBaseQuery):
"""Management API Web Property Feed query class.
Example Usage:
queryUrl = WebPropertyQuery()
queryUrl = WebPropertyQuery('123', {'max-results': 100})
queryUrl = WebPropertyQuery(acct_id='123',
query={'max-results': 100})
queryUrl2 = WebPropertyQuery()
queryUrl2.acct_id = '1234'
queryUrl2.query['max-results'] = 100
Args:
acct_id: string (optional) The account ID to filter results.
Default is ~all.
query: dict (optional) A dictionary of query parameters.
"""
def __init__(self, acct_id='~all', query={}, **kwargs):
self.acct_id = acct_id
self.query = query
gdata.client.GDQuery(self, **kwargs)
@property
def path(self):
"""Wrapper for path attribute."""
return ('/analytics/feeds/datasources/ga/accounts/%s/webproperties' %
self.acct_id)
class ProfileQuery(AnalyticsBaseQuery):
"""Management API Profile Feed query class.
Example Usage:
queryUrl = ProfileQuery()
queryUrl = ProfileQuery('123', 'UA-123-1', {'max-results': 100})
queryUrl = ProfileQuery(acct_id='123',
web_prop_id='UA-123-1',
query={'max-results': 100})
queryUrl2 = ProfileQuery()
queryUrl2.acct_id = '123'
queryUrl2.web_prop_id = 'UA-123-1'
queryUrl2.query['max-results'] = 100
Args:
acct_id: string (optional) The account ID to filter results.
Default is ~all.
web_prop_id: string (optional) The web property ID to filter results.
Default is ~all.
query: dict (optional) A dictionary of query parameters.
"""
def __init__(self, acct_id='~all', web_prop_id='~all', query={}, **kwargs):
self.acct_id = acct_id
self.web_prop_id = web_prop_id
self.query = query
gdata.client.GDQuery(self, **kwargs)
@property
def path(self):
"""Wrapper for path attribute."""
return ('/analytics/feeds/datasources/ga/accounts/%s/webproperties'
'/%s/profiles' % (self.acct_id, self.web_prop_id))
class GoalQuery(AnalyticsBaseQuery):
"""Management API Goal Feed query class.
Example Usage:
queryUrl = GoalQuery()
queryUrl = GoalQuery('123', 'UA-123-1', '555',
{'max-results': 100})
queryUrl = GoalQuery(acct_id='123',
web_prop_id='UA-123-1',
profile_id='555',
query={'max-results': 100})
queryUrl2 = GoalQuery()
queryUrl2.acct_id = '123'
queryUrl2.web_prop_id = 'UA-123-1'
queryUrl2.query['max-results'] = 100
Args:
acct_id: string (optional) The account ID to filter results.
Default is ~all.
web_prop_id: string (optional) The web property ID to filter results.
Default is ~all.
profile_id: string (optional) The profile ID to filter results.
Default is ~all.
query: dict (optional) A dictionary of query parameters.
"""
def __init__(self, acct_id='~all', web_prop_id='~all', profile_id='~all',
query={}, **kwargs):
self.acct_id = acct_id
self.web_prop_id = web_prop_id
self.profile_id = profile_id
self.query = query or {}
gdata.client.GDQuery(self, **kwargs)
@property
def path(self):
"""Wrapper for path attribute."""
return ('/analytics/feeds/datasources/ga/accounts/%s/webproperties'
'/%s/profiles/%s/goals' % (self.acct_id, self.web_prop_id,
self.profile_id))
class AdvSegQuery(AnalyticsBaseQuery):
"""Management API Goal Feed query class.
Example Usage:
queryUrl = AdvSegQuery()
queryUrl = AdvSegQuery({'max-results': 100})
queryUrl1 = AdvSegQuery()
queryUrl1.query['max-results'] = 100
Args:
query: dict (optional) A dictionary of query parameters.
"""
path = '/analytics/feeds/datasources/ga/segments'
def __init__(self, query={}, **kwargs):
self.query = query
gdata.client.GDQuery(self, **kwargs)

379
gdata/analytics/analytics/data.py Normal file
View File

@ -0,0 +1,379 @@
#!/usr/bin/python
#
# Copyright 2010 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Data model classes for parsing and generating XML for both the
Google Analytics Data Export and Management APIs. Although both APIs
operate on different parts of Google Analytics, they share common XML
elements and are released in the same module.
The Management API supports 5 feeds all using the same ManagementFeed
data class.
"""
__author__ = 'api.nickm@google.com (Nick Mihailovski)'
import gdata.data
import atom.core
import atom.data
# XML Namespace used in Google Analytics API entities.
DXP_NS = '{http://schemas.google.com/analytics/2009}%s'
GA_NS = '{http://schemas.google.com/ga/2009}%s'
GD_NS = '{http://schemas.google.com/g/2005}%s'
class GetProperty(object):
"""Utility class to simplify retrieving Property objects."""
def get_property(self, name):
"""Helper method to return a propery object by its name attribute.
Args:
name: string The name of the <dxp:property> element to retrieve.
Returns:
A property object corresponding to the matching <dxp:property> element.
if no property is found, None is returned.
"""
for prop in self.property:
if prop.name == name:
return prop
return None
GetProperty = get_property
class GetMetric(object):
"""Utility class to simplify retrieving Metric objects."""
def get_metric(self, name):
"""Helper method to return a propery value by its name attribute
Args:
name: string The name of the <dxp:metric> element to retrieve.
Returns:
A property object corresponding to the matching <dxp:metric> element.
if no property is found, None is returned.
"""
for met in self.metric:
if met.name == name:
return met
return None
GetMetric = get_metric
class GetDimension(object):
"""Utility class to simplify retrieving Dimension objects."""
def get_dimension(self, name):
"""Helper method to return a dimention object by its name attribute
Args:
name: string The name of the <dxp:dimension> element to retrieve.
Returns:
A dimension object corresponding to the matching <dxp:dimension> element.
if no dimension is found, None is returned.
"""
for dim in self.dimension:
if dim.name == name:
return dim
return None
GetDimension = get_dimension
class GaLinkFinder(object):
"""Utility class to return specific links in Google Analytics feeds."""
def get_parent_links(self):
"""Returns a list of all the parent links in an entry."""
links = []
for link in self.link:
if link.rel == link.parent():
links.append(link)
return links
GetParentLinks = get_parent_links
def get_child_links(self):
"""Returns a list of all the child links in an entry."""
links = []
for link in self.link:
if link.rel == link.child():
links.append(link)
return links
GetChildLinks = get_child_links
def get_child_link(self, target_kind):
"""Utility method to return one child link.
Returns:
A child link with the given target_kind. None if the target_kind was
not found.
"""
for link in self.link:
if link.rel == link.child() and link.target_kind == target_kind:
return link
return None
GetChildLink = get_child_link
class StartDate(atom.core.XmlElement):
"""Analytics Feed <dxp:startDate>"""
_qname = DXP_NS % 'startDate'
class EndDate(atom.core.XmlElement):
"""Analytics Feed <dxp:endDate>"""
_qname = DXP_NS % 'endDate'
class Metric(atom.core.XmlElement):
"""Analytics Feed <dxp:metric>"""
_qname = DXP_NS % 'metric'
name = 'name'
type = 'type'
value = 'value'
confidence_interval = 'confidenceInterval'
class Aggregates(atom.core.XmlElement, GetMetric):
"""Analytics Data Feed <dxp:aggregates>"""
_qname = DXP_NS % 'aggregates'
metric = [Metric]
class ContainsSampledData(atom.core.XmlElement):
"""Analytics Data Feed <dxp:containsSampledData>"""
_qname = DXP_NS % 'containsSampledData'
class TableId(atom.core.XmlElement):
"""Analytics Feed <dxp:tableId>"""
_qname = DXP_NS % 'tableId'
class TableName(atom.core.XmlElement):
"""Analytics Feed <dxp:tableName>"""
_qname = DXP_NS % 'tableName'
class Property(atom.core.XmlElement):
"""Analytics Feed <dxp:property>"""
_qname = DXP_NS % 'property'
name = 'name'
value = 'value'
class Definition(atom.core.XmlElement):
"""Analytics Feed <dxp:definition>"""
_qname = DXP_NS % 'definition'
class Segment(atom.core.XmlElement):
"""Analytics Feed <dxp:segment>"""
_qname = DXP_NS % 'segment'
id = 'id'
name = 'name'
definition = Definition
class Engagement(atom.core.XmlElement):
"""Analytics Feed <dxp:engagement>"""
_qname = GA_NS % 'engagement'
type = 'type'
comparison = 'comparison'
threshold_value = 'thresholdValue'
class Step(atom.core.XmlElement):
"""Analytics Feed <dxp:step>"""
_qname = GA_NS % 'step'
number = 'number'
name = 'name'
path = 'path'
class Destination(atom.core.XmlElement):
"""Analytics Feed <dxp:destination>"""
_qname = GA_NS % 'destination'
step = [Step]
expression = 'expression'
case_sensitive = 'caseSensitive'
match_type = 'matchType'
step1_required = 'step1Required'
class Goal(atom.core.XmlElement):
"""Analytics Feed <dxp:goal>"""
_qname = GA_NS % 'goal'
destination = Destination
engagement = Engagement
number = 'number'
name = 'name'
value = 'value'
active = 'active'
class CustomVariable(atom.core.XmlElement):
"""Analytics Data Feed <dxp:customVariable>"""
_qname = GA_NS % 'customVariable'
index = 'index'
name = 'name'
scope = 'scope'
class DataSource(atom.core.XmlElement, GetProperty):
"""Analytics Data Feed <dxp:dataSource>"""
_qname = DXP_NS % 'dataSource'
table_id = TableId
table_name = TableName
property = [Property]
class Dimension(atom.core.XmlElement):
"""Analytics Feed <dxp:dimension>"""
_qname = DXP_NS % 'dimension'
name = 'name'
value = 'value'
class AnalyticsLink(atom.data.Link):
"""Subclass of link <link>"""
target_kind = GD_NS % 'targetKind'
@classmethod
def parent(cls):
"""Parent target_kind"""
return '%s#parent' % GA_NS[1:-3]
@classmethod
def child(cls):
"""Child target_kind"""
return '%s#child' % GA_NS[1:-3]
# Account Feed.
class AccountEntry(gdata.data.GDEntry, GetProperty):
"""Analytics Account Feed <entry>"""
_qname = atom.data.ATOM_TEMPLATE % 'entry'
table_id = TableId
property = [Property]
goal = [Goal]
custom_variable = [CustomVariable]
class AccountFeed(gdata.data.GDFeed):
"""Analytics Account Feed <feed>"""
_qname = atom.data.ATOM_TEMPLATE % 'feed'
segment = [Segment]
entry = [AccountEntry]
# Data Feed.
class DataEntry(gdata.data.GDEntry, GetMetric, GetDimension):
"""Analytics Data Feed <entry>"""
_qname = atom.data.ATOM_TEMPLATE % 'entry'
dimension = [Dimension]
metric = [Metric]
def get_object(self, name):
"""Returns either a Dimension or Metric object with the same name as the
name parameter.
Args:
name: string The name of the object to retrieve.
Returns:
Either a Dimension or Object that has the same as the name parameter.
"""
output = self.GetDimension(name)
if not output:
output = self.GetMetric(name)
return output
GetObject = get_object
class DataFeed(gdata.data.GDFeed):
"""Analytics Data Feed <feed>.
Although there is only one datasource, it is stored in an array to replicate
the design of the Java client library and ensure backwards compatibility if
new data sources are added in the future.
"""
_qname = atom.data.ATOM_TEMPLATE % 'feed'
start_date = StartDate
end_date = EndDate
aggregates = Aggregates
contains_sampled_data = ContainsSampledData
data_source = [DataSource]
entry = [DataEntry]
segment = Segment
def has_sampled_data(self):
"""Returns whether this feed has sampled data."""
if (self.contains_sampled_data.text == 'true'):
return True
return False
HasSampledData = has_sampled_data
# Management Feed.
class ManagementEntry(gdata.data.GDEntry, GetProperty, GaLinkFinder):
"""Analytics Managememt Entry <entry>."""
_qname = atom.data.ATOM_TEMPLATE % 'entry'
kind = GD_NS % 'kind'
property = [Property]
goal = Goal
segment = Segment
link = [AnalyticsLink]
class ManagementFeed(gdata.data.GDFeed):
"""Analytics Management Feed <feed>.
This class holds the data for all 5 Management API feeds: Account,
Web Property, Profile, Goal, and Advanced Segment Feeds.
"""
_qname = atom.data.ATOM_TEMPLATE % 'feed'
entry = [ManagementEntry]
kind = GD_NS % 'kind'

331
gdata/analytics/analytics/service.py Normal file
View File

@ -0,0 +1,331 @@
#!/usr/bin/python
#
# Copyright (C) 2006 Google Inc.
# Refactored in 2009 to work for Google Analytics by Sal Uryasev at Juice Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
AccountsService extends the GDataService to streamline Google Analytics
account information operations.
AnalyticsDataService: Provides methods to query google analytics data feeds.
Extends GDataService.
DataQuery: Queries a Google Analytics Data list feed.
AccountQuery: Queries a Google Analytics Account list feed.
"""
__author__ = 'api.suryasev (Sal Uryasev)'
import urllib
import atom
import gdata.service
import gdata.analytics
class AccountsService(gdata.service.GDataService):
"""Client extension for the Google Analytics Account List feed."""
def __init__(self, email="", password=None, source=None,
server='www.google.com/analytics', additional_headers=None,
**kwargs):
"""Creates a client for the Google Analytics service.
Args:
email: string (optional) The user's email address, used for
authentication.
password: string (optional) The user's password.
source: string (optional) The name of the user's application.
server: string (optional) The name of the server to which a connection
will be opened.
**kwargs: The other parameters to pass to gdata.service.GDataService
constructor.
"""
gdata.service.GDataService.__init__(
self, email=email, password=password, service='analytics',
source=source, server=server, additional_headers=additional_headers,
**kwargs)
def QueryAccountListFeed(self, uri):
"""Retrieves an AccountListFeed by retrieving a URI based off the Document
List feed, including any query parameters. An AccountListFeed object
can be used to construct these parameters.
Args:
uri: string The URI of the feed being retrieved possibly with query
parameters.
Returns:
An AccountListFeed object representing the feed returned by the server.
"""
return self.Get(uri, converter=gdata.analytics.AccountListFeedFromString)
def GetAccountListEntry(self, uri):
"""Retrieves a particular AccountListEntry by its unique URI.
Args:
uri: string The unique URI of an entry in an Account List feed.
Returns:
An AccountLisFeed object representing the retrieved entry.
"""
return self.Get(uri, converter=gdata.analytics.AccountListEntryFromString)
def GetAccountList(self, max_results=1000, text_query=None,
params=None, categories=None):
"""Retrieves a feed containing all of a user's accounts and profiles."""
q = gdata.analytics.service.AccountQuery(max_results=max_results,
text_query=text_query,
params=params,
categories=categories);
return self.QueryAccountListFeed(q.ToUri())
class AnalyticsDataService(gdata.service.GDataService):
"""Client extension for the Google Analytics service Data List feed."""
def __init__(self, email=None, password=None, source=None,
server='www.google.com/analytics', additional_headers=None,
**kwargs):
"""Creates a client for the Google Analytics service.
Args:
email: string (optional) The user's email address, used for
authentication.
password: string (optional) The user's password.
source: string (optional) The name of the user's application.
server: string (optional) The name of the server to which a connection
will be opened. Default value: 'docs.google.com'.
**kwargs: The other parameters to pass to gdata.service.GDataService
constructor.
"""
gdata.service.GDataService.__init__(self,
email=email, password=password, service='analytics', source=source,
server=server, additional_headers=additional_headers, **kwargs)
def GetData(self, ids='', dimensions='', metrics='',
sort='', filters='', start_date='',
end_date='', start_index='',
max_results=''):
"""Retrieves a feed containing a user's data
ids: comma-separated string of analytics accounts.
dimensions: comma-separated string of dimensions.
metrics: comma-separated string of metrics.
sort: comma-separated string of dimensions and metrics for sorting.
This may be previxed with a minus to sort in reverse order.
(e.g. '-ga:keyword')
If ommited, the first dimension passed in will be used.
filters: comma-separated string of filter parameters.
(e.g. 'ga:keyword==google')
start_date: start date for data pull.
end_date: end date for data pull.
start_index: used in combination with max_results to pull more than 1000
entries. This defaults to 1.
max_results: maximum results that the pull will return. This defaults
to, and maxes out at 1000.
"""
q = gdata.analytics.service.DataQuery(ids=ids,
dimensions=dimensions,
metrics=metrics,
filters=filters,
sort=sort,
start_date=start_date,
end_date=end_date,
start_index=start_index,
max_results=max_results);
return self.AnalyticsDataFeed(q.ToUri())
def AnalyticsDataFeed(self, uri):
"""Retrieves an AnalyticsListFeed by retrieving a URI based off the
Document List feed, including any query parameters. An
AnalyticsListFeed object can be used to construct these parameters.
Args:
uri: string The URI of the feed being retrieved possibly with query
parameters.
Returns:
An AnalyticsListFeed object representing the feed returned by the
server.
"""
return self.Get(uri,
converter=gdata.analytics.AnalyticsDataFeedFromString)
"""
Account Fetching
"""
def QueryAccountListFeed(self, uri):
"""Retrieves an Account ListFeed by retrieving a URI based off the Account
List feed, including any query parameters. A AccountQuery object can
be used to construct these parameters.
Args:
uri: string The URI of the feed being retrieved possibly with query
parameters.
Returns:
An AccountListFeed object representing the feed returned by the server.
"""
return self.Get(uri, converter=gdata.analytics.AccountListFeedFromString)
def GetAccountListEntry(self, uri):
"""Retrieves a particular AccountListEntry by its unique URI.
Args:
uri: string The unique URI of an entry in an Account List feed.
Returns:
An AccountListEntry object representing the retrieved entry.
"""
return self.Get(uri, converter=gdata.analytics.AccountListEntryFromString)
def GetAccountList(self, username="default", max_results=1000,
start_index=1):
"""Retrieves a feed containing all of a user's accounts and profiles.
The username parameter is soon to be deprecated, with 'default'
becoming the only allowed parameter.
"""
if not username:
raise Exception("username is a required parameter")
q = gdata.analytics.service.AccountQuery(username=username,
max_results=max_results,
start_index=start_index);
return self.QueryAccountListFeed(q.ToUri())
class DataQuery(gdata.service.Query):
"""Object used to construct a URI to a data feed"""
def __init__(self, feed='/feeds/data', text_query=None,
params=None, categories=None, ids="",
dimensions="", metrics="", sort="", filters="",
start_date="", end_date="", start_index="",
max_results=""):
"""Constructor for Analytics List Query
Args:
feed: string (optional) The path for the feed. (e.g. '/feeds/data')
text_query: string (optional) The contents of the q query parameter.
This string is URL escaped upon conversion to a URI.
params: dict (optional) Parameter value string pairs which become URL
params when translated to a URI. These parameters are added to
the query's items.
categories: list (optional) List of category strings which should be
included as query categories. See gdata.service.Query for
additional documentation.
ids: comma-separated string of analytics accounts.
dimensions: comma-separated string of dimensions.
metrics: comma-separated string of metrics.
sort: comma-separated string of dimensions and metrics.
This may be previxed with a minus to sort in reverse order
(e.g. '-ga:keyword').
If ommited, the first dimension passed in will be used.
filters: comma-separated string of filter parameters
(e.g. 'ga:keyword==google').
start_date: start date for data pull.
end_date: end date for data pull.
start_index: used in combination with max_results to pull more than 1000
entries. This defaults to 1.
max_results: maximum results that the pull will return. This defaults
to, and maxes out at 1000.
Yields:
A DocumentQuery object used to construct a URI based on the Document
List feed.
"""
self.elements = {'ids': ids,
'dimensions': dimensions,
'metrics': metrics,
'sort': sort,
'filters': filters,
'start-date': start_date,
'end-date': end_date,
'start-index': start_index,
'max-results': max_results}
gdata.service.Query.__init__(self, feed, text_query, params, categories)
def ToUri(self):
"""Generates a URI from the query parameters set in the object.
Returns:
A string containing the URI used to retrieve entries from the Analytics
List feed.
"""
old_feed = self.feed
self.feed = '/'.join([old_feed]) + '?' + \
urllib.urlencode(dict([(key, value) for key, value in \
self.elements.iteritems() if value]))
new_feed = gdata.service.Query.ToUri(self)
self.feed = old_feed
return new_feed
class AccountQuery(gdata.service.Query):
"""Object used to construct a URI to query the Google Account List feed"""
def __init__(self, feed='/feeds/accounts', start_index=1,
max_results=1000, username='default', text_query=None,
params=None, categories=None):
"""Constructor for Account List Query
Args:
feed: string (optional) The path for the feed. (e.g. '/feeds/documents')
visibility: string (optional) The visibility chosen for the current
feed.
projection: string (optional) The projection chosen for the current
feed.
text_query: string (optional) The contents of the q query parameter.
This string is URL escaped upon conversion to a URI.
params: dict (optional) Parameter value string pairs which become URL
params when translated to a URI. These parameters are added to
the query's items.
categories: list (optional) List of category strings which should be
included as query categories. See gdata.service.Query for
additional documentation.
username: string (deprecated) This value should now always be passed as
'default'.
Yields:
A DocumentQuery object used to construct a URI based on the Document
List feed.
"""
self.max_results = max_results
self.start_index = start_index
self.username = username
gdata.service.Query.__init__(self, feed, text_query, params, categories)
def ToUri(self):
"""Generates a URI from the query parameters set in the object.
Returns:
A string containing the URI used to retrieve entries from the Account
List feed.
"""
old_feed = self.feed
self.feed = '/'.join([old_feed, self.username]) + '?' + \
'&'.join(['max-results=' + str(self.max_results),
'start-index=' + str(self.start_index)])
new_feed = self.feed
self.feed = old_feed
return new_feed

526
gdata/analytics/apps/__init__.py Normal file
View File

@ -0,0 +1,526 @@
#!/usr/bin/python
#
# Copyright (C) 2007 SIOS Technology, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Contains objects used with Google Apps."""
__author__ = 'tmatsuo@sios.com (Takashi MATSUO)'
import atom
import gdata
# XML namespaces which are often used in Google Apps entity.
APPS_NAMESPACE = 'http://schemas.google.com/apps/2006'
APPS_TEMPLATE = '{http://schemas.google.com/apps/2006}%s'
class EmailList(atom.AtomBase):
"""The Google Apps EmailList element"""
_tag = 'emailList'
_namespace = APPS_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['name'] = 'name'
def __init__(self, name=None, extension_elements=None,
extension_attributes=None, text=None):
self.name = name
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def EmailListFromString(xml_string):
return atom.CreateClassFromXMLString(EmailList, xml_string)
class Who(atom.AtomBase):
"""The Google Apps Who element"""
_tag = 'who'
_namespace = gdata.GDATA_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['rel'] = 'rel'
_attributes['email'] = 'email'
def __init__(self, rel=None, email=None, extension_elements=None,
extension_attributes=None, text=None):
self.rel = rel
self.email = email
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def WhoFromString(xml_string):
return atom.CreateClassFromXMLString(Who, xml_string)
class Login(atom.AtomBase):
"""The Google Apps Login element"""
_tag = 'login'
_namespace = APPS_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['userName'] = 'user_name'
_attributes['password'] = 'password'
_attributes['suspended'] = 'suspended'
_attributes['admin'] = 'admin'
_attributes['changePasswordAtNextLogin'] = 'change_password'
_attributes['agreedToTerms'] = 'agreed_to_terms'
_attributes['ipWhitelisted'] = 'ip_whitelisted'
_attributes['hashFunctionName'] = 'hash_function_name'
def __init__(self, user_name=None, password=None, suspended=None,
ip_whitelisted=None, hash_function_name=None,
admin=None, change_password=None, agreed_to_terms=None,
extension_elements=None, extension_attributes=None,
text=None):
self.user_name = user_name
self.password = password
self.suspended = suspended
self.admin = admin
self.change_password = change_password
self.agreed_to_terms = agreed_to_terms
self.ip_whitelisted = ip_whitelisted
self.hash_function_name = hash_function_name
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def LoginFromString(xml_string):
return atom.CreateClassFromXMLString(Login, xml_string)
class Quota(atom.AtomBase):
"""The Google Apps Quota element"""
_tag = 'quota'
_namespace = APPS_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['limit'] = 'limit'
def __init__(self, limit=None, extension_elements=None,
extension_attributes=None, text=None):
self.limit = limit
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def QuotaFromString(xml_string):
return atom.CreateClassFromXMLString(Quota, xml_string)
class Name(atom.AtomBase):
"""The Google Apps Name element"""
_tag = 'name'
_namespace = APPS_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['familyName'] = 'family_name'
_attributes['givenName'] = 'given_name'
def __init__(self, family_name=None, given_name=None,
extension_elements=None, extension_attributes=None, text=None):
self.family_name = family_name
self.given_name = given_name
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def NameFromString(xml_string):
return atom.CreateClassFromXMLString(Name, xml_string)
class Nickname(atom.AtomBase):
"""The Google Apps Nickname element"""
_tag = 'nickname'
_namespace = APPS_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['name'] = 'name'
def __init__(self, name=None,
extension_elements=None, extension_attributes=None, text=None):
self.name = name
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def NicknameFromString(xml_string):
return atom.CreateClassFromXMLString(Nickname, xml_string)
class NicknameEntry(gdata.GDataEntry):
"""A Google Apps flavor of an Atom Entry for Nickname"""
_tag = 'entry'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataEntry._children.copy()
_attributes = gdata.GDataEntry._attributes.copy()
_children['{%s}login' % APPS_NAMESPACE] = ('login', Login)
_children['{%s}nickname' % APPS_NAMESPACE] = ('nickname', Nickname)
def __init__(self, author=None, category=None, content=None,
atom_id=None, link=None, published=None,
title=None, updated=None,
login=None, nickname=None,
extended_property=None,
extension_elements=None, extension_attributes=None, text=None):
gdata.GDataEntry.__init__(self, author=author, category=category,
content=content,
atom_id=atom_id, link=link, published=published,
title=title, updated=updated)
self.login = login
self.nickname = nickname
self.extended_property = extended_property or []
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def NicknameEntryFromString(xml_string):
return atom.CreateClassFromXMLString(NicknameEntry, xml_string)
class NicknameFeed(gdata.GDataFeed, gdata.LinkFinder):
"""A Google Apps Nickname feed flavor of an Atom Feed"""
_tag = 'feed'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataFeed._children.copy()
_attributes = gdata.GDataFeed._attributes.copy()
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', [NicknameEntry])
def __init__(self, author=None, category=None, contributor=None,
generator=None, icon=None, atom_id=None, link=None, logo=None,
rights=None, subtitle=None, title=None, updated=None,
entry=None, total_results=None, start_index=None,
items_per_page=None, extension_elements=None,
extension_attributes=None, text=None):
gdata.GDataFeed.__init__(self, author=author, category=category,
contributor=contributor, generator=generator,
icon=icon, atom_id=atom_id, link=link,
logo=logo, rights=rights, subtitle=subtitle,
title=title, updated=updated, entry=entry,
total_results=total_results,
start_index=start_index,
items_per_page=items_per_page,
extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def NicknameFeedFromString(xml_string):
return atom.CreateClassFromXMLString(NicknameFeed, xml_string)
class UserEntry(gdata.GDataEntry):
"""A Google Apps flavor of an Atom Entry"""
_tag = 'entry'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataEntry._children.copy()
_attributes = gdata.GDataEntry._attributes.copy()
_children['{%s}login' % APPS_NAMESPACE] = ('login', Login)
_children['{%s}name' % APPS_NAMESPACE] = ('name', Name)
_children['{%s}quota' % APPS_NAMESPACE] = ('quota', Quota)
# This child may already be defined in GDataEntry, confirm before removing.
_children['{%s}feedLink' % gdata.GDATA_NAMESPACE] = ('feed_link',
[gdata.FeedLink])
_children['{%s}who' % gdata.GDATA_NAMESPACE] = ('who', Who)
def __init__(self, author=None, category=None, content=None,
atom_id=None, link=None, published=None,
title=None, updated=None,
login=None, name=None, quota=None, who=None, feed_link=None,
extended_property=None,
extension_elements=None, extension_attributes=None, text=None):
gdata.GDataEntry.__init__(self, author=author, category=category,
content=content,
atom_id=atom_id, link=link, published=published,
title=title, updated=updated)
self.login = login
self.name = name
self.quota = quota
self.who = who
self.feed_link = feed_link or []
self.extended_property = extended_property or []
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def UserEntryFromString(xml_string):
return atom.CreateClassFromXMLString(UserEntry, xml_string)
class UserFeed(gdata.GDataFeed, gdata.LinkFinder):
"""A Google Apps User feed flavor of an Atom Feed"""
_tag = 'feed'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataFeed._children.copy()
_attributes = gdata.GDataFeed._attributes.copy()
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', [UserEntry])
def __init__(self, author=None, category=None, contributor=None,
generator=None, icon=None, atom_id=None, link=None, logo=None,
rights=None, subtitle=None, title=None, updated=None,
entry=None, total_results=None, start_index=None,
items_per_page=None, extension_elements=None,
extension_attributes=None, text=None):
gdata.GDataFeed.__init__(self, author=author, category=category,
contributor=contributor, generator=generator,
icon=icon, atom_id=atom_id, link=link,
logo=logo, rights=rights, subtitle=subtitle,
title=title, updated=updated, entry=entry,
total_results=total_results,
start_index=start_index,
items_per_page=items_per_page,
extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def UserFeedFromString(xml_string):
return atom.CreateClassFromXMLString(UserFeed, xml_string)
class EmailListEntry(gdata.GDataEntry):
"""A Google Apps EmailList flavor of an Atom Entry"""
_tag = 'entry'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataEntry._children.copy()
_attributes = gdata.GDataEntry._attributes.copy()
_children['{%s}emailList' % APPS_NAMESPACE] = ('email_list', EmailList)
# Might be able to remove this _children entry.
_children['{%s}feedLink' % gdata.GDATA_NAMESPACE] = ('feed_link',
[gdata.FeedLink])
def __init__(self, author=None, category=None, content=None,
atom_id=None, link=None, published=None,
title=None, updated=None,
email_list=None, feed_link=None,
extended_property=None,
extension_elements=None, extension_attributes=None, text=None):
gdata.GDataEntry.__init__(self, author=author, category=category,
content=content,
atom_id=atom_id, link=link, published=published,
title=title, updated=updated)
self.email_list = email_list
self.feed_link = feed_link or []
self.extended_property = extended_property or []
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def EmailListEntryFromString(xml_string):
return atom.CreateClassFromXMLString(EmailListEntry, xml_string)
class EmailListFeed(gdata.GDataFeed, gdata.LinkFinder):
"""A Google Apps EmailList feed flavor of an Atom Feed"""
_tag = 'feed'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataFeed._children.copy()
_attributes = gdata.GDataFeed._attributes.copy()
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', [EmailListEntry])
def __init__(self, author=None, category=None, contributor=None,
generator=None, icon=None, atom_id=None, link=None, logo=None,
rights=None, subtitle=None, title=None, updated=None,
entry=None, total_results=None, start_index=None,
items_per_page=None, extension_elements=None,
extension_attributes=None, text=None):
gdata.GDataFeed.__init__(self, author=author, category=category,
contributor=contributor, generator=generator,
icon=icon, atom_id=atom_id, link=link,
logo=logo, rights=rights, subtitle=subtitle,
title=title, updated=updated, entry=entry,
total_results=total_results,
start_index=start_index,
items_per_page=items_per_page,
extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def EmailListFeedFromString(xml_string):
return atom.CreateClassFromXMLString(EmailListFeed, xml_string)
class EmailListRecipientEntry(gdata.GDataEntry):
"""A Google Apps EmailListRecipient flavor of an Atom Entry"""
_tag = 'entry'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataEntry._children.copy()
_attributes = gdata.GDataEntry._attributes.copy()
_children['{%s}who' % gdata.GDATA_NAMESPACE] = ('who', Who)
def __init__(self, author=None, category=None, content=None,
atom_id=None, link=None, published=None,
title=None, updated=None,
who=None,
extended_property=None,
extension_elements=None, extension_attributes=None, text=None):
gdata.GDataEntry.__init__(self, author=author, category=category,
content=content,
atom_id=atom_id, link=link, published=published,
title=title, updated=updated)
self.who = who
self.extended_property = extended_property or []
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def EmailListRecipientEntryFromString(xml_string):
return atom.CreateClassFromXMLString(EmailListRecipientEntry, xml_string)
class EmailListRecipientFeed(gdata.GDataFeed, gdata.LinkFinder):
"""A Google Apps EmailListRecipient feed flavor of an Atom Feed"""
_tag = 'feed'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataFeed._children.copy()
_attributes = gdata.GDataFeed._attributes.copy()
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry',
[EmailListRecipientEntry])
def __init__(self, author=None, category=None, contributor=None,
generator=None, icon=None, atom_id=None, link=None, logo=None,
rights=None, subtitle=None, title=None, updated=None,
entry=None, total_results=None, start_index=None,
items_per_page=None, extension_elements=None,
extension_attributes=None, text=None):
gdata.GDataFeed.__init__(self, author=author, category=category,
contributor=contributor, generator=generator,
icon=icon, atom_id=atom_id, link=link,
logo=logo, rights=rights, subtitle=subtitle,
title=title, updated=updated, entry=entry,
total_results=total_results,
start_index=start_index,
items_per_page=items_per_page,
extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def EmailListRecipientFeedFromString(xml_string):
return atom.CreateClassFromXMLString(EmailListRecipientFeed, xml_string)
class Property(atom.AtomBase):
"""The Google Apps Property element"""
_tag = 'property'
_namespace = APPS_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['name'] = 'name'
_attributes['value'] = 'value'
def __init__(self, name=None, value=None, extension_elements=None,
extension_attributes=None, text=None):
self.name = name
self.value = value
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def PropertyFromString(xml_string):
return atom.CreateClassFromXMLString(Property, xml_string)
class PropertyEntry(gdata.GDataEntry):
"""A Google Apps Property flavor of an Atom Entry"""
_tag = 'entry'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataEntry._children.copy()
_attributes = gdata.GDataEntry._attributes.copy()
_children['{%s}property' % APPS_NAMESPACE] = ('property', [Property])
def __init__(self, author=None, category=None, content=None,
atom_id=None, link=None, published=None,
title=None, updated=None,
property=None,
extended_property=None,
extension_elements=None, extension_attributes=None, text=None):
gdata.GDataEntry.__init__(self, author=author, category=category,
content=content,
atom_id=atom_id, link=link, published=published,
title=title, updated=updated)
self.property = property
self.extended_property = extended_property or []
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def PropertyEntryFromString(xml_string):
return atom.CreateClassFromXMLString(PropertyEntry, xml_string)
class PropertyFeed(gdata.GDataFeed, gdata.LinkFinder):
"""A Google Apps Property feed flavor of an Atom Feed"""
_tag = 'feed'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataFeed._children.copy()
_attributes = gdata.GDataFeed._attributes.copy()
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', [PropertyEntry])
def __init__(self, author=None, category=None, contributor=None,
generator=None, icon=None, atom_id=None, link=None, logo=None,
rights=None, subtitle=None, title=None, updated=None,
entry=None, total_results=None, start_index=None,
items_per_page=None, extension_elements=None,
extension_attributes=None, text=None):
gdata.GDataFeed.__init__(self, author=author, category=category,
contributor=contributor, generator=generator,
icon=icon, atom_id=atom_id, link=link,
logo=logo, rights=rights, subtitle=subtitle,
title=title, updated=updated, entry=entry,
total_results=total_results,
start_index=start_index,
items_per_page=items_per_page,
extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
def PropertyFeedFromString(xml_string):
return atom.CreateClassFromXMLString(PropertyFeed, xml_string)

16
gdata/analytics/apps/adminsettings/__init__.py Normal file
View File

@ -0,0 +1,16 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

471
gdata/analytics/apps/adminsettings/service.py Normal file
View File

@ -0,0 +1,471 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Allow Google Apps domain administrators to set domain admin settings.
AdminSettingsService: Set admin settings."""
__author__ = 'jlee@pbu.edu'
import gdata.apps
import gdata.apps.service
import gdata.service
API_VER='2.0'
class AdminSettingsService(gdata.apps.service.PropertyService):
"""Client for the Google Apps Admin Settings service."""
def _serviceUrl(self, setting_id, domain=None):
if domain is None:
domain = self.domain
return '/a/feeds/domain/%s/%s/%s' % (API_VER, domain, setting_id)
def genericGet(self, location):
"""Generic HTTP Get Wrapper
Args:
location: relative uri to Get
Returns:
A dict containing the result of the get operation."""
uri = self._serviceUrl(location)
try:
return self._GetProperties(uri)
except gdata.service.RequestError, e:
raise AppsForYourDomainException(e.args[0])
def GetDefaultLanguage(self):
"""Gets Domain Default Language
Args:
None
Returns:
Default Language as a string. All possible values are listed at:
http://code.google.com/apis/apps/email_settings/developers_guide_protocol.html#GA_email_language_tags"""
result = self.genericGet('general/defaultLanguage')
return result['defaultLanguage']
def UpdateDefaultLanguage(self, defaultLanguage):
"""Updates Domain Default Language
Args:
defaultLanguage: Domain Language to set
possible values are at:
http://code.google.com/apis/apps/email_settings/developers_guide_protocol.html#GA_email_language_tags
Returns:
A dict containing the result of the put operation"""
uri = self._serviceUrl('general/defaultLanguage')
properties = {'defaultLanguage': defaultLanguage}
return self._PutProperties(uri, properties)
def GetOrganizationName(self):
"""Gets Domain Default Language
Args:
None
Returns:
Organization Name as a string."""
result = self.genericGet('general/organizationName')
return result['organizationName']
def UpdateOrganizationName(self, organizationName):
"""Updates Organization Name
Args:
organizationName: Name of organization
Returns:
A dict containing the result of the put operation"""
uri = self._serviceUrl('general/organizationName')
properties = {'organizationName': organizationName}
return self._PutProperties(uri, properties)
def GetMaximumNumberOfUsers(self):
"""Gets Maximum Number of Users Allowed
Args:
None
Returns: An integer, the maximum number of users"""
result = self.genericGet('general/maximumNumberOfUsers')
return int(result['maximumNumberOfUsers'])
def GetCurrentNumberOfUsers(self):
"""Gets Current Number of Users
Args:
None
Returns: An integer, the current number of users"""
result = self.genericGet('general/currentNumberOfUsers')
return int(result['currentNumberOfUsers'])
def IsDomainVerified(self):
"""Is the domain verified
Args:
None
Returns: Boolean, is domain verified"""
result = self.genericGet('accountInformation/isVerified')
if result['isVerified'] == 'true':
return True
else:
return False
def GetSupportPIN(self):
"""Gets Support PIN
Args:
None
Returns: A string, the Support PIN"""
result = self.genericGet('accountInformation/supportPIN')
return result['supportPIN']
def GetEdition(self):
"""Gets Google Apps Domain Edition
Args:
None
Returns: A string, the domain's edition (premier, education, partner)"""
result = self.genericGet('accountInformation/edition')
return result['edition']
def GetCustomerPIN(self):
"""Gets Customer PIN
Args:
None
Returns: A string, the customer PIN"""
result = self.genericGet('accountInformation/customerPIN')
return result['customerPIN']
def GetCreationTime(self):
"""Gets Domain Creation Time
Args:
None
Returns: A string, the domain's creation time"""
result = self.genericGet('accountInformation/creationTime')
return result['creationTime']
def GetCountryCode(self):
"""Gets Domain Country Code
Args:
None
Returns: A string, the domain's country code. Possible values at:
http://www.iso.org/iso/country_codes/iso_3166_code_lists/english_country_names_and_code_elements.htm"""
result = self.genericGet('accountInformation/countryCode')
return result['countryCode']
def GetAdminSecondaryEmail(self):
"""Gets Domain Admin Secondary Email Address
Args:
None
Returns: A string, the secondary email address for domain admin"""
result = self.genericGet('accountInformation/adminSecondaryEmail')
return result['adminSecondaryEmail']
def UpdateAdminSecondaryEmail(self, adminSecondaryEmail):
"""Gets Domain Creation Time
Args:
adminSecondaryEmail: string, secondary email address of admin
Returns: A dict containing the result of the put operation"""
uri = self._serviceUrl('accountInformation/adminSecondaryEmail')
properties = {'adminSecondaryEmail': adminSecondaryEmail}
return self._PutProperties(uri, properties)
def GetDomainLogo(self):
"""Gets Domain Logo
This function does not make use of the Google Apps Admin Settings API,
it does an HTTP Get of a url specific to the Google Apps domain. It is
included for completeness sake.
Args:
None
Returns: binary image file"""
import urllib
url = 'http://www.google.com/a/cpanel/'+self.domain+'/images/logo.gif'
response = urllib.urlopen(url)
return response.read()
def UpdateDomainLogo(self, logoImage):
"""Update Domain's Custom Logo
Args:
logoImage: binary image data
Returns: A dict containing the result of the put operation"""
from base64 import base64encode
uri = self._serviceUrl('appearance/customLogo')
properties = {'logoImage': base64encode(logoImage)}
return self._PutProperties(uri, properties)
def GetCNAMEVerificationStatus(self):
"""Gets Domain CNAME Verification Status
Args:
None
Returns: A dict {recordName, verified, verifiedMethod}"""
return self.genericGet('verification/cname')
def UpdateCNAMEVerificationStatus(self, verified):
"""Updates CNAME Verification Status
Args:
verified: boolean, True will retry verification process
Returns: A dict containing the result of the put operation"""
uri = self._serviceUrl('verification/cname')
properties = self.GetCNAMEVerificationStatus()
properties['verified'] = verified
return self._PutProperties(uri, properties)
def GetMXVerificationStatus(self):
"""Gets Domain MX Verification Status
Args:
None
Returns: A dict {verified, verifiedMethod}"""
return self.genericGet('verification/mx')
def UpdateMXVerificationStatus(self, verified):
"""Updates MX Verification Status
Args:
verified: boolean, True will retry verification process
Returns: A dict containing the result of the put operation"""
uri = self._serviceUrl('verification/mx')
properties = self.GetMXVerificationStatus()
properties['verified'] = verified
return self._PutProperties(uri, properties)
def GetSSOSettings(self):
"""Gets Domain Single Sign-On Settings
Args:
None
Returns: A dict {samlSignonUri, samlLogoutUri, changePasswordUri, enableSSO, ssoWhitelist, useDomainSpecificIssuer}"""
return self.genericGet('sso/general')
def UpdateSSOSettings(self, enableSSO=None, samlSignonUri=None,
samlLogoutUri=None, changePasswordUri=None,
ssoWhitelist=None, useDomainSpecificIssuer=None):
"""Update SSO Settings.
Args:
enableSSO: boolean, SSO Master on/off switch
samlSignonUri: string, SSO Login Page
samlLogoutUri: string, SSO Logout Page
samlPasswordUri: string, SSO Password Change Page
ssoWhitelist: string, Range of IP Addresses which will see SSO
useDomainSpecificIssuer: boolean, Include Google Apps Domain in Issuer
Returns:
A dict containing the result of the update operation.
"""
uri = self._serviceUrl('sso/general')
#Get current settings, replace Nones with ''
properties = self.GetSSOSettings()
if properties['samlSignonUri'] == None:
properties['samlSignonUri'] = ''
if properties['samlLogoutUri'] == None:
properties['samlLogoutUri'] = ''
if properties['changePasswordUri'] == None:
properties['changePasswordUri'] = ''
if properties['ssoWhitelist'] == None:
properties['ssoWhitelist'] = ''
#update only the values we were passed
if enableSSO != None:
properties['enableSSO'] = gdata.apps.service._bool2str(enableSSO)
if samlSignonUri != None:
properties['samlSignonUri'] = samlSignonUri
if samlLogoutUri != None:
properties['samlLogoutUri'] = samlLogoutUri
if changePasswordUri != None:
properties['changePasswordUri'] = changePasswordUri
if ssoWhitelist != None:
properties['ssoWhitelist'] = ssoWhitelist
if useDomainSpecificIssuer != None:
properties['useDomainSpecificIssuer'] = gdata.apps.service._bool2str(useDomainSpecificIssuer)
return self._PutProperties(uri, properties)
def GetSSOKey(self):
"""Gets Domain Single Sign-On Signing Key
Args:
None
Returns: A dict {modulus, exponent, algorithm, format}"""
return self.genericGet('sso/signingkey')
def UpdateSSOKey(self, signingKey):
"""Update SSO Settings.
Args:
signingKey: string, public key to be uploaded
Returns:
A dict containing the result of the update operation."""
uri = self._serviceUrl('sso/signingkey')
properties = {'signingKey': signingKey}
return self._PutProperties(uri, properties)
def IsUserMigrationEnabled(self):
"""Is User Migration Enabled
Args:
None
Returns:
boolean, is user migration enabled"""
result = self.genericGet('email/migration')
if result['enableUserMigration'] == 'true':
return True
else:
return False
def UpdateUserMigrationStatus(self, enableUserMigration):
"""Update User Migration Status
Args:
enableUserMigration: boolean, user migration enable/disable
Returns:
A dict containing the result of the update operation."""
uri = self._serviceUrl('email/migration')
properties = {'enableUserMigration': enableUserMigration}
return self._PutProperties(uri, properties)
def GetOutboundGatewaySettings(self):
"""Get Outbound Gateway Settings
Args:
None
Returns:
A dict {smartHost, smtpMode}"""
uri = self._serviceUrl('email/gateway')
try:
return self._GetProperties(uri)
except gdata.service.RequestError, e:
raise AppsForYourDomainException(e.args[0])
except TypeError:
#if no outbound gateway is set, we get a TypeError,
#catch it and return nothing...
return {'smartHost': None, 'smtpMode': None}
def UpdateOutboundGatewaySettings(self, smartHost=None, smtpMode=None):
"""Update Outbound Gateway Settings
Args:
smartHost: string, ip address or hostname of outbound gateway
smtpMode: string, SMTP or SMTP_TLS
Returns:
A dict containing the result of the update operation."""
uri = self._serviceUrl('email/gateway')
#Get current settings, replace Nones with ''
properties = GetOutboundGatewaySettings()
if properties['smartHost'] == None:
properties['smartHost'] = ''
if properties['smtpMode'] == None:
properties['smtpMode'] = ''
#If we were passed new values for smartHost or smtpMode, update them
if smartHost != None:
properties['smartHost'] = smartHost
if smtpMode != None:
properties['smtpMode'] = smtpMode
return self._PutProperties(uri, properties)
def AddEmailRoute(self, routeDestination, routeRewriteTo, routeEnabled, bounceNotifications, accountHandling):
"""Adds Domain Email Route
Args:
routeDestination: string, destination ip address or hostname
routeRewriteTo: boolean, rewrite smtp envelop To:
routeEnabled: boolean, enable disable email routing
bounceNotifications: boolean, send bound notificiations to sender
accountHandling: string, which to route, "allAccounts", "provisionedAccounts", "unknownAccounts"
Returns:
A dict containing the result of the update operation."""
uri = self._serviceUrl('emailrouting')
properties = {}
properties['routeDestination'] = routeDestination
properties['routeRewriteTo'] = gdata.apps.service._bool2str(routeRewriteTo)
properties['routeEnabled'] = gdata.apps.service._bool2str(routeEnabled)
properties['bounceNotifications'] = gdata.apps.service._bool2str(bounceNotifications)
properties['accountHandling'] = accountHandling
return self._PostProperties(uri, properties)

1
gdata/analytics/apps/audit/__init__.py Normal file
View File

@ -0,0 +1 @@

277
gdata/analytics/apps/audit/service.py Normal file
View File

@ -0,0 +1,277 @@
# Copyright (C) 2008 Google, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Allow Google Apps domain administrators to audit user data.
AuditService: Set auditing."""
__author__ = 'jlee@pbu.edu'
from base64 import b64encode
import gdata.apps
import gdata.apps.service
import gdata.service
class AuditService(gdata.apps.service.PropertyService):
"""Client for the Google Apps Audit service."""
def _serviceUrl(self, setting_id, domain=None, user=None):
if domain is None:
domain = self.domain
if user is None:
return '/a/feeds/compliance/audit/%s/%s' % (setting_id, domain)
else:
return '/a/feeds/compliance/audit/%s/%s/%s' % (setting_id, domain, user)
def updatePGPKey(self, pgpkey):
"""Updates Public PGP Key Google uses to encrypt audit data
Args:
pgpkey: string, ASCII text of PGP Public Key to be used
Returns:
A dict containing the result of the POST operation."""
uri = self._serviceUrl('publickey')
b64pgpkey = b64encode(pgpkey)
properties = {}
properties['publicKey'] = b64pgpkey
return self._PostProperties(uri, properties)
def createEmailMonitor(self, source_user, destination_user, end_date,
begin_date=None, incoming_headers_only=False,
outgoing_headers_only=False, drafts=False,
drafts_headers_only=False, chats=False,
chats_headers_only=False):
"""Creates a email monitor, forwarding the source_users emails/chats
Args:
source_user: string, the user whose email will be audited
destination_user: string, the user to receive the audited email
end_date: string, the date the audit will end in
"yyyy-MM-dd HH:mm" format, required
begin_date: string, the date the audit will start in
"yyyy-MM-dd HH:mm" format, leave blank to use current time
incoming_headers_only: boolean, whether to audit only the headers of
mail delivered to source user
outgoing_headers_only: boolean, whether to audit only the headers of
mail sent from the source user
drafts: boolean, whether to audit draft messages of the source user
drafts_headers_only: boolean, whether to audit only the headers of
mail drafts saved by the user
chats: boolean, whether to audit archived chats of the source user
chats_headers_only: boolean, whether to audit only the headers of
archived chats of the source user
Returns:
A dict containing the result of the POST operation."""
uri = self._serviceUrl('mail/monitor', user=source_user)
properties = {}
properties['destUserName'] = destination_user
if begin_date is not None:
properties['beginDate'] = begin_date
properties['endDate'] = end_date
if incoming_headers_only:
properties['incomingEmailMonitorLevel'] = 'HEADER_ONLY'
else:
properties['incomingEmailMonitorLevel'] = 'FULL_MESSAGE'
if outgoing_headers_only:
properties['outgoingEmailMonitorLevel'] = 'HEADER_ONLY'
else:
properties['outgoingEmailMonitorLevel'] = 'FULL_MESSAGE'
if drafts:
if drafts_headers_only:
properties['draftMonitorLevel'] = 'HEADER_ONLY'
else:
properties['draftMonitorLevel'] = 'FULL_MESSAGE'
if chats:
if chats_headers_only:
properties['chatMonitorLevel'] = 'HEADER_ONLY'
else:
properties['chatMonitorLevel'] = 'FULL_MESSAGE'
return self._PostProperties(uri, properties)
def getEmailMonitors(self, user):
""""Gets the email monitors for the given user
Args:
user: string, the user to retrieve email monitors for
Returns:
list results of the POST operation
"""
uri = self._serviceUrl('mail/monitor', user=user)
return self._GetPropertiesList(uri)
def deleteEmailMonitor(self, source_user, destination_user):
"""Deletes the email monitor for the given user
Args:
source_user: string, the user who is being monitored
destination_user: string, theuser who recieves the monitored emails
Returns:
Nothing
"""
uri = self._serviceUrl('mail/monitor', user=source_user+'/'+destination_user)
try:
return self._DeleteProperties(uri)
except gdata.service.RequestError, e:
raise AppsForYourDomainException(e.args[0])
def createAccountInformationRequest(self, user):
"""Creates a request for account auditing details
Args:
user: string, the user to request account information for
Returns:
A dict containing the result of the post operation."""
uri = self._serviceUrl('account', user=user)
properties = {}
#XML Body is left empty
try:
return self._PostProperties(uri, properties)
except gdata.service.RequestError, e:
raise AppsForYourDomainException(e.args[0])
def getAccountInformationRequestStatus(self, user, request_id):
"""Gets the status of an account auditing request
Args:
user: string, the user whose account auditing details were requested
request_id: string, the request_id
Returns:
A dict containing the result of the get operation."""
uri = self._serviceUrl('account', user=user+'/'+request_id)
try:
return self._GetProperties(uri)
except gdata.service.RequestError, e:
raise AppsForYourDomainException(e.args[0])
def getAllAccountInformationRequestsStatus(self):
"""Gets the status of all account auditing requests for the domain
Args:
None
Returns:
list results of the POST operation
"""
uri = self._serviceUrl('account')
return self._GetPropertiesList(uri)
def deleteAccountInformationRequest(self, user, request_id):
"""Deletes the request for account auditing information
Args:
user: string, the user whose account auditing details were requested
request_id: string, the request_id
Returns:
Nothing
"""
uri = self._serviceUrl('account', user=user+'/'+request_id)
try:
return self._DeleteProperties(uri)
except gdata.service.RequestError, e:
raise AppsForYourDomainException(e.args[0])
def createMailboxExportRequest(self, user, begin_date=None, end_date=None, include_deleted=False, search_query=None, headers_only=False):
"""Creates a mailbox export request
Args:
user: string, the user whose mailbox export is being requested
begin_date: string, date of earliest emails to export, optional, defaults to date of account creation
format is 'yyyy-MM-dd HH:mm'
end_date: string, date of latest emails to export, optional, defaults to current date
format is 'yyyy-MM-dd HH:mm'
include_deleted: boolean, whether to include deleted emails in export, mutually exclusive with search_query
search_query: string, gmail style search query, matched emails will be exported, mutually exclusive with include_deleted
Returns:
A dict containing the result of the post operation."""
uri = self._serviceUrl('mail/export', user=user)
properties = {}
if begin_date is not None:
properties['beginDate'] = begin_date
if end_date is not None:
properties['endDate'] = end_date
if include_deleted is not None:
properties['includeDeleted'] = gdata.apps.service._bool2str(include_deleted)
if search_query is not None:
properties['searchQuery'] = search_query
if headers_only is True:
properties['packageContent'] = 'HEADER_ONLY'
else:
properties['packageContent'] = 'FULL_MESSAGE'
return self._PostProperties(uri, properties)
def getMailboxExportRequestStatus(self, user, request_id):
"""Gets the status of an mailbox export request
Args:
user: string, the user whose mailbox were requested
request_id: string, the request_id
Returns:
A dict containing the result of the get operation."""
uri = self._serviceUrl('mail/export', user=user+'/'+request_id)
try:
return self._GetProperties(uri)
except gdata.service.RequestError, e:
raise AppsForYourDomainException(e.args[0])
def getAllMailboxExportRequestsStatus(self):
"""Gets the status of all mailbox export requests for the domain
Args:
None
Returns:
list results of the POST operation
"""
uri = self._serviceUrl('mail/export')
return self._GetPropertiesList(uri)
def deleteMailboxExportRequest(self, user, request_id):
"""Deletes the request for mailbox export
Args:
user: string, the user whose mailbox were requested
request_id: string, the request_id
Returns:
Nothing
"""
uri = self._serviceUrl('mail/export', user=user+'/'+request_id)
try:
return self._DeleteProperties(uri)
except gdata.service.RequestError, e:
raise AppsForYourDomainException(e.args[0])

15
gdata/analytics/apps/emailsettings/__init__.py Normal file
View File

@ -0,0 +1,15 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

557
gdata/analytics/apps/emailsettings/client.py Normal file
View File

@ -0,0 +1,557 @@
#!/usr/bin/python2.4
#
# Copyright 2010 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""EmailSettingsClient simplifies Email Settings API calls.
EmailSettingsClient extends gdata.client.GDClient to ease interaction with
the Google Apps Email Settings API. These interactions include the ability
to create labels, filters, aliases, and update web-clip, forwarding, POP,
IMAP, vacation-responder, signature, language, and general settings, and
retrieve labels, send-as, forwarding, pop, imap, vacation and signature
settings.
"""
__author__ = 'Claudio Cherubino <ccherubino@google.com>'
import gdata.apps.emailsettings.data
import gdata.client
# Email Settings URI template
# The strings in this template are eventually replaced with the API version,
# Google Apps domain name, username, and settingID, respectively.
EMAIL_SETTINGS_URI_TEMPLATE = '/a/feeds/emailsettings/%s/%s/%s/%s'
# The settingID value for the label requests
SETTING_ID_LABEL = 'label'
# The settingID value for the filter requests
SETTING_ID_FILTER = 'filter'
# The settingID value for the send-as requests
SETTING_ID_SENDAS = 'sendas'
# The settingID value for the webclip requests
SETTING_ID_WEBCLIP = 'webclip'
# The settingID value for the forwarding requests
SETTING_ID_FORWARDING = 'forwarding'
# The settingID value for the POP requests
SETTING_ID_POP = 'pop'
# The settingID value for the IMAP requests
SETTING_ID_IMAP = 'imap'
# The settingID value for the vacation responder requests
SETTING_ID_VACATION_RESPONDER = 'vacation'
# The settingID value for the signature requests
SETTING_ID_SIGNATURE = 'signature'
# The settingID value for the language requests
SETTING_ID_LANGUAGE = 'language'
# The settingID value for the general requests
SETTING_ID_GENERAL = 'general'
# The settingID value for the delegation requests
SETTING_ID_DELEGATION = 'delegation'
# The KEEP action for the email settings
ACTION_KEEP = 'KEEP'
# The ARCHIVE action for the email settings
ACTION_ARCHIVE = 'ARCHIVE'
# The DELETE action for the email settings
ACTION_DELETE = 'DELETE'
# The ALL_MAIL setting for POP enable_for property
POP_ENABLE_FOR_ALL_MAIL = 'ALL_MAIL'
# The MAIL_FROM_NOW_ON setting for POP enable_for property
POP_ENABLE_FOR_MAIL_FROM_NOW_ON = 'MAIL_FROM_NOW_ON'
class EmailSettingsClient(gdata.client.GDClient):
"""Client extension for the Google Email Settings API service.
Attributes:
host: string The hostname for the Email Settings API service.
api_version: string The version of the Email Settings API.
"""
host = 'apps-apis.google.com'
api_version = '2.0'
auth_service = 'apps'
auth_scopes = gdata.gauth.AUTH_SCOPES['apps']
ssl = True
def __init__(self, domain, auth_token=None, **kwargs):
"""Constructs a new client for the Email Settings API.
Args:
domain: string The Google Apps domain with Email Settings.
auth_token: (optional) gdata.gauth.ClientLoginToken, AuthSubToken, or
OAuthToken which authorizes this client to edit the email settings.
kwargs: The other parameters to pass to the gdata.client.GDClient
constructor.
"""
gdata.client.GDClient.__init__(self, auth_token=auth_token, **kwargs)
self.domain = domain
def make_email_settings_uri(self, username, setting_id):
"""Creates the URI for the Email Settings API call.
Using this client's Google Apps domain, create the URI to setup
email settings for the given user in that domain. If params are provided,
append them as GET params.
Args:
username: string The name of the user affected by this setting.
setting_id: string The key of the setting to be configured.
Returns:
A string giving the URI for Email Settings API calls for this client's
Google Apps domain.
"""
if '@' in username:
username, domain = username.split('@', 1)
else:
domain = self.domain
uri = EMAIL_SETTINGS_URI_TEMPLATE % (self.api_version, domain,
username, setting_id)
return uri
MakeEmailSettingsUri = make_email_settings_uri
def create_label(self, username, name, **kwargs):
"""Creates a label with the given properties.
Args:
username: string The name of the user.
name: string The name of the label.
kwargs: The other parameters to pass to gdata.client.GDClient.post().
Returns:
gdata.apps.emailsettings.data.EmailSettingsLabel of the new resource.
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_LABEL)
new_label = gdata.apps.emailsettings.data.EmailSettingsLabel(
uri=uri, name=name)
return self.post(new_label, uri, **kwargs)
CreateLabel = create_label
def retrieve_labels(self, username, **kwargs):
"""Retrieves email labels for the specified username
Args:
username: string The name of the user to get the labels for
Returns:
A gdata.data.GDFeed of the user's email labels
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_LABEL)
return self.GetFeed(uri, auth_token=None, query=None, **kwargs)
RetrieveLabels = retrieve_labels
def create_filter(self, username, from_address=None,
to_address=None, subject=None, has_the_word=None,
does_not_have_the_word=None, has_attachments=None,
label=None, mark_as_read=None, archive=None, **kwargs):
"""Creates a filter with the given properties.
Args:
username: string The name of the user.
from_address: string The source email address for the filter.
to_address: string (optional) The destination email address for
the filter.
subject: string (optional) The value the email must have in its
subject to be filtered.
has_the_word: string (optional) The value the email must have
in its subject or body to be filtered.
does_not_have_the_word: string (optional) The value the email
cannot have in its subject or body to be filtered.
has_attachments: string (optional) A boolean string representing
whether the email must have an attachment to be filtered.
label: string (optional) The name of the label to apply to
messages matching the filter criteria.
mark_as_read: Boolean (optional) Whether or not to mark
messages matching the filter criteria as read.
archive: Boolean (optional) Whether or not to move messages
matching to Archived state.
kwargs: The other parameters to pass to gdata.client.GDClient.post().
Returns:
gdata.apps.emailsettings.data.EmailSettingsFilter of the new resource.
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_FILTER)
new_filter = gdata.apps.emailsettings.data.EmailSettingsFilter(
uri=uri, from_address=from_address,
to_address=to_address, subject=subject,
has_the_word=has_the_word,
does_not_have_the_word=does_not_have_the_word,
has_attachments=has_attachments, label=label,
mark_as_read=mark_as_read, archive=archive)
return self.post(new_filter, uri, **kwargs)
CreateFilter = create_filter
def create_send_as(self, username, name, address, reply_to=None,
make_default=None, **kwargs):
"""Creates a send-as alias with the given properties.
Args:
username: string The name of the user.
name: string The name that will appear in the "From" field.
address: string The email address that appears as the
origination address for emails sent by this user.
reply_to: string (optional) The address to be used as the reply-to
address in email sent using the alias.
make_default: Boolean (optional) Whether or not this alias should
become the default alias for this user.
kwargs: The other parameters to pass to gdata.client.GDClient.post().
Returns:
gdata.apps.emailsettings.data.EmailSettingsSendAsAlias of the
new resource.
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_SENDAS)
new_alias = gdata.apps.emailsettings.data.EmailSettingsSendAsAlias(
uri=uri, name=name, address=address,
reply_to=reply_to, make_default=make_default)
return self.post(new_alias, uri, **kwargs)
CreateSendAs = create_send_as
def retrieve_send_as(self, username, **kwargs):
"""Retrieves send-as aliases for the specified username
Args:
username: string The name of the user to get the send-as for
Returns:
A gdata.data.GDFeed of the user's send-as alias settings
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_SENDAS)
return self.GetFeed(uri, auth_token=None, query=None, **kwargs)
RetrieveSendAs = retrieve_send_as
def update_webclip(self, username, enable, **kwargs):
"""Enable/Disable Google Mail web clip.
Args:
username: string The name of the user.
enable: Boolean Whether to enable showing Web clips.
kwargs: The other parameters to pass to the update method.
Returns:
gdata.apps.emailsettings.data.EmailSettingsWebClip of the
updated resource.
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_WEBCLIP)
new_webclip = gdata.apps.emailsettings.data.EmailSettingsWebClip(
uri=uri, enable=enable)
return self.update(new_webclip, **kwargs)
UpdateWebclip = update_webclip
def update_forwarding(self, username, enable, forward_to=None,
action=None, **kwargs):
"""Update Google Mail Forwarding settings.
Args:
username: string The name of the user.
enable: Boolean Whether to enable incoming email forwarding.
forward_to: (optional) string The address email will be forwarded to.
action: string (optional) The action to perform after forwarding
an email (ACTION_KEEP, ACTION_ARCHIVE, ACTION_DELETE).
kwargs: The other parameters to pass to the update method.
Returns:
gdata.apps.emailsettings.data.EmailSettingsForwarding of the
updated resource
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_FORWARDING)
new_forwarding = gdata.apps.emailsettings.data.EmailSettingsForwarding(
uri=uri, enable=enable, forward_to=forward_to, action=action)
return self.update(new_forwarding, **kwargs)
UpdateForwarding = update_forwarding
def retrieve_forwarding(self, username, **kwargs):
"""Retrieves forwarding settings for the specified username
Args:
username: string The name of the user to get the forwarding settings for
Returns:
A gdata.data.GDEntry of the user's email forwarding settings
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_FORWARDING)
return self.GetEntry(uri, auth_token=None, query=None, **kwargs)
RetrieveForwarding = retrieve_forwarding
def update_pop(self, username, enable, enable_for=None, action=None,
**kwargs):
"""Update Google Mail POP settings.
Args:
username: string The name of the user.
enable: Boolean Whether to enable incoming POP3 access.
enable_for: string (optional) Whether to enable POP3 for all mail
(POP_ENABLE_FOR_ALL_MAIL), or mail from now on
(POP_ENABLE_FOR_MAIL_FROM_NOW_ON).
action: string (optional) What Google Mail should do with its copy
of the email after it is retrieved using POP (ACTION_KEEP,
ACTION_ARCHIVE, ACTION_DELETE).
kwargs: The other parameters to pass to the update method.
Returns:
gdata.apps.emailsettings.data.EmailSettingsPop of the updated resource.
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_POP)
new_pop = gdata.apps.emailsettings.data.EmailSettingsPop(
uri=uri, enable=enable,
enable_for=enable_for, action=action)
return self.update(new_pop, **kwargs)
UpdatePop = update_pop
def retrieve_pop(self, username, **kwargs):
"""Retrieves POP settings for the specified username
Args:
username: string The name of the user to get the POP settings for
Returns:
A gdata.data.GDEntry of the user's POP settings
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_POP)
return self.GetEntry(uri, auth_token=None, query=None, **kwargs)
RetrievePop = retrieve_pop
def update_imap(self, username, enable, **kwargs):
"""Update Google Mail IMAP settings.
Args:
username: string The name of the user.
enable: Boolean Whether to enable IMAP access.language
kwargs: The other parameters to pass to the update method.
Returns:
gdata.apps.emailsettings.data.EmailSettingsImap of the updated resource.
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_IMAP)
new_imap = gdata.apps.emailsettings.data.EmailSettingsImap(
uri=uri, enable=enable)
return self.update(new_imap, **kwargs)
UpdateImap = update_imap
def retrieve_imap(self, username, **kwargs):
"""Retrieves imap settings for the specified username
Args:
username: string The name of the user to get the imap settings for
Returns:
A gdata.data.GDEntry of the user's IMAP settings
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_IMAP)
return self.GetEntry(uri, auth_token=None, query=None, **kwargs)
RetrieveImap = retrieve_imap
def update_vacation(self, username, enable, subject=None, message=None,
contacts_only=None, **kwargs):
"""Update Google Mail vacation-responder settings.
Args:
username: string The name of the user.
enable: Boolean Whether to enable the vacation responder.
subject: string (optional) The subject line of the vacation responder
autoresponse.
message: string (optional) The message body of the vacation responder
autoresponse.
contacts_only: Boolean (optional) Whether to only send autoresponses
to known contacts.
kwargs: The other parameters to pass to the update method.
Returns:
gdata.apps.emailsettings.data.EmailSettingsVacationResponder of the
updated resource.
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_VACATION_RESPONDER)
new_vacation = gdata.apps.emailsettings.data.EmailSettingsVacationResponder(
uri=uri, enable=enable, subject=subject,
message=message, contacts_only=contacts_only)
return self.update(new_vacation, **kwargs)
UpdateVacation = update_vacation
def retrieve_vacation(self, username, **kwargs):
"""Retrieves vacation settings for the specified username
Args:
username: string The name of the user to get the vacation settings for
Returns:
A gdata.data.GDEntry of the user's vacation auto-responder settings
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_VACATION_RESPONDER)
return self.GetEntry(uri, auth_token=None, query=None, **kwargs)
RetrieveVacation = retrieve_vacation
def update_signature(self, username, signature, **kwargs):
"""Update Google Mail signature.
Args:
username: string The name of the user.
signature: string The signature to be appended to outgoing messages.
kwargs: The other parameters to pass to the update method.
Returns:
gdata.apps.emailsettings.data.EmailSettingsSignature of the
updated resource.
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_SIGNATURE)
new_signature = gdata.apps.emailsettings.data.EmailSettingsSignature(
uri=uri, signature=signature)
return self.update(new_signature, **kwargs)
UpdateSignature = update_signature
def retrieve_signature(self, username, **kwargs):
"""Retrieves signature settings for the specified username
Args:
username: string The name of the user to get the signature settings for
Returns:
A gdata.data.GDEntry of the user's signature settings
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_SIGNATURE)
return self.GetEntry(uri, auth_token=None, query=None, **kwargs)
RetrieveSignature = retrieve_signature
def update_language(self, username, language, **kwargs):
"""Update Google Mail language settings.
Args:
username: string The name of the user.
language: string The language tag for Google Mail's display language.
kwargs: The other parameters to pass to the update method.
Returns:
gdata.apps.emailsettings.data.EmailSettingsLanguage of the
updated resource.
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_LANGUAGE)
new_language = gdata.apps.emailsettings.data.EmailSettingsLanguage(
uri=uri, language=language)
return self.update(new_language, **kwargs)
UpdateLanguage = update_language
def update_general_settings(self, username, page_size=None, shortcuts=None,
arrows=None, snippets=None, use_unicode=None,
**kwargs):
"""Update Google Mail general settings.
Args:
username: string The name of the user.
page_size: int (optional) The number of conversations to be shown per
page.
shortcuts: Boolean (optional) Whether to enable keyboard shortcuts.
arrows: Boolean (optional) Whether to display arrow-shaped personal
indicators next to email sent specifically to the user.
snippets: Boolean (optional) Whether to display snippets of the messages
in the inbox and when searching.
use_unicode: Boolean (optional) Whether to use UTF-8 (unicode) encoding
for all outgoing messages.
kwargs: The other parameters to pass to the update method.
Returns:
gdata.apps.emailsettings.data.EmailSettingsGeneral of the
updated resource.
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_GENERAL)
new_general = gdata.apps.emailsettings.data.EmailSettingsGeneral(
uri=uri, page_size=page_size, shortcuts=shortcuts,
arrows=arrows, snippets=snippets, use_unicode=use_unicode)
return self.update(new_general, **kwargs)
UpdateGeneralSettings = update_general_settings
def add_email_delegate(self, username, address, **kwargs):
"""Add an email delegate to the mail account
Args:
username: string The name of the user
address: string The email address of the delegated account
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_DELEGATION)
new_delegation = gdata.apps.emailsettings.data.EmailSettingsDelegation(
uri=uri, address=address)
return self.post(new_delegation, uri, **kwargs)
AddEmailDelegate = add_email_delegate
def retrieve_email_delegates(self, username, **kwargs):
"""Retrieve a feed of the email delegates for the specified username
Args:
username: string The name of the user to get the email delegates for
Returns:
A gdata.data.GDFeed of the user's email delegates
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_DELEGATION)
return self.GetFeed(uri, auth_token=None, query=None, **kwargs)
RetrieveEmailDelegates = retrieve_email_delegates
def delete_email_delegate(self, username, address, **kwargs):
"""Delete an email delegate from the specified account
Args:
username: string The name of the user
address: string The email address of the delegated account
"""
uri = self.MakeEmailSettingsUri(username=username,
setting_id=SETTING_ID_DELEGATION)
uri = uri + '/' + address
return self.delete(uri, **kwargs)
DeleteEmailDelegate = delete_email_delegate

1174
gdata/analytics/apps/emailsettings/data.py Normal file
View File

File diff suppressed because it is too large Load Diff

264
gdata/analytics/apps/emailsettings/service.py Normal file
View File

@ -0,0 +1,264 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Allow Google Apps domain administrators to set users' email settings.
EmailSettingsService: Set various email settings.
"""
__author__ = 'google-apps-apis@googlegroups.com'
import gdata.apps
import gdata.apps.service
import gdata.service
API_VER='2.0'
# Forwarding and POP3 options
KEEP='KEEP'
ARCHIVE='ARCHIVE'
DELETE='DELETE'
ALL_MAIL='ALL_MAIL'
MAIL_FROM_NOW_ON='MAIL_FROM_NOW_ON'
class EmailSettingsService(gdata.apps.service.PropertyService):
"""Client for the Google Apps Email Settings service."""
def _serviceUrl(self, setting_id, username, domain=None):
if domain is None:
domain = self.domain
return '/a/feeds/emailsettings/%s/%s/%s/%s' % (API_VER, domain, username,
setting_id)
def CreateLabel(self, username, label):
"""Create a label.
Args:
username: User to create label for.
label: Label to create.
Returns:
A dict containing the result of the create operation.
"""
uri = self._serviceUrl('label', username)
properties = {'label': label}
return self._PostProperties(uri, properties)
def CreateFilter(self, username, from_=None, to=None, subject=None,
has_the_word=None, does_not_have_the_word=None,
has_attachment=None, label=None, should_mark_as_read=None,
should_archive=None):
"""Create a filter.
Args:
username: User to create filter for.
from_: Filter from string.
to: Filter to string.
subject: Filter subject.
has_the_word: Words to filter in.
does_not_have_the_word: Words to filter out.
has_attachment: Boolean for message having attachment.
label: Label to apply.
should_mark_as_read: Boolean for marking message as read.
should_archive: Boolean for archiving message.
Returns:
A dict containing the result of the create operation.
"""
uri = self._serviceUrl('filter', username)
properties = {}
properties['from'] = from_
properties['to'] = to
properties['subject'] = subject
properties['hasTheWord'] = has_the_word
properties['doesNotHaveTheWord'] = does_not_have_the_word
properties['hasAttachment'] = gdata.apps.service._bool2str(has_attachment)
properties['label'] = label
properties['shouldMarkAsRead'] = gdata.apps.service._bool2str(should_mark_as_read)
properties['shouldArchive'] = gdata.apps.service._bool2str(should_archive)
return self._PostProperties(uri, properties)
def CreateSendAsAlias(self, username, name, address, reply_to=None,
make_default=None):
"""Create alias to send mail as.
Args:
username: User to create alias for.
name: Name of alias.
address: Email address to send from.
reply_to: Email address to reply to.
make_default: Boolean for whether this is the new default sending alias.
Returns:
A dict containing the result of the create operation.
"""
uri = self._serviceUrl('sendas', username)
properties = {}
properties['name'] = name
properties['address'] = address
properties['replyTo'] = reply_to
properties['makeDefault'] = gdata.apps.service._bool2str(make_default)
return self._PostProperties(uri, properties)
def UpdateWebClipSettings(self, username, enable):
"""Update WebClip Settings
Args:
username: User to update forwarding for.
enable: Boolean whether to enable Web Clip.
Returns:
A dict containing the result of the update operation.
"""
uri = self._serviceUrl('webclip', username)
properties = {}
properties['enable'] = gdata.apps.service._bool2str(enable)
return self._PutProperties(uri, properties)
def UpdateForwarding(self, username, enable, forward_to=None, action=None):
"""Update forwarding settings.
Args:
username: User to update forwarding for.
enable: Boolean whether to enable this forwarding rule.
forward_to: Email address to forward to.
action: Action to take after forwarding.
Returns:
A dict containing the result of the update operation.
"""
uri = self._serviceUrl('forwarding', username)
properties = {}
properties['enable'] = gdata.apps.service._bool2str(enable)
if enable is True:
properties['forwardTo'] = forward_to
properties['action'] = action
return self._PutProperties(uri, properties)
def UpdatePop(self, username, enable, enable_for=None, action=None):
"""Update POP3 settings.
Args:
username: User to update POP3 settings for.
enable: Boolean whether to enable POP3.
enable_for: Which messages to make available via POP3.
action: Action to take after user retrieves email via POP3.
Returns:
A dict containing the result of the update operation.
"""
uri = self._serviceUrl('pop', username)
properties = {}
properties['enable'] = gdata.apps.service._bool2str(enable)
if enable is True:
properties['enableFor'] = enable_for
properties['action'] = action
return self._PutProperties(uri, properties)
def UpdateImap(self, username, enable):
"""Update IMAP settings.
Args:
username: User to update IMAP settings for.
enable: Boolean whether to enable IMAP.
Returns:
A dict containing the result of the update operation.
"""
uri = self._serviceUrl('imap', username)
properties = {'enable': gdata.apps.service._bool2str(enable)}
return self._PutProperties(uri, properties)
def UpdateVacation(self, username, enable, subject=None, message=None,
contacts_only=None):
"""Update vacation settings.
Args:
username: User to update vacation settings for.
enable: Boolean whether to enable vacation responses.
subject: Vacation message subject.
message: Vacation message body.
contacts_only: Boolean whether to send message only to contacts.
Returns:
A dict containing the result of the update operation.
"""
uri = self._serviceUrl('vacation', username)
properties = {}
properties['enable'] = gdata.apps.service._bool2str(enable)
if enable is True:
properties['subject'] = subject
properties['message'] = message
properties['contactsOnly'] = gdata.apps.service._bool2str(contacts_only)
return self._PutProperties(uri, properties)
def UpdateSignature(self, username, signature):
"""Update signature.
Args:
username: User to update signature for.
signature: Signature string.
Returns:
A dict containing the result of the update operation.
"""
uri = self._serviceUrl('signature', username)
properties = {'signature': signature}
return self._PutProperties(uri, properties)
def UpdateLanguage(self, username, language):
"""Update user interface language.
Args:
username: User to update language for.
language: Language code.
Returns:
A dict containing the result of the update operation.
"""
uri = self._serviceUrl('language', username)
properties = {'language': language}
return self._PutProperties(uri, properties)
def UpdateGeneral(self, username, page_size=None, shortcuts=None, arrows=None,
snippets=None, unicode=None):
"""Update general settings.
Args:
username: User to update general settings for.
page_size: Number of messages to show.
shortcuts: Boolean whether shortcuts are enabled.
arrows: Boolean whether arrows are enabled.
snippets: Boolean whether snippets are enabled.
unicode: Wheter unicode is enabled.
Returns:
A dict containing the result of the update operation.
"""
uri = self._serviceUrl('general', username)
properties = {}
if page_size != None:
properties['pageSize'] = str(page_size)
if shortcuts != None:
properties['shortcuts'] = gdata.apps.service._bool2str(shortcuts)
if arrows != None:
properties['arrows'] = gdata.apps.service._bool2str(arrows)
if snippets != None:
properties['snippets'] = gdata.apps.service._bool2str(snippets)
if unicode != None:
properties['unicode'] = gdata.apps.service._bool2str(unicode)
return self._PutProperties(uri, properties)

0
gdata/analytics/apps/groups/__init__.py Normal file
View File

387
gdata/analytics/apps/groups/service.py Normal file
View File

@ -0,0 +1,387 @@
#!/usr/bin/python
#
# Copyright (C) 2008 Google, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Allow Google Apps domain administrators to manage groups, group members and group owners.
GroupsService: Provides methods to manage groups, members and owners.
"""
__author__ = 'google-apps-apis@googlegroups.com'
import urllib
import gdata.apps
import gdata.apps.service
import gdata.service
API_VER = '2.0'
BASE_URL = '/a/feeds/group/' + API_VER + '/%s'
GROUP_MEMBER_URL = BASE_URL + '?member=%s'
GROUP_MEMBER_DIRECT_URL = GROUP_MEMBER_URL + '&directOnly=%s'
GROUP_ID_URL = BASE_URL + '/%s'
MEMBER_URL = BASE_URL + '/%s/member'
MEMBER_WITH_SUSPENDED_URL = MEMBER_URL + '?includeSuspendedUsers=%s'
MEMBER_ID_URL = MEMBER_URL + '/%s'
OWNER_URL = BASE_URL + '/%s/owner'
OWNER_WITH_SUSPENDED_URL = OWNER_URL + '?includeSuspendedUsers=%s'
OWNER_ID_URL = OWNER_URL + '/%s'
PERMISSION_OWNER = 'Owner'
PERMISSION_MEMBER = 'Member'
PERMISSION_DOMAIN = 'Domain'
PERMISSION_ANYONE = 'Anyone'
class GroupsService(gdata.apps.service.PropertyService):
"""Client for the Google Apps Groups service."""
def _ServiceUrl(self, service_type, is_existed, group_id, member_id, owner_email,
direct_only=False, domain=None, suspended_users=False):
if domain is None:
domain = self.domain
if service_type == 'group':
if group_id != '' and is_existed:
return GROUP_ID_URL % (domain, group_id)
elif member_id != '':
if direct_only:
return GROUP_MEMBER_DIRECT_URL % (domain, urllib.quote_plus(member_id),
self._Bool2Str(direct_only))
else:
return GROUP_MEMBER_URL % (domain, urllib.quote_plus(member_id))
else:
return BASE_URL % (domain)
if service_type == 'member':
if member_id != '' and is_existed:
return MEMBER_ID_URL % (domain, group_id, urllib.quote_plus(member_id))
elif suspended_users:
return MEMBER_WITH_SUSPENDED_URL % (domain, group_id,
self._Bool2Str(suspended_users))
else:
return MEMBER_URL % (domain, group_id)
if service_type == 'owner':
if owner_email != '' and is_existed:
return OWNER_ID_URL % (domain, group_id, urllib.quote_plus(owner_email))
elif suspended_users:
return OWNER_WITH_SUSPENDED_URL % (domain, group_id,
self._Bool2Str(suspended_users))
else:
return OWNER_URL % (domain, group_id)
def _Bool2Str(self, b):
if b is None:
return None
return str(b is True).lower()
def _IsExisted(self, uri):
try:
self._GetProperties(uri)
return True
except gdata.apps.service.AppsForYourDomainException, e:
if e.error_code == gdata.apps.service.ENTITY_DOES_NOT_EXIST:
return False
else:
raise e
def CreateGroup(self, group_id, group_name, description, email_permission):
"""Create a group.
Args:
group_id: The ID of the group (e.g. us-sales).
group_name: The name of the group.
description: A description of the group
email_permission: The subscription permission of the group.
Returns:
A dict containing the result of the create operation.
"""
uri = self._ServiceUrl('group', False, group_id, '', '')
properties = {}
properties['groupId'] = group_id
properties['groupName'] = group_name
properties['description'] = description
properties['emailPermission'] = email_permission
return self._PostProperties(uri, properties)
def UpdateGroup(self, group_id, group_name, description, email_permission):
"""Update a group's name, description and/or permission.
Args:
group_id: The ID of the group (e.g. us-sales).
group_name: The name of the group.
description: A description of the group
email_permission: The subscription permission of the group.
Returns:
A dict containing the result of the update operation.
"""
uri = self._ServiceUrl('group', True, group_id, '', '')
properties = {}
properties['groupId'] = group_id
properties['groupName'] = group_name
properties['description'] = description
properties['emailPermission'] = email_permission
return self._PutProperties(uri, properties)
def RetrieveGroup(self, group_id):
"""Retrieve a group based on its ID.
Args:
group_id: The ID of the group (e.g. us-sales).
Returns:
A dict containing the result of the retrieve operation.
"""
uri = self._ServiceUrl('group', True, group_id, '', '')
return self._GetProperties(uri)
def RetrieveAllGroups(self):
"""Retrieve all groups in the domain.
Args:
None
Returns:
A list containing the result of the retrieve operation.
"""
uri = self._ServiceUrl('group', True, '', '', '')
return self._GetPropertiesList(uri)
def RetrievePageOfGroups(self, start_group=None):
"""Retrieve one page of groups in the domain.
Args:
start_group: The key to continue for pagination through all groups.
Returns:
A feed object containing the result of the retrieve operation.
"""
uri = self._ServiceUrl('group', True, '', '', '')
if start_group is not None:
uri += "?start="+start_group
property_feed = self._GetPropertyFeed(uri)
return property_feed
def RetrieveGroups(self, member_id, direct_only=False):
"""Retrieve all groups that belong to the given member_id.
Args:
member_id: The member's email address (e.g. member@example.com).
direct_only: Boolean whether only return groups that this member directly belongs to.
Returns:
A list containing the result of the retrieve operation.
"""
uri = self._ServiceUrl('group', True, '', member_id, '', direct_only=direct_only)
return self._GetPropertiesList(uri)
def DeleteGroup(self, group_id):
"""Delete a group based on its ID.
Args:
group_id: The ID of the group (e.g. us-sales).
Returns:
A dict containing the result of the delete operation.
"""
uri = self._ServiceUrl('group', True, group_id, '', '')
return self._DeleteProperties(uri)
def AddMemberToGroup(self, member_id, group_id):
"""Add a member to a group.
Args:
member_id: The member's email address (e.g. member@example.com).
group_id: The ID of the group (e.g. us-sales).
Returns:
A dict containing the result of the add operation.
"""
uri = self._ServiceUrl('member', False, group_id, member_id, '')
properties = {}
properties['memberId'] = member_id
return self._PostProperties(uri, properties)
def IsMember(self, member_id, group_id):
"""Check whether the given member already exists in the given group.
Args:
member_id: The member's email address (e.g. member@example.com).
group_id: The ID of the group (e.g. us-sales).
Returns:
True if the member exists in the group. False otherwise.
"""
uri = self._ServiceUrl('member', True, group_id, member_id, '')
return self._IsExisted(uri)
def RetrieveMember(self, member_id, group_id):
"""Retrieve the given member in the given group.
Args:
member_id: The member's email address (e.g. member@example.com).
group_id: The ID of the group (e.g. us-sales).
Returns:
A dict containing the result of the retrieve operation.
"""
uri = self._ServiceUrl('member', True, group_id, member_id, '')
return self._GetProperties(uri)
def RetrieveAllMembers(self, group_id, suspended_users=False):
"""Retrieve all members in the given group.
Args:
group_id: The ID of the group (e.g. us-sales).
suspended_users: A boolean; should we include any suspended users in
the membership list returned?
Returns:
A list containing the result of the retrieve operation.
"""
uri = self._ServiceUrl('member', True, group_id, '', '',
suspended_users=suspended_users)
return self._GetPropertiesList(uri)
def RetrievePageOfMembers(self, group_id, suspended_users=False, start=None):
"""Retrieve one page of members of a given group.
Args:
group_id: The ID of the group (e.g. us-sales).
suspended_users: A boolean; should we include any suspended users in
the membership list returned?
start: The key to continue for pagination through all members.
Returns:
A feed object containing the result of the retrieve operation.
"""
uri = self._ServiceUrl('member', True, group_id, '', '',
suspended_users=suspended_users)
if start is not None:
if suspended_users:
uri += "&start="+start
else:
uri += "?start="+start
property_feed = self._GetPropertyFeed(uri)
return property_feed
def RemoveMemberFromGroup(self, member_id, group_id):
"""Remove the given member from the given group.
Args:
member_id: The member's email address (e.g. member@example.com).
group_id: The ID of the group (e.g. us-sales).
Returns:
A dict containing the result of the remove operation.
"""
uri = self._ServiceUrl('member', True, group_id, member_id, '')
return self._DeleteProperties(uri)
def AddOwnerToGroup(self, owner_email, group_id):
"""Add an owner to a group.
Args:
owner_email: The email address of a group owner.
group_id: The ID of the group (e.g. us-sales).
Returns:
A dict containing the result of the add operation.
"""
uri = self._ServiceUrl('owner', False, group_id, '', owner_email)
properties = {}
properties['email'] = owner_email
return self._PostProperties(uri, properties)
def IsOwner(self, owner_email, group_id):
"""Check whether the given member an owner of the given group.
Args:
owner_email: The email address of a group owner.
group_id: The ID of the group (e.g. us-sales).
Returns:
True if the member is an owner of the given group. False otherwise.
"""
uri = self._ServiceUrl('owner', True, group_id, '', owner_email)
return self._IsExisted(uri)
def RetrieveOwner(self, owner_email, group_id):
"""Retrieve the given owner in the given group.
Args:
owner_email: The email address of a group owner.
group_id: The ID of the group (e.g. us-sales).
Returns:
A dict containing the result of the retrieve operation.
"""
uri = self._ServiceUrl('owner', True, group_id, '', owner_email)
return self._GetProperties(uri)
def RetrieveAllOwners(self, group_id, suspended_users=False):
"""Retrieve all owners of the given group.
Args:
group_id: The ID of the group (e.g. us-sales).
suspended_users: A boolean; should we include any suspended users in
the ownership list returned?
Returns:
A list containing the result of the retrieve operation.
"""
uri = self._ServiceUrl('owner', True, group_id, '', '',
suspended_users=suspended_users)
return self._GetPropertiesList(uri)
def RetrievePageOfOwners(self, group_id, suspended_users=False, start=None):
"""Retrieve one page of owners of the given group.
Args:
group_id: The ID of the group (e.g. us-sales).
suspended_users: A boolean; should we include any suspended users in
the ownership list returned?
start: The key to continue for pagination through all owners.
Returns:
A feed object containing the result of the retrieve operation.
"""
uri = self._ServiceUrl('owner', True, group_id, '', '',
suspended_users=suspended_users)
if start is not None:
if suspended_users:
uri += "&start="+start
else:
uri += "?start="+start
property_feed = self._GetPropertyFeed(uri)
return property_feed
def RemoveOwnerFromGroup(self, owner_email, group_id):
"""Remove the given owner from the given group.
Args:
owner_email: The email address of a group owner.
group_id: The ID of the group (e.g. us-sales).
Returns:
A dict containing the result of the remove operation.
"""
uri = self._ServiceUrl('owner', True, group_id, '', owner_email)
return self._DeleteProperties(uri)

223
gdata/analytics/apps/migration/__init__.py Normal file
View File

@ -0,0 +1,223 @@
#!/usr/bin/python2.4
#
# Copyright 2008 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Contains objects used with Google Apps."""
__author__ = 'google-apps-apis@googlegroups.com'
import atom
import gdata
# XML namespaces which are often used in Google Apps entity.
APPS_NAMESPACE = 'http://schemas.google.com/apps/2006'
APPS_TEMPLATE = '{http://schemas.google.com/apps/2006}%s'
class Rfc822Msg(atom.AtomBase):
"""The Migration rfc822Msg element."""
_tag = 'rfc822Msg'
_namespace = APPS_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['encoding'] = 'encoding'
def __init__(self, extension_elements=None,
extension_attributes=None, text=None):
self.text = text
self.encoding = 'base64'
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def Rfc822MsgFromString(xml_string):
"""Parse in the Rrc822 message from the XML definition."""
return atom.CreateClassFromXMLString(Rfc822Msg, xml_string)
class MailItemProperty(atom.AtomBase):
"""The Migration mailItemProperty element."""
_tag = 'mailItemProperty'
_namespace = APPS_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['value'] = 'value'
def __init__(self, value=None, extension_elements=None,
extension_attributes=None, text=None):
self.value = value
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def MailItemPropertyFromString(xml_string):
"""Parse in the MailItemProperiy from the XML definition."""
return atom.CreateClassFromXMLString(MailItemProperty, xml_string)
class Label(atom.AtomBase):
"""The Migration label element."""
_tag = 'label'
_namespace = APPS_NAMESPACE
_children = atom.AtomBase._children.copy()
_attributes = atom.AtomBase._attributes.copy()
_attributes['labelName'] = 'label_name'
def __init__(self, label_name=None,
extension_elements=None, extension_attributes=None,
text=None):
self.label_name = label_name
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def LabelFromString(xml_string):
"""Parse in the mailItemProperty from the XML definition."""
return atom.CreateClassFromXMLString(Label, xml_string)
class MailEntry(gdata.GDataEntry):
"""A Google Migration flavor of an Atom Entry."""
_tag = 'entry'
_namespace = atom.ATOM_NAMESPACE
_children = gdata.GDataEntry._children.copy()
_attributes = gdata.GDataEntry._attributes.copy()
_children['{%s}rfc822Msg' % APPS_NAMESPACE] = ('rfc822_msg', Rfc822Msg)
_children['{%s}mailItemProperty' % APPS_NAMESPACE] = ('mail_item_property',
[MailItemProperty])
_children['{%s}label' % APPS_NAMESPACE] = ('label', [Label])
def __init__(self, author=None, category=None, content=None,
atom_id=None, link=None, published=None,
title=None, updated=None,
rfc822_msg=None, mail_item_property=None, label=None,
extended_property=None,
extension_elements=None, extension_attributes=None, text=None):
gdata.GDataEntry.__init__(self, author=author, category=category,
content=content,
atom_id=atom_id, link=link, published=published,
title=title, updated=updated)
self.rfc822_msg = rfc822_msg
self.mail_item_property = mail_item_property
self.label = label
self.extended_property = extended_property or []
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def MailEntryFromString(xml_string):
"""Parse in the MailEntry from the XML definition."""
return atom.CreateClassFromXMLString(MailEntry, xml_string)
class BatchMailEntry(gdata.BatchEntry):
"""A Google Migration flavor of an Atom Entry."""
_tag = gdata.BatchEntry._tag
_namespace = gdata.BatchEntry._namespace
_children = gdata.BatchEntry._children.copy()
_attributes = gdata.BatchEntry._attributes.copy()
_children['{%s}rfc822Msg' % APPS_NAMESPACE] = ('rfc822_msg', Rfc822Msg)
_children['{%s}mailItemProperty' % APPS_NAMESPACE] = ('mail_item_property',
[MailItemProperty])
_children['{%s}label' % APPS_NAMESPACE] = ('label', [Label])
def __init__(self, author=None, category=None, content=None,
atom_id=None, link=None, published=None,
title=None, updated=None,
rfc822_msg=None, mail_item_property=None, label=None,
batch_operation=None, batch_id=None, batch_status=None,
extended_property=None,
extension_elements=None, extension_attributes=None, text=None):
gdata.BatchEntry.__init__(self, author=author, category=category,
content=content,
atom_id=atom_id, link=link, published=published,
batch_operation=batch_operation,
batch_id=batch_id, batch_status=batch_status,
title=title, updated=updated)
self.rfc822_msg = rfc822_msg or None
self.mail_item_property = mail_item_property or []
self.label = label or []
self.extended_property = extended_property or []
self.text = text
self.extension_elements = extension_elements or []
self.extension_attributes = extension_attributes or {}
def BatchMailEntryFromString(xml_string):
"""Parse in the BatchMailEntry from the XML definition."""
return atom.CreateClassFromXMLString(BatchMailEntry, xml_string)
class BatchMailEventFeed(gdata.BatchFeed):
"""A Migration event feed flavor of an Atom Feed."""
_tag = gdata.BatchFeed._tag
_namespace = gdata.BatchFeed._namespace
_children = gdata.BatchFeed._children.copy()
_attributes = gdata.BatchFeed._attributes.copy()
_children['{%s}entry' % atom.ATOM_NAMESPACE] = ('entry', [BatchMailEntry])
def __init__(self, author=None, category=None, contributor=None,
generator=None, icon=None, atom_id=None, link=None, logo=None,
rights=None, subtitle=None, title=None, updated=None,
entry=None, total_results=None, start_index=None,
items_per_page=None, interrupted=None, extension_elements=None,
extension_attributes=None, text=None):
gdata.BatchFeed.__init__(self, author=author, category=category,
contributor=contributor, generator=generator,
icon=icon, atom_id=atom_id, link=link,
logo=logo, rights=rights, subtitle=subtitle,
title=title, updated=updated, entry=entry,
total_results=total_results,
start_index=start_index,
items_per_page=items_per_page,
interrupted=interrupted,
extension_elements=extension_elements,
extension_attributes=extension_attributes,
text=text)
class MailEntryProperties(object):
"""Represents a mail message and its attributes."""
def __init__(self, mail_message=None, mail_item_properties=None,
mail_labels=None, identifier=None):
self.mail_message = mail_message
self.mail_item_properties = mail_item_properties or []
self.mail_labels = mail_labels or []
self.identifier = identifier
def BatchMailEventFeedFromString(xml_string):
"""Parse in the BatchMailEventFeed from the XML definition."""
return atom.CreateClassFromXMLString(BatchMailEventFeed, xml_string)

218
gdata/analytics/apps/migration/service.py Normal file
View File

@ -0,0 +1,218 @@
#!/usr/bin/python2.4
#
# Copyright 2008 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Contains the methods to import mail via Google Apps Email Migration API.
MigrationService: Provides methods to import mail.
"""
__author__ = ('google-apps-apis@googlegroups.com',
'pti@google.com (Prashant Tiwari)')
import base64
import threading
import time
from atom.service import deprecation
from gdata.apps import migration
from gdata.apps.migration import MailEntryProperties
import gdata.apps.service
import gdata.service
API_VER = '2.0'
class MigrationService(gdata.apps.service.AppsService):
"""Client for the EMAPI migration service. Use either ImportMail to import
one message at a time, or AddMailEntry and ImportMultipleMails to import a
bunch of messages at a time.
"""
def __init__(self, email=None, password=None, domain=None, source=None,
server='apps-apis.google.com', additional_headers=None):
gdata.apps.service.AppsService.__init__(
self, email=email, password=password, domain=domain, source=source,
server=server, additional_headers=additional_headers)
self.mail_batch = migration.BatchMailEventFeed()
self.mail_entries = []
self.exceptions = 0
def _BaseURL(self):
return '/a/feeds/migration/%s/%s' % (API_VER, self.domain)
def ImportMail(self, user_name, mail_message, mail_item_properties,
mail_labels):
"""Imports a single mail message.
Args:
user_name: The username to import messages to.
mail_message: An RFC822 format email message.
mail_item_properties: A list of Gmail properties to apply to the message.
mail_labels: A list of labels to apply to the message.
Returns:
A MailEntry representing the successfully imported message.
Raises:
AppsForYourDomainException: An error occurred importing the message.
"""
uri = '%s/%s/mail' % (self._BaseURL(), user_name)
mail_entry = migration.MailEntry()
mail_entry.rfc822_msg = migration.Rfc822Msg(text=(base64.b64encode(
mail_message)))
mail_entry.rfc822_msg.encoding = 'base64'
mail_entry.mail_item_property = map(
lambda x: migration.MailItemProperty(value=x), mail_item_properties)
mail_entry.label = map(lambda x: migration.Label(label_name=x),
mail_labels)
try:
return migration.MailEntryFromString(str(self.Post(mail_entry, uri)))
except gdata.service.RequestError, e:
# Store the number of failed imports when importing several at a time
self.exceptions += 1
raise gdata.apps.service.AppsForYourDomainException(e.args[0])
def AddBatchEntry(self, mail_message, mail_item_properties,
mail_labels):
"""Adds a message to the current batch that you later will submit.
Deprecated, use AddMailEntry instead
Args:
mail_message: An RFC822 format email message.
mail_item_properties: A list of Gmail properties to apply to the message.
mail_labels: A list of labels to apply to the message.
Returns:
The length of the MailEntry representing the message.
"""
deprecation("calling deprecated method AddBatchEntry")
mail_entry = migration.BatchMailEntry()
mail_entry.rfc822_msg = migration.Rfc822Msg(text=(base64.b64encode(
mail_message)))
mail_entry.rfc822_msg.encoding = 'base64'
mail_entry.mail_item_property = map(
lambda x: migration.MailItemProperty(value=x), mail_item_properties)
mail_entry.label = map(lambda x: migration.Label(label_name=x),
mail_labels)
self.mail_batch.AddBatchEntry(mail_entry)
return len(str(mail_entry))
def SubmitBatch(self, user_name):
"""Sends all the mail items you have added to the batch to the server.
Deprecated, use ImportMultipleMails instead
Args:
user_name: The username to import messages to.
Returns:
An HTTPResponse from the web service call.
Raises:
AppsForYourDomainException: An error occurred importing the batch.
"""
deprecation("calling deprecated method SubmitBatch")
uri = '%s/%s/mail/batch' % (self._BaseURL(), user_name)
try:
self.result = self.Post(self.mail_batch, uri,
converter=migration.BatchMailEventFeedFromString)
except gdata.service.RequestError, e:
raise gdata.apps.service.AppsForYourDomainException(e.args[0])
self.mail_batch = migration.BatchMailEventFeed()
return self.result
def AddMailEntry(self, mail_message, mail_item_properties=None,
mail_labels=None, identifier=None):
"""Prepares a list of mail messages to import using ImportMultipleMails.
Args:
mail_message: An RFC822 format email message as a string.
mail_item_properties: List of Gmail properties to apply to the
message.
mail_labels: List of Gmail labels to apply to the message.
identifier: The optional file identifier string
Returns:
The number of email messages to be imported.
"""
mail_entry_properties = MailEntryProperties(
mail_message=mail_message,
mail_item_properties=mail_item_properties,
mail_labels=mail_labels,
identifier=identifier)
self.mail_entries.append(mail_entry_properties)
return len(self.mail_entries)
def ImportMultipleMails(self, user_name, threads_per_batch=20):
"""Launches separate threads to import every message added by AddMailEntry.
Args:
user_name: The user account name to import messages to.
threads_per_batch: Number of messages to import at a time.
Returns:
The number of email messages that were successfully migrated.
Raises:
Exception: An error occurred while importing mails.
"""
num_entries = len(self.mail_entries)
if not num_entries:
return 0
threads = []
for mail_entry_properties in self.mail_entries:
t = threading.Thread(name=mail_entry_properties.identifier,
target=self.ImportMail,
args=(user_name, mail_entry_properties.mail_message,
mail_entry_properties.mail_item_properties,
mail_entry_properties.mail_labels))
threads.append(t)
try:
# Determine the number of batches needed with threads_per_batch in each
batches = num_entries / threads_per_batch + (
0 if num_entries % threads_per_batch == 0 else 1)
batch_min = 0
# Start the threads, one batch at a time
for batch in range(batches):
batch_max = ((batch + 1) * threads_per_batch
if (batch + 1) * threads_per_batch < num_entries
else num_entries)
for i in range(batch_min, batch_max):
threads[i].start()
time.sleep(1)
for i in range(batch_min, batch_max):
threads[i].join()
batch_min = batch_max
self.mail_entries = []
except Exception, e:
raise Exception(e.args[0])
else:
return num_entries - self.exceptions

Some files were not shown because too many files have changed in this diff Show More