Domains and Data Transfer APIs added

"updating user..." should be in stdout, not stderr

.gitignore

@@ -64,3 +64,7 @@ nobrowser.txt
 nocache.txt
 noverifyssl.txt
 gamcache/
+gam/
+gam-64/
+gam.spec
+*.zip

README.md

@@ -1,7 +1,6 @@
-GAM, the Google Apps Manager
+Dito GAM
 ============================
-
-Dito GAM is a free, open source command line tool for
+GAM is a free, open source command line tool for
 Google Apps Administrators to efficiently manage
 domain and user settings quickly and easily. GAM has support
 for many features, such as
@@ -18,68 +17,50 @@ for many features, such as
 * generating detailed reports for users, groups, resources,
   account activity, email clients, and quotas.
 
-
-Resources
-========
-
-There are a number of GAM resources available via several different
-websites.
-
-Source Repository
------------------
-
-The official GAM source repository is on [Github][github].
-
-
 Downloads
 ---------
+You can download current GAM from 
+the [GitHub Releases] page.
 
-You can download current and pre-released versions of GAM from 
-the [Github Releases][github releases] page.  Final releases
-are also available on [Google Drive][google drive]
-
-
-Wiki Documentation
-----
-
-The GAM documentation is currently hosted on Google Code
-and can be found at the [Gam Getting Started Wiki][gam wiki]
+Documentation
+------------------
+The GAM documentation is currently hosted in the [GitHub Wiki]
 
 Mailing List / Discussion group
 -------------------------------
-
 The GAM mailing list / discussion group is hosted
 on [Google Groups].  You can join the list and interact
 via email, or just post from the web itself.
 
+Source Repository
+-----------------
+
+The official GAM source repository is on [GitHub].
 
 Author
-======
+------
 
 GAM is maintained by <a href="mailto:jay0lee@gmail.com">Jay Lee</a>.
 
-THANKS TO
-=========
+Thanks To
+---------
 
 GAM is made possible and maintained by the work of Dito.
 Who is Dito?
 
-Dito is solely focused on moving organizations to Google's
+*Dito is solely focused on moving organizations to Google's
 cloud. After hundreds of successful deployments over the
 last 5 years, we have gained notoriety for our complete
 understanding of the platform, our change management &
 training ability, and our rock-star deployment engineers.
-We are known worldwide as the Google Apps Experts.
+We are known worldwide as the Google Apps Experts.*
 
 Need a Google Apps Expert? 
 [Contact Dito](http://ditoweb.com/contact), which offers
 [free premium GAM support](http://www.ditoweb.com/dito-gam)
 for domains that sign up through Dito.
 
-
-[github releases]: https://github.com/jay0lee/GAM/releases
-[github]: https://github.com/jay0lee/GAM/
-[google code downloads]: https://code.google.com/p/google-apps-manager/wiki/Downloads
-[google drive]: https://googledrive.com/host/0B0YvUuHHn3MnbFl6N0k1UXcwdVk/
-[gam wiki]: https://code.google.com/p/google-apps-manager/wiki/GAM3GettingStarted
-[google groups]: http://groups.google.com/group/google-apps-manager
+[GitHub Releases]: https://github.com/jay0lee/GAM/releases
+[GitHub]: https://github.com/jay0lee/GAM/
+[GitHub Wiki]: https://github.com/jay0lee/GAM/wiki/
+[Google Groups]: http://groups.google.com/group/google-apps-manager

apiclient/push.py

@@ -1,274 +0,0 @@
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Push notifications support.
-
-This code is based on experimental APIs and is subject to change.
-"""
-
-__author__ = 'afshar@google.com (Ali Afshar)'
-
-import binascii
-import collections
-import os
-import urllib
-
-SUBSCRIBE = 'X-GOOG-SUBSCRIBE'
-SUBSCRIPTION_ID = 'X-GOOG-SUBSCRIPTION-ID'
-TOPIC_ID = 'X-GOOG-TOPIC-ID'
-TOPIC_URI = 'X-GOOG-TOPIC-URI'
-CLIENT_TOKEN = 'X-GOOG-CLIENT-TOKEN'
-EVENT_TYPE = 'X-GOOG-EVENT-TYPE'
-UNSUBSCRIBE = 'X-GOOG-UNSUBSCRIBE'
-
-
-class InvalidSubscriptionRequestError(ValueError):
-  """The request cannot be subscribed."""
-
-
-def new_token():
-  """Gets a random token for use as a client_token in push notifications.
-
-  Returns:
-    str, a new random token.
-  """
-  return binascii.hexlify(os.urandom(32))
-
-
-class Channel(object):
-  """Base class for channel types."""
-
-  def __init__(self, channel_type, channel_args):
-    """Create a new Channel.
-
-    You probably won't need to create this channel manually, since there are
-    subclassed Channel for each specific type with a more customized set of
-    arguments to pass. However, you may wish to just create it manually here.
-
-    Args:
-      channel_type: str, the type of channel.
-      channel_args: dict, arguments to pass to the channel.
-    """
-    self.channel_type = channel_type
-    self.channel_args = channel_args
-
-  def as_header_value(self):
-    """Create the appropriate header for this channel.
-
-    Returns:
-      str encoded channel description suitable for use as a header.
-    """
-    return '%s?%s' % (self.channel_type, urllib.urlencode(self.channel_args))
-
-  def write_header(self, headers):
-    """Write the appropriate subscribe header to a headers dict.
-
-    Args:
-      headers: dict, headers to add subscribe header to.
-    """
-    headers[SUBSCRIBE] = self.as_header_value()
-
-
-class WebhookChannel(Channel):
-  """Channel for registering web hook notifications."""
-
-  def __init__(self, url, app_engine=False):
-    """Create a new WebhookChannel
-
-    Args:
-      url: str, URL to post notifications to.
-      app_engine: bool, default=False, whether the destination for the
-      notifications is an App Engine application.
-    """
-    super(WebhookChannel, self).__init__(
-        channel_type='web_hook',
-        channel_args={
-            'url': url,
-            'app_engine': app_engine and 'true' or 'false',
-        }
-    )
-
-
-class Headers(collections.defaultdict):
-  """Headers for managing subscriptions."""
-
-
-  ALL_HEADERS = set([SUBSCRIBE, SUBSCRIPTION_ID, TOPIC_ID, TOPIC_URI,
-                     CLIENT_TOKEN, EVENT_TYPE, UNSUBSCRIBE])
-
-  def __init__(self):
-    """Create a new subscription configuration instance."""
-    collections.defaultdict.__init__(self, str)
-
-  def __setitem__(self, key, value):
-    """Set a header value, ensuring the key is an allowed value.
-
-    Args:
-      key: str, the header key.
-      value: str, the header value.
-    Raises:
-      ValueError if key is not one of the accepted headers.
-    """
-    normal_key = self._normalize_key(key)
-    if normal_key not in self.ALL_HEADERS:
-      raise ValueError('Header name must be one of %s.' % self.ALL_HEADERS)
-    else:
-      return collections.defaultdict.__setitem__(self, normal_key, value)
-
-  def __getitem__(self, key):
-    """Get a header value, normalizing the key case.
-
-    Args:
-      key: str, the header key.
-    Returns:
-      String header value.
-    Raises:
-      KeyError if the key is not one of the accepted headers.
-    """
-    normal_key = self._normalize_key(key)
-    if normal_key not in self.ALL_HEADERS:
-      raise ValueError('Header name must be one of %s.' % self.ALL_HEADERS)
-    else:
-      return collections.defaultdict.__getitem__(self, normal_key)
-
-  def _normalize_key(self, key):
-    """Normalize a header name for use as a key."""
-    return key.upper()
-
-  def items(self):
-    """Generator for each header."""
-    for header in self.ALL_HEADERS:
-      value = self[header]
-      if value:
-        yield header, value
-
-  def write(self, headers):
-    """Applies the subscription headers.
-
-    Args:
-      headers: dict of headers to insert values into.
-    """
-    for header, value in self.items():
-      headers[header.lower()] = value
-
-  def read(self, headers):
-    """Read from headers.
-
-    Args:
-      headers: dict of headers to read from.
-    """
-    for header in self.ALL_HEADERS:
-      if header.lower() in headers:
-        self[header] = headers[header.lower()]
-
-
-class Subscription(object):
-  """Information about a subscription."""
-
-  def __init__(self):
-    """Create a new Subscription."""
-    self.headers = Headers()
-
-  @classmethod
-  def for_request(cls, request, channel, client_token=None):
-    """Creates a subscription and attaches it to a request.
-
-    Args:
-      request: An http.HttpRequest to modify for making a subscription.
-      channel: A apiclient.push.Channel describing the subscription to
-               create.
-      client_token: (optional) client token to verify the notification.
-
-    Returns:
-      New subscription object.
-    """
-    subscription = cls.for_channel(channel=channel, client_token=client_token)
-    subscription.headers.write(request.headers)
-    if request.method != 'GET':
-      raise InvalidSubscriptionRequestError(
-          'Can only subscribe to requests which are GET.')
-    request.method = 'POST'
-
-    def _on_response(response, subscription=subscription):
-      """Called with the response headers. Reads the subscription headers."""
-      subscription.headers.read(response)
-
-    request.add_response_callback(_on_response)
-    return subscription
-
-  @classmethod
-  def for_channel(cls, channel, client_token=None):
-    """Alternate constructor to create a subscription from a channel.
-
-    Args:
-      channel: A apiclient.push.Channel describing the subscription to
-               create.
-      client_token: (optional) client token to verify the notification.
-
-    Returns:
-      New subscription object.
-    """
-    subscription = cls()
-    channel.write_header(subscription.headers)
-    if client_token is None:
-      client_token = new_token()
-    subscription.headers[SUBSCRIPTION_ID] = new_token()
-    subscription.headers[CLIENT_TOKEN] = client_token
-    return subscription
-
-  def verify(self, headers):
-    """Verifies that a webhook notification has the correct client_token.
-
-    Args:
-      headers: dict of request headers for a push notification.
-
-    Returns:
-      Boolean value indicating whether the notification is verified.
-    """
-    new_subscription = Subscription()
-    new_subscription.headers.read(headers)
-    return new_subscription.client_token == self.client_token
-
-  @property
-  def subscribe(self):
-    """Subscribe header value."""
-    return self.headers[SUBSCRIBE]
-
-  @property
-  def subscription_id(self):
-    """Subscription ID header value."""
-    return self.headers[SUBSCRIPTION_ID]
-
-  @property
-  def topic_id(self):
-    """Topic ID header value."""
-    return self.headers[TOPIC_ID]
-
-  @property
-  def topic_uri(self):
-    """Topic URI header value."""
-    return self.headers[TOPIC_URI]
-
-  @property
-  def client_token(self):
-    """Client Token header value."""
-    return self.headers[CLIENT_TOKEN]
-
-  @property
-  def event_type(self):
-    """Event Type header value."""
-    return self.headers[EVENT_TYPE]
-
-  @property
-  def unsubscribe(self):
-    """Unsuscribe header value."""
-    return self.headers[UNSUBSCRIBE]

build.bat

@@ -1,23 +1,21 @@
 rmdir /q /s gam
 rmdir /q /s gam-64
-rmdir /q /s python-src-%1
 rmdir /q /s build
 rmdir /q /s dist
-del /q /f gam-%1-python-src.zip
 del /q /f gam-%1-windows.zip
 del /q /f gam-%1-windows-x64.zip
 
-\python27-32\python.exe setup.py py2exe
+c:\python27-32\scripts\pyinstaller -F --distpath=gam gam.spec
 xcopy LICENSE gam\
 xcopy whatsnew.txt gam\
-xcopy cacert.pem gam\
 xcopy admin-settings-v1.json gam\
+xcopy cloudprint-v2.json gam\
 del gam\w9xpopen.exe
 "%ProgramFiles(x86)%\7-Zip\7z.exe" a -tzip gam-%1-windows.zip gam\ -xr!.svn
 
-\python27\python.exe setup-64.py py2exe
+c:\python27\scripts\pyinstaller -F --distpath=gam-64 gam.spec
 xcopy LICENSE gam-64\
 xcopy whatsnew.txt gam-64\
-xcopy cacert.pem gam-64\
 xcopy admin-settings-v1.json gam-64\
+xcopy cloudprint-v2.json gam-64\
 "%ProgramFiles(x86)%\7-Zip\7z.exe" a -tzip gam-%1-windows-x64.zip gam-64\ -xr!.svn

cloudprint-v2.json

@@ -0,0 +1,486 @@
+{
+ "kind": "discovery#restDescription",
+ "discoveryVersion": "v1",
+ "id": "cloudprint:v2",
+ "name": "cloudprint",
+ "version": "v2",
+ "revision": "20150605",
+ "title": "Cloud Print API",
+ "description": "Lets you access Cloud Print Printers",
+ "ownerDomain": "google.com",
+ "ownerName": "Google",
+ "icons": {
+  "x16": "http://www.google.com/images/icons/product/search-16.gif",
+  "x32": "http://www.google.com/images/icons/product/search-32.gif"
+ },
+ "documentationLink": "https://developers.google.com/cloud-print",
+ "protocol": "rest",
+ "baseUrl": "https://www.google.com/",
+ "basePath": "/cloudprint/",
+ "rootUrl": "https://www.google.com/",
+ "servicePath": "/cloudprint/",
+ "parameters": {
+  "prettyPrint": {
+   "type": "boolean",
+   "description": "Returns response with indentations and line breaks.",
+   "default": "true",
+   "location": "query"
+  }
+ },
+ "auth": {
+  "oauth2": {
+   "scopes": {
+    "https://www.googleapis.com/auth/cloudprint": {
+     "description": "Manage Cloud Print"
+    }
+   }
+  }
+ },
+ "schemas": {
+  "Job": {
+   "id": "Job",
+   "type": "object",
+   "description": "Job Object",
+   "properties": {
+    "title": {
+     "type": "string",
+     "description": "Job Title"
+    },
+    "id": {
+     "type": "string",
+     "description": "Unique ID"
+    }
+   }
+  },
+  "Jobs": {
+   "id": "Jobs",
+   "type": "object",
+   "description": "List of Jobs.",
+   "properties": {
+    "jobs": {
+     "type": "array",
+     "description": "List of job objects.",
+     "items": {
+      "$ref": "Job"
+     }
+    }
+   }
+  },
+  "Printer": {
+   "id": "Printer",
+   "type": "object",
+   "description": "Printer Object",
+   "properties": {
+    "displayName": {
+     "type": "string",
+     "description": "Display Name"
+    },
+    "id": {
+     "type": "string",
+     "description": "Unique ID"
+    }
+   }
+  },
+  "Printers": {
+   "id": "Printers",
+   "type": "object",
+   "description": "List of Printers.",
+   "properties": {
+    "printers": {
+     "type": "array",
+     "description": "List of printer objects.",
+     "items": {
+      "$ref": "Printer"
+     }
+    }
+   }
+  }
+ },
+ "resources": {
+  "jobs": {
+   "methods": {
+    "delete": {
+     "id": "cloudprint.jobs.delete",
+     "path": "deletejob",
+     "httpMethod": "GET",
+     "parameters": {
+      "jobid": {
+       "type": "string",
+       "location": "query",
+       "required": "true"
+      }
+     }
+    },
+    "fetch": {
+     "id": "cloudprint.jobs.fetch",
+     "path": "fetch",
+     "httpMethod": "GET",
+     "parameters": {
+      "printerid": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      }
+     },
+     "response": {
+      "$ref": "Jobs"
+     }
+    },
+    "getticket": {
+     "id": "cloudprint.jobs.getticket",
+     "path": "ticket",
+     "httpMethod": "GET",
+     "parameters": {
+      "jobid": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      },
+      "use_cjt": {
+       "type": "boolean",
+       "required": "true",
+       "location": "query"
+      }
+     }
+    }, 
+    "list": {
+     "id": "cloudprint.jobs.list",
+     "path": "jobs",
+     "httpMethod": "GET",
+     "parameters": {
+      "printerid": {
+       "type": "string",
+       "location": "query"
+      },
+      "owner": {
+       "type": "string",
+       "location": "query"
+      },
+      "status": {
+       "type": "string",
+       "location": "query"
+      },
+      "q": {
+       "type": "string",
+       "location": "query"
+      },
+      "offset": {
+       "type": "string",
+       "location": "query"
+      },
+      "limit": {
+       "type": "string",
+       "location": "query"
+      },
+      "sortorder": {
+       "type": "string",
+       "location": "query"
+      }
+     },
+     "response": {
+      "$ref": "Jobs"
+     }
+    },
+    "update": {
+     "id": "cloudprint.jobs.update",
+     "path": "control",
+     "httpMethod": "GET",
+     "parameters": {
+      "jobid": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      },
+      "semantic_state_diff": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      }
+     },
+     "response": {
+      "$ref": "Jobs"
+     }
+    },
+    "resubmit": {
+     "id": "cloudprint.jobs.resubmit",
+     "path": "resubmit",
+     "httpMethod": "POST",
+     "description": "resubmit a job to new printer.",
+     "parameters": {
+      "printerid": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      },
+      "jobid": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      },
+      "ticket": {
+       "type": "string",
+       "location": "query"
+      }
+     },
+     "response": {
+      "$ref": "Job"
+     }
+    },
+    "submit": {
+     "id": "cloudprint.jobs.submit",
+     "path": "submit",
+     "httpMethod": "POST",
+     "description": "Send a print job to cloud print.",
+     "request": {
+      "printerid": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      },
+      "title": {
+       "type": "string",
+       "location": "query"
+      },
+      "ticket": {
+       "type": "string",
+       "location": "query"
+      },
+      "content": {
+       "type": "string",
+       "location": "query"
+      },
+      "contentType": {
+       "type": "string",
+       "location": "query"
+      },
+      "tag": {
+       "type": "string",
+       "location": "query"
+      }
+     },
+     "response": {
+      "$ref": "Job"
+     }
+    }
+   }
+  },
+  "printers": {
+   "methods": {
+    "get": {
+     "id": "cloudprint.printers.get",
+     "path": "printer",
+     "httpMethod": "GET",
+     "parameters": {
+      "printerid": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      },
+      "extra_fields": {
+       "type": "string",
+       "location": "query"
+      }
+     },
+     "response": {
+      "$ref": "Printer"
+     }
+    },
+    "list": {
+     "id": "cloudprint.printers.list",
+     "path": "search",
+     "httpMethod": "GET",
+     "description": "List all printers",
+     "parameters": {
+      "q": {
+       "type": "string",
+       "description": "Query list of printers",
+       "location": "query"
+      },
+      "type": {
+       "type": "string",
+       "description": "limit results to printers of type",
+       "location": "query"
+       },
+      "connection_status": {
+       "type": "string",
+       "description": "limit results to printers with this status",
+       "location": "query"
+      },
+      "extra_fields": {
+       "type": "string",
+       "description": "include extra fields",
+       "location": "query"
+      }
+     },
+     "response": {
+      "$ref": "Printers"
+     }
+    },
+    "share": {
+     "id": "cloudprint.printers.share",
+     "path": "share",
+     "httpMethod": "GET",
+     "description": "Share printer with user, group or domain",
+     "parameters": {
+      "printerid": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      },
+      "scope": {
+       "type": "string",
+       "location": "query"
+      },
+      "role": {
+       "type": "string",
+       "location": "query"
+      },
+      "type": {
+        "type": "string",
+        "location": "query"
+      },
+      "skip_notification": {
+       "type": "boolean",
+       "location": "query"
+      },
+      "public": {
+       "type": "boolean",
+       "location": "query"
+      }
+     } 
+    },
+    "unshare": {
+     "id": "cloudprint.printers.unshare",
+     "path": "unshare",
+     "httpMethod": "GET",
+     "description": "unshare printer with user, group or domain",
+     "parameters": {
+      "printerid": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      },
+      "scope": {
+       "type": "string",
+       "location": "query"
+      },
+      "public": {
+       "type": "string",
+       "location": "query"
+      }
+     }
+    },
+    "delete": {
+     "id": "cloudprint.printers.delete",
+     "path": "delete",
+     "httpMethod": "GET",
+     "description": "delete a printer",
+     "parameters": {
+      "printerid": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      }
+     }
+    },
+    "update": {
+     "id": "cloudprint.printers.update",
+     "path": "update",
+     "httpMethod": "GET",
+     "description": "update a printer",
+     "parameters": {
+      "isTosAccepted": {
+       "type": "boolean",
+       "location": "query"
+      },
+      "gcpVersion": {
+       "type": "string",
+       "location": "query"
+      },
+      "setupUrl": {
+       "type": "string",
+       "location": "query"
+      },
+      "supportUrl": {
+       "type": "string",
+       "location": "query"
+      },
+      "firmware": {
+       "type": "string",
+       "location": "query"
+      },
+      "currentQuota": {
+       "type": "string",
+       "location": "query"
+      },
+      "type": {
+       "type": "string",
+       "location": "query"
+      },
+      "public": {
+       "type": "boolean",
+       "location": "query"
+      },
+      "status": {
+       "type": "string",
+       "location": "query"
+      },
+      "proxy": {
+       "type": "string",
+       "location": "query"
+      },
+      "manufacturer": {
+       "type": "string",
+       "location": "query"
+      },
+      "defaultDisplayName": {
+       "type": "string",
+       "location": "query"
+      },
+      "displayName": {
+       "type": "string",
+       "location": "query"
+      },
+      "name": {
+       "type": "string",
+       "location": "query"
+      },
+      "uuid": {
+       "type": "string",
+       "location": "query"
+      },
+      "updateUrl": {
+       "type": "string",
+       "location": "query"
+      },
+      "ownerId": {
+       "type": "string",
+       "location": "query"
+      },
+      "model": {
+       "type": "string",
+       "location": "query"
+      },
+      "description": {
+       "type": "string",
+       "location": "query"
+      },
+      "printerid": {
+       "type": "string",
+       "required": "true",
+       "location": "query"
+      },
+      "quotaEnabled": {
+       "type": "boolean",
+       "location": "query"
+      },
+      "dailyQuota": {
+       "type": "string",
+       "location": "query"
+      }
+     }
+    } 
+   }
+  }
+ }
+}

googleapiclient/__init__.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2012 Google Inc.
+# Copyright 2014 Google Inc. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -12,4 +12,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-__version__ = "1.2"
+__version__ = "1.4.1"

googleapiclient/channel.py

@@ -55,12 +55,14 @@ Example of unsubscribing.
 
   service.channels().stop(channel.body())
 """
+from __future__ import absolute_import
 
 import datetime
 import uuid
 
-from apiclient import errors
+from googleapiclient import errors
 from oauth2client import util
+import six
 
 
 # The unix time epoch starts at midnight 1970.
@@ -88,7 +90,7 @@ X_GOOG_RESOURCE_ID    = 'X-GOOG-RESOURCE-ID'
 
 def _upper_header_keys(headers):
   new_headers = {}
-  for k, v in headers.iteritems():
+  for k, v in six.iteritems(headers):
     new_headers[k.upper()] = v
   return new_headers
 
@@ -218,7 +220,7 @@ class Channel(object):
     Args:
       resp: dict, The response from a watch() method.
     """
-    for json_name, param_name in CHANNEL_PARAMS.iteritems():
+    for json_name, param_name in six.iteritems(CHANNEL_PARAMS):
       value = resp.get(json_name)
       if value is not None:
         setattr(self, param_name, value)

googleapiclient/discovery.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2010 Google Inc.
+# Copyright 2014 Google Inc. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -16,6 +16,9 @@
 
 A client library for Google's discovery based APIs.
 """
+from __future__ import absolute_import
+import six
+from six.moves import zip
 
 __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 __all__ = [
@@ -25,44 +28,43 @@ __all__ = [
     'key2param',
     ]
 
+from six import StringIO
+from six.moves.urllib.parse import urlencode, urlparse, urljoin, \
+  urlunparse, parse_qsl
 
 # Standard library imports
 import copy
+from email.generator import Generator
 from email.mime.multipart import MIMEMultipart
 from email.mime.nonmultipart import MIMENonMultipart
+import json
 import keyword
 import logging
 import mimetypes
 import os
 import re
-import urllib
-import urlparse
-
-try:
-  from urlparse import parse_qsl
-except ImportError:
-  from cgi import parse_qsl
 
 # Third-party imports
 import httplib2
-import mimeparse
 import uritemplate
 
 # Local imports
-from apiclient.errors import HttpError
-from apiclient.errors import InvalidJsonError
-from apiclient.errors import MediaUploadSizeError
-from apiclient.errors import UnacceptableMimeTypeError
-from apiclient.errors import UnknownApiNameOrVersion
-from apiclient.errors import UnknownFileType
-from apiclient.http import HttpRequest
-from apiclient.http import MediaFileUpload
-from apiclient.http import MediaUpload
-from apiclient.model import JsonModel
-from apiclient.model import MediaModel
-from apiclient.model import RawModel
-from apiclient.schema import Schemas
-from oauth2client.anyjson import simplejson
+from googleapiclient import mimeparse
+from googleapiclient.errors import HttpError
+from googleapiclient.errors import InvalidJsonError
+from googleapiclient.errors import MediaUploadSizeError
+from googleapiclient.errors import UnacceptableMimeTypeError
+from googleapiclient.errors import UnknownApiNameOrVersion
+from googleapiclient.errors import UnknownFileType
+from googleapiclient.http import BatchHttpRequest
+from googleapiclient.http import HttpRequest
+from googleapiclient.http import MediaFileUpload
+from googleapiclient.http import MediaUpload
+from googleapiclient.model import JsonModel
+from googleapiclient.model import MediaModel
+from googleapiclient.model import RawModel
+from googleapiclient.schema import Schemas
+from oauth2client.client import GoogleCredentials
 from oauth2client.util import _add_query_parameter
 from oauth2client.util import positional
 
@@ -74,12 +76,8 @@ logger = logging.getLogger(__name__)
 
 URITEMPLATE = re.compile('{[^}]*}')
 VARNAME = re.compile('[a-zA-Z0-9_-]+')
-if httplib2.debuglevel > 0:
-  prettyPrint = 'true'
-else:
-  prettyPrint = 'false'
 DISCOVERY_URI = ('https://www.googleapis.com/discovery/v1/apis/'
-                 '{api}/{apiVersion}/rest?prettyPrint=%s' % prettyPrint)
+                 '{api}/{apiVersion}/rest')
 DEFAULT_METHOD_DOC = 'A description of how to use this function'
 HTTP_PAYLOAD_METHODS = frozenset(['PUT', 'POST', 'PATCH'])
 _MEDIA_SIZE_BIT_SHIFTS = {'KB': 10, 'MB': 20, 'GB': 30, 'TB': 40}
@@ -150,7 +148,8 @@ def build(serviceName,
           discoveryServiceUrl=DISCOVERY_URI,
           developerKey=None,
           model=None,
-          requestBuilder=HttpRequest):
+          requestBuilder=HttpRequest,
+          credentials=None):
   """Construct a Resource for interacting with an API.
 
   Construct a Resource object for interacting with an API. The serviceName and
@@ -167,9 +166,11 @@ def build(serviceName,
       document for that service.
     developerKey: string, key obtained from
       https://code.google.com/apis/console.
-    model: apiclient.Model, converts to and from the wire format.
-    requestBuilder: apiclient.http.HttpRequest, encapsulator for an HTTP
+    model: googleapiclient.Model, converts to and from the wire format.
+    requestBuilder: googleapiclient.http.HttpRequest, encapsulator for an HTTP
       request.
+    credentials: oauth2client.Credentials, credentials to be used for
+      authentication.
 
   Returns:
     A Resource object with methods for interacting with the service.
@@ -191,7 +192,7 @@ def build(serviceName,
   if 'REMOTE_ADDR' in os.environ:
     requested_url = _add_query_parameter(requested_url, 'userIp',
                                          os.environ['REMOTE_ADDR'])
-  logger.info('URL being requested: %s' % requested_url)
+  logger.info('URL being requested: GET %s' % requested_url)
 
   resp, content = http.request(requested_url)
 
@@ -202,13 +203,19 @@ def build(serviceName,
     raise HttpError(resp, content, uri=requested_url)
 
   try:
-    service = simplejson.loads(content)
-  except ValueError, e:
+    content = content.decode('utf-8')
+  except AttributeError:
+    pass
+
+  try:
+    service = json.loads(content)
+  except ValueError as e:
     logger.error('Failed to parse as JSON: ' + content)
     raise InvalidJsonError()
 
   return build_from_document(content, base=discoveryServiceUrl, http=http,
-      developerKey=developerKey, model=model, requestBuilder=requestBuilder)
+      developerKey=developerKey, model=model, requestBuilder=requestBuilder,
+      credentials=credentials)
 
 
 @positional(1)
@@ -219,7 +226,8 @@ def build_from_document(
     http=None,
     developerKey=None,
     model=None,
-    requestBuilder=HttpRequest):
+    requestBuilder=HttpRequest,
+    credentials=None):
   """Create a Resource for interacting with an API.
 
   Same as `build()`, but constructs the Resource object from a discovery
@@ -240,6 +248,7 @@ def build_from_document(
     model: Model class instance that serializes and de-serializes requests and
       responses.
     requestBuilder: Takes an http request and packages it up to be executed.
+    credentials: object, credentials to be used for authentication.
 
   Returns:
     A Resource object with methods for interacting with the service.
@@ -248,11 +257,33 @@ def build_from_document(
   # future is no longer used.
   future = {}
 
-  if isinstance(service, basestring):
-    service = simplejson.loads(service)
-  base = urlparse.urljoin(service['rootUrl'], service['servicePath'])
+  if isinstance(service, six.string_types):
+    service = json.loads(service)
+  base = urljoin(service['rootUrl'], service['servicePath'])
   schema = Schemas(service)
 
+  if credentials:
+    # If credentials were passed in, we could have two cases:
+    # 1. the scopes were specified, in which case the given credentials
+    #    are used for authorizing the http;
+    # 2. the scopes were not provided (meaning the Application Default
+    #    Credentials are to be used). In this case, the Application Default
+    #    Credentials are built and used instead of the original credentials.
+    #    If there are no scopes found (meaning the given service requires no
+    #    authentication), there is no authorization of the http.
+    if (isinstance(credentials, GoogleCredentials) and
+        credentials.create_scoped_required()):
+      scopes = service.get('auth', {}).get('oauth2', {}).get('scopes', {})
+      if scopes:
+        credentials = credentials.create_scoped(list(scopes.keys()))
+      else:
+        # No need to authorize the http object
+        # if the service does not require authentication.
+        credentials = None
+
+    if credentials:
+      http = credentials.authorize(http)
+
   if model is None:
     features = service.get('features', [])
     model = JsonModel('dataWrapper' in features)
@@ -302,13 +333,13 @@ def _media_size_to_long(maxSize):
     The size as an integer value.
   """
   if len(maxSize) < 2:
-    return 0L
+    return 0
   units = maxSize[-2:].upper()
   bit_shift = _MEDIA_SIZE_BIT_SHIFTS.get(units)
   if bit_shift is not None:
-    return long(maxSize[:-2]) << bit_shift
+    return int(maxSize[:-2]) << bit_shift
   else:
-    return long(maxSize)
+    return int(maxSize)
 
 
 def _media_path_url_from_info(root_desc, path_url):
@@ -358,7 +389,7 @@ def _fix_up_parameters(method_desc, root_desc, http_method):
   parameters = method_desc.setdefault('parameters', {})
 
   # Add in the parameters common to all methods.
-  for name, description in root_desc.get('parameters', {}).iteritems():
+  for name, description in six.iteritems(root_desc.get('parameters', {})):
     parameters[name] = description
 
   # Add in undocumented query parameters.
@@ -464,6 +495,23 @@ def _fix_up_method_description(method_desc, root_desc):
   return path_url, http_method, method_id, accept, max_size, media_path_url
 
 
+def _urljoin(base, url):
+  """Custom urljoin replacement supporting : before / in url."""
+  # In general, it's unsafe to simply join base and url. However, for
+  # the case of discovery documents, we know:
+  #  * base will never contain params, query, or fragment
+  #  * url will never contain a scheme or net_loc.
+  # In general, this means we can safely join on /; we just need to
+  # ensure we end up with precisely one / joining base and url. The
+  # exception here is the case of media uploads, where url will be an
+  # absolute url.
+  if url.startswith('http://') or url.startswith('https://'):
+    return urljoin(base, url)
+  new_base = base if base.endswith('/') else base + '/'
+  new_url = url[1:] if url.startswith('/') else url
+  return new_base + new_url
+
+
 # TODO(dhermes): Convert this class to ResourceMethod and make it callable
 class ResourceMethodParameters(object):
   """Represents the parameters associated with a method.
@@ -524,7 +572,7 @@ class ResourceMethodParameters(object):
           comes from the dictionary of methods stored in the 'methods' key in
           the deserialized discovery document.
     """
-    for arg, desc in method_desc.get('parameters', {}).iteritems():
+    for arg, desc in six.iteritems(method_desc.get('parameters', {})):
       param = key2param(arg)
       self.argmap[param] = arg
 
@@ -572,12 +620,12 @@ def createMethod(methodName, methodDesc, rootDesc, schema):
   def method(self, **kwargs):
     # Don't bother with doc string, it will be over-written by createMethod.
 
-    for name in kwargs.iterkeys():
+    for name in six.iterkeys(kwargs):
       if name not in parameters.argmap:
         raise TypeError('Got an unexpected keyword argument "%s"' % name)
 
     # Remove args that have a value of None.
-    keys = kwargs.keys()
+    keys = list(kwargs.keys())
     for name in keys:
       if kwargs[name] is None:
         del kwargs[name]
@@ -586,9 +634,9 @@ def createMethod(methodName, methodDesc, rootDesc, schema):
       if name not in kwargs:
         raise TypeError('Missing required parameter "%s"' % name)
 
-    for name, regex in parameters.pattern_params.iteritems():
+    for name, regex in six.iteritems(parameters.pattern_params):
       if name in kwargs:
-        if isinstance(kwargs[name], basestring):
+        if isinstance(kwargs[name], six.string_types):
           pvalues = [kwargs[name]]
         else:
           pvalues = kwargs[name]
@@ -598,13 +646,13 @@ def createMethod(methodName, methodDesc, rootDesc, schema):
                 'Parameter "%s" value "%s" does not match the pattern "%s"' %
                 (name, pvalue, regex))
 
-    for name, enums in parameters.enum_params.iteritems():
+    for name, enums in six.iteritems(parameters.enum_params):
       if name in kwargs:
         # We need to handle the case of a repeated enum
         # name differently, since we want to handle both
         # arg='value' and arg=['value1', 'value2']
         if (name in parameters.repeated_params and
-            not isinstance(kwargs[name], basestring)):
+            not isinstance(kwargs[name], six.string_types)):
           values = kwargs[name]
         else:
           values = [kwargs[name]]
@@ -616,7 +664,7 @@ def createMethod(methodName, methodDesc, rootDesc, schema):
 
     actual_query_params = {}
     actual_path_params = {}
-    for key, value in kwargs.iteritems():
+    for key, value in six.iteritems(kwargs):
       to_type = parameters.param_types.get(key, 'string')
       # For repeated parameters we cast each member of the list.
       if key in parameters.repeated_params and type(value) == type([]):
@@ -644,14 +692,14 @@ def createMethod(methodName, methodDesc, rootDesc, schema):
         actual_path_params, actual_query_params, body_value)
 
     expanded_url = uritemplate.expand(pathUrl, params)
-    url = urlparse.urljoin(self._baseUrl, expanded_url + query)
+    url = _urljoin(self._baseUrl, expanded_url + query)
 
     resumable = None
     multipart_boundary = ''
 
     if media_filename:
       # Ensure we end up with a valid MediaUpload object.
-      if isinstance(media_filename, basestring):
+      if isinstance(media_filename, six.string_types):
         (media_mime_type, encoding) = mimetypes.guess_type(media_filename)
         if media_mime_type is None:
           raise UnknownFileType(media_filename)
@@ -665,12 +713,12 @@ def createMethod(methodName, methodDesc, rootDesc, schema):
         raise TypeError('media_filename must be str or MediaUpload.')
 
       # Check the maxSize
-      if maxSize > 0 and media_upload.size() > maxSize:
+      if media_upload.size() is not None and media_upload.size() > maxSize > 0:
         raise MediaUploadSizeError("Media larger than: %s" % maxSize)
 
       # Use the media path uri for media uploads
       expanded_url = uritemplate.expand(mediaPathUrl, params)
-      url = urlparse.urljoin(self._baseUrl, expanded_url + query)
+      url = _urljoin(self._baseUrl, expanded_url + query)
       if media_upload.resumable():
         url = _add_query_parameter(url, 'uploadType', 'resumable')
 
@@ -703,14 +751,19 @@ def createMethod(methodName, methodDesc, rootDesc, schema):
           payload = media_upload.getbytes(0, media_upload.size())
           msg.set_payload(payload)
           msgRoot.attach(msg)
-          body = msgRoot.as_string()
+          # encode the body: note that we can't use `as_string`, because
+          # it plays games with `From ` lines.
+          fp = StringIO()
+          g = Generator(fp, mangle_from_=False)
+          g.flatten(msgRoot, unixfrom=False)
+          body = fp.getvalue()
 
           multipart_boundary = msgRoot.get_boundary()
           headers['content-type'] = ('multipart/related; '
                                      'boundary="%s"') % multipart_boundary
           url = _add_query_parameter(url, 'uploadType', 'multipart')
 
-    logger.info('URL being requested: %s' % url)
+    logger.info('URL being requested: %s %s' % (httpMethod,url))
     return self._requestBuilder(self._http,
                                 model.response,
                                 url,
@@ -725,10 +778,10 @@ def createMethod(methodName, methodDesc, rootDesc, schema):
     docs.append('Args:\n')
 
   # Skip undocumented params and params common to all methods.
-  skip_parameters = rootDesc.get('parameters', {}).keys()
+  skip_parameters = list(rootDesc.get('parameters', {}).keys())
   skip_parameters.extend(STACK_QUERY_PARAMETERS)
 
-  all_args = parameters.argmap.keys()
+  all_args = list(parameters.argmap.keys())
   args_ordered = [key2param(s) for s in methodDesc.get('parameterOrder', [])]
 
   # Move body to the front of the line.
@@ -807,18 +860,18 @@ Returns:
     request = copy.copy(previous_request)
 
     pageToken = previous_response['nextPageToken']
-    parsed = list(urlparse.urlparse(request.uri))
+    parsed = list(urlparse(request.uri))
     q = parse_qsl(parsed[4])
 
     # Find and remove old 'pageToken' value from URI
     newq = [(key, value) for (key, value) in q if key != 'pageToken']
     newq.append(('pageToken', pageToken))
-    parsed[4] = urllib.urlencode(newq)
-    uri = urlparse.urlunparse(parsed)
+    parsed[4] = urlencode(newq)
+    uri = urlunparse(parsed)
 
     request.uri = uri
 
-    logger.info('URL being requested: %s' % uri)
+    logger.info('URL being requested: %s %s' % (methodName,uri))
 
     return request
 
@@ -836,9 +889,9 @@ class Resource(object):
       http: httplib2.Http, Object to make http requests with.
       baseUrl: string, base URL for the API. All requests are relative to this
           URI.
-      model: apiclient.Model, converts to and from the wire format.
+      model: googleapiclient.Model, converts to and from the wire format.
       requestBuilder: class or callable that instantiates an
-          apiclient.HttpRequest object.
+          googleapiclient.HttpRequest object.
       developerKey: string, key obtained from
           https://code.google.com/apis/console
       resourceDesc: object, section of deserialized discovery document that
@@ -898,9 +951,30 @@ class Resource(object):
     self._add_next_methods(self._resourceDesc, self._schema)
 
   def _add_basic_methods(self, resourceDesc, rootDesc, schema):
+    # If this is the root Resource, add a new_batch_http_request() method.
+    if resourceDesc == rootDesc:
+      batch_uri = '%s%s' % (
+        rootDesc['rootUrl'], rootDesc.get('batchPath', 'batch'))
+      def new_batch_http_request(callback=None):
+        """Create a BatchHttpRequest object based on the discovery document.
+
+        Args:
+          callback: callable, A callback to be called for each response, of the
+            form callback(id, response, exception). The first parameter is the
+            request id, and the second is the deserialized response object. The
+            third is an apiclient.errors.HttpError exception object if an HTTP
+            error occurred while processing the request, or None if no error
+            occurred.
+
+        Returns:
+          A BatchHttpRequest object based on the discovery document.
+        """
+        return BatchHttpRequest(callback=callback, batch_uri=batch_uri)
+      self._set_dynamic_attr('new_batch_http_request', new_batch_http_request)
+
     # Add basic methods to Resource
     if 'methods' in resourceDesc:
-      for methodName, methodDesc in resourceDesc['methods'].iteritems():
+      for methodName, methodDesc in six.iteritems(resourceDesc['methods']):
         fixedMethodName, method = createMethod(
             methodName, methodDesc, rootDesc, schema)
         self._set_dynamic_attr(fixedMethodName,
@@ -939,7 +1013,7 @@ class Resource(object):
 
         return (methodName, methodResource)
 
-      for methodName, methodDesc in resourceDesc['resources'].iteritems():
+      for methodName, methodDesc in six.iteritems(resourceDesc['resources']):
         fixedMethodName, method = createResourceMethod(methodName, methodDesc)
         self._set_dynamic_attr(fixedMethodName,
                                method.__get__(self, self.__class__))
@@ -949,7 +1023,7 @@ class Resource(object):
     # Look for response bodies in schema that contain nextPageToken, and methods
     # that take a pageToken parameter.
     if 'methods' in resourceDesc:
-      for methodName, methodDesc in resourceDesc['methods'].iteritems():
+      for methodName, methodDesc in six.iteritems(resourceDesc['methods']):
         if 'response' in methodDesc:
           responseSchema = methodDesc['response']
           if '$ref' in responseSchema:

googleapiclient/errors.py

@@ -1,6 +1,4 @@
-#!/usr/bin/python2.4
-#
-# Copyright (C) 2010 Google Inc.
+# Copyright 2014 Google Inc. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -19,12 +17,13 @@
 All exceptions defined by the library
 should be defined in this file.
 """
+from __future__ import absolute_import
 
 __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 
+import json
 
 from oauth2client import util
-from oauth2client.anyjson import simplejson
 
 
 class Error(Exception):
@@ -38,6 +37,8 @@ class HttpError(Error):
   @util.positional(3)
   def __init__(self, resp, content, uri=None):
     self.resp = resp
+    if not isinstance(content, bytes):
+        raise TypeError("HTTP content should be bytes")
     self.content = content
     self.uri = uri
 
@@ -45,7 +46,7 @@ class HttpError(Error):
     """Calculate the reason for the error from the response content."""
     reason = self.resp.reason
     try:
-      data = simplejson.loads(self.content)
+      data = json.loads(self.content.decode('utf-8'))
       reason = data['error']['message']
     except (ValueError, KeyError):
       pass

googleapiclient/http.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2012 Google Inc.
+# Copyright 2014 Google Inc. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -18,38 +18,42 @@ The classes implement a command pattern, with every
 object supporting an execute() method that does the
 actuall HTTP request.
 """
+from __future__ import absolute_import
+import six
+from six.moves import range
 
 __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 
-import StringIO
+from six import BytesIO, StringIO
+from six.moves.urllib.parse import urlparse, urlunparse, quote, unquote
+
 import base64
 import copy
 import gzip
 import httplib2
+import json
 import logging
-import mimeparse
 import mimetypes
 import os
 import random
 import sys
 import time
-import urllib
-import urlparse
 import uuid
 
 from email.generator import Generator
 from email.mime.multipart import MIMEMultipart
 from email.mime.nonmultipart import MIMENonMultipart
 from email.parser import FeedParser
-from errors import BatchError
-from errors import HttpError
-from errors import InvalidChunkSizeError
-from errors import ResumableUploadError
-from errors import UnexpectedBodyError
-from errors import UnexpectedMethodError
-from model import JsonModel
+
+from googleapiclient import mimeparse
+from googleapiclient.errors import BatchError
+from googleapiclient.errors import HttpError
+from googleapiclient.errors import InvalidChunkSizeError
+from googleapiclient.errors import ResumableUploadError
+from googleapiclient.errors import UnexpectedBodyError
+from googleapiclient.errors import UnexpectedMethodError
+from googleapiclient.model import JsonModel
 from oauth2client import util
-from oauth2client.anyjson import simplejson
 
 
 DEFAULT_CHUNK_SIZE = 512*1024
@@ -221,7 +225,7 @@ class MediaUpload(object):
         del d[member]
     d['_class'] = t.__name__
     d['_module'] = t.__module__
-    return simplejson.dumps(d)
+    return json.dumps(d)
 
   def to_json(self):
     """Create a JSON representation of an instance of MediaUpload.
@@ -244,7 +248,7 @@ class MediaUpload(object):
       An instance of the subclass of MediaUpload that was serialized with
       to_json().
     """
-    data = simplejson.loads(s)
+    data = json.loads(s)
     # Find and call the right classmethod from_json() to restore the object.
     module = data['_module']
     m = __import__(module, fromlist=module.split('.')[:-1])
@@ -259,7 +263,7 @@ class MediaIoBaseUpload(MediaUpload):
   Note that the Python file object is compatible with io.Base and can be used
   with this class also.
 
-    fh = io.BytesIO('...Some data to upload...')
+    fh = BytesIO('...Some data to upload...')
     media = MediaIoBaseUpload(fh, mimetype='image/png',
       chunksize=1024*1024, resumable=True)
     farm.animals().insert(
@@ -436,7 +440,7 @@ class MediaFileUpload(MediaIoBaseUpload):
 
   @staticmethod
   def from_json(s):
-    d = simplejson.loads(s)
+    d = json.loads(s)
     return MediaFileUpload(d['_filename'], mimetype=d['_mimetype'],
                            chunksize=d['_chunksize'], resumable=d['_resumable'])
 
@@ -465,7 +469,7 @@ class MediaInMemoryUpload(MediaIoBaseUpload):
     resumable: bool, True if this is a resumable upload. False means upload
       in a single request.
     """
-    fd = StringIO.StringIO(body)
+    fd = BytesIO(body)
     super(MediaInMemoryUpload, self).__init__(fd, mimetype, chunksize=chunksize,
                                               resumable=resumable)
 
@@ -497,7 +501,7 @@ class MediaIoBaseDownload(object):
     Args:
       fd: io.Base or file object, The stream in which to write the downloaded
         bytes.
-      request: apiclient.http.HttpRequest, the media request to perform in
+      request: googleapiclient.http.HttpRequest, the media request to perform in
         chunks.
       chunksize: int, File will be downloaded in chunks of this many bytes.
     """
@@ -529,7 +533,7 @@ class MediaIoBaseDownload(object):
          downloaded.
 
     Raises:
-      apiclient.errors.HttpError if the response was not a 2xx.
+      googleapiclient.errors.HttpError if the response was not a 2xx.
       httplib2.HttpLib2Error if a transport error has occured.
     """
     headers = {
@@ -538,7 +542,7 @@ class MediaIoBaseDownload(object):
         }
     http = self._request.http
 
-    for retry_num in xrange(num_retries + 1):
+    for retry_num in range(num_retries + 1):
       if retry_num > 0:
         self._sleep(self._rand() * 2**retry_num)
         logging.warning(
@@ -559,6 +563,8 @@ class MediaIoBaseDownload(object):
         content_range = resp['content-range']
         length = content_range.rsplit('/', 1)[1]
         self._total_size = int(length)
+      elif 'content-length' in resp:
+        self._total_size = int(resp['content-length'])
 
       if self._progress == self._total_size:
         self._done = True
@@ -676,7 +682,7 @@ class HttpRequest(object):
       by the postproc.
 
     Raises:
-      apiclient.errors.HttpError if the response was not a 2xx.
+      googleapiclient.errors.HttpError if the response was not a 2xx.
       httplib2.HttpLib2Error if a transport error has occured.
     """
     if http is None:
@@ -697,8 +703,8 @@ class HttpRequest(object):
       self.method = 'POST'
       self.headers['x-http-method-override'] = 'GET'
       self.headers['content-type'] = 'application/x-www-form-urlencoded'
-      parsed = urlparse.urlparse(self.uri)
-      self.uri = urlparse.urlunparse(
+      parsed = urlparse(self.uri)
+      self.uri = urlunparse(
           (parsed.scheme, parsed.netloc, parsed.path, parsed.params, None,
            None)
           )
@@ -706,7 +712,7 @@ class HttpRequest(object):
       self.headers['content-length'] = str(len(self.body))
 
     # Handle retries for server-side errors.
-    for retry_num in xrange(num_retries + 1):
+    for retry_num in range(num_retries + 1):
       if retry_num > 0:
         self._sleep(self._rand() * 2**retry_num)
         logging.warning('Retry #%d for request: %s %s, following status: %d'
@@ -771,7 +777,7 @@ class HttpRequest(object):
          The body will be None until the resumable media is fully uploaded.
 
     Raises:
-      apiclient.errors.HttpError if the response was not a 2xx.
+      googleapiclient.errors.HttpError if the response was not a 2xx.
       httplib2.HttpLib2Error if a transport error has occured.
     """
     if http is None:
@@ -789,7 +795,7 @@ class HttpRequest(object):
         start_headers['X-Upload-Content-Length'] = size
       start_headers['content-length'] = str(self.body_size)
 
-      for retry_num in xrange(num_retries + 1):
+      for retry_num in range(num_retries + 1):
         if retry_num > 0:
           self._sleep(self._rand() * 2**retry_num)
           logging.warning(
@@ -854,7 +860,7 @@ class HttpRequest(object):
         'Content-Length': str(chunk_end - self.resumable_progress + 1)
         }
 
-    for retry_num in xrange(num_retries + 1):
+    for retry_num in range(num_retries + 1):
       if retry_num > 0:
         self._sleep(self._rand() * 2**retry_num)
         logging.warning(
@@ -885,7 +891,7 @@ class HttpRequest(object):
          The body will be None until the resumable media is fully uploaded.
 
     Raises:
-      apiclient.errors.HttpError if the response was not a 2xx or a 308.
+      googleapiclient.errors.HttpError if the response was not a 2xx or a 308.
     """
     if resp.status in [200, 201]:
       self._in_error_state = False
@@ -913,12 +919,12 @@ class HttpRequest(object):
     del d['_sleep']
     del d['_rand']
 
-    return simplejson.dumps(d)
+    return json.dumps(d)
 
   @staticmethod
   def from_json(s, http, postproc):
     """Returns an HttpRequest populated with info from a JSON object."""
-    d = simplejson.loads(s)
+    d = json.loads(s)
     if d['resumable'] is not None:
       d['resumable'] = MediaUpload.new_from_json(d['resumable'])
     return HttpRequest(
@@ -936,7 +942,7 @@ class BatchHttpRequest(object):
   """Batches multiple HttpRequest objects into a single HTTP request.
 
   Example:
-    from apiclient.http import BatchHttpRequest
+    from googleapiclient.http import BatchHttpRequest
 
     def list_animals(request_id, response, exception):
       \"\"\"Do something with the animals list response.\"\"\"
@@ -973,7 +979,7 @@ class BatchHttpRequest(object):
       callback: callable, A callback to be called for each response, of the
         form callback(id, response, exception). The first parameter is the
         request id, and the second is the deserialized response object. The
-        third is an apiclient.errors.HttpError exception object if an HTTP error
+        third is an googleapiclient.errors.HttpError exception object if an HTTP error
         occurred while processing the request, or None if no error occurred.
       batch_uri: string, URI to send batch requests to.
     """
@@ -1046,7 +1052,7 @@ class BatchHttpRequest(object):
     if self._base_id is None:
       self._base_id = uuid.uuid4()
 
-    return '<%s+%s>' % (self._base_id, urllib.quote(id_))
+    return '<%s+%s>' % (self._base_id, quote(id_))
 
   def _header_to_id(self, header):
     """Convert a Content-ID header value to an id.
@@ -1069,7 +1075,7 @@ class BatchHttpRequest(object):
       raise BatchError("Invalid value for Content-ID: %s" % header)
     base, id_ = header[1:-1].rsplit('+', 1)
 
-    return urllib.unquote(id_)
+    return unquote(id_)
 
   def _serialize_request(self, request):
     """Convert an HttpRequest object into a string.
@@ -1081,9 +1087,9 @@ class BatchHttpRequest(object):
       The request as a string in application/http format.
     """
     # Construct status line
-    parsed = urlparse.urlparse(request.uri)
-    request_line = urlparse.urlunparse(
-        (None, None, parsed.path, parsed.params, parsed.query, None)
+    parsed = urlparse(request.uri)
+    request_line = urlunparse(
+        ('', '', parsed.path, parsed.params, parsed.query, '')
         )
     status_line = request.method + ' ' + request_line + ' HTTP/1.1\n'
     major, minor = request.headers.get('content-type', 'application/json').split('/')
@@ -1098,7 +1104,7 @@ class BatchHttpRequest(object):
     if 'content-type' in headers:
       del headers['content-type']
 
-    for key, value in headers.iteritems():
+    for key, value in six.iteritems(headers):
       msg[key] = value
     msg['Host'] = parsed.netloc
     msg.set_unixfrom(None)
@@ -1108,17 +1114,13 @@ class BatchHttpRequest(object):
       msg['content-length'] = str(len(request.body))
 
     # Serialize the mime message.
-    fp = StringIO.StringIO()
+    fp = StringIO()
     # maxheaderlen=0 means don't line wrap headers.
     g = Generator(fp, maxheaderlen=0)
     g.flatten(msg, unixfrom=False)
     body = fp.getvalue()
 
-    # Strip off the \n\n that the MIME lib tacks onto the end of the payload.
-    if request.body is None:
-      body = body[:-2]
-
-    return status_line.encode('utf-8') + body
+    return status_line + body
 
   def _deserialize_response(self, payload):
     """Convert string into httplib2 response and content.
@@ -1178,7 +1180,7 @@ class BatchHttpRequest(object):
       callback: callable, A callback to be called for this response, of the
         form callback(id, response, exception). The first parameter is the
         request id, and the second is the deserialized response object. The
-        third is an apiclient.errors.HttpError exception object if an HTTP error
+        third is an googleapiclient.errors.HttpError exception object if an HTTP error
         occurred while processing the request, or None if no errors occurred.
       request_id: string, A unique id for the request. The id will be passed to
         the callback with the response.
@@ -1211,7 +1213,7 @@ class BatchHttpRequest(object):
 
     Raises:
       httplib2.HttpLib2Error if a transport error has occured.
-      apiclient.errors.BatchError if the response is the wrong format.
+      googleapiclient.errors.BatchError if the response is the wrong format.
     """
     message = MIMEMultipart('mixed')
     # Message should not write out it's own headers.
@@ -1229,7 +1231,12 @@ class BatchHttpRequest(object):
       msg.set_payload(body)
       message.attach(msg)
 
-    body = message.as_string()
+    # encode the body: note that we can't use `as_string`, because
+    # it plays games with `From ` lines.
+    fp = StringIO()
+    g = Generator(fp, mangle_from_=False)
+    g.flatten(message, unixfrom=False)
+    body = fp.getvalue()
 
     headers = {}
     headers['content-type'] = ('multipart/mixed; '
@@ -1241,11 +1248,12 @@ class BatchHttpRequest(object):
     if resp.status >= 300:
       raise HttpError(resp, content, uri=self._batch_uri)
 
-    # Now break out the individual responses and store each one.
-    boundary, _ = content.split(None, 1)
-
     # Prepend with a content-type header so FeedParser can handle it.
     header = 'content-type: %s\r\n\r\n' % resp['content-type']
+    # PY3's FeedParser only accepts unicode. So we should decode content
+    # here, and encode each payload again.
+    if six.PY3:
+      content = content.decode('utf-8')
     for_parser = header + content
 
     parser = FeedParser()
@@ -1259,6 +1267,9 @@ class BatchHttpRequest(object):
     for part in mime_response.get_payload():
       request_id = self._header_to_id(part['Content-ID'])
       response, content = self._deserialize_response(part.get_payload())
+      # We encode content here to emulate normal http response.
+      if isinstance(content, six.text_type):
+        content = content.encode('utf-8')
       self._responses[request_id] = (response, content)
 
   @util.positional(1)
@@ -1275,7 +1286,7 @@ class BatchHttpRequest(object):
 
     Raises:
       httplib2.HttpLib2Error if a transport error has occured.
-      apiclient.errors.BatchError if the response is the wrong format.
+      googleapiclient.errors.BatchError if the response is the wrong format.
     """
 
     # If http is not supplied use the first valid one given in the requests.
@@ -1323,7 +1334,7 @@ class BatchHttpRequest(object):
         if resp.status >= 300:
           raise HttpError(resp, content, uri=request.uri)
         response = request.postproc(resp, content)
-      except HttpError, e:
+      except HttpError as e:
         exception = e
 
       if callback is not None:
@@ -1381,7 +1392,7 @@ class RequestMockBuilder(object):
           'plus.activities.get': (None, response),
         }
       )
-      apiclient.discovery.build("plus", "v1", requestBuilder=requestBuilder)
+      googleapiclient.discovery.build("plus", "v1", requestBuilder=requestBuilder)
 
     Methods that you do not supply a response for will return a
     200 OK with an empty string as the response content or raise an excpetion
@@ -1425,8 +1436,8 @@ class RequestMockBuilder(object):
           # or expecting a body and not provided one.
           raise UnexpectedBodyError(expected_body, body)
         if isinstance(expected_body, str):
-          expected_body = simplejson.loads(expected_body)
-        body = simplejson.loads(body)
+          expected_body = json.loads(expected_body)
+        body = json.loads(body)
         if body != expected_body:
           raise UnexpectedBodyError(expected_body, body)
       return HttpRequestMock(resp, content, postproc)
@@ -1447,9 +1458,9 @@ class HttpMock(object):
       headers: dict, header to return with response
     """
     if headers is None:
-      headers = {'status': '200 OK'}
+      headers = {'status': '200'}
     if filename:
-      f = file(filename, 'r')
+      f = open(filename, 'r')
       self.data = f.read()
       f.close()
     else:
@@ -1517,7 +1528,7 @@ class HttpMockSequence(object):
     if content == 'echo_request_headers':
       content = headers
     elif content == 'echo_request_headers_as_json':
-      content = simplejson.dumps(headers)
+      content = json.dumps(headers)
     elif content == 'echo_request_body':
       if hasattr(body, 'read'):
         content = body.read()
@@ -1525,6 +1536,8 @@ class HttpMockSequence(object):
         content = body
     elif content == 'echo_request_uri':
       content = uri
+    if isinstance(content, six.text_type):
+      content = content.encode('utf-8')
     return httplib2.Response(resp), content
 
 

googleapiclient/mimeparse.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2007 Joe Gregorio
+# Copyright 2014 Joe Gregorio
 #
 # Licensed under the MIT License
 
@@ -21,6 +21,9 @@ Contents:
  - best_match():        Choose the mime-type with the highest quality ('q')
                           from a list of candidates.
 """
+from __future__ import absolute_import
+from functools import reduce
+import six
 
 __version__ = '0.1.3'
 __author__ = 'Joe Gregorio'
@@ -68,7 +71,7 @@ def parse_media_range(range):
     necessary.
     """
     (type, subtype, params) = parse_mime_type(range)
-    if not params.has_key('q') or not params['q'] or \
+    if 'q' not in params or not params['q'] or \
             not float(params['q']) or float(params['q']) > 1\
             or float(params['q']) < 0:
         params['q'] = '1'
@@ -98,8 +101,8 @@ def fitness_and_quality_parsed(mime_type, parsed_ranges):
                          target_subtype == '*')
         if type_match and subtype_match:
             param_matches = reduce(lambda x, y: x + y, [1 for (key, value) in \
-                    target_params.iteritems() if key != 'q' and \
-                    params.has_key(key) and value == params[key]], 0)
+                    six.iteritems(target_params) if key != 'q' and \
+                    key in params and value == params[key]], 0)
             fitness = (type == target_type) and 100 or 0
             fitness += (subtype == target_subtype) and 10 or 0
             fitness += param_matches

googleapiclient/model.py

@@ -1,6 +1,4 @@
-#!/usr/bin/python2.4
-#
-# Copyright (C) 2010 Google Inc.
+# Copyright 2014 Google Inc. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -21,15 +19,18 @@ as JSON, Atom, etc. The model classes are responsible
 for converting between the wire format and the Python
 object representation.
 """
+from __future__ import absolute_import
+import six
 
 __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 
+import json
 import logging
-import urllib
 
-from apiclient import __version__
-from errors import HttpError
-from oauth2client.anyjson import simplejson
+from six.moves.urllib.parse import urlencode
+
+from googleapiclient import __version__
+from googleapiclient.errors import HttpError
 
 
 dump_request_response = False
@@ -77,7 +78,7 @@ class Model(object):
       The body de-serialized as a Python object.
 
     Raises:
-      apiclient.errors.HttpError if a non 2xx response is received.
+      googleapiclient.errors.HttpError if a non 2xx response is received.
     """
     _abstract()
 
@@ -106,11 +107,11 @@ class BaseModel(Model):
     if dump_request_response:
       logging.info('--request-start--')
       logging.info('-headers-start-')
-      for h, v in headers.iteritems():
+      for h, v in six.iteritems(headers):
         logging.info('%s: %s', h, v)
       logging.info('-headers-end-')
       logging.info('-path-parameters-start-')
-      for h, v in path_params.iteritems():
+      for h, v in six.iteritems(path_params):
         logging.info('%s: %s', h, v)
       logging.info('-path-parameters-end-')
       logging.info('body: %s', body)
@@ -125,7 +126,7 @@ class BaseModel(Model):
       path_params: dict, parameters that appear in the request path
       query_params: dict, parameters that appear in the query
       body_value: object, the request body as a Python object, which must be
-                  serializable by simplejson.
+                  serializable by json.
     Returns:
       A tuple of (headers, path_params, query, body)
 
@@ -161,22 +162,22 @@ class BaseModel(Model):
     if self.alt_param is not None:
       params.update({'alt': self.alt_param})
     astuples = []
-    for key, value in params.iteritems():
+    for key, value in six.iteritems(params):
       if type(value) == type([]):
         for x in value:
           x = x.encode('utf-8')
           astuples.append((key, x))
       else:
-        if getattr(value, 'encode', False) and callable(value.encode):
+        if isinstance(value, six.text_type) and callable(value.encode):
           value = value.encode('utf-8')
         astuples.append((key, value))
-    return '?' + urllib.urlencode(astuples)
+    return '?' + urlencode(astuples)
 
   def _log_response(self, resp, content):
     """Logs debugging information about the response if requested."""
     if dump_request_response:
       logging.info('--response-start--')
-      for h, v in resp.iteritems():
+      for h, v in six.iteritems(resp):
         logging.info('%s: %s', h, v)
       if content:
         logging.info(content)
@@ -193,7 +194,7 @@ class BaseModel(Model):
       The body de-serialized as a Python object.
 
     Raises:
-      apiclient.errors.HttpError if a non 2xx response is received.
+      googleapiclient.errors.HttpError if a non 2xx response is received.
     """
     self._log_response(resp, content)
     # Error handling is TBD, for example, do we retry
@@ -254,11 +255,14 @@ class JsonModel(BaseModel):
     if (isinstance(body_value, dict) and 'data' not in body_value and
         self._data_wrapper):
       body_value = {'data': body_value}
-    return simplejson.dumps(body_value)
+    return json.dumps(body_value)
 
   def deserialize(self, content):
-    content = content.decode('utf-8')
-    body = simplejson.loads(content)
+    try:
+        content = content.decode('utf-8')
+    except AttributeError:
+        pass
+    body = json.loads(content)
     if self._data_wrapper and isinstance(body, dict) and 'data' in body:
       body = body['data']
     return body
@@ -361,7 +365,7 @@ def makepatch(original, modified):
       body=makepatch(original, item)).execute()
   """
   patch = {}
-  for key, original_value in original.iteritems():
+  for key, original_value in six.iteritems(original):
     modified_value = modified.get(key, None)
     if modified_value is None:
       # Use None to signal that the element is deleted

googleapiclient/sample_tools.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2013 Google Inc.
+# Copyright 2014 Google Inc. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -16,6 +16,7 @@
 
 Consolidates a lot of code commonly repeated in sample applications.
 """
+from __future__ import absolute_import
 
 __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 __all__ = ['init']
@@ -25,13 +26,13 @@ import argparse
 import httplib2
 import os
 
-from apiclient import discovery
+from googleapiclient import discovery
 from oauth2client import client
 from oauth2client import file
 from oauth2client import tools
 
 
-def init(argv, name, version, doc, filename, scope=None, parents=[]):
+def init(argv, name, version, doc, filename, scope=None, parents=[], discovery_filename=None):
   """A common initialization routine for samples.
 
   Many of the sample applications do the same initialization, which has now
@@ -49,6 +50,7 @@ def init(argv, name, version, doc, filename, scope=None, parents=[]):
     file: string, filename of the application. Usually set to __file__.
     parents: list of argparse.ArgumentParser, additional command-line flags.
     scope: string, The OAuth scope used.
+    discovery_filename: string, name of local discovery file (JSON). Use when discovery doc not available via URL.
 
   Returns:
     A tuple of (service, flags), where service is the service object and flags
@@ -88,6 +90,14 @@ def init(argv, name, version, doc, filename, scope=None, parents=[]):
     credentials = tools.run_flow(flow, storage, flags)
   http = credentials.authorize(http = httplib2.Http())
 
-  # Construct a service object via the discovery service.
-  service = discovery.build(name, version, http=http)
+  if discovery_filename is None:
+    # Construct a service object via the discovery service.
+    service = discovery.build(name, version, http=http)
+  else:
+    # Construct a service object using a local discovery document file.
+    with open(discovery_filename) as discovery_file:
+      service = discovery.build_from_document(
+          discovery_file.read(),
+          base='https://www.googleapis.com/',
+          http=http)
   return (service, flags)

googleapiclient/schema.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2010 Google Inc.
+# Copyright 2014 Google Inc. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -56,6 +56,8 @@ For example, given the schema:
 
 The constructor takes a discovery document in which to look up named schema.
 """
+from __future__ import absolute_import
+import six
 
 # TODO(jcgregorio) support format, enum, minimum, maximum
 
@@ -64,7 +66,6 @@ __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 import copy
 
 from oauth2client import util
-from oauth2client.anyjson import simplejson
 
 
 class Schemas(object):
@@ -250,7 +251,7 @@ class _SchemaToStruct(object):
       self.emitEnd('{', schema.get('description', ''))
       self.indent()
       if 'properties' in schema:
-        for pname, pschema in schema.get('properties', {}).iteritems():
+        for pname, pschema in six.iteritems(schema.get('properties', {})):
           self.emitBegin('"%s": ' % pname)
           self._to_str_impl(pschema)
       elif 'additionalProperties' in schema:

httplib2/__init__.py

@@ -22,7 +22,7 @@ __contributors__ = ["Thomas Broyer (t.broyer@ltgt.net)",
                     "Sam Ruby",
                     "Louis Nyffenegger"]
 __license__ = "MIT"
-__version__ = "0.9"
+__version__ = "0.9.1"
 
 import re
 import sys
@@ -749,12 +749,27 @@ class ProxyInfo(object):
     bypass_hosts = ()
 
     def __init__(self, proxy_type, proxy_host, proxy_port,
-                 proxy_rdns=None, proxy_user=None, proxy_pass=None):
-        """The parameter proxy_type must be set to one of socks.PROXY_TYPE_XXX
-        constants. For example:
+                 proxy_rdns=True, proxy_user=None, proxy_pass=None):
+        """
+        Args:
+          proxy_type: The type of proxy server.  This must be set to one of
+          socks.PROXY_TYPE_XXX constants.  For example:
 
-        p = ProxyInfo(proxy_type=socks.PROXY_TYPE_HTTP,
-            proxy_host='localhost', proxy_port=8000)
+            p = ProxyInfo(proxy_type=socks.PROXY_TYPE_HTTP,
+              proxy_host='localhost', proxy_port=8000)
+
+          proxy_host: The hostname or IP address of the proxy server.
+
+          proxy_port: The port that the proxy server is running on.
+
+          proxy_rdns: If True (default), DNS queries will not be performed
+          locally, and instead, handed to the proxy to resolve.  This is useful
+          if the network does not allow resolution of non-local names.  In
+          httplib2 0.9 and earlier, this defaulted to False.
+
+          proxy_user: The username used to authenticate with the proxy server.
+
+          proxy_pass: The password used to authenticate with the proxy server.
         """
         self.proxy_type = proxy_type
         self.proxy_host = proxy_host
@@ -871,12 +886,12 @@ class HTTPConnectionWithTimeout(httplib.HTTPConnection):
         if self.proxy_info and self.proxy_info.isgood():
             use_proxy = True
             proxy_type, proxy_host, proxy_port, proxy_rdns, proxy_user, proxy_pass = self.proxy_info.astuple()
-        else:
-            use_proxy = False
-        if use_proxy and proxy_rdns:
+
             host = proxy_host
             port = proxy_port
         else:
+            use_proxy = False
+
             host = self.host
             port = self.port
 
@@ -993,12 +1008,12 @@ class HTTPSConnectionWithTimeout(httplib.HTTPSConnection):
         if self.proxy_info and self.proxy_info.isgood():
             use_proxy = True
             proxy_type, proxy_host, proxy_port, proxy_rdns, proxy_user, proxy_pass = self.proxy_info.astuple()
-        else:
-            use_proxy = False
-        if use_proxy and proxy_rdns:
+
             host = proxy_host
             port = proxy_port
         else:
+            use_proxy = False
+
             host = self.host
             port = self.port
 
@@ -1481,7 +1496,7 @@ class Http(object):
             info = email.Message.Message()
             cached_value = None
             if self.cache:
-                cachekey = defrag_uri
+                cachekey = defrag_uri.encode('utf-8')
                 cached_value = self.cache.get(cachekey)
                 if cached_value:
                     # info = email.message_from_string(cached_value)

oauth2client/__init__.py

@@ -1,5 +1,8 @@
-__version__ = "1.2"
+"""Client library for using OAuth2, especially with Google APIs."""
+
+__version__ = '1.4.7'
 
 GOOGLE_AUTH_URI = 'https://accounts.google.com/o/oauth2/auth'
+GOOGLE_DEVICE_URI = 'https://accounts.google.com/o/oauth2/device/code'
 GOOGLE_REVOKE_URI = 'https://accounts.google.com/o/oauth2/revoke'
 GOOGLE_TOKEN_URI = 'https://accounts.google.com/o/oauth2/token'

oauth2client/anyjson.py

@@ -1,32 +0,0 @@
-# Copyright (C) 2010 Google Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Utility module to import a JSON module
-
-Hides all the messy details of exactly where
-we get a simplejson module from.
-"""
-
-__author__ = 'jcgregorio@google.com (Joe Gregorio)'
-
-
-try: # pragma: no cover
-  # Should work for Python2.6 and higher.
-  import json as simplejson
-except ImportError: # pragma: no cover
-  try:
-    import simplejson
-  except ImportError:
-    # Try to import from django, should work on App Engine
-    from django.utils import simplejson

oauth2client/appengine.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2010 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -19,14 +19,14 @@ Utilities for making it easier to use OAuth 2.0 on Google App Engine.
 
 __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 
-import base64
 import cgi
-import httplib2
+import json
 import logging
 import os
 import pickle
 import threading
-import time
+
+import httplib2
 
 from google.appengine.api import app_identity
 from google.appengine.api import memcache
@@ -41,7 +41,6 @@ from oauth2client import GOOGLE_TOKEN_URI
 from oauth2client import clientsecrets
 from oauth2client import util
 from oauth2client import xsrfutil
-from oauth2client.anyjson import simplejson
 from oauth2client.client import AccessTokenRefreshError
 from oauth2client.client import AssertionCredentials
 from oauth2client.client import Credentials
@@ -159,15 +158,20 @@ class AppAssertionCredentials(AssertionCredentials):
     Args:
       scope: string or iterable of strings, scope(s) of the credentials being
         requested.
+      **kwargs: optional keyword args, including:
+        service_account_id: service account id of the application. If None or
+          unspecified, the default service account for the app is used.
     """
     self.scope = util.scopes_to_string(scope)
+    self._kwargs = kwargs
+    self.service_account_id = kwargs.get('service_account_id', None)
 
     # Assertion type is no longer used, but still in the parent class signature.
     super(AppAssertionCredentials, self).__init__(None)
 
   @classmethod
-  def from_json(cls, json):
-    data = simplejson.loads(json)
+  def from_json(cls, json_data):
+    data = json.loads(json_data)
     return AppAssertionCredentials(data['scope'])
 
   def _refresh(self, http_request):
@@ -186,11 +190,22 @@ class AppAssertionCredentials(AssertionCredentials):
     """
     try:
       scopes = self.scope.split()
-      (token, _) = app_identity.get_access_token(scopes)
-    except app_identity.Error, e:
+      (token, _) = app_identity.get_access_token(
+          scopes, service_account_id=self.service_account_id)
+    except app_identity.Error as e:
       raise AccessTokenRefreshError(str(e))
     self.access_token = token
 
+  @property
+  def serialization_data(self):
+    raise NotImplementedError('Cannot serialize credentials for AppEngine.')
+
+  def create_scoped_required(self):
+    return not self.scope
+
+  def create_scoped(self, scopes):
+    return AppAssertionCredentials(scopes, **self._kwargs)
+
 
 class FlowProperty(db.Property):
   """App Engine datastore Property for Flow.
@@ -434,6 +449,7 @@ class StorageByKeyName(Storage):
       entity_key = db.Key.from_path(self._model.kind(), self._key_name)
       db.delete(entity_key)
 
+  @db.non_transactional(allow_existing=True)
   def locked_get(self):
     """Retrieve Credential from datastore.
 
@@ -456,6 +472,7 @@ class StorageByKeyName(Storage):
       credentials.set_store(self)
     return credentials
 
+  @db.non_transactional(allow_existing=True)
   def locked_put(self, credentials):
     """Write a Credentials to the datastore.
 
@@ -468,6 +485,7 @@ class StorageByKeyName(Storage):
     if self._cache:
       self._cache.set(self._key_name, credentials.to_json())
 
+  @db.non_transactional(allow_existing=True)
   def locked_delete(self):
     """Delete Credential from datastore."""
 
@@ -553,16 +571,14 @@ class OAuth2Decorator(object):
   Instantiate and then use with oauth_required or oauth_aware
   as decorators on webapp.RequestHandler methods.
 
-  Example:
+  ::
 
     decorator = OAuth2Decorator(
         client_id='837...ent.com',
         client_secret='Qh...wwI',
         scope='https://www.googleapis.com/auth/plus')
 
-
     class MainHandler(webapp.RequestHandler):
-
       @decorator.oauth_required
       def get(self):
         http = decorator.http()
@@ -650,8 +666,9 @@ class OAuth2Decorator(object):
         provided to this constructor. A string indicating the name of the field
         on the _credentials_class where a Credentials object will be stored.
         Defaults to 'credentials'.
-      **kwargs: dict, Keyword arguments are be passed along as kwargs to the
-        OAuth2WebServerFlow constructor.
+      **kwargs: dict, Keyword arguments are passed along as kwargs to
+        the OAuth2WebServerFlow constructor.
+
     """
     self._tls = threading.local()
     self.flow = None
@@ -798,14 +815,18 @@ class OAuth2Decorator(object):
     url = self.flow.step1_get_authorize_url()
     return str(url)
 
-  def http(self):
+  def http(self, *args, **kwargs):
     """Returns an authorized http instance.
 
     Must only be called from within an @oauth_required decorated method, or
     from within an @oauth_aware decorated method where has_credentials()
     returns True.
+
+    Args:
+        *args: Positional arguments passed to httplib2.Http constructor.
+        **kwargs: Positional arguments passed to httplib2.Http constructor.
     """
-    return self.credentials.authorize(httplib2.Http())
+    return self.credentials.authorize(httplib2.Http(*args, **kwargs))
 
   @property
   def callback_path(self):
@@ -824,7 +845,8 @@ class OAuth2Decorator(object):
   def callback_handler(self):
     """RequestHandler for the OAuth 2.0 redirect callback.
 
-    Usage:
+    Usage::
+
        app = webapp.WSGIApplication([
          ('/index', MyIndexHandler),
          ...,
@@ -858,7 +880,7 @@ class OAuth2Decorator(object):
                                             user)
 
           if decorator._token_response_param and credentials.token_response:
-            resp_json = simplejson.dumps(credentials.token_response)
+            resp_json = json.dumps(credentials.token_response)
             redirect_uri = util._add_query_parameter(
                 redirect_uri, decorator._token_response_param, resp_json)
 
@@ -887,24 +909,23 @@ class OAuth2DecoratorFromClientSecrets(OAuth2Decorator):
   Uses a clientsecrets file as the source for all the information when
   constructing an OAuth2Decorator.
 
-  Example:
+  ::
 
     decorator = OAuth2DecoratorFromClientSecrets(
       os.path.join(os.path.dirname(__file__), 'client_secrets.json')
       scope='https://www.googleapis.com/auth/plus')
 
-
     class MainHandler(webapp.RequestHandler):
-
       @decorator.oauth_required
       def get(self):
         http = decorator.http()
         # http is authorized with the user's Credentials and can be used
         # in API calls
+
   """
 
   @util.positional(3)
-  def __init__(self, filename, scope, message=None, cache=None):
+  def __init__(self, filename, scope, message=None, cache=None, **kwargs):
     """Constructor
 
     Args:
@@ -917,17 +938,20 @@ class OAuth2DecoratorFromClientSecrets(OAuth2Decorator):
         decorator.
       cache: An optional cache service client that implements get() and set()
         methods. See clientsecrets.loadfile() for details.
+      **kwargs: dict, Keyword arguments are passed along as kwargs to
+        the OAuth2WebServerFlow constructor.
     """
     client_type, client_info = clientsecrets.loadfile(filename, cache=cache)
     if client_type not in [
         clientsecrets.TYPE_WEB, clientsecrets.TYPE_INSTALLED]:
       raise InvalidClientSecretsError(
-          'OAuth2Decorator doesn\'t support this OAuth 2.0 flow.')
-    constructor_kwargs = {
-      'auth_uri': client_info['auth_uri'],
-      'token_uri': client_info['token_uri'],
-      'message': message,
-    }
+          "OAuth2Decorator doesn't support this OAuth 2.0 flow.")
+    constructor_kwargs = dict(kwargs)
+    constructor_kwargs.update({
+        'auth_uri': client_info['auth_uri'],
+        'token_uri': client_info['token_uri'],
+        'message': message,
+    })
     revoke_uri = client_info.get('revoke_uri')
     if revoke_uri is not None:
       constructor_kwargs['revoke_uri'] = revoke_uri

oauth2client/clientsecrets.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2011 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -20,8 +20,9 @@ an OAuth 2.0 protected service.
 
 __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 
+import json
+import six
 
-from anyjson import simplejson
 
 # Properties that make a client_secrets.json file valid.
 TYPE_WEB = 'web'
@@ -68,11 +69,21 @@ class InvalidClientSecretsError(Error):
 
 
 def _validate_clientsecrets(obj):
-  if obj is None or len(obj) != 1:
-    raise InvalidClientSecretsError('Invalid file format.')
-  client_type = obj.keys()[0]
-  if client_type not in VALID_CLIENT.keys():
-    raise InvalidClientSecretsError('Unknown client type: %s.' % client_type)
+  _INVALID_FILE_FORMAT_MSG = (
+    'Invalid file format. See '
+    'https://developers.google.com/api-client-library/'
+    'python/guide/aaa_client_secrets')
+
+  if obj is None:
+    raise InvalidClientSecretsError(_INVALID_FILE_FORMAT_MSG)
+  if len(obj) != 1:
+    raise InvalidClientSecretsError(
+      _INVALID_FILE_FORMAT_MSG + ' '
+      'Expected a JSON object with a single property for a "web" or '
+      '"installed" application')
+  client_type = tuple(obj)[0]
+  if client_type not in VALID_CLIENT:
+    raise InvalidClientSecretsError('Unknown client type: %s.' % (client_type,))
   client_info = obj[client_type]
   for prop_name in VALID_CLIENT[client_type]['required']:
     if prop_name not in client_info:
@@ -87,22 +98,19 @@ def _validate_clientsecrets(obj):
 
 
 def load(fp):
-  obj = simplejson.load(fp)
+  obj = json.load(fp)
   return _validate_clientsecrets(obj)
 
 
 def loads(s):
-  obj = simplejson.loads(s)
+  obj = json.loads(s)
   return _validate_clientsecrets(obj)
 
 
 def _loadfile(filename):
   try:
-    fp = file(filename, 'r')
-    try:
-      obj = simplejson.load(fp)
-    finally:
-      fp.close()
+    with open(filename, 'r') as fp:
+      obj = json.load(fp)
   except IOError:
     raise InvalidClientSecretsError('File not found: "%s"' % filename)
   return _validate_clientsecrets(obj)
@@ -114,10 +122,12 @@ def loadfile(filename, cache=None):
   Typical cache storage would be App Engine memcache service,
   but you can pass in any other cache client that implements
   these methods:
-    - get(key, namespace=ns)
-    - set(key, value, namespace=ns)
 
-  Usage:
+  * ``get(key, namespace=ns)``
+  * ``set(key, value, namespace=ns)``
+
+  Usage::
+
     # without caching
     client_type, client_info = loadfile('secrets.json')
     # using App Engine memcache service
@@ -150,4 +160,4 @@ def loadfile(filename, cache=None):
     obj = {client_type: client_info}
     cache.set(filename, obj, namespace=_SECRET_NAMESPACE)
 
-  return obj.iteritems().next()
+  return next(six.iteritems(obj))

oauth2client/crypt.py

@@ -1,7 +1,6 @@
-#!/usr/bin/python2.4
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2011 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -14,13 +13,15 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+"""Crypto-related routines for oauth2client."""
 
 import base64
-import hashlib
+import json
 import logging
+import sys
 import time
 
-from anyjson import simplejson
+import six
 
 
 CLOCK_SKEW_SECS = 300  # 5 minutes in seconds
@@ -38,7 +39,6 @@ class AppIdentityError(Exception):
 try:
   from OpenSSL import crypto
 
-
   class OpenSSLVerifier(object):
     """Verifies the signature on a message."""
 
@@ -62,6 +62,8 @@ try:
         key that this object was constructed with.
       """
       try:
+        if isinstance(message, six.text_type):
+          message = message.encode('utf-8')
         crypto.verify(self._pubkey, signature, message, 'sha256')
         return True
       except:
@@ -104,15 +106,17 @@ try:
       """Signs a message.
 
       Args:
-        message: string, Message to be signed.
+        message: bytes, Message to be signed.
 
       Returns:
         string, The signature of the message for the given key.
       """
+      if isinstance(message, six.text_type):
+        message = message.encode('utf-8')
       return crypto.sign(self._key, message, 'sha256')
 
     @staticmethod
-    def from_string(key, password='notasecret'):
+    def from_string(key, password=b'notasecret'):
       """Construct a Signer instance from a string.
 
       Args:
@@ -125,21 +129,45 @@ try:
       Raises:
         OpenSSL.crypto.Error if the key can't be parsed.
       """
-      if key.startswith('-----BEGIN '):
-        pkey = crypto.load_privatekey(crypto.FILETYPE_PEM, key)
+      parsed_pem_key = _parse_pem_key(key)
+      if parsed_pem_key:
+        pkey = crypto.load_privatekey(crypto.FILETYPE_PEM, parsed_pem_key)
       else:
+        if isinstance(password, six.text_type):
+          password = password.encode('utf-8')
         pkey = crypto.load_pkcs12(key, password).get_privatekey()
       return OpenSSLSigner(pkey)
 
+
+  def pkcs12_key_as_pem(private_key_text, private_key_password):
+    """Convert the contents of a PKCS12 key to PEM using OpenSSL.
+
+    Args:
+      private_key_text: String. Private key.
+      private_key_password: String. Password for PKCS12.
+
+    Returns:
+      String. PEM contents of ``private_key_text``.
+    """
+    decoded_body = base64.b64decode(private_key_text)
+    if isinstance(private_key_password, six.string_types):
+      private_key_password = private_key_password.encode('ascii')
+
+    pkcs12 = crypto.load_pkcs12(decoded_body, private_key_password)
+    return crypto.dump_privatekey(crypto.FILETYPE_PEM,
+                                  pkcs12.get_privatekey())
 except ImportError:
   OpenSSLVerifier = None
   OpenSSLSigner = None
+  def pkcs12_key_as_pem(*args, **kwargs):
+    raise NotImplementedError('pkcs12_key_as_pem requires OpenSSL.')
 
 
 try:
   from Crypto.PublicKey import RSA
   from Crypto.Hash import SHA256
   from Crypto.Signature import PKCS1_v1_5
+  from Crypto.Util.asn1 import DerSequence
 
 
   class PyCryptoVerifier(object):
@@ -181,14 +209,17 @@ try:
 
       Returns:
         Verifier instance.
-
-      Raises:
-        NotImplementedError if is_x509_cert is true.
       """
       if is_x509_cert:
-        raise NotImplementedError(
-            'X509 certs are not supported by the PyCrypto library. '
-            'Try using PyOpenSSL if native code is an option.')
+        if isinstance(key_pem, six.text_type):
+          key_pem = key_pem.encode('ascii')
+        pemLines = key_pem.replace(b' ', b'').split()
+        certDer = _urlsafe_b64decode(b''.join(pemLines[1:-1]))
+        certSeq = DerSequence()
+        certSeq.decode(certDer)
+        tbsSeq = DerSequence()
+        tbsSeq.decode(certSeq[0])
+        pubkey = RSA.importKey(tbsSeq[6])
       else:
         pubkey = RSA.importKey(key_pem)
       return PyCryptoVerifier(pubkey)
@@ -214,6 +245,8 @@ try:
       Returns:
         string, The signature of the message for the given key.
       """
+      if isinstance(message, six.text_type):
+        message = message.encode('utf-8')
       return PKCS1_v1_5.new(self._key).sign(SHA256.new(message))
 
     @staticmethod
@@ -230,11 +263,12 @@ try:
       Raises:
         NotImplementedError if they key isn't in PEM format.
       """
-      if key.startswith('-----BEGIN '):
-        pkey = RSA.importKey(key)
+      parsed_pem_key = _parse_pem_key(key)
+      if parsed_pem_key:
+        pkey = RSA.importKey(parsed_pem_key)
       else:
         raise NotImplementedError(
-            'PKCS12 format is not supported by the PyCrpto library. '
+            'PKCS12 format is not supported by the PyCrypto library. '
             'Try converting to a "PEM" '
             '(openssl pkcs12 -in xxxxx.p12 -nodes -nocerts > privatekey.pem) '
             'or using PyOpenSSL if native code is an option.')
@@ -256,19 +290,39 @@ else:
                     'PyOpenSSL, or PyCrypto 2.6 or later')
 
 
+def _parse_pem_key(raw_key_input):
+  """Identify and extract PEM keys.
+
+  Determines whether the given key is in the format of PEM key, and extracts
+  the relevant part of the key if it is.
+
+  Args:
+    raw_key_input: The contents of a private key file (either PEM or PKCS12).
+
+  Returns:
+    string, The actual key if the contents are from a PEM file, or else None.
+  """
+  offset = raw_key_input.find(b'-----BEGIN ')
+  if offset != -1:
+    return raw_key_input[offset:]
+
+
 def _urlsafe_b64encode(raw_bytes):
-  return base64.urlsafe_b64encode(raw_bytes).rstrip('=')
+  if isinstance(raw_bytes, six.text_type):
+    raw_bytes = raw_bytes.encode('utf-8')
+  return base64.urlsafe_b64encode(raw_bytes).decode('ascii').rstrip('=')
 
 
 def _urlsafe_b64decode(b64string):
   # Guard against unicode strings, which base64 can't handle.
-  b64string = b64string.encode('ascii')
-  padded = b64string + '=' * (4 - len(b64string) % 4)
+  if isinstance(b64string, six.text_type):
+    b64string = b64string.encode('ascii')
+  padded = b64string + b'=' * (4 - len(b64string) % 4)
   return base64.urlsafe_b64decode(padded)
 
 
 def _json_encode(data):
-  return simplejson.dumps(data, separators = (',', ':'))
+  return json.dumps(data, separators=(',', ':'))
 
 
 def make_signed_jwt(signer, payload):
@@ -286,8 +340,8 @@ def make_signed_jwt(signer, payload):
   header = {'typ': 'JWT', 'alg': 'RS256'}
 
   segments = [
-          _urlsafe_b64encode(_json_encode(header)),
-          _urlsafe_b64encode(_json_encode(payload)),
+      _urlsafe_b64encode(_json_encode(header)),
+      _urlsafe_b64encode(_json_encode(payload)),
   ]
   signing_input = '.'.join(segments)
 
@@ -318,9 +372,8 @@ def verify_signed_jwt_with_certs(jwt, certs, audience):
   """
   segments = jwt.split('.')
 
-  if (len(segments) != 3):
-    raise AppIdentityError(
-      'Wrong number of segments in token: %s' % jwt)
+  if len(segments) != 3:
+    raise AppIdentityError('Wrong number of segments in token: %s' % jwt)
   signed = '%s.%s' % (segments[0], segments[1])
 
   signature = _urlsafe_b64decode(segments[2])
@@ -328,15 +381,15 @@ def verify_signed_jwt_with_certs(jwt, certs, audience):
   # Parse token.
   json_body = _urlsafe_b64decode(segments[1])
   try:
-    parsed = simplejson.loads(json_body)
+    parsed = json.loads(json_body.decode('utf-8'))
   except:
     raise AppIdentityError('Can\'t parse token: %s' % json_body)
 
   # Check signature.
   verified = False
-  for (keyname, pem) in certs.items():
+  for pem in certs.values():
     verifier = Verifier.from_string(pem, True)
-    if (verifier.verify(signed, signature)):
+    if verifier.verify(signed, signature):
       verified = True
       break
   if not verified:
@@ -349,21 +402,20 @@ def verify_signed_jwt_with_certs(jwt, certs, audience):
   earliest = iat - CLOCK_SKEW_SECS
 
   # Check expiration timestamp.
-  now = long(time.time())
+  now = int(time.time())
   exp = parsed.get('exp')
   if exp is None:
     raise AppIdentityError('No exp field in token: %s' % json_body)
   if exp >= now + MAX_TOKEN_LIFETIME_SECS:
-    raise AppIdentityError(
-      'exp field too far in future: %s' % json_body)
+    raise AppIdentityError('exp field too far in future: %s' % json_body)
   latest = exp + CLOCK_SKEW_SECS
 
   if now < earliest:
     raise AppIdentityError('Token used too early, %d < %d: %s' %
-      (now, earliest, json_body))
+                           (now, earliest, json_body))
   if now > latest:
     raise AppIdentityError('Token used too late, %d > %d: %s' %
-      (now, latest, json_body))
+                           (now, latest, json_body))
 
   # Check audience.
   if audience is not None:
@@ -372,6 +424,6 @@ def verify_signed_jwt_with_certs(jwt, certs, audience):
       raise AppIdentityError('No aud field in token: %s' % json_body)
     if aud != audience:
       raise AppIdentityError('Wrong recipient, %s != %s: %s' %
-          (aud, audience, json_body))
+                             (aud, audience, json_body))
 
   return parsed

oauth2client/devshell.py

@@ -0,0 +1,136 @@
+# Copyright 2015 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""OAuth 2.0 utitilies for Google Developer Shell environment."""
+
+import json
+import os
+
+from oauth2client import client
+
+
+DEVSHELL_ENV = 'DEVSHELL_CLIENT_PORT'
+
+
+class Error(Exception):
+  """Errors for this module."""
+  pass
+
+
+class CommunicationError(Error):
+  """Errors for communication with the Developer Shell server."""
+
+
+class NoDevshellServer(Error):
+  """Error when no Developer Shell server can be contacted."""
+
+
+# The request for credential information to the Developer Shell client socket is
+# always an empty PBLite-formatted JSON object, so just define it as a constant.
+CREDENTIAL_INFO_REQUEST_JSON = '[]'
+
+
+class CredentialInfoResponse(object):
+  """Credential information response from Developer Shell server.
+
+  The credential information response from Developer Shell socket is a
+  PBLite-formatted JSON array with fields encoded by their index in the array:
+  * Index 0 - user email
+  * Index 1 - default project ID. None if the project context is not known.
+  * Index 2 - OAuth2 access token. None if there is no valid auth context.
+  """
+
+  def __init__(self, json_string):
+    """Initialize the response data from JSON PBLite array."""
+    pbl = json.loads(json_string)
+    if not isinstance(pbl, list):
+      raise ValueError('Not a list: ' + str(pbl))
+    pbl_len = len(pbl)
+    self.user_email = pbl[0] if pbl_len > 0 else None
+    self.project_id = pbl[1] if pbl_len > 1 else None
+    self.access_token = pbl[2] if pbl_len > 2 else None
+
+
+def _SendRecv():
+  """Communicate with the Developer Shell server socket."""
+
+  port = int(os.getenv(DEVSHELL_ENV, 0))
+  if port == 0:
+    raise NoDevshellServer()
+
+  import socket
+
+  sock = socket.socket()
+  sock.connect(('localhost', port))
+
+  data = CREDENTIAL_INFO_REQUEST_JSON
+  msg = '%s\n%s' % (len(data), data)
+  sock.sendall(msg.encode())
+
+  header = sock.recv(6).decode()
+  if '\n' not in header:
+    raise CommunicationError('saw no newline in the first 6 bytes')
+  len_str, json_str = header.split('\n', 1)
+  to_read = int(len_str) - len(json_str)
+  if to_read > 0:
+    json_str += sock.recv(to_read, socket.MSG_WAITALL).decode()
+
+  return CredentialInfoResponse(json_str)
+
+
+class DevshellCredentials(client.GoogleCredentials):
+  """Credentials object for Google Developer Shell environment.
+
+  This object will allow a Google Developer Shell session to identify its user
+  to Google and other OAuth 2.0 servers that can verify assertions. It can be
+  used for the purpose of accessing data stored under the user account.
+
+  This credential does not require a flow to instantiate because it represents
+  a two legged flow, and therefore has all of the required information to
+  generate and refresh its own access tokens.
+  """
+
+  def __init__(self, user_agent=None):
+    super(DevshellCredentials, self).__init__(
+        None,  # access_token, initialized below
+        None,  # client_id
+        None,  # client_secret
+        None,  # refresh_token
+        None,  # token_expiry
+        None,  # token_uri
+        user_agent)
+    self._refresh(None)
+
+  def _refresh(self, http_request):
+    self.devshell_response = _SendRecv()
+    self.access_token = self.devshell_response.access_token
+
+  @property
+  def user_email(self):
+    return self.devshell_response.user_email
+
+  @property
+  def project_id(self):
+    return self.devshell_response.project_id
+
+  @classmethod
+  def from_json(cls, json_data):
+    raise NotImplementedError(
+        'Cannot load Developer Shell credentials from JSON.')
+
+  @property
+  def serialization_data(self):
+    raise NotImplementedError(
+        'Cannot serialize Developer Shell credentials.')
+

oauth2client/django_orm.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2010 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -116,14 +116,21 @@ class Storage(BaseStorage):
         credential.set_store(self)
     return credential
 
-  def locked_put(self, credentials):
+  def locked_put(self, credentials, overwrite=False):
     """Write a Credentials to the datastore.
 
     Args:
       credentials: Credentials, the credentials to store.
+      overwrite: Boolean, indicates whether you would like these credentials to
+                          overwrite any existing stored credentials.
     """
     args = {self.key_name: self.key_value}
-    entity = self.model_class(**args)
+
+    if overwrite:
+      entity, unused_is_new = self.model_class.objects.get_or_create(**args)
+    else:
+      entity = self.model_class(**args)
+
     setattr(entity, self.property_name, credentials)
     entity.save()
 

oauth2client/file.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2010 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -21,12 +21,10 @@ credentials.
 __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 
 import os
-import stat
 import threading
 
-from anyjson import simplejson
-from client import Storage as BaseStorage
-from client import Credentials
+from oauth2client.client import Credentials
+from oauth2client.client import Storage as BaseStorage
 
 
 class CredentialsFileSymbolicLinkError(Exception):
@@ -41,7 +39,6 @@ class Storage(BaseStorage):
     self._lock = threading.Lock()
 
   def _validate_file(self):
-    return
     if os.path.islink(self._filename):
       raise CredentialsFileSymbolicLinkError(
           'File: %s is a symbolic link.' % self._filename)
@@ -93,7 +90,7 @@ class Storage(BaseStorage):
     simple version of "touch" to ensure the file has been created.
     """
     if not os.path.exists(self._filename):
-      old_umask = os.umask(0177)
+      old_umask = os.umask(0o177)
       try:
         open(self._filename, 'a+b').close()
       finally:
@@ -111,7 +108,7 @@ class Storage(BaseStorage):
 
     self._create_file_if_needed()
     self._validate_file()
-    f = open(self._filename, 'wb')
+    f = open(self._filename, 'w')
     f.write(credentials.to_json())
     f.close()
 

oauth2client/gce.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2012 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -19,12 +19,11 @@ Utilities for making it easier to use OAuth 2.0 on Google Compute Engine.
 
 __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 
-import httplib2
+import json
 import logging
-import uritemplate
+from six.moves import urllib
 
 from oauth2client import util
-from oauth2client.anyjson import simplejson
 from oauth2client.client import AccessTokenRefreshError
 from oauth2client.client import AssertionCredentials
 
@@ -57,13 +56,14 @@ class AppAssertionCredentials(AssertionCredentials):
         requested.
     """
     self.scope = util.scopes_to_string(scope)
+    self.kwargs = kwargs
 
     # Assertion type is no longer used, but still in the parent class signature.
     super(AppAssertionCredentials, self).__init__(None)
 
   @classmethod
-  def from_json(cls, json):
-    data = simplejson.loads(json)
+  def from_json(cls, json_data):
+    data = json.loads(json_data)
     return AppAssertionCredentials(data['scope'])
 
   def _refresh(self, http_request):
@@ -78,13 +78,28 @@ class AppAssertionCredentials(AssertionCredentials):
     Raises:
       AccessTokenRefreshError: When the refresh fails.
     """
-    uri = uritemplate.expand(META, {'scope': self.scope})
+    query = '?scope=%s' % urllib.parse.quote(self.scope, '')
+    uri = META.replace('{?scope}', query)
     response, content = http_request(uri)
     if response.status == 200:
       try:
-        d = simplejson.loads(content)
-      except StandardError, e:
+        d = json.loads(content)
+      except Exception as e:
         raise AccessTokenRefreshError(str(e))
       self.access_token = d['accessToken']
     else:
+      if response.status == 404:
+        content += (' This can occur if a VM was created'
+                    ' with no service account or scopes.')
       raise AccessTokenRefreshError(content)
+
+  @property
+  def serialization_data(self):
+    raise NotImplementedError(
+        'Cannot serialize credentials for GCE service accounts.')
+
+  def create_scoped_required(self):
+    return not self.scope
+
+  def create_scoped(self, scopes):
+    return AppAssertionCredentials(scopes, **self.kwargs)

oauth2client/keyring_storage.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2012 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -19,11 +19,12 @@ A Storage for Credentials that uses the keyring module.
 
 __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 
-import keyring
 import threading
 
-from client import Storage as BaseStorage
-from client import Credentials
+import keyring
+
+from oauth2client.client import Credentials
+from oauth2client.client import Storage as BaseStorage
 
 
 class Storage(BaseStorage):

oauth2client/locked_file.py

@@ -1,4 +1,4 @@
-# Copyright 2011 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -17,17 +17,21 @@
 This module first tries to use fcntl locking to ensure serialized access
 to a file, then falls back on a lock file if that is unavialable.
 
-Usage:
+Usage::
+
     f = LockedFile('filename', 'r+b', 'rb')
     f.open_and_lock()
     if f.is_locked():
-      print 'Acquired filename with r+b mode'
+      print('Acquired filename with r+b mode')
       f.file_handle().write('locked data')
     else:
-      print 'Aquired filename with rb mode'
+      print('Acquired filename with rb mode')
     f.unlock_and_close()
+
 """
 
+from __future__ import print_function
+
 __author__ = 'cache@google.com (David T McWherter)'
 
 import errno
@@ -70,6 +74,7 @@ class _Opener(object):
     self._mode = mode
     self._fallback_mode = fallback_mode
     self._fh = None
+    self._lock_fd = None
 
   def is_locked(self):
     """Was the file locked."""
@@ -122,7 +127,7 @@ class _PosixOpener(_Opener):
     validate_file(self._filename)
     try:
       self._fh = open(self._filename, self._mode)
-    except IOError, e:
+    except IOError as e:
       # If we can't access with _mode, try _fallback_mode and don't lock.
       if e.errno == errno.EACCES:
         self._fh = open(self._filename, self._fallback_mode)
@@ -137,12 +142,12 @@ class _PosixOpener(_Opener):
         self._locked = True
         break
 
-      except OSError, e:
+      except OSError as e:
         if e.errno != errno.EEXIST:
           raise
         if (time.time() - start_time) >= timeout:
-          logger.warn('Could not acquire lock %s in %s seconds' % (
-              lock_filename, timeout))
+          logger.warn('Could not acquire lock %s in %s seconds',
+                      lock_filename, timeout)
           # Close the file and open in fallback_mode.
           if self._fh:
             self._fh.close()
@@ -192,9 +197,9 @@ try:
       validate_file(self._filename)
       try:
         self._fh = open(self._filename, self._mode)
-      except IOError, e:
+      except IOError as e:
         # If we can't access with _mode, try _fallback_mode and don't lock.
-        if e.errno == errno.EACCES:
+        if e.errno in (errno.EPERM, errno.EACCES):
           self._fh = open(self._filename, self._fallback_mode)
           return
 
@@ -204,16 +209,16 @@ try:
           fcntl.lockf(self._fh.fileno(), fcntl.LOCK_EX)
           self._locked = True
           return
-        except IOError, e:
+        except IOError as e:
           # If not retrying, then just pass on the error.
           if timeout == 0:
-            raise e
+            raise
           if e.errno != errno.EACCES:
-            raise e
+            raise
           # We could not acquire the lock. Try again.
           if (time.time() - start_time) >= timeout:
-            logger.warn('Could not lock %s in %s seconds' % (
-                self._filename, timeout))
+            logger.warn('Could not lock %s in %s seconds',
+                        self._filename, timeout)
             if self._fh:
               self._fh.close()
             self._fh = open(self._filename, self._fallback_mode)
@@ -267,7 +272,7 @@ try:
       validate_file(self._filename)
       try:
         self._fh = open(self._filename, self._mode)
-      except IOError, e:
+      except IOError as e:
         # If we can't access with _mode, try _fallback_mode and don't lock.
         if e.errno == errno.EACCES:
           self._fh = open(self._filename, self._fallback_mode)
@@ -284,9 +289,9 @@ try:
               pywintypes.OVERLAPPED())
           self._locked = True
           return
-        except pywintypes.error, e:
+        except pywintypes.error as e:
           if timeout == 0:
-            raise e
+            raise
 
           # If the error is not that the file is already in use, raise.
           if e[0] != _Win32Opener.FILE_IN_USE_ERROR:
@@ -308,7 +313,7 @@ try:
         try:
           hfile = win32file._get_osfhandle(self._fh.fileno())
           win32file.UnlockFileEx(hfile, 0, -0x10000, pywintypes.OVERLAPPED())
-        except pywintypes.error, e:
+        except pywintypes.error as e:
           if e[0] != _Win32Opener.FILE_ALREADY_UNLOCKED_ERROR:
             raise
       self._locked = False

oauth2client/multistore_file.py

@@ -1,4 +1,4 @@
-# Copyright 2011 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -19,41 +19,43 @@ credentials can be stored in one file. That file supports locking
 both in a single process and across processes.
 
 The credential themselves are keyed off of:
+
 * client_id
 * user_agent
 * scope
 
-The format of the stored data is like so:
-{
-  'file_version': 1,
-  'data': [
-    {
-      'key': {
-        'clientId': '<client id>',
-        'userAgent': '<user agent>',
-        'scope': '<scope>'
-      },
-      'credential': {
-        # JSON serialized Credentials.
+The format of the stored data is like so::
+
+  {
+    'file_version': 1,
+    'data': [
+      {
+        'key': {
+          'clientId': '<client id>',
+          'userAgent': '<user agent>',
+          'scope': '<scope>'
+        },
+        'credential': {
+          # JSON serialized Credentials.
+        }
       }
-    }
-  ]
-}
+    ]
+  }
+
 """
 
 __author__ = 'jbeda@google.com (Joe Beda)'
 
-import base64
 import errno
+import json
 import logging
 import os
 import threading
 
-from anyjson import simplejson
-from oauth2client.client import Storage as BaseStorage
 from oauth2client.client import Credentials
+from oauth2client.client import Storage as BaseStorage
 from oauth2client import util
-from locked_file import LockedFile
+from oauth2client.locked_file import LockedFile
 
 logger = logging.getLogger(__name__)
 
@@ -64,12 +66,10 @@ _multistores_lock = threading.Lock()
 
 class Error(Exception):
   """Base error for this module."""
-  pass
 
 
 class NewerCredentialStoreError(Error):
-  """The credential store is a newer version that supported."""
-  pass
+  """The credential store is a newer version than supported."""
 
 
 @util.positional(4)
@@ -193,7 +193,7 @@ class _MultiStore(object):
 
     This will create the file if necessary.
     """
-    self._file = LockedFile(filename, 'r+b', 'rb')
+    self._file = LockedFile(filename, 'r+', 'r')
     self._thread_lock = threading.Lock()
     self._read_only = False
     self._warn_on_readonly = warn_on_readonly
@@ -271,7 +271,7 @@ class _MultiStore(object):
     simple version of "touch" to ensure the file has been created.
     """
     if not os.path.exists(self._file.filename()):
-      old_umask = os.umask(0177)
+      old_umask = os.umask(0o177)
       try:
         open(self._file.filename(), 'a+b').close()
       finally:
@@ -280,13 +280,23 @@ class _MultiStore(object):
   def _lock(self):
     """Lock the entire multistore."""
     self._thread_lock.acquire()
-    self._file.open_and_lock()
+    try:
+      self._file.open_and_lock()
+    except IOError as e:
+      if e.errno == errno.ENOSYS:
+        logger.warn('File system does not support locking the credentials '
+                    'file.')
+      elif e.errno ==  errno.ENOLCK:
+        logger.warn('File system is out of resources for writing the '
+                    'credentials file (is your disk full?).')
+      else:
+        raise
     if not self._file.is_locked():
       self._read_only = True
       if self._warn_on_readonly:
         logger.warn('The credentials file (%s) is not writable. Opening in '
                     'read-only mode. Any refreshed credentials will only be '
-                    'valid for this run.' % self._file.filename())
+                    'valid for this run.', self._file.filename())
     if os.path.getsize(self._file.filename()) == 0:
       logger.debug('Initializing empty multistore file')
       # The multistore is empty so write out an empty file.
@@ -315,7 +325,7 @@ class _MultiStore(object):
     """
     assert self._thread_lock.locked()
     self._file.file_handle().seek(0)
-    return simplejson.load(self._file.file_handle())
+    return json.load(self._file.file_handle())
 
   def _locked_json_write(self, data):
     """Write a JSON serializable data structure to the multistore.
@@ -329,7 +339,7 @@ class _MultiStore(object):
     if self._read_only:
       return
     self._file.file_handle().seek(0)
-    simplejson.dump(data, self._file.file_handle(), sort_keys=True, indent=2)
+    json.dump(data, self._file.file_handle(), sort_keys=True, indent=2, separators=(',', ': '))
     self._file.file_handle().truncate()
 
   def _refresh_data_cache(self):
@@ -387,7 +397,7 @@ class _MultiStore(object):
     raw_key = cred_entry['key']
     key = util.dict_to_tuple_key(raw_key)
     credential = None
-    credential = Credentials.new_from_json(simplejson.dumps(cred_entry['credential']))
+    credential = Credentials.new_from_json(json.dumps(cred_entry['credential']))
     return (key, credential)
 
   def _write(self):
@@ -400,7 +410,7 @@ class _MultiStore(object):
     raw_data['data'] = raw_creds
     for (cred_key, cred) in self._data.items():
       raw_key = dict(cred_key)
-      raw_cred = simplejson.loads(cred.to_json())
+      raw_cred = json.loads(cred.to_json())
       raw_creds.append({'key': raw_key, 'credential': raw_cred})
     self._locked_json_write(raw_data)
 

oauth2client/old_run.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2013 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -15,6 +15,7 @@
 """This module holds the old run() function which is deprecated, the
 tools.run_flow() function should be used in its place."""
 
+from __future__ import print_function
 
 import logging
 import socket
@@ -22,11 +23,12 @@ import sys
 import webbrowser
 
 import gflags
+from six.moves import input
 
 from oauth2client import client
 from oauth2client import util
-from tools import ClientRedirectHandler
-from tools import ClientRedirectServer
+from oauth2client.tools import ClientRedirectHandler
+from oauth2client.tools import ClientRedirectServer
 
 
 FLAGS = gflags.FLAGS
@@ -48,39 +50,38 @@ gflags.DEFINE_multi_int('auth_host_port', [8080, 8090],
 def run(flow, storage, http=None):
   """Core code for a command-line application.
 
-  The run() function is called from your application and runs through all
-  the steps to obtain credentials. It takes a Flow argument and attempts to
-  open an authorization server page in the user's default web browser. The
-  server asks the user to grant your application access to the user's data.
-  If the user grants access, the run() function returns new credentials. The
-  new credentials are also stored in the Storage argument, which updates the
-  file associated with the Storage object.
+  The ``run()`` function is called from your application and runs
+  through all the steps to obtain credentials. It takes a ``Flow``
+  argument and attempts to open an authorization server page in the
+  user's default web browser. The server asks the user to grant your
+  application access to the user's data. If the user grants access,
+  the ``run()`` function returns new credentials. The new credentials
+  are also stored in the ``storage`` argument, which updates the file
+  associated with the ``Storage`` object.
 
   It presumes it is run from a command-line application and supports the
   following flags:
 
-    --auth_host_name: Host name to use when running a local web server
-      to handle redirects during OAuth authorization.
-      (default: 'localhost')
+    ``--auth_host_name`` (string, default: ``localhost``)
+       Host name to use when running a local web server to handle
+       redirects during OAuth authorization.
 
-    --auth_host_port: Port to use when running a local web server to handle
-    redirects during OAuth authorization.;
-      repeat this option to specify a list of values
-      (default: '[8080, 8090]')
-      (an integer)
+    ``--auth_host_port`` (integer, default: ``[8080, 8090]``)
+       Port to use when running a local web server to handle redirects
+       during OAuth authorization. Repeat this option to specify a list
+       of values.
 
-    --[no]auth_local_webserver: Run a local web server to handle redirects
-      during OAuth authorization.
-      (default: 'true')
+    ``--[no]auth_local_webserver`` (boolean, default: ``True``)
+       Run a local web server to handle redirects during OAuth authorization.
 
-  Since it uses flags make sure to initialize the gflags module before
-  calling run().
+  Since it uses flags make sure to initialize the ``gflags`` module before
+  calling ``run()``.
 
   Args:
     flow: Flow, an OAuth 2.0 Flow to step through.
-    storage: Storage, a Storage to store the credential in.
-    http: An instance of httplib2.Http.request
-         or something that acts like it.
+    storage: Storage, a ``Storage`` to store the credential in.
+    http: An instance of ``httplib2.Http.request`` or something that acts
+        like it.
 
   Returns:
     Credentials, the obtained credential.
@@ -96,20 +97,20 @@ def run(flow, storage, http=None):
       try:
         httpd = ClientRedirectServer((FLAGS.auth_host_name, port),
                                      ClientRedirectHandler)
-      except socket.error, e:
+      except socket.error as e:
         pass
       else:
         success = True
         break
     FLAGS.auth_local_webserver = success
     if not success:
-      print 'Failed to start a local webserver listening on either port 8080'
-      print 'or port 9090. Please check your firewall settings and locally'
-      print 'running programs that may be blocking or using those ports.'
-      print
-      print 'Falling back to --noauth_local_webserver and continuing with',
-      print 'authorization.'
-      print
+      print('Failed to start a local webserver listening on either port 8080')
+      print('or port 9090. Please check your firewall settings and locally')
+      print('running programs that may be blocking or using those ports.')
+      print()
+      print('Falling back to --noauth_local_webserver and continuing with')
+      print('authorization.')
+      print()
 
   if FLAGS.auth_local_webserver:
     oauth_callback = 'http://%s:%s/' % (FLAGS.auth_host_name, port_number)
@@ -120,20 +121,20 @@ def run(flow, storage, http=None):
 
   if FLAGS.auth_local_webserver:
     webbrowser.open(authorize_url, new=1, autoraise=True)
-    print 'Your browser has been opened to visit:'
-    print
-    print '    ' + authorize_url
-    print
-    print 'If your browser is on a different machine then exit and re-run'
-    print 'this application with the command-line parameter '
-    print
-    print '  --noauth_local_webserver'
-    print
+    print('Your browser has been opened to visit:')
+    print()
+    print('    ' + authorize_url)
+    print()
+    print('If your browser is on a different machine then exit and re-run')
+    print('this application with the command-line parameter ')
+    print()
+    print('  --noauth_local_webserver')
+    print()
   else:
-    print 'Go to the following link in your browser:'
-    print
-    print '    ' + authorize_url
-    print
+    print('Go to the following link in your browser:')
+    print()
+    print('    ' + authorize_url)
+    print()
 
   code = None
   if FLAGS.auth_local_webserver:
@@ -143,18 +144,18 @@ def run(flow, storage, http=None):
     if 'code' in httpd.query_params:
       code = httpd.query_params['code']
     else:
-      print 'Failed to find "code" in the query parameters of the redirect.'
+      print('Failed to find "code" in the query parameters of the redirect.')
       sys.exit('Try running with --noauth_local_webserver.')
   else:
-    code = raw_input('Enter verification code: ').strip()
+    code = input('Enter verification code: ').strip()
 
   try:
     credential = flow.step2_exchange(code, http=http)
-  except client.FlowExchangeError, e:
+  except client.FlowExchangeError as e:
     sys.exit('Authentication has failed: %s' % e)
 
   storage.put(credential)
   credential.set_store(storage)
-  print 'Authentication successful.'
+  print('Authentication successful.')
 
   return credential

oauth2client/service_account.py

@@ -0,0 +1,139 @@
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""A service account credentials class.
+
+This credentials class is implemented on top of rsa library.
+"""
+
+import base64
+import json
+import six
+import time
+
+from pyasn1.codec.ber import decoder
+from pyasn1_modules.rfc5208 import PrivateKeyInfo
+import rsa
+
+from oauth2client import GOOGLE_REVOKE_URI
+from oauth2client import GOOGLE_TOKEN_URI
+from oauth2client import util
+from oauth2client.client import AssertionCredentials
+
+
+class _ServiceAccountCredentials(AssertionCredentials):
+  """Class representing a service account (signed JWT) credential."""
+
+  MAX_TOKEN_LIFETIME_SECS = 3600 # 1 hour in seconds
+
+  def __init__(self, service_account_id, service_account_email, private_key_id,
+               private_key_pkcs8_text, scopes, user_agent=None,
+               token_uri=GOOGLE_TOKEN_URI, revoke_uri=GOOGLE_REVOKE_URI,
+               **kwargs):
+
+    super(_ServiceAccountCredentials, self).__init__(
+        None, user_agent=user_agent, token_uri=token_uri, revoke_uri=revoke_uri)
+
+    self._service_account_id = service_account_id
+    self._service_account_email = service_account_email
+    self._private_key_id = private_key_id
+    self._private_key = _get_private_key(private_key_pkcs8_text)
+    self._private_key_pkcs8_text = private_key_pkcs8_text
+    self._scopes = util.scopes_to_string(scopes)
+    self._user_agent = user_agent
+    self._token_uri = token_uri
+    self._revoke_uri = revoke_uri
+    self._kwargs = kwargs
+
+  def _generate_assertion(self):
+    """Generate the assertion that will be used in the request."""
+
+    header = {
+        'alg': 'RS256',
+        'typ': 'JWT',
+        'kid': self._private_key_id
+    }
+
+    now = int(time.time())
+    payload = {
+        'aud': self._token_uri,
+        'scope': self._scopes,
+        'iat': now,
+        'exp': now + _ServiceAccountCredentials.MAX_TOKEN_LIFETIME_SECS,
+        'iss': self._service_account_email
+    }
+    payload.update(self._kwargs)
+
+    assertion_input = (_urlsafe_b64encode(header) + b'.' +
+                       _urlsafe_b64encode(payload))
+
+    # Sign the assertion.
+    rsa_bytes = rsa.pkcs1.sign(assertion_input, self._private_key, 'SHA-256')
+    signature = base64.urlsafe_b64encode(rsa_bytes).rstrip(b'=')
+
+    return assertion_input + b'.' + signature
+
+  def sign_blob(self, blob):
+    # Ensure that it is bytes
+    try:
+      blob = blob.encode('utf-8')
+    except AttributeError:
+      pass
+    return (self._private_key_id,
+            rsa.pkcs1.sign(blob, self._private_key, 'SHA-256'))
+
+  @property
+  def service_account_email(self):
+    return self._service_account_email
+
+  @property
+  def serialization_data(self):
+    return {
+        'type': 'service_account',
+        'client_id': self._service_account_id,
+        'client_email': self._service_account_email,
+        'private_key_id': self._private_key_id,
+        'private_key': self._private_key_pkcs8_text
+    }
+
+  def create_scoped_required(self):
+    return not self._scopes
+
+  def create_scoped(self, scopes):
+    return _ServiceAccountCredentials(self._service_account_id,
+                                      self._service_account_email,
+                                      self._private_key_id,
+                                      self._private_key_pkcs8_text,
+                                      scopes,
+                                      user_agent=self._user_agent,
+                                      token_uri=self._token_uri,
+                                      revoke_uri=self._revoke_uri,
+                                      **self._kwargs)
+
+
+def _urlsafe_b64encode(data):
+  return base64.urlsafe_b64encode(
+      json.dumps(data, separators=(',', ':')).encode('UTF-8')).rstrip(b'=')
+
+
+def _get_private_key(private_key_pkcs8_text):
+  """Get an RSA private key object from a pkcs8 representation."""
+
+  if not isinstance(private_key_pkcs8_text, six.binary_type):
+    private_key_pkcs8_text = private_key_pkcs8_text.encode('ascii')
+  der = rsa.pem.load_pem(private_key_pkcs8_text, 'PRIVATE KEY')
+  asn1_private_key, _ = decoder.decode(der, asn1Spec=PrivateKeyInfo())
+  return rsa.PrivateKey.load_pkcs1(
+      asn1_private_key.getComponentByName('privateKey').asOctets(),
+      format='DER')

oauth2client/tools.py

@@ -1,4 +1,4 @@
-# Copyright (C) 2013 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -19,27 +19,22 @@ generated credentials in a common file that is used by other example apps in
 the same directory.
 """
 
+from __future__ import print_function
+
 __author__ = 'jcgregorio@google.com (Joe Gregorio)'
 __all__ = ['argparser', 'run_flow', 'run', 'message_if_missing']
 
-
-import BaseHTTPServer
-#import argparse
-import httplib2
 import logging
-import os
 import socket
 import sys
-import webbrowser
+
+from six.moves import BaseHTTPServer
+from six.moves import urllib
+from six.moves import input
 
 from oauth2client import client
-from oauth2client import file
 from oauth2client import util
 
-try:
-  from urlparse import parse_qsl
-except ImportError:
-  from cgi import parse_qsl
 
 _CLIENT_SECRETS_MESSAGE = """WARNING: Please configure OAuth 2.0
 
@@ -52,20 +47,27 @@ with information from the APIs Console <https://code.google.com/apis/console>.
 
 """
 
-# run_parser is an ArgumentParser that contains command-line options expected
+def _CreateArgumentParser():
+  try:
+    import argparse
+  except ImportError:
+    return None
+  parser = argparse.ArgumentParser(add_help=False)
+  parser.add_argument('--auth_host_name', default='localhost',
+                      help='Hostname when running a local web server.')
+  parser.add_argument('--noauth_local_webserver', action='store_true',
+                      default=False, help='Do not run a local web server.')
+  parser.add_argument('--auth_host_port', default=[8080, 8090], type=int,
+                      nargs='*', help='Port web server should listen on.')
+  parser.add_argument('--logging_level', default='ERROR',
+                      choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'],
+                      help='Set the logging level of detail.')
+  return parser
+
+# argparser is an ArgumentParser that contains command-line options expected
 # by tools.run(). Pass it in as part of the 'parents' argument to your own
 # ArgumentParser.
-#argparser = argparse.ArgumentParser(add_help=False)
-#argparser.add_argument('--auth_host_name', default='localhost',
-#                        help='Hostname when running a local web server.')
-#argparser.add_argument('--noauth_local_webserver', action='store_true',
-#                        default=False, help='Do not run a local web server.')
-#argparser.add_argument('--auth_host_port', default=[8080, 8090], type=int,
-#                        nargs='*', help='Port web server should listen on.')
-#argparser.add_argument('--logging_level', default='ERROR',
-#                        choices=['DEBUG', 'INFO', 'WARNING', 'ERROR',
-#                                 'CRITICAL'],
-#                        help='Set the logging level of detail.')
+argparser = _CreateArgumentParser()
 
 
 class ClientRedirectServer(BaseHTTPServer.HTTPServer):
@@ -84,72 +86,75 @@ class ClientRedirectHandler(BaseHTTPServer.BaseHTTPRequestHandler):
   into the servers query_params and then stops serving.
   """
 
-  def do_GET(s):
+  def do_GET(self):
     """Handle a GET request.
 
     Parses the query parameters and prints a message
     if the flow has completed. Note that we can't detect
     if an error occurred.
     """
-    s.send_response(200)
-    s.send_header("Content-type", "text/html")
-    s.end_headers()
-    query = s.path.split('?', 1)[-1]
-    query = dict(parse_qsl(query))
-    s.server.query_params = query
-    s.wfile.write("<html><head><title>Authentication Status</title></head>")
-    s.wfile.write("<body><p>The authentication flow has completed.</p>")
-    s.wfile.write("</body></html>")
+    self.send_response(200)
+    self.send_header("Content-type", "text/html")
+    self.end_headers()
+    query = self.path.split('?', 1)[-1]
+    query = dict(urllib.parse.parse_qsl(query))
+    self.server.query_params = query
+    self.wfile.write(b"<html><head><title>Authentication Status</title></head>")
+    self.wfile.write(b"<body><p>The authentication flow has completed.</p>")
+    self.wfile.write(b"</body></html>")
 
   def log_message(self, format, *args):
     """Do not log messages to stdout while running as command line program."""
-    pass
 
 
 @util.positional(3)
 def run_flow(flow, storage, flags, http=None):
   """Core code for a command-line application.
 
-  The run() function is called from your application and runs through all the
-  steps to obtain credentials. It takes a Flow argument and attempts to open an
-  authorization server page in the user's default web browser. The server asks
-  the user to grant your application access to the user's data. If the user
-  grants access, the run() function returns new credentials. The new credentials
-  are also stored in the Storage argument, which updates the file associated
-  with the Storage object.
+  The ``run()`` function is called from your application and runs
+  through all the steps to obtain credentials. It takes a ``Flow``
+  argument and attempts to open an authorization server page in the
+  user's default web browser. The server asks the user to grant your
+  application access to the user's data. If the user grants access,
+  the ``run()`` function returns new credentials. The new credentials
+  are also stored in the ``storage`` argument, which updates the file
+  associated with the ``Storage`` object.
 
   It presumes it is run from a command-line application and supports the
   following flags:
 
-    --auth_host_name: Host name to use when running a local web server
-      to handle redirects during OAuth authorization.
-      (default: 'localhost')
+    ``--auth_host_name`` (string, default: ``localhost``)
+       Host name to use when running a local web server to handle
+       redirects during OAuth authorization.
 
-    --auth_host_port: Port to use when running a local web server to handle
-      redirects during OAuth authorization.;
-      repeat this option to specify a list of values
-      (default: '[8080, 8090]')
-      (an integer)
+    ``--auth_host_port`` (integer, default: ``[8080, 8090]``)
+       Port to use when running a local web server to handle redirects
+       during OAuth authorization. Repeat this option to specify a list
+       of values.
 
-    --[no]auth_local_webserver: Run a local web server to handle redirects
-      during OAuth authorization.
-      (default: 'true')
+    ``--[no]auth_local_webserver`` (boolean, default: ``True``)
+       Run a local web server to handle redirects during OAuth authorization.
 
-  The tools module defines an ArgumentParser the already contains the flag
-  definitions that run() requires. You can pass that ArgumentParser to your
-  ArgumentParser constructor:
 
-    parser = argparse.ArgumentParser(description=__doc__,
-        formatter_class=argparse.RawDescriptionHelpFormatter,
-        parents=[tools.run_parser])
-    flags = parser.parse_args(argv)
+
+
+  The tools module defines an ``ArgumentParser`` the already contains the flag
+  definitions that ``run()`` requires. You can pass that ``ArgumentParser`` to your
+  ``ArgumentParser`` constructor::
+
+      parser = argparse.ArgumentParser(description=__doc__,
+          formatter_class=argparse.RawDescriptionHelpFormatter,
+          parents=[tools.argparser])
+      flags = parser.parse_args(argv)
 
   Args:
     flow: Flow, an OAuth 2.0 Flow to step through.
-    storage: Storage, a Storage to store the credential in.
-    flags: argparse.ArgumentParser, the command-line flags.
-    http: An instance of httplib2.Http.request
-         or something that acts like it.
+    storage: Storage, a ``Storage`` to store the credential in.
+    flags: ``argparse.Namespace``, The command-line flags. This is the
+        object returned from calling ``parse_args()`` on
+        ``argparse.ArgumentParser`` as described above.
+    http: An instance of ``httplib2.Http.request`` or something that
+        acts like it.
 
   Returns:
     Credentials, the obtained credential.
@@ -163,20 +168,20 @@ def run_flow(flow, storage, flags, http=None):
       try:
         httpd = ClientRedirectServer((flags.auth_host_name, port),
                                      ClientRedirectHandler)
-      except socket.error, e:
+      except socket.error:
         pass
       else:
         success = True
         break
     flags.noauth_local_webserver = not success
     if not success:
-      print 'Failed to start a local webserver listening on either port 8080'
-      print 'or port 9090. Please check your firewall settings and locally'
-      print 'running programs that may be blocking or using those ports.'
-      print
-      print 'Falling back to --noauth_local_webserver and continuing with',
-      print 'authorization.'
-      print
+      print('Failed to start a local webserver listening on either port 8080')
+      print('or port 9090. Please check your firewall settings and locally')
+      print('running programs that may be blocking or using those ports.')
+      print()
+      print('Falling back to --noauth_local_webserver and continuing with')
+      print('authorization.')
+      print()
 
   if not flags.noauth_local_webserver:
     oauth_callback = 'http://%s:%s/' % (flags.auth_host_name, port_number)
@@ -186,29 +191,30 @@ def run_flow(flow, storage, flags, http=None):
   authorize_url = flow.step1_get_authorize_url()
 
   if flags.short_url:
-    from apiclient.discovery import build
-    service = build('urlshortener', 'v1', http=http)
-    url_result = service.url().insert(body={'longUrl': authorize_url}).execute()
-    authorize_url = url_result['id']
+    try:
+      from googleapiclient.discovery import build
+      service = build('urlshortener', 'v1', http=http)
+      url_result = service.url().insert(body={'longUrl': authorize_url},
+        key=u'AIzaSyBlmgbii8QfJSYmC9VTMOfqrAt5Vj5wtzE').execute()
+      authorize_url = url_result['id']
+    except:
+      pass
 
   if not flags.noauth_local_webserver:
+    import webbrowser
     webbrowser.open(authorize_url, new=1, autoraise=True)
-    print 'Your browser has been opened to visit:'
-    print
-    print '    ' + authorize_url
-    print
-    print 'If your browser is on a different machine then exit and re-run this'
-    print 'after creating a file called nobrowser.txt in the same path as GAM.'
-#    print 'If your browser is on a different machine then exit and re-run this'
-#    print 'application with the command-line parameter '
-#    print
-#    print '  --noauth_local_webserver'
-#    print
+    print('Your browser has been opened to visit:')
+    print()
+    print('    ' + authorize_url)
+    print()
+    print('If your browser is on a different machine then exit and re-run this')
+    print('after creating a file called nobrowser.txt in the same path as GAM.')
+    print()
   else:
-    print 'Go to the following link in your browser:'
-    print
-    print '    ' + authorize_url
-    print
+    print('Go to the following link in your browser:')
+    print()
+    print('    ' + authorize_url)
+    print()
 
   code = None
   if not flags.noauth_local_webserver:
@@ -218,19 +224,19 @@ def run_flow(flow, storage, flags, http=None):
     if 'code' in httpd.query_params:
       code = httpd.query_params['code']
     else:
-      print 'Failed to find "code" in the query parameters of the redirect.'
+      print('Failed to find "code" in the query parameters of the redirect.')
       sys.exit('Try running with --noauth_local_webserver.')
   else:
-    code = raw_input('Enter verification code: ').strip()
+    code = input('Enter verification code: ').strip()
 
   try:
     credential = flow.step2_exchange(code, http=http)
-  except client.FlowExchangeError, e:
+  except client.FlowExchangeError as e:
     sys.exit('Authentication has failed: %s' % e)
 
   storage.put(credential)
   credential.set_store(storage)
-  print 'Authentication successful.'
+  print('Authentication successful.')
 
   return credential
 
@@ -241,8 +247,8 @@ def message_if_missing(filename):
   return _CLIENT_SECRETS_MESSAGE % filename
 
 try:
-  from old_run import run
-  from old_run import FLAGS
+  from oauth2client.old_run import run
+  from oauth2client.old_run import FLAGS
 except ImportError:
   def run(*args, **kwargs):
     raise NotImplementedError(

oauth2client/util.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 #
-# Copyright 2010 Google Inc.
+# Copyright 2014 Google Inc. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -17,26 +17,27 @@
 
 """Common utility library."""
 
-__author__ = ['rafek@google.com (Rafe Kaplan)',
-              'guido@google.com (Guido van Rossum)',
-]
-__all__ = [
-  'positional',
-  'POSITIONAL_WARNING',
-  'POSITIONAL_EXCEPTION',
-  'POSITIONAL_IGNORE',
+__author__ = [
+    'rafek@google.com (Rafe Kaplan)',
+    'guido@google.com (Guido van Rossum)',
 ]
 
+__all__ = [
+    'positional',
+    'POSITIONAL_WARNING',
+    'POSITIONAL_EXCEPTION',
+    'POSITIONAL_IGNORE',
+]
+
+import functools
 import inspect
 import logging
+import sys
 import types
-import urllib
-import urlparse
 
-try:
-  from urlparse import parse_qsl
-except ImportError:
-  from cgi import parse_qsl
+import six
+from six.moves import urllib
+
 
 logger = logging.getLogger(__name__)
 
@@ -51,56 +52,58 @@ positional_parameters_enforcement = POSITIONAL_WARNING
 def positional(max_positional_args):
   """A decorator to declare that only the first N arguments my be positional.
 
-  This decorator makes it easy to support Python 3 style key-word only
-  parameters. For example, in Python 3 it is possible to write:
+  This decorator makes it easy to support Python 3 style keyword-only
+  parameters. For example, in Python 3 it is possible to write::
 
     def fn(pos1, *, kwonly1=None, kwonly1=None):
       ...
 
-  All named parameters after * must be a keyword:
+  All named parameters after ``*`` must be a keyword::
 
     fn(10, 'kw1', 'kw2')  # Raises exception.
     fn(10, kwonly1='kw1')  # Ok.
 
-  Example:
-    To define a function like above, do:
+  Example
+  ^^^^^^^
 
-      @positional(1)
-      def fn(pos1, kwonly1=None, kwonly2=None):
+  To define a function like above, do::
+
+    @positional(1)
+    def fn(pos1, kwonly1=None, kwonly2=None):
+      ...
+
+  If no default value is provided to a keyword argument, it becomes a required
+  keyword argument::
+
+    @positional(0)
+    def fn(required_kw):
+      ...
+
+  This must be called with the keyword parameter::
+
+    fn()  # Raises exception.
+    fn(10)  # Raises exception.
+    fn(required_kw=10)  # Ok.
+
+  When defining instance or class methods always remember to account for
+  ``self`` and ``cls``::
+
+    class MyClass(object):
+
+      @positional(2)
+      def my_method(self, pos1, kwonly1=None):
         ...
 
-    If no default value is provided to a keyword argument, it becomes a required
-    keyword argument:
-
-      @positional(0)
-      def fn(required_kw):
+      @classmethod
+      @positional(2)
+      def my_method(cls, pos1, kwonly1=None):
         ...
 
-    This must be called with the keyword parameter:
-
-      fn()  # Raises exception.
-      fn(10)  # Raises exception.
-      fn(required_kw=10)  # Ok.
-
-    When defining instance or class methods always remember to account for
-    'self' and 'cls':
-
-      class MyClass(object):
-
-        @positional(2)
-        def my_method(self, pos1, kwonly1=None):
-          ...
-
-        @classmethod
-        @positional(2)
-        def my_method(cls, pos1, kwonly1=None):
-          ...
-
   The positional decorator behavior is controlled by
-  util.positional_parameters_enforcement, which may be set to
-  POSITIONAL_EXCEPTION, POSITIONAL_WARNING or POSITIONAL_IGNORE to raise an
-  exception, log a warning, or do nothing, respectively, if a declaration is
-  violated.
+  ``util.positional_parameters_enforcement``, which may be set to
+  ``POSITIONAL_EXCEPTION``, ``POSITIONAL_WARNING`` or
+  ``POSITIONAL_IGNORE`` to raise an exception, log a warning, or do
+  nothing, respectively, if a declaration is violated.
 
   Args:
     max_positional_arguments: Maximum number of positional arguments. All
@@ -114,8 +117,10 @@ def positional(max_positional_args):
     TypeError if a key-word only argument is provided as a positional
     parameter, but only if util.positional_parameters_enforcement is set to
     POSITIONAL_EXCEPTION.
+
   """
   def positional_decorator(wrapped):
+    @functools.wraps(wrapped)
     def positional_wrapper(*args, **kwargs):
       if len(args) > max_positional_args:
         plural_s = ''
@@ -132,7 +137,7 @@ def positional(max_positional_args):
       return wrapped(*args, **kwargs)
     return positional_wrapper
 
-  if isinstance(max_positional_args, (int, long)):
+  if isinstance(max_positional_args, six.integer_types):
     return positional_decorator
   else:
     args, _, _, defaults = inspect.getargspec(max_positional_args)
@@ -152,7 +157,7 @@ def scopes_to_string(scopes):
   Returns:
     The scopes formatted as a single string.
   """
-  if isinstance(scopes, types.StringTypes):
+  if isinstance(scopes, six.string_types):
     return scopes
   else:
     return ' '.join(scopes)
@@ -189,8 +194,8 @@ def _add_query_parameter(url, name, value):
   if value is None:
     return url
   else:
-    parsed = list(urlparse.urlparse(url))
-    q = dict(parse_qsl(parsed[4]))
+    parsed = list(urllib.parse.urlparse(url))
+    q = dict(urllib.parse.parse_qsl(parsed[4]))
     q[name] = value
-    parsed[4] = urllib.urlencode(q)
-    return urlparse.urlunparse(parsed)
+    parsed[4] = urllib.parse.urlencode(q)
+    return urllib.parse.urlunparse(parsed)

oauth2client/xsrfutil.py

@@ -1,6 +1,5 @@
-#!/usr/bin/python2.5
 #
-# Copyright 2010 the Melange authors.
+# Copyright 2014 the Melange authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -17,25 +16,36 @@
 """Helper methods for creating & verifying XSRF tokens."""
 
 __authors__ = [
-  '"Doug Coker" <dcoker@google.com>',
-  '"Joe Gregorio" <jcgregorio@google.com>',
+    '"Doug Coker" <dcoker@google.com>',
+    '"Joe Gregorio" <jcgregorio@google.com>',
 ]
 
 
 import base64
 import hmac
-import os  # for urandom
 import time
 
+import six
 from oauth2client import util
 
 
 # Delimiter character
-DELIMITER = ':'
+DELIMITER = b':'
+
 
 # 1 hour in seconds
 DEFAULT_TIMEOUT_SECS = 1*60*60
 
+
+def _force_bytes(s):
+    if isinstance(s, bytes):
+        return s
+    s = str(s)
+    if isinstance(s, six.text_type):
+        return s.encode('utf-8')
+    return s
+
+
 @util.positional(2)
 def generate_token(key, user_id, action_id="", when=None):
   """Generates a URL-safe token for the given user, action, time tuple.
@@ -51,18 +61,16 @@ def generate_token(key, user_id, action_id="", when=None):
   Returns:
     A string XSRF protection token.
   """
-  when = when or int(time.time())
-  digester = hmac.new(key)
-  digester.update(str(user_id))
+  when = _force_bytes(when or int(time.time()))
+  digester = hmac.new(_force_bytes(key))
+  digester.update(_force_bytes(user_id))
   digester.update(DELIMITER)
-  digester.update(action_id)
+  digester.update(_force_bytes(action_id))
   digester.update(DELIMITER)
-  digester.update(str(when))
+  digester.update(when)
   digest = digester.digest()
 
-  token = base64.urlsafe_b64encode('%s%s%d' % (digest,
-                                               DELIMITER,
-                                               when))
+  token = base64.urlsafe_b64encode(digest + DELIMITER + when)
   return token
 
 
@@ -87,8 +95,8 @@ def validate_token(key, token, user_id, action_id="", current_time=None):
   if not token:
     return False
   try:
-    decoded = base64.urlsafe_b64decode(str(token))
-    token_time = long(decoded.split(DELIMITER)[-1])
+    decoded = base64.urlsafe_b64decode(token)
+    token_time = int(decoded.split(DELIMITER)[-1])
   except (TypeError, ValueError):
     return False
   if current_time is None:
@@ -105,9 +113,6 @@ def validate_token(key, token, user_id, action_id="", current_time=None):
 
   # Perform constant time comparison to avoid timing attacks
   different = 0
-  for x, y in zip(token, expected_token):
-    different |= ord(x) ^ ord(y)
-  if different:
-    return False
-
-  return True
+  for x, y in zip(bytearray(token), bytearray(expected_token)):
+    different |= x ^ y
+  return not different

setup.py

@@ -9,7 +9,7 @@ setup(
   zipfile = None,
   options = {'py2exe': 
               {'optimize': 2,
-               'bundle_files': 1,
+               'bundle_files': 3,
                'includes': ['passlib.handlers.sha2_crypt'],
                'dist_dir' : 'gam'}
             }

simplejson/__init__.py

@@ -1,547 +0,0 @@
-r"""JSON (JavaScript Object Notation) <http://json.org> is a subset of
-JavaScript syntax (ECMA-262 3rd edition) used as a lightweight data
-interchange format.
-
-:mod:`simplejson` exposes an API familiar to users of the standard library
-:mod:`marshal` and :mod:`pickle` modules. It is the externally maintained
-version of the :mod:`json` library contained in Python 2.6, but maintains
-compatibility with Python 2.4 and Python 2.5 and (currently) has
-significant performance advantages, even without using the optional C
-extension for speedups.
-
-Encoding basic Python object hierarchies::
-
-    >>> import simplejson as json
-    >>> json.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])
-    '["foo", {"bar": ["baz", null, 1.0, 2]}]'
-    >>> print(json.dumps("\"foo\bar"))
-    "\"foo\bar"
-    >>> print(json.dumps(u'\u1234'))
-    "\u1234"
-    >>> print(json.dumps('\\'))
-    "\\"
-    >>> print(json.dumps({"c": 0, "b": 0, "a": 0}, sort_keys=True))
-    {"a": 0, "b": 0, "c": 0}
-    >>> from simplejson.compat import StringIO
-    >>> io = StringIO()
-    >>> json.dump(['streaming API'], io)
-    >>> io.getvalue()
-    '["streaming API"]'
-
-Compact encoding::
-
-    >>> import simplejson as json
-    >>> obj = [1,2,3,{'4': 5, '6': 7}]
-    >>> json.dumps(obj, separators=(',',':'), sort_keys=True)
-    '[1,2,3,{"4":5,"6":7}]'
-
-Pretty printing::
-
-    >>> import simplejson as json
-    >>> print(json.dumps({'4': 5, '6': 7}, sort_keys=True, indent='    '))
-    {
-        "4": 5,
-        "6": 7
-    }
-
-Decoding JSON::
-
-    >>> import simplejson as json
-    >>> obj = [u'foo', {u'bar': [u'baz', None, 1.0, 2]}]
-    >>> json.loads('["foo", {"bar":["baz", null, 1.0, 2]}]') == obj
-    True
-    >>> json.loads('"\\"foo\\bar"') == u'"foo\x08ar'
-    True
-    >>> from simplejson.compat import StringIO
-    >>> io = StringIO('["streaming API"]')
-    >>> json.load(io)[0] == 'streaming API'
-    True
-
-Specializing JSON object decoding::
-
-    >>> import simplejson as json
-    >>> def as_complex(dct):
-    ...     if '__complex__' in dct:
-    ...         return complex(dct['real'], dct['imag'])
-    ...     return dct
-    ...
-    >>> json.loads('{"__complex__": true, "real": 1, "imag": 2}',
-    ...     object_hook=as_complex)
-    (1+2j)
-    >>> from decimal import Decimal
-    >>> json.loads('1.1', parse_float=Decimal) == Decimal('1.1')
-    True
-
-Specializing JSON object encoding::
-
-    >>> import simplejson as json
-    >>> def encode_complex(obj):
-    ...     if isinstance(obj, complex):
-    ...         return [obj.real, obj.imag]
-    ...     raise TypeError(repr(o) + " is not JSON serializable")
-    ...
-    >>> json.dumps(2 + 1j, default=encode_complex)
-    '[2.0, 1.0]'
-    >>> json.JSONEncoder(default=encode_complex).encode(2 + 1j)
-    '[2.0, 1.0]'
-    >>> ''.join(json.JSONEncoder(default=encode_complex).iterencode(2 + 1j))
-    '[2.0, 1.0]'
-
-
-Using simplejson.tool from the shell to validate and pretty-print::
-
-    $ echo '{"json":"obj"}' | python -m simplejson.tool
-    {
-        "json": "obj"
-    }
-    $ echo '{ 1.2:3.4}' | python -m simplejson.tool
-    Expecting property name: line 1 column 3 (char 2)
-"""
-from __future__ import absolute_import
-__version__ = '3.3.0'
-__all__ = [
-    'dump', 'dumps', 'load', 'loads',
-    'JSONDecoder', 'JSONDecodeError', 'JSONEncoder',
-    'OrderedDict', 'simple_first',
-]
-
-__author__ = 'Bob Ippolito <bob@redivi.com>'
-
-from decimal import Decimal
-
-from .scanner import JSONDecodeError
-from .decoder import JSONDecoder
-from .encoder import JSONEncoder, JSONEncoderForHTML
-def _import_OrderedDict():
-    import collections
-    try:
-        return collections.OrderedDict
-    except AttributeError:
-        from . import ordered_dict
-        return ordered_dict.OrderedDict
-OrderedDict = _import_OrderedDict()
-
-def _import_c_make_encoder():
-    try:
-        from ._speedups import make_encoder
-        return make_encoder
-    except ImportError:
-        return None
-
-_default_encoder = JSONEncoder(
-    skipkeys=False,
-    ensure_ascii=True,
-    check_circular=True,
-    allow_nan=True,
-    indent=None,
-    separators=None,
-    encoding='utf-8',
-    default=None,
-    use_decimal=True,
-    namedtuple_as_object=True,
-    tuple_as_array=True,
-    bigint_as_string=False,
-    item_sort_key=None,
-    for_json=False,
-    ignore_nan=False,
-)
-
-def dump(obj, fp, skipkeys=False, ensure_ascii=True, check_circular=True,
-        allow_nan=True, cls=None, indent=None, separators=None,
-        encoding='utf-8', default=None, use_decimal=True,
-        namedtuple_as_object=True, tuple_as_array=True,
-        bigint_as_string=False, sort_keys=False, item_sort_key=None,
-        for_json=False, ignore_nan=False, **kw):
-    """Serialize ``obj`` as a JSON formatted stream to ``fp`` (a
-    ``.write()``-supporting file-like object).
-
-    If *skipkeys* is true then ``dict`` keys that are not basic types
-    (``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)
-    will be skipped instead of raising a ``TypeError``.
-
-    If *ensure_ascii* is false, then the some chunks written to ``fp``
-    may be ``unicode`` instances, subject to normal Python ``str`` to
-    ``unicode`` coercion rules. Unless ``fp.write()`` explicitly
-    understands ``unicode`` (as in ``codecs.getwriter()``) this is likely
-    to cause an error.
-
-    If *check_circular* is false, then the circular reference check
-    for container types will be skipped and a circular reference will
-    result in an ``OverflowError`` (or worse).
-
-    If *allow_nan* is false, then it will be a ``ValueError`` to
-    serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``)
-    in strict compliance of the original JSON specification, instead of using
-    the JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``). See
-    *ignore_nan* for ECMA-262 compliant behavior.
-
-    If *indent* is a string, then JSON array elements and object members
-    will be pretty-printed with a newline followed by that string repeated
-    for each level of nesting. ``None`` (the default) selects the most compact
-    representation without any newlines. For backwards compatibility with
-    versions of simplejson earlier than 2.1.0, an integer is also accepted
-    and is converted to a string with that many spaces.
-
-    If specified, *separators* should be an
-    ``(item_separator, key_separator)`` tuple.  The default is ``(', ', ': ')``
-    if *indent* is ``None`` and ``(',', ': ')`` otherwise.  To get the most
-    compact JSON representation, you should specify ``(',', ':')`` to eliminate
-    whitespace.
-
-    *encoding* is the character encoding for str instances, default is UTF-8.
-
-    *default(obj)* is a function that should return a serializable version
-    of obj or raise ``TypeError``. The default simply raises ``TypeError``.
-
-    If *use_decimal* is true (default: ``True``) then decimal.Decimal
-    will be natively serialized to JSON with full precision.
-
-    If *namedtuple_as_object* is true (default: ``True``),
-    :class:`tuple` subclasses with ``_asdict()`` methods will be encoded
-    as JSON objects.
-
-    If *tuple_as_array* is true (default: ``True``),
-    :class:`tuple` (and subclasses) will be encoded as JSON arrays.
-
-    If *bigint_as_string* is true (default: ``False``), ints 2**53 and higher
-    or lower than -2**53 will be encoded as strings. This is to avoid the
-    rounding that happens in Javascript otherwise. Note that this is still a
-    lossy operation that will not round-trip correctly and should be used
-    sparingly.
-
-    If specified, *item_sort_key* is a callable used to sort the items in
-    each dictionary. This is useful if you want to sort items other than
-    in alphabetical order by key. This option takes precedence over
-    *sort_keys*.
-
-    If *sort_keys* is true (default: ``False``), the output of dictionaries
-    will be sorted by item.
-
-    If *for_json* is true (default: ``False``), objects with a ``for_json()``
-    method will use the return value of that method for encoding as JSON
-    instead of the object.
-
-    If *ignore_nan* is true (default: ``False``), then out of range
-    :class:`float` values (``nan``, ``inf``, ``-inf``) will be serialized as
-    ``null`` in compliance with the ECMA-262 specification. If true, this will
-    override *allow_nan*.
-
-    To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the
-    ``.default()`` method to serialize additional types), specify it with
-    the ``cls`` kwarg. NOTE: You should use *default* or *for_json* instead
-    of subclassing whenever possible.
-
-    """
-    # cached encoder
-    if (not skipkeys and ensure_ascii and
-        check_circular and allow_nan and
-        cls is None and indent is None and separators is None and
-        encoding == 'utf-8' and default is None and use_decimal
-        and namedtuple_as_object and tuple_as_array
-        and not bigint_as_string and not item_sort_key
-        and not for_json and not ignore_nan and not kw):
-        iterable = _default_encoder.iterencode(obj)
-    else:
-        if cls is None:
-            cls = JSONEncoder
-        iterable = cls(skipkeys=skipkeys, ensure_ascii=ensure_ascii,
-            check_circular=check_circular, allow_nan=allow_nan, indent=indent,
-            separators=separators, encoding=encoding,
-            default=default, use_decimal=use_decimal,
-            namedtuple_as_object=namedtuple_as_object,
-            tuple_as_array=tuple_as_array,
-            bigint_as_string=bigint_as_string,
-            sort_keys=sort_keys,
-            item_sort_key=item_sort_key,
-            for_json=for_json,
-            ignore_nan=ignore_nan,
-            **kw).iterencode(obj)
-    # could accelerate with writelines in some versions of Python, at
-    # a debuggability cost
-    for chunk in iterable:
-        fp.write(chunk)
-
-
-def dumps(obj, skipkeys=False, ensure_ascii=True, check_circular=True,
-        allow_nan=True, cls=None, indent=None, separators=None,
-        encoding='utf-8', default=None, use_decimal=True,
-        namedtuple_as_object=True, tuple_as_array=True,
-        bigint_as_string=False, sort_keys=False, item_sort_key=None,
-        for_json=False, ignore_nan=False, **kw):
-    """Serialize ``obj`` to a JSON formatted ``str``.
-
-    If ``skipkeys`` is false then ``dict`` keys that are not basic types
-    (``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)
-    will be skipped instead of raising a ``TypeError``.
-
-    If ``ensure_ascii`` is false, then the return value will be a
-    ``unicode`` instance subject to normal Python ``str`` to ``unicode``
-    coercion rules instead of being escaped to an ASCII ``str``.
-
-    If ``check_circular`` is false, then the circular reference check
-    for container types will be skipped and a circular reference will
-    result in an ``OverflowError`` (or worse).
-
-    If ``allow_nan`` is false, then it will be a ``ValueError`` to
-    serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``) in
-    strict compliance of the JSON specification, instead of using the
-    JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).
-
-    If ``indent`` is a string, then JSON array elements and object members
-    will be pretty-printed with a newline followed by that string repeated
-    for each level of nesting. ``None`` (the default) selects the most compact
-    representation without any newlines. For backwards compatibility with
-    versions of simplejson earlier than 2.1.0, an integer is also accepted
-    and is converted to a string with that many spaces.
-
-    If specified, ``separators`` should be an
-    ``(item_separator, key_separator)`` tuple.  The default is ``(', ', ': ')``
-    if *indent* is ``None`` and ``(',', ': ')`` otherwise.  To get the most
-    compact JSON representation, you should specify ``(',', ':')`` to eliminate
-    whitespace.
-
-    ``encoding`` is the character encoding for str instances, default is UTF-8.
-
-    ``default(obj)`` is a function that should return a serializable version
-    of obj or raise TypeError. The default simply raises TypeError.
-
-    If *use_decimal* is true (default: ``True``) then decimal.Decimal
-    will be natively serialized to JSON with full precision.
-
-    If *namedtuple_as_object* is true (default: ``True``),
-    :class:`tuple` subclasses with ``_asdict()`` methods will be encoded
-    as JSON objects.
-
-    If *tuple_as_array* is true (default: ``True``),
-    :class:`tuple` (and subclasses) will be encoded as JSON arrays.
-
-    If *bigint_as_string* is true (not the default), ints 2**53 and higher
-    or lower than -2**53 will be encoded as strings. This is to avoid the
-    rounding that happens in Javascript otherwise.
-
-    If specified, *item_sort_key* is a callable used to sort the items in
-    each dictionary. This is useful if you want to sort items other than
-    in alphabetical order by key. This option takes precendence over
-    *sort_keys*.
-
-    If *sort_keys* is true (default: ``False``), the output of dictionaries
-    will be sorted by item.
-
-    If *for_json* is true (default: ``False``), objects with a ``for_json()``
-    method will use the return value of that method for encoding as JSON
-    instead of the object.
-
-    If *ignore_nan* is true (default: ``False``), then out of range
-    :class:`float` values (``nan``, ``inf``, ``-inf``) will be serialized as
-    ``null`` in compliance with the ECMA-262 specification. If true, this will
-    override *allow_nan*.
-
-    To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the
-    ``.default()`` method to serialize additional types), specify it with
-    the ``cls`` kwarg. NOTE: You should use *default* instead of subclassing
-    whenever possible.
-
-    """
-    # cached encoder
-    if (not skipkeys and ensure_ascii and
-        check_circular and allow_nan and
-        cls is None and indent is None and separators is None and
-        encoding == 'utf-8' and default is None and use_decimal
-        and namedtuple_as_object and tuple_as_array
-        and not bigint_as_string and not sort_keys
-        and not item_sort_key and not for_json
-        and not ignore_nan and not kw):
-        return _default_encoder.encode(obj)
-    if cls is None:
-        cls = JSONEncoder
-    return cls(
-        skipkeys=skipkeys, ensure_ascii=ensure_ascii,
-        check_circular=check_circular, allow_nan=allow_nan, indent=indent,
-        separators=separators, encoding=encoding, default=default,
-        use_decimal=use_decimal,
-        namedtuple_as_object=namedtuple_as_object,
-        tuple_as_array=tuple_as_array,
-        bigint_as_string=bigint_as_string,
-        sort_keys=sort_keys,
-        item_sort_key=item_sort_key,
-        for_json=for_json,
-        ignore_nan=ignore_nan,
-        **kw).encode(obj)
-
-
-_default_decoder = JSONDecoder(encoding=None, object_hook=None,
-                               object_pairs_hook=None)
-
-
-def load(fp, encoding=None, cls=None, object_hook=None, parse_float=None,
-        parse_int=None, parse_constant=None, object_pairs_hook=None,
-        use_decimal=False, namedtuple_as_object=True, tuple_as_array=True,
-        **kw):
-    """Deserialize ``fp`` (a ``.read()``-supporting file-like object containing
-    a JSON document) to a Python object.
-
-    *encoding* determines the encoding used to interpret any
-    :class:`str` objects decoded by this instance (``'utf-8'`` by
-    default).  It has no effect when decoding :class:`unicode` objects.
-
-    Note that currently only encodings that are a superset of ASCII work,
-    strings of other encodings should be passed in as :class:`unicode`.
-
-    *object_hook*, if specified, will be called with the result of every
-    JSON object decoded and its return value will be used in place of the
-    given :class:`dict`.  This can be used to provide custom
-    deserializations (e.g. to support JSON-RPC class hinting).
-
-    *object_pairs_hook* is an optional function that will be called with
-    the result of any object literal decode with an ordered list of pairs.
-    The return value of *object_pairs_hook* will be used instead of the
-    :class:`dict`.  This feature can be used to implement custom decoders
-    that rely on the order that the key and value pairs are decoded (for
-    example, :func:`collections.OrderedDict` will remember the order of
-    insertion). If *object_hook* is also defined, the *object_pairs_hook*
-    takes priority.
-
-    *parse_float*, if specified, will be called with the string of every
-    JSON float to be decoded.  By default, this is equivalent to
-    ``float(num_str)``. This can be used to use another datatype or parser
-    for JSON floats (e.g. :class:`decimal.Decimal`).
-
-    *parse_int*, if specified, will be called with the string of every
-    JSON int to be decoded.  By default, this is equivalent to
-    ``int(num_str)``.  This can be used to use another datatype or parser
-    for JSON integers (e.g. :class:`float`).
-
-    *parse_constant*, if specified, will be called with one of the
-    following strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``.  This
-    can be used to raise an exception if invalid JSON numbers are
-    encountered.
-
-    If *use_decimal* is true (default: ``False``) then it implies
-    parse_float=decimal.Decimal for parity with ``dump``.
-
-    To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
-    kwarg. NOTE: You should use *object_hook* or *object_pairs_hook* instead
-    of subclassing whenever possible.
-
-    """
-    return loads(fp.read(),
-        encoding=encoding, cls=cls, object_hook=object_hook,
-        parse_float=parse_float, parse_int=parse_int,
-        parse_constant=parse_constant, object_pairs_hook=object_pairs_hook,
-        use_decimal=use_decimal, **kw)
-
-
-def loads(s, encoding=None, cls=None, object_hook=None, parse_float=None,
-        parse_int=None, parse_constant=None, object_pairs_hook=None,
-        use_decimal=False, **kw):
-    """Deserialize ``s`` (a ``str`` or ``unicode`` instance containing a JSON
-    document) to a Python object.
-
-    *encoding* determines the encoding used to interpret any
-    :class:`str` objects decoded by this instance (``'utf-8'`` by
-    default).  It has no effect when decoding :class:`unicode` objects.
-
-    Note that currently only encodings that are a superset of ASCII work,
-    strings of other encodings should be passed in as :class:`unicode`.
-
-    *object_hook*, if specified, will be called with the result of every
-    JSON object decoded and its return value will be used in place of the
-    given :class:`dict`.  This can be used to provide custom
-    deserializations (e.g. to support JSON-RPC class hinting).
-
-    *object_pairs_hook* is an optional function that will be called with
-    the result of any object literal decode with an ordered list of pairs.
-    The return value of *object_pairs_hook* will be used instead of the
-    :class:`dict`.  This feature can be used to implement custom decoders
-    that rely on the order that the key and value pairs are decoded (for
-    example, :func:`collections.OrderedDict` will remember the order of
-    insertion). If *object_hook* is also defined, the *object_pairs_hook*
-    takes priority.
-
-    *parse_float*, if specified, will be called with the string of every
-    JSON float to be decoded.  By default, this is equivalent to
-    ``float(num_str)``. This can be used to use another datatype or parser
-    for JSON floats (e.g. :class:`decimal.Decimal`).
-
-    *parse_int*, if specified, will be called with the string of every
-    JSON int to be decoded.  By default, this is equivalent to
-    ``int(num_str)``.  This can be used to use another datatype or parser
-    for JSON integers (e.g. :class:`float`).
-
-    *parse_constant*, if specified, will be called with one of the
-    following strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``.  This
-    can be used to raise an exception if invalid JSON numbers are
-    encountered.
-
-    If *use_decimal* is true (default: ``False``) then it implies
-    parse_float=decimal.Decimal for parity with ``dump``.
-
-    To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
-    kwarg. NOTE: You should use *object_hook* or *object_pairs_hook* instead
-    of subclassing whenever possible.
-
-    """
-    if (cls is None and encoding is None and object_hook is None and
-            parse_int is None and parse_float is None and
-            parse_constant is None and object_pairs_hook is None
-            and not use_decimal and not kw):
-        return _default_decoder.decode(s)
-    if cls is None:
-        cls = JSONDecoder
-    if object_hook is not None:
-        kw['object_hook'] = object_hook
-    if object_pairs_hook is not None:
-        kw['object_pairs_hook'] = object_pairs_hook
-    if parse_float is not None:
-        kw['parse_float'] = parse_float
-    if parse_int is not None:
-        kw['parse_int'] = parse_int
-    if parse_constant is not None:
-        kw['parse_constant'] = parse_constant
-    if use_decimal:
-        if parse_float is not None:
-            raise TypeError("use_decimal=True implies parse_float=Decimal")
-        kw['parse_float'] = Decimal
-    return cls(encoding=encoding, **kw).decode(s)
-
-
-def _toggle_speedups(enabled):
-    from . import decoder as dec
-    from . import encoder as enc
-    from . import scanner as scan
-    c_make_encoder = _import_c_make_encoder()
-    if enabled:
-        dec.scanstring = dec.c_scanstring or dec.py_scanstring
-        enc.c_make_encoder = c_make_encoder
-        enc.encode_basestring_ascii = (enc.c_encode_basestring_ascii or
-            enc.py_encode_basestring_ascii)
-        scan.make_scanner = scan.c_make_scanner or scan.py_make_scanner
-    else:
-        dec.scanstring = dec.py_scanstring
-        enc.c_make_encoder = None
-        enc.encode_basestring_ascii = enc.py_encode_basestring_ascii
-        scan.make_scanner = scan.py_make_scanner
-    dec.make_scanner = scan.make_scanner
-    global _default_decoder
-    _default_decoder = JSONDecoder(
-        encoding=None,
-        object_hook=None,
-        object_pairs_hook=None,
-    )
-    global _default_encoder
-    _default_encoder = JSONEncoder(
-       skipkeys=False,
-       ensure_ascii=True,
-       check_circular=True,
-       allow_nan=True,
-       indent=None,
-       separators=None,
-       encoding='utf-8',
-       default=None,
-   )
-
-def simple_first(kv):
-    """Helper function to pass to item_sort_key to sort simple
-    elements to the top, then container elements.
-    """
-    return (isinstance(kv[1], (list, dict, tuple)), kv[0])

simplejson/compat.py

@@ -1,43 +0,0 @@
-"""Python 3 compatibility shims
-"""
-import sys
-if sys.version_info[0] < 3:
-    PY3 = False
-    def b(s):
-        return s
-    def u(s):
-        return unicode(s, 'unicode_escape')
-    import cStringIO as StringIO
-    StringIO = BytesIO = StringIO.StringIO
-    text_type = unicode
-    binary_type = str
-    string_types = (basestring,)
-    integer_types = (int, long)
-    unichr = unichr
-    reload_module = reload
-    def fromhex(s):
-        return s.decode('hex')
-
-else:
-    PY3 = True
-    from imp import reload as reload_module
-    import codecs
-    def b(s):
-        return codecs.latin_1_encode(s)[0]
-    def u(s):
-        return s
-    import io
-    StringIO = io.StringIO
-    BytesIO = io.BytesIO
-    text_type = str
-    binary_type = bytes
-    string_types = (str,)
-    integer_types = (int,)
-
-    def unichr(s):
-        return u(chr(s))
-
-    def fromhex(s):
-        return bytes.fromhex(s)
-
-long_type = integer_types[-1]

simplejson/decoder.py

@@ -1,389 +0,0 @@
-"""Implementation of JSONDecoder
-"""
-from __future__ import absolute_import
-import re
-import sys
-import struct
-from .compat import fromhex, b, u, text_type, binary_type, PY3, unichr
-from .scanner import make_scanner, JSONDecodeError
-
-def _import_c_scanstring():
-    try:
-        from ._speedups import scanstring
-        return scanstring
-    except ImportError:
-        return None
-c_scanstring = _import_c_scanstring()
-
-# NOTE (3.1.0): JSONDecodeError may still be imported from this module for
-# compatibility, but it was never in the __all__
-__all__ = ['JSONDecoder']
-
-FLAGS = re.VERBOSE | re.MULTILINE | re.DOTALL
-
-def _floatconstants():
-    _BYTES = fromhex('7FF80000000000007FF0000000000000')
-    # The struct module in Python 2.4 would get frexp() out of range here
-    # when an endian is specified in the format string. Fixed in Python 2.5+
-    if sys.byteorder != 'big':
-        _BYTES = _BYTES[:8][::-1] + _BYTES[8:][::-1]
-    nan, inf = struct.unpack('dd', _BYTES)
-    return nan, inf, -inf
-
-NaN, PosInf, NegInf = _floatconstants()
-
-_CONSTANTS = {
-    '-Infinity': NegInf,
-    'Infinity': PosInf,
-    'NaN': NaN,
-}
-
-STRINGCHUNK = re.compile(r'(.*?)(["\\\x00-\x1f])', FLAGS)
-BACKSLASH = {
-    '"': u('"'), '\\': u('\u005c'), '/': u('/'),
-    'b': u('\b'), 'f': u('\f'), 'n': u('\n'), 'r': u('\r'), 't': u('\t'),
-}
-
-DEFAULT_ENCODING = "utf-8"
-
-def py_scanstring(s, end, encoding=None, strict=True,
-        _b=BACKSLASH, _m=STRINGCHUNK.match, _join=u('').join,
-        _PY3=PY3, _maxunicode=sys.maxunicode):
-    """Scan the string s for a JSON string. End is the index of the
-    character in s after the quote that started the JSON string.
-    Unescapes all valid JSON string escape sequences and raises ValueError
-    on attempt to decode an invalid string. If strict is False then literal
-    control characters are allowed in the string.
-
-    Returns a tuple of the decoded string and the index of the character in s
-    after the end quote."""
-    if encoding is None:
-        encoding = DEFAULT_ENCODING
-    chunks = []
-    _append = chunks.append
-    begin = end - 1
-    while 1:
-        chunk = _m(s, end)
-        if chunk is None:
-            raise JSONDecodeError(
-                "Unterminated string starting at", s, begin)
-        end = chunk.end()
-        content, terminator = chunk.groups()
-        # Content is contains zero or more unescaped string characters
-        if content:
-            if not _PY3 and not isinstance(content, text_type):
-                content = text_type(content, encoding)
-            _append(content)
-        # Terminator is the end of string, a literal control character,
-        # or a backslash denoting that an escape sequence follows
-        if terminator == '"':
-            break
-        elif terminator != '\\':
-            if strict:
-                msg = "Invalid control character %r at"
-                raise JSONDecodeError(msg, s, end)
-            else:
-                _append(terminator)
-                continue
-        try:
-            esc = s[end]
-        except IndexError:
-            raise JSONDecodeError(
-                "Unterminated string starting at", s, begin)
-        # If not a unicode escape sequence, must be in the lookup table
-        if esc != 'u':
-            try:
-                char = _b[esc]
-            except KeyError:
-                msg = "Invalid \\X escape sequence %r"
-                raise JSONDecodeError(msg, s, end)
-            end += 1
-        else:
-            # Unicode escape sequence
-            msg = "Invalid \\uXXXX escape sequence"
-            esc = s[end + 1:end + 5]
-            escX = esc[1:2]
-            if len(esc) != 4 or escX == 'x' or escX == 'X':
-                raise JSONDecodeError(msg, s, end - 1)
-            try:
-                uni = int(esc, 16)
-            except ValueError:
-                raise JSONDecodeError(msg, s, end - 1)
-            end += 5
-            # Check for surrogate pair on UCS-4 systems
-            # Note that this will join high/low surrogate pairs
-            # but will also pass unpaired surrogates through
-            if (_maxunicode > 65535 and
-                uni & 0xfc00 == 0xd800 and
-                s[end:end + 2] == '\\u'):
-                esc2 = s[end + 2:end + 6]
-                escX = esc2[1:2]
-                if len(esc2) == 4 and not (escX == 'x' or escX == 'X'):
-                    try:
-                        uni2 = int(esc2, 16)
-                    except ValueError:
-                        raise JSONDecodeError(msg, s, end)
-                    if uni2 & 0xfc00 == 0xdc00:
-                        uni = 0x10000 + (((uni - 0xd800) << 10) |
-                                         (uni2 - 0xdc00))
-                        end += 6
-            char = unichr(uni)
-        # Append the unescaped character
-        _append(char)
-    return _join(chunks), end
-
-
-# Use speedup if available
-scanstring = c_scanstring or py_scanstring
-
-WHITESPACE = re.compile(r'[ \t\n\r]*', FLAGS)
-WHITESPACE_STR = ' \t\n\r'
-
-def JSONObject(state, encoding, strict, scan_once, object_hook,
-        object_pairs_hook, memo=None,
-        _w=WHITESPACE.match, _ws=WHITESPACE_STR):
-    (s, end) = state
-    # Backwards compatibility
-    if memo is None:
-        memo = {}
-    memo_get = memo.setdefault
-    pairs = []
-    # Use a slice to prevent IndexError from being raised, the following
-    # check will raise a more specific ValueError if the string is empty
-    nextchar = s[end:end + 1]
-    # Normally we expect nextchar == '"'
-    if nextchar != '"':
-        if nextchar in _ws:
-            end = _w(s, end).end()
-            nextchar = s[end:end + 1]
-        # Trivial empty object
-        if nextchar == '}':
-            if object_pairs_hook is not None:
-                result = object_pairs_hook(pairs)
-                return result, end + 1
-            pairs = {}
-            if object_hook is not None:
-                pairs = object_hook(pairs)
-            return pairs, end + 1
-        elif nextchar != '"':
-            raise JSONDecodeError(
-                "Expecting property name enclosed in double quotes",
-                s, end)
-    end += 1
-    while True:
-        key, end = scanstring(s, end, encoding, strict)
-        key = memo_get(key, key)
-
-        # To skip some function call overhead we optimize the fast paths where
-        # the JSON key separator is ": " or just ":".
-        if s[end:end + 1] != ':':
-            end = _w(s, end).end()
-            if s[end:end + 1] != ':':
-                raise JSONDecodeError("Expecting ':' delimiter", s, end)
-
-        end += 1
-
-        try:
-            if s[end] in _ws:
-                end += 1
-                if s[end] in _ws:
-                    end = _w(s, end + 1).end()
-        except IndexError:
-            pass
-
-        value, end = scan_once(s, end)
-        pairs.append((key, value))
-
-        try:
-            nextchar = s[end]
-            if nextchar in _ws:
-                end = _w(s, end + 1).end()
-                nextchar = s[end]
-        except IndexError:
-            nextchar = ''
-        end += 1
-
-        if nextchar == '}':
-            break
-        elif nextchar != ',':
-            raise JSONDecodeError("Expecting ',' delimiter or '}'", s, end - 1)
-
-        try:
-            nextchar = s[end]
-            if nextchar in _ws:
-                end += 1
-                nextchar = s[end]
-                if nextchar in _ws:
-                    end = _w(s, end + 1).end()
-                    nextchar = s[end]
-        except IndexError:
-            nextchar = ''
-
-        end += 1
-        if nextchar != '"':
-            raise JSONDecodeError(
-                "Expecting property name enclosed in double quotes",
-                s, end - 1)
-
-    if object_pairs_hook is not None:
-        result = object_pairs_hook(pairs)
-        return result, end
-    pairs = dict(pairs)
-    if object_hook is not None:
-        pairs = object_hook(pairs)
-    return pairs, end
-
-def JSONArray(state, scan_once, _w=WHITESPACE.match, _ws=WHITESPACE_STR):
-    (s, end) = state
-    values = []
-    nextchar = s[end:end + 1]
-    if nextchar in _ws:
-        end = _w(s, end + 1).end()
-        nextchar = s[end:end + 1]
-    # Look-ahead for trivial empty array
-    if nextchar == ']':
-        return values, end + 1
-    elif nextchar == '':
-        raise JSONDecodeError("Expecting value or ']'", s, end)
-    _append = values.append
-    while True:
-        value, end = scan_once(s, end)
-        _append(value)
-        nextchar = s[end:end + 1]
-        if nextchar in _ws:
-            end = _w(s, end + 1).end()
-            nextchar = s[end:end + 1]
-        end += 1
-        if nextchar == ']':
-            break
-        elif nextchar != ',':
-            raise JSONDecodeError("Expecting ',' delimiter or ']'", s, end - 1)
-
-        try:
-            if s[end] in _ws:
-                end += 1
-                if s[end] in _ws:
-                    end = _w(s, end + 1).end()
-        except IndexError:
-            pass
-
-    return values, end
-
-class JSONDecoder(object):
-    """Simple JSON <http://json.org> decoder
-
-    Performs the following translations in decoding by default:
-
-    +---------------+-------------------+
-    | JSON          | Python            |
-    +===============+===================+
-    | object        | dict              |
-    +---------------+-------------------+
-    | array         | list              |
-    +---------------+-------------------+
-    | string        | unicode           |
-    +---------------+-------------------+
-    | number (int)  | int, long         |
-    +---------------+-------------------+
-    | number (real) | float             |
-    +---------------+-------------------+
-    | true          | True              |
-    +---------------+-------------------+
-    | false         | False             |
-    +---------------+-------------------+
-    | null          | None              |
-    +---------------+-------------------+
-
-    It also understands ``NaN``, ``Infinity``, and ``-Infinity`` as
-    their corresponding ``float`` values, which is outside the JSON spec.
-
-    """
-
-    def __init__(self, encoding=None, object_hook=None, parse_float=None,
-            parse_int=None, parse_constant=None, strict=True,
-            object_pairs_hook=None):
-        """
-        *encoding* determines the encoding used to interpret any
-        :class:`str` objects decoded by this instance (``'utf-8'`` by
-        default).  It has no effect when decoding :class:`unicode` objects.
-
-        Note that currently only encodings that are a superset of ASCII work,
-        strings of other encodings should be passed in as :class:`unicode`.
-
-        *object_hook*, if specified, will be called with the result of every
-        JSON object decoded and its return value will be used in place of the
-        given :class:`dict`.  This can be used to provide custom
-        deserializations (e.g. to support JSON-RPC class hinting).
-
-        *object_pairs_hook* is an optional function that will be called with
-        the result of any object literal decode with an ordered list of pairs.
-        The return value of *object_pairs_hook* will be used instead of the
-        :class:`dict`.  This feature can be used to implement custom decoders
-        that rely on the order that the key and value pairs are decoded (for
-        example, :func:`collections.OrderedDict` will remember the order of
-        insertion). If *object_hook* is also defined, the *object_pairs_hook*
-        takes priority.
-
-        *parse_float*, if specified, will be called with the string of every
-        JSON float to be decoded.  By default, this is equivalent to
-        ``float(num_str)``. This can be used to use another datatype or parser
-        for JSON floats (e.g. :class:`decimal.Decimal`).
-
-        *parse_int*, if specified, will be called with the string of every
-        JSON int to be decoded.  By default, this is equivalent to
-        ``int(num_str)``.  This can be used to use another datatype or parser
-        for JSON integers (e.g. :class:`float`).
-
-        *parse_constant*, if specified, will be called with one of the
-        following strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``.  This
-        can be used to raise an exception if invalid JSON numbers are
-        encountered.
-
-        *strict* controls the parser's behavior when it encounters an
-        invalid control character in a string. The default setting of
-        ``True`` means that unescaped control characters are parse errors, if
-        ``False`` then control characters will be allowed in strings.
-
-        """
-        if encoding is None:
-            encoding = DEFAULT_ENCODING
-        self.encoding = encoding
-        self.object_hook = object_hook
-        self.object_pairs_hook = object_pairs_hook
-        self.parse_float = parse_float or float
-        self.parse_int = parse_int or int
-        self.parse_constant = parse_constant or _CONSTANTS.__getitem__
-        self.strict = strict
-        self.parse_object = JSONObject
-        self.parse_array = JSONArray
-        self.parse_string = scanstring
-        self.memo = {}
-        self.scan_once = make_scanner(self)
-
-    def decode(self, s, _w=WHITESPACE.match, _PY3=PY3):
-        """Return the Python representation of ``s`` (a ``str`` or ``unicode``
-        instance containing a JSON document)
-
-        """
-        if _PY3 and isinstance(s, binary_type):
-            s = s.decode(self.encoding)
-        obj, end = self.raw_decode(s)
-        end = _w(s, end).end()
-        if end != len(s):
-            raise JSONDecodeError("Extra data", s, end, len(s))
-        return obj
-
-    def raw_decode(self, s, idx=0, _w=WHITESPACE.match, _PY3=PY3):
-        """Decode a JSON document from ``s`` (a ``str`` or ``unicode``
-        beginning with a JSON document) and return a 2-tuple of the Python
-        representation and the index in ``s`` where the document ended.
-        Optionally, ``idx`` can be used to specify an offset in ``s`` where
-        the JSON document begins.
-
-        This can be used to decode a JSON document from a string that may
-        have extraneous data at the end.
-
-        """
-        if _PY3 and not isinstance(s, text_type):
-            raise TypeError("Input string must be text, not bytes")
-        return self.scan_once(s, idx=_w(s, idx).end())

simplejson/encoder.py

@@ -1,628 +0,0 @@
-"""Implementation of JSONEncoder
-"""
-from __future__ import absolute_import
-import re
-from operator import itemgetter
-from decimal import Decimal
-from .compat import u, unichr, binary_type, string_types, integer_types, PY3
-def _import_speedups():
-    try:
-        from . import _speedups
-        return _speedups.encode_basestring_ascii, _speedups.make_encoder
-    except ImportError:
-        return None, None
-c_encode_basestring_ascii, c_make_encoder = _import_speedups()
-
-from simplejson.decoder import PosInf
-
-#ESCAPE = re.compile(ur'[\x00-\x1f\\"\b\f\n\r\t\u2028\u2029]')
-# This is required because u() will mangle the string and ur'' isn't valid
-# python3 syntax
-ESCAPE = re.compile(u'[\\x00-\\x1f\\\\"\\b\\f\\n\\r\\t\u2028\u2029]')
-ESCAPE_ASCII = re.compile(r'([\\"]|[^\ -~])')
-HAS_UTF8 = re.compile(r'[\x80-\xff]')
-ESCAPE_DCT = {
-    '\\': '\\\\',
-    '"': '\\"',
-    '\b': '\\b',
-    '\f': '\\f',
-    '\n': '\\n',
-    '\r': '\\r',
-    '\t': '\\t',
-}
-for i in range(0x20):
-    #ESCAPE_DCT.setdefault(chr(i), '\\u{0:04x}'.format(i))
-    ESCAPE_DCT.setdefault(chr(i), '\\u%04x' % (i,))
-for i in [0x2028, 0x2029]:
-    ESCAPE_DCT.setdefault(unichr(i), '\\u%04x' % (i,))
-
-FLOAT_REPR = repr
-
-def encode_basestring(s, _PY3=PY3, _q=u('"')):
-    """Return a JSON representation of a Python string
-
-    """
-    if _PY3:
-        if isinstance(s, binary_type):
-            s = s.decode('utf-8')
-    else:
-        if isinstance(s, str) and HAS_UTF8.search(s) is not None:
-            s = s.decode('utf-8')
-    def replace(match):
-        return ESCAPE_DCT[match.group(0)]
-    return _q + ESCAPE.sub(replace, s) + _q
-
-
-def py_encode_basestring_ascii(s, _PY3=PY3):
-    """Return an ASCII-only JSON representation of a Python string
-
-    """
-    if _PY3:
-        if isinstance(s, binary_type):
-            s = s.decode('utf-8')
-    else:
-        if isinstance(s, str) and HAS_UTF8.search(s) is not None:
-            s = s.decode('utf-8')
-    def replace(match):
-        s = match.group(0)
-        try:
-            return ESCAPE_DCT[s]
-        except KeyError:
-            n = ord(s)
-            if n < 0x10000:
-                #return '\\u{0:04x}'.format(n)
-                return '\\u%04x' % (n,)
-            else:
-                # surrogate pair
-                n -= 0x10000
-                s1 = 0xd800 | ((n >> 10) & 0x3ff)
-                s2 = 0xdc00 | (n & 0x3ff)
-                #return '\\u{0:04x}\\u{1:04x}'.format(s1, s2)
-                return '\\u%04x\\u%04x' % (s1, s2)
-    return '"' + str(ESCAPE_ASCII.sub(replace, s)) + '"'
-
-
-encode_basestring_ascii = (
-    c_encode_basestring_ascii or py_encode_basestring_ascii)
-
-class JSONEncoder(object):
-    """Extensible JSON <http://json.org> encoder for Python data structures.
-
-    Supports the following objects and types by default:
-
-    +-------------------+---------------+
-    | Python            | JSON          |
-    +===================+===============+
-    | dict, namedtuple  | object        |
-    +-------------------+---------------+
-    | list, tuple       | array         |
-    +-------------------+---------------+
-    | str, unicode      | string        |
-    +-------------------+---------------+
-    | int, long, float  | number        |
-    +-------------------+---------------+
-    | True              | true          |
-    +-------------------+---------------+
-    | False             | false         |
-    +-------------------+---------------+
-    | None              | null          |
-    +-------------------+---------------+
-
-    To extend this to recognize other objects, subclass and implement a
-    ``.default()`` method with another method that returns a serializable
-    object for ``o`` if possible, otherwise it should call the superclass
-    implementation (to raise ``TypeError``).
-
-    """
-    item_separator = ', '
-    key_separator = ': '
-    def __init__(self, skipkeys=False, ensure_ascii=True,
-            check_circular=True, allow_nan=True, sort_keys=False,
-            indent=None, separators=None, encoding='utf-8', default=None,
-            use_decimal=True, namedtuple_as_object=True,
-            tuple_as_array=True, bigint_as_string=False,
-            item_sort_key=None, for_json=False, ignore_nan=False):
-        """Constructor for JSONEncoder, with sensible defaults.
-
-        If skipkeys is false, then it is a TypeError to attempt
-        encoding of keys that are not str, int, long, float or None.  If
-        skipkeys is True, such items are simply skipped.
-
-        If ensure_ascii is true, the output is guaranteed to be str
-        objects with all incoming unicode characters escaped.  If
-        ensure_ascii is false, the output will be unicode object.
-
-        If check_circular is true, then lists, dicts, and custom encoded
-        objects will be checked for circular references during encoding to
-        prevent an infinite recursion (which would cause an OverflowError).
-        Otherwise, no such check takes place.
-
-        If allow_nan is true, then NaN, Infinity, and -Infinity will be
-        encoded as such.  This behavior is not JSON specification compliant,
-        but is consistent with most JavaScript based encoders and decoders.
-        Otherwise, it will be a ValueError to encode such floats.
-
-        If sort_keys is true, then the output of dictionaries will be
-        sorted by key; this is useful for regression tests to ensure
-        that JSON serializations can be compared on a day-to-day basis.
-
-        If indent is a string, then JSON array elements and object members
-        will be pretty-printed with a newline followed by that string repeated
-        for each level of nesting. ``None`` (the default) selects the most compact
-        representation without any newlines. For backwards compatibility with
-        versions of simplejson earlier than 2.1.0, an integer is also accepted
-        and is converted to a string with that many spaces.
-
-        If specified, separators should be an (item_separator, key_separator)
-        tuple.  The default is (', ', ': ') if *indent* is ``None`` and
-        (',', ': ') otherwise.  To get the most compact JSON representation,
-        you should specify (',', ':') to eliminate whitespace.
-
-        If specified, default is a function that gets called for objects
-        that can't otherwise be serialized.  It should return a JSON encodable
-        version of the object or raise a ``TypeError``.
-
-        If encoding is not None, then all input strings will be
-        transformed into unicode using that encoding prior to JSON-encoding.
-        The default is UTF-8.
-
-        If use_decimal is true (not the default), ``decimal.Decimal`` will
-        be supported directly by the encoder. For the inverse, decode JSON
-        with ``parse_float=decimal.Decimal``.
-
-        If namedtuple_as_object is true (the default), objects with
-        ``_asdict()`` methods will be encoded as JSON objects.
-
-        If tuple_as_array is true (the default), tuple (and subclasses) will
-        be encoded as JSON arrays.
-
-        If bigint_as_string is true (not the default), ints 2**53 and higher
-        or lower than -2**53 will be encoded as strings. This is to avoid the
-        rounding that happens in Javascript otherwise.
-
-        If specified, item_sort_key is a callable used to sort the items in
-        each dictionary. This is useful if you want to sort items other than
-        in alphabetical order by key.
-
-        If for_json is true (not the default), objects with a ``for_json()``
-        method will use the return value of that method for encoding as JSON
-        instead of the object.
-
-        If *ignore_nan* is true (default: ``False``), then out of range
-        :class:`float` values (``nan``, ``inf``, ``-inf``) will be serialized
-        as ``null`` in compliance with the ECMA-262 specification. If true,
-        this will override *allow_nan*.
-
-        """
-
-        self.skipkeys = skipkeys
-        self.ensure_ascii = ensure_ascii
-        self.check_circular = check_circular
-        self.allow_nan = allow_nan
-        self.sort_keys = sort_keys
-        self.use_decimal = use_decimal
-        self.namedtuple_as_object = namedtuple_as_object
-        self.tuple_as_array = tuple_as_array
-        self.bigint_as_string = bigint_as_string
-        self.item_sort_key = item_sort_key
-        self.for_json = for_json
-        self.ignore_nan = ignore_nan
-        if indent is not None and not isinstance(indent, string_types):
-            indent = indent * ' '
-        self.indent = indent
-        if separators is not None:
-            self.item_separator, self.key_separator = separators
-        elif indent is not None:
-            self.item_separator = ','
-        if default is not None:
-            self.default = default
-        self.encoding = encoding
-
-    def default(self, o):
-        """Implement this method in a subclass such that it returns
-        a serializable object for ``o``, or calls the base implementation
-        (to raise a ``TypeError``).
-
-        For example, to support arbitrary iterators, you could
-        implement default like this::
-
-            def default(self, o):
-                try:
-                    iterable = iter(o)
-                except TypeError:
-                    pass
-                else:
-                    return list(iterable)
-                return JSONEncoder.default(self, o)
-
-        """
-        raise TypeError(repr(o) + " is not JSON serializable")
-
-    def encode(self, o):
-        """Return a JSON string representation of a Python data structure.
-
-        >>> from simplejson import JSONEncoder
-        >>> JSONEncoder().encode({"foo": ["bar", "baz"]})
-        '{"foo": ["bar", "baz"]}'
-
-        """
-        # This is for extremely simple cases and benchmarks.
-        if isinstance(o, binary_type):
-            _encoding = self.encoding
-            if (_encoding is not None and not (_encoding == 'utf-8')):
-                o = o.decode(_encoding)
-        if isinstance(o, string_types):
-            if self.ensure_ascii:
-                return encode_basestring_ascii(o)
-            else:
-                return encode_basestring(o)
-        # This doesn't pass the iterator directly to ''.join() because the
-        # exceptions aren't as detailed.  The list call should be roughly
-        # equivalent to the PySequence_Fast that ''.join() would do.
-        chunks = self.iterencode(o, _one_shot=True)
-        if not isinstance(chunks, (list, tuple)):
-            chunks = list(chunks)
-        if self.ensure_ascii:
-            return ''.join(chunks)
-        else:
-            return u''.join(chunks)
-
-    def iterencode(self, o, _one_shot=False):
-        """Encode the given object and yield each string
-        representation as available.
-
-        For example::
-
-            for chunk in JSONEncoder().iterencode(bigobject):
-                mysocket.write(chunk)
-
-        """
-        if self.check_circular:
-            markers = {}
-        else:
-            markers = None
-        if self.ensure_ascii:
-            _encoder = encode_basestring_ascii
-        else:
-            _encoder = encode_basestring
-        if self.encoding != 'utf-8':
-            def _encoder(o, _orig_encoder=_encoder, _encoding=self.encoding):
-                if isinstance(o, binary_type):
-                    o = o.decode(_encoding)
-                return _orig_encoder(o)
-
-        def floatstr(o, allow_nan=self.allow_nan, ignore_nan=self.ignore_nan,
-                _repr=FLOAT_REPR, _inf=PosInf, _neginf=-PosInf):
-            # Check for specials. Note that this type of test is processor
-            # and/or platform-specific, so do tests which don't depend on
-            # the internals.
-
-            if o != o:
-                text = 'NaN'
-            elif o == _inf:
-                text = 'Infinity'
-            elif o == _neginf:
-                text = '-Infinity'
-            else:
-                return _repr(o)
-
-            if ignore_nan:
-                text = 'null'
-            elif not allow_nan:
-                raise ValueError(
-                    "Out of range float values are not JSON compliant: " +
-                    repr(o))
-
-            return text
-
-
-        key_memo = {}
-        if (_one_shot and c_make_encoder is not None
-                and self.indent is None):
-            _iterencode = c_make_encoder(
-                markers, self.default, _encoder, self.indent,
-                self.key_separator, self.item_separator, self.sort_keys,
-                self.skipkeys, self.allow_nan, key_memo, self.use_decimal,
-                self.namedtuple_as_object, self.tuple_as_array,
-                self.bigint_as_string, self.item_sort_key,
-                self.encoding, self.for_json, self.ignore_nan,
-                Decimal)
-        else:
-            _iterencode = _make_iterencode(
-                markers, self.default, _encoder, self.indent, floatstr,
-                self.key_separator, self.item_separator, self.sort_keys,
-                self.skipkeys, _one_shot, self.use_decimal,
-                self.namedtuple_as_object, self.tuple_as_array,
-                self.bigint_as_string, self.item_sort_key,
-                self.encoding, self.for_json,
-                Decimal=Decimal)
-        try:
-            return _iterencode(o, 0)
-        finally:
-            key_memo.clear()
-
-
-class JSONEncoderForHTML(JSONEncoder):
-    """An encoder that produces JSON safe to embed in HTML.
-
-    To embed JSON content in, say, a script tag on a web page, the
-    characters &, < and > should be escaped. They cannot be escaped
-    with the usual entities (e.g. &amp;) because they are not expanded
-    within <script> tags.
-    """
-
-    def encode(self, o):
-        # Override JSONEncoder.encode because it has hacks for
-        # performance that make things more complicated.
-        chunks = self.iterencode(o, True)
-        if self.ensure_ascii:
-            return ''.join(chunks)
-        else:
-            return u''.join(chunks)
-
-    def iterencode(self, o, _one_shot=False):
-        chunks = super(JSONEncoderForHTML, self).iterencode(o, _one_shot)
-        for chunk in chunks:
-            chunk = chunk.replace('&', '\\u0026')
-            chunk = chunk.replace('<', '\\u003c')
-            chunk = chunk.replace('>', '\\u003e')
-            yield chunk
-
-
-def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,
-        _key_separator, _item_separator, _sort_keys, _skipkeys, _one_shot,
-        _use_decimal, _namedtuple_as_object, _tuple_as_array,
-        _bigint_as_string, _item_sort_key, _encoding, _for_json,
-        ## HACK: hand-optimized bytecode; turn globals into locals
-        _PY3=PY3,
-        ValueError=ValueError,
-        string_types=string_types,
-        Decimal=Decimal,
-        dict=dict,
-        float=float,
-        id=id,
-        integer_types=integer_types,
-        isinstance=isinstance,
-        list=list,
-        str=str,
-        tuple=tuple,
-    ):
-    if _item_sort_key and not callable(_item_sort_key):
-        raise TypeError("item_sort_key must be None or callable")
-    elif _sort_keys and not _item_sort_key:
-        _item_sort_key = itemgetter(0)
-
-    def _iterencode_list(lst, _current_indent_level):
-        if not lst:
-            yield '[]'
-            return
-        if markers is not None:
-            markerid = id(lst)
-            if markerid in markers:
-                raise ValueError("Circular reference detected")
-            markers[markerid] = lst
-        buf = '['
-        if _indent is not None:
-            _current_indent_level += 1
-            newline_indent = '\n' + (_indent * _current_indent_level)
-            separator = _item_separator + newline_indent
-            buf += newline_indent
-        else:
-            newline_indent = None
-            separator = _item_separator
-        first = True
-        for value in lst:
-            if first:
-                first = False
-            else:
-                buf = separator
-            if (isinstance(value, string_types) or
-                (_PY3 and isinstance(value, binary_type))):
-                yield buf + _encoder(value)
-            elif value is None:
-                yield buf + 'null'
-            elif value is True:
-                yield buf + 'true'
-            elif value is False:
-                yield buf + 'false'
-            elif isinstance(value, integer_types):
-                yield ((buf + str(value))
-                       if (not _bigint_as_string or
-                           (-1 << 53) < value < (1 << 53))
-                           else (buf + '"' + str(value) + '"'))
-            elif isinstance(value, float):
-                yield buf + _floatstr(value)
-            elif _use_decimal and isinstance(value, Decimal):
-                yield buf + str(value)
-            else:
-                yield buf
-                for_json = _for_json and getattr(value, 'for_json', None)
-                if for_json and callable(for_json):
-                    chunks = _iterencode(for_json(), _current_indent_level)
-                elif isinstance(value, list):
-                    chunks = _iterencode_list(value, _current_indent_level)
-                else:
-                    _asdict = _namedtuple_as_object and getattr(value, '_asdict', None)
-                    if _asdict and callable(_asdict):
-                        chunks = _iterencode_dict(_asdict(),
-                                                  _current_indent_level)
-                    elif _tuple_as_array and isinstance(value, tuple):
-                        chunks = _iterencode_list(value, _current_indent_level)
-                    elif isinstance(value, dict):
-                        chunks = _iterencode_dict(value, _current_indent_level)
-                    else:
-                        chunks = _iterencode(value, _current_indent_level)
-                for chunk in chunks:
-                    yield chunk
-        if newline_indent is not None:
-            _current_indent_level -= 1
-            yield '\n' + (_indent * _current_indent_level)
-        yield ']'
-        if markers is not None:
-            del markers[markerid]
-
-    def _stringify_key(key):
-        if isinstance(key, string_types): # pragma: no cover
-            pass
-        elif isinstance(key, binary_type):
-            key = key.decode(_encoding)
-        elif isinstance(key, float):
-            key = _floatstr(key)
-        elif key is True:
-            key = 'true'
-        elif key is False:
-            key = 'false'
-        elif key is None:
-            key = 'null'
-        elif isinstance(key, integer_types):
-            key = str(key)
-        elif _use_decimal and isinstance(key, Decimal):
-            key = str(key)
-        elif _skipkeys:
-            key = None
-        else:
-            raise TypeError("key " + repr(key) + " is not a string")
-        return key
-
-    def _iterencode_dict(dct, _current_indent_level):
-        if not dct:
-            yield '{}'
-            return
-        if markers is not None:
-            markerid = id(dct)
-            if markerid in markers:
-                raise ValueError("Circular reference detected")
-            markers[markerid] = dct
-        yield '{'
-        if _indent is not None:
-            _current_indent_level += 1
-            newline_indent = '\n' + (_indent * _current_indent_level)
-            item_separator = _item_separator + newline_indent
-            yield newline_indent
-        else:
-            newline_indent = None
-            item_separator = _item_separator
-        first = True
-        if _PY3:
-            iteritems = dct.items()
-        else:
-            iteritems = dct.iteritems()
-        if _item_sort_key:
-            items = []
-            for k, v in dct.items():
-                if not isinstance(k, string_types):
-                    k = _stringify_key(k)
-                    if k is None:
-                        continue
-                items.append((k, v))
-            items.sort(key=_item_sort_key)
-        else:
-            items = iteritems
-        for key, value in items:
-            if not (_item_sort_key or isinstance(key, string_types)):
-                key = _stringify_key(key)
-                if key is None:
-                    # _skipkeys must be True
-                    continue
-            if first:
-                first = False
-            else:
-                yield item_separator
-            yield _encoder(key)
-            yield _key_separator
-            if (isinstance(value, string_types) or
-                (_PY3 and isinstance(value, binary_type))):
-                yield _encoder(value)
-            elif value is None:
-                yield 'null'
-            elif value is True:
-                yield 'true'
-            elif value is False:
-                yield 'false'
-            elif isinstance(value, integer_types):
-                yield (str(value)
-                       if (not _bigint_as_string or
-                           (-1 << 53) < value < (1 << 53))
-                           else ('"' + str(value) + '"'))
-            elif isinstance(value, float):
-                yield _floatstr(value)
-            elif _use_decimal and isinstance(value, Decimal):
-                yield str(value)
-            else:
-                for_json = _for_json and getattr(value, 'for_json', None)
-                if for_json and callable(for_json):
-                    chunks = _iterencode(for_json(), _current_indent_level)
-                elif isinstance(value, list):
-                    chunks = _iterencode_list(value, _current_indent_level)
-                else:
-                    _asdict = _namedtuple_as_object and getattr(value, '_asdict', None)
-                    if _asdict and callable(_asdict):
-                        chunks = _iterencode_dict(_asdict(),
-                                                  _current_indent_level)
-                    elif _tuple_as_array and isinstance(value, tuple):
-                        chunks = _iterencode_list(value, _current_indent_level)
-                    elif isinstance(value, dict):
-                        chunks = _iterencode_dict(value, _current_indent_level)
-                    else:
-                        chunks = _iterencode(value, _current_indent_level)
-                for chunk in chunks:
-                    yield chunk
-        if newline_indent is not None:
-            _current_indent_level -= 1
-            yield '\n' + (_indent * _current_indent_level)
-        yield '}'
-        if markers is not None:
-            del markers[markerid]
-
-    def _iterencode(o, _current_indent_level):
-        if (isinstance(o, string_types) or
-            (_PY3 and isinstance(o, binary_type))):
-            yield _encoder(o)
-        elif o is None:
-            yield 'null'
-        elif o is True:
-            yield 'true'
-        elif o is False:
-            yield 'false'
-        elif isinstance(o, integer_types):
-            yield (str(o)
-                   if (not _bigint_as_string or
-                       (-1 << 53) < o < (1 << 53))
-                       else ('"' + str(o) + '"'))
-        elif isinstance(o, float):
-            yield _floatstr(o)
-        else:
-            for_json = _for_json and getattr(o, 'for_json', None)
-            if for_json and callable(for_json):
-                for chunk in _iterencode(for_json(), _current_indent_level):
-                    yield chunk
-            elif isinstance(o, list):
-                for chunk in _iterencode_list(o, _current_indent_level):
-                    yield chunk
-            else:
-                _asdict = _namedtuple_as_object and getattr(o, '_asdict', None)
-                if _asdict and callable(_asdict):
-                    for chunk in _iterencode_dict(_asdict(),
-                            _current_indent_level):
-                        yield chunk
-                elif (_tuple_as_array and isinstance(o, tuple)):
-                    for chunk in _iterencode_list(o, _current_indent_level):
-                        yield chunk
-                elif isinstance(o, dict):
-                    for chunk in _iterencode_dict(o, _current_indent_level):
-                        yield chunk
-                elif _use_decimal and isinstance(o, Decimal):
-                    yield str(o)
-                else:
-                    if markers is not None:
-                        markerid = id(o)
-                        if markerid in markers:
-                            raise ValueError("Circular reference detected")
-                        markers[markerid] = o
-                    o = _default(o)
-                    for chunk in _iterencode(o, _current_indent_level):
-                        yield chunk
-                    if markers is not None:
-                        del markers[markerid]
-
-    return _iterencode

simplejson/ordered_dict.py

@@ -1,119 +0,0 @@
-"""Drop-in replacement for collections.OrderedDict by Raymond Hettinger
-
-http://code.activestate.com/recipes/576693/
-
-"""
-from UserDict import DictMixin
-
-# Modified from original to support Python 2.4, see
-# http://code.google.com/p/simplejson/issues/detail?id=53
-try:
-    all
-except NameError:
-    def all(seq):
-        for elem in seq:
-            if not elem:
-                return False
-        return True
-
-class OrderedDict(dict, DictMixin):
-
-    def __init__(self, *args, **kwds):
-        if len(args) > 1:
-            raise TypeError('expected at most 1 arguments, got %d' % len(args))
-        try:
-            self.__end
-        except AttributeError:
-            self.clear()
-        self.update(*args, **kwds)
-
-    def clear(self):
-        self.__end = end = []
-        end += [None, end, end]         # sentinel node for doubly linked list
-        self.__map = {}                 # key --> [key, prev, next]
-        dict.clear(self)
-
-    def __setitem__(self, key, value):
-        if key not in self:
-            end = self.__end
-            curr = end[1]
-            curr[2] = end[1] = self.__map[key] = [key, curr, end]
-        dict.__setitem__(self, key, value)
-
-    def __delitem__(self, key):
-        dict.__delitem__(self, key)
-        key, prev, next = self.__map.pop(key)
-        prev[2] = next
-        next[1] = prev
-
-    def __iter__(self):
-        end = self.__end
-        curr = end[2]
-        while curr is not end:
-            yield curr[0]
-            curr = curr[2]
-
-    def __reversed__(self):
-        end = self.__end
-        curr = end[1]
-        while curr is not end:
-            yield curr[0]
-            curr = curr[1]
-
-    def popitem(self, last=True):
-        if not self:
-            raise KeyError('dictionary is empty')
-        # Modified from original to support Python 2.4, see
-        # http://code.google.com/p/simplejson/issues/detail?id=53
-        if last:
-            key = reversed(self).next()
-        else:
-            key = iter(self).next()
-        value = self.pop(key)
-        return key, value
-
-    def __reduce__(self):
-        items = [[k, self[k]] for k in self]
-        tmp = self.__map, self.__end
-        del self.__map, self.__end
-        inst_dict = vars(self).copy()
-        self.__map, self.__end = tmp
-        if inst_dict:
-            return (self.__class__, (items,), inst_dict)
-        return self.__class__, (items,)
-
-    def keys(self):
-        return list(self)
-
-    setdefault = DictMixin.setdefault
-    update = DictMixin.update
-    pop = DictMixin.pop
-    values = DictMixin.values
-    items = DictMixin.items
-    iterkeys = DictMixin.iterkeys
-    itervalues = DictMixin.itervalues
-    iteritems = DictMixin.iteritems
-
-    def __repr__(self):
-        if not self:
-            return '%s()' % (self.__class__.__name__,)
-        return '%s(%r)' % (self.__class__.__name__, self.items())
-
-    def copy(self):
-        return self.__class__(self)
-
-    @classmethod
-    def fromkeys(cls, iterable, value=None):
-        d = cls()
-        for key in iterable:
-            d[key] = value
-        return d
-
-    def __eq__(self, other):
-        if isinstance(other, OrderedDict):
-            return len(self)==len(other) and \
-                   all(p==q for p, q in  zip(self.items(), other.items()))
-        return dict.__eq__(self, other)
-
-    def __ne__(self, other):
-        return not self == other

simplejson/scanner.py

@@ -1,125 +0,0 @@
-"""JSON token scanner
-"""
-import re
-def _import_c_make_scanner():
-    try:
-        from simplejson._speedups import make_scanner
-        return make_scanner
-    except ImportError:
-        return None
-c_make_scanner = _import_c_make_scanner()
-
-__all__ = ['make_scanner', 'JSONDecodeError']
-
-NUMBER_RE = re.compile(
-    r'(-?(?:0|[1-9]\d*))(\.\d+)?([eE][-+]?\d+)?',
-    (re.VERBOSE | re.MULTILINE | re.DOTALL))
-
-class JSONDecodeError(ValueError):
-    """Subclass of ValueError with the following additional properties:
-
-    msg: The unformatted error message
-    doc: The JSON document being parsed
-    pos: The start index of doc where parsing failed
-    end: The end index of doc where parsing failed (may be None)
-    lineno: The line corresponding to pos
-    colno: The column corresponding to pos
-    endlineno: The line corresponding to end (may be None)
-    endcolno: The column corresponding to end (may be None)
-
-    """
-    # Note that this exception is used from _speedups
-    def __init__(self, msg, doc, pos, end=None):
-        ValueError.__init__(self, errmsg(msg, doc, pos, end=end))
-        self.msg = msg
-        self.doc = doc
-        self.pos = pos
-        self.end = end
-        self.lineno, self.colno = linecol(doc, pos)
-        if end is not None:
-            self.endlineno, self.endcolno = linecol(doc, end)
-        else:
-            self.endlineno, self.endcolno = None, None
-
-
-def linecol(doc, pos):
-    lineno = doc.count('\n', 0, pos) + 1
-    if lineno == 1:
-        colno = pos + 1
-    else:
-        colno = pos - doc.rindex('\n', 0, pos)
-    return lineno, colno
-
-
-def errmsg(msg, doc, pos, end=None):
-    lineno, colno = linecol(doc, pos)
-    msg = msg.replace('%r', repr(doc[pos:pos + 1]))
-    if end is None:
-        fmt = '%s: line %d column %d (char %d)'
-        return fmt % (msg, lineno, colno, pos)
-    endlineno, endcolno = linecol(doc, end)
-    fmt = '%s: line %d column %d - line %d column %d (char %d - %d)'
-    return fmt % (msg, lineno, colno, endlineno, endcolno, pos, end)
-
-
-def py_make_scanner(context):
-    parse_object = context.parse_object
-    parse_array = context.parse_array
-    parse_string = context.parse_string
-    match_number = NUMBER_RE.match
-    encoding = context.encoding
-    strict = context.strict
-    parse_float = context.parse_float
-    parse_int = context.parse_int
-    parse_constant = context.parse_constant
-    object_hook = context.object_hook
-    object_pairs_hook = context.object_pairs_hook
-    memo = context.memo
-
-    def _scan_once(string, idx):
-        errmsg = 'Expecting value'
-        try:
-            nextchar = string[idx]
-        except IndexError:
-            raise JSONDecodeError(errmsg, string, idx)
-
-        if nextchar == '"':
-            return parse_string(string, idx + 1, encoding, strict)
-        elif nextchar == '{':
-            return parse_object((string, idx + 1), encoding, strict,
-                _scan_once, object_hook, object_pairs_hook, memo)
-        elif nextchar == '[':
-            return parse_array((string, idx + 1), _scan_once)
-        elif nextchar == 'n' and string[idx:idx + 4] == 'null':
-            return None, idx + 4
-        elif nextchar == 't' and string[idx:idx + 4] == 'true':
-            return True, idx + 4
-        elif nextchar == 'f' and string[idx:idx + 5] == 'false':
-            return False, idx + 5
-
-        m = match_number(string, idx)
-        if m is not None:
-            integer, frac, exp = m.groups()
-            if frac or exp:
-                res = parse_float(integer + (frac or '') + (exp or ''))
-            else:
-                res = parse_int(integer)
-            return res, m.end()
-        elif nextchar == 'N' and string[idx:idx + 3] == 'NaN':
-            return parse_constant('NaN'), idx + 3
-        elif nextchar == 'I' and string[idx:idx + 8] == 'Infinity':
-            return parse_constant('Infinity'), idx + 8
-        elif nextchar == '-' and string[idx:idx + 9] == '-Infinity':
-            return parse_constant('-Infinity'), idx + 9
-        else:
-            raise JSONDecodeError(errmsg, string, idx)
-
-    def scan_once(string, idx):
-        try:
-            return _scan_once(string, idx)
-        finally:
-            memo.clear()
-
-    return scan_once
-
-make_scanner = c_make_scanner or py_make_scanner

simplejson/tests/__init__.py

@@ -1,77 +0,0 @@
-from __future__ import absolute_import
-import unittest
-import doctest
-import sys
-
-class OptionalExtensionTestSuite(unittest.TestSuite):
-    def run(self, result):
-        import simplejson
-        run = unittest.TestSuite.run
-        run(self, result)
-        if simplejson._import_c_make_encoder() is None:
-            TestMissingSpeedups().run(result)
-        else:
-            simplejson._toggle_speedups(False)
-            run(self, result)
-            simplejson._toggle_speedups(True)
-        return result
-
-class TestMissingSpeedups(unittest.TestCase):
-    def runTest(self):
-        if hasattr(sys, 'pypy_translation_info'):
-            "PyPy doesn't need speedups! :)"
-        elif hasattr(self, 'skipTest'):
-            self.skipTest('_speedups.so is missing!')
-
-def additional_tests(suite=None):
-    import simplejson
-    import simplejson.encoder
-    import simplejson.decoder
-    if suite is None:
-        suite = unittest.TestSuite()
-    for mod in (simplejson, simplejson.encoder, simplejson.decoder):
-        suite.addTest(doctest.DocTestSuite(mod))
-    suite.addTest(doctest.DocFileSuite('../../index.rst'))
-    return suite
-
-
-def all_tests_suite():
-    suite = unittest.TestLoader().loadTestsFromNames([
-        'simplejson.tests.test_bigint_as_string',
-        'simplejson.tests.test_check_circular',
-        'simplejson.tests.test_decode',
-        'simplejson.tests.test_default',
-        'simplejson.tests.test_dump',
-        'simplejson.tests.test_encode_basestring_ascii',
-        'simplejson.tests.test_encode_for_html',
-        'simplejson.tests.test_errors',
-        'simplejson.tests.test_fail',
-        'simplejson.tests.test_float',
-        'simplejson.tests.test_indent',
-        'simplejson.tests.test_pass1',
-        'simplejson.tests.test_pass2',
-        'simplejson.tests.test_pass3',
-        'simplejson.tests.test_recursion',
-        'simplejson.tests.test_scanstring',
-        'simplejson.tests.test_separators',
-        'simplejson.tests.test_speedups',
-        'simplejson.tests.test_unicode',
-        'simplejson.tests.test_decimal',
-        'simplejson.tests.test_tuple',
-        'simplejson.tests.test_namedtuple',
-    ])
-    suite = additional_tests(suite)
-    return OptionalExtensionTestSuite([suite])
-
-
-def main():
-    runner = unittest.TextTestRunner(verbosity=1 + sys.argv.count('-v'))
-    suite = all_tests_suite()
-    raise SystemExit(not runner.run(suite).wasSuccessful())
-
-
-if __name__ == '__main__':
-    import os
-    import sys
-    sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
-    main()

simplejson/tests/test_bigint_as_string.py

@@ -1,58 +0,0 @@
-from unittest import TestCase
-
-import simplejson as json
-from simplejson.compat import long_type
-
-class TestBigintAsString(TestCase):
-    # Python 2.5, at least the one that ships on Mac OS X, calculates
-    # 2 ** 53 as 0! It manages to calculate 1 << 53 correctly.
-    values = [(200, 200),
-              ((1 << 53) - 1, 9007199254740991),
-              ((1 << 53), '9007199254740992'),
-              ((1 << 53) + 1, '9007199254740993'),
-              (-100, -100),
-              ((-1 << 53), '-9007199254740992'),
-              ((-1 << 53) - 1, '-9007199254740993'),
-              ((-1 << 53) + 1, -9007199254740991)]
-
-    def test_ints(self):
-        for val, expect in self.values:
-            self.assertEqual(
-                val,
-                json.loads(json.dumps(val)))
-            self.assertEqual(
-                expect,
-                json.loads(json.dumps(val, bigint_as_string=True)))
-
-    def test_lists(self):
-        for val, expect in self.values:
-            val = [val, val]
-            expect = [expect, expect]
-            self.assertEqual(
-                val,
-                json.loads(json.dumps(val)))
-            self.assertEqual(
-                expect,
-                json.loads(json.dumps(val, bigint_as_string=True)))
-
-    def test_dicts(self):
-        for val, expect in self.values:
-            val = {'k': val}
-            expect = {'k': expect}
-            self.assertEqual(
-                val,
-                json.loads(json.dumps(val)))
-            self.assertEqual(
-                expect,
-                json.loads(json.dumps(val, bigint_as_string=True)))
-
-    def test_dict_keys(self):
-        for val, _ in self.values:
-            expect = {str(val): 'value'}
-            val = {val: 'value'}
-            self.assertEqual(
-                expect,
-                json.loads(json.dumps(val)))
-            self.assertEqual(
-                expect,
-                json.loads(json.dumps(val, bigint_as_string=True)))

simplejson/tests/test_check_circular.py

@@ -1,30 +0,0 @@
-from unittest import TestCase
-import simplejson as json
-
-def default_iterable(obj):
-    return list(obj)
-
-class TestCheckCircular(TestCase):
-    def test_circular_dict(self):
-        dct = {}
-        dct['a'] = dct
-        self.assertRaises(ValueError, json.dumps, dct)
-
-    def test_circular_list(self):
-        lst = []
-        lst.append(lst)
-        self.assertRaises(ValueError, json.dumps, lst)
-
-    def test_circular_composite(self):
-        dct2 = {}
-        dct2['a'] = []
-        dct2['a'].append(dct2)
-        self.assertRaises(ValueError, json.dumps, dct2)
-
-    def test_circular_default(self):
-        json.dumps([set()], default=default_iterable)
-        self.assertRaises(TypeError, json.dumps, [set()])
-
-    def test_circular_off_default(self):
-        json.dumps([set()], default=default_iterable, check_circular=False)
-        self.assertRaises(TypeError, json.dumps, [set()], check_circular=False)

simplejson/tests/test_decimal.py

@@ -1,71 +0,0 @@
-import decimal
-from decimal import Decimal
-from unittest import TestCase
-from simplejson.compat import StringIO, reload_module
-
-import simplejson as json
-
-class TestDecimal(TestCase):
-    NUMS = "1.0", "10.00", "1.1", "1234567890.1234567890", "500"
-    def dumps(self, obj, **kw):
-        sio = StringIO()
-        json.dump(obj, sio, **kw)
-        res = json.dumps(obj, **kw)
-        self.assertEqual(res, sio.getvalue())
-        return res
-
-    def loads(self, s, **kw):
-        sio = StringIO(s)
-        res = json.loads(s, **kw)
-        self.assertEqual(res, json.load(sio, **kw))
-        return res
-
-    def test_decimal_encode(self):
-        for d in map(Decimal, self.NUMS):
-            self.assertEqual(self.dumps(d, use_decimal=True), str(d))
-
-    def test_decimal_decode(self):
-        for s in self.NUMS:
-            self.assertEqual(self.loads(s, parse_float=Decimal), Decimal(s))
-
-    def test_stringify_key(self):
-        for d in map(Decimal, self.NUMS):
-            v = {d: d}
-            self.assertEqual(
-                self.loads(
-                    self.dumps(v, use_decimal=True), parse_float=Decimal),
-                {str(d): d})
-
-    def test_decimal_roundtrip(self):
-        for d in map(Decimal, self.NUMS):
-            # The type might not be the same (int and Decimal) but they
-            # should still compare equal.
-            for v in [d, [d], {'': d}]:
-                self.assertEqual(
-                    self.loads(
-                        self.dumps(v, use_decimal=True), parse_float=Decimal),
-                    v)
-
-    def test_decimal_defaults(self):
-        d = Decimal('1.1')
-        # use_decimal=True is the default
-        self.assertRaises(TypeError, json.dumps, d, use_decimal=False)
-        self.assertEqual('1.1', json.dumps(d))
-        self.assertEqual('1.1', json.dumps(d, use_decimal=True))
-        self.assertRaises(TypeError, json.dump, d, StringIO(),
-                          use_decimal=False)
-        sio = StringIO()
-        json.dump(d, sio)
-        self.assertEqual('1.1', sio.getvalue())
-        sio = StringIO()
-        json.dump(d, sio, use_decimal=True)
-        self.assertEqual('1.1', sio.getvalue())
-
-    def test_decimal_reload(self):
-        # Simulate a subinterpreter that reloads the Python modules but not
-        # the C code https://github.com/simplejson/simplejson/issues/34
-        global Decimal
-        Decimal = reload_module(decimal).Decimal
-        import simplejson.encoder
-        simplejson.encoder.Decimal = Decimal
-        self.test_decimal_roundtrip()

simplejson/tests/test_decode.py

@@ -1,88 +0,0 @@
-from __future__ import absolute_import
-import decimal
-from unittest import TestCase
-
-import simplejson as json
-from simplejson.compat import StringIO
-from simplejson import OrderedDict
-
-class TestDecode(TestCase):
-    if not hasattr(TestCase, 'assertIs'):
-        def assertIs(self, a, b):
-            self.assertTrue(a is b, '%r is %r' % (a, b))
-
-    def test_decimal(self):
-        rval = json.loads('1.1', parse_float=decimal.Decimal)
-        self.assertTrue(isinstance(rval, decimal.Decimal))
-        self.assertEqual(rval, decimal.Decimal('1.1'))
-
-    def test_float(self):
-        rval = json.loads('1', parse_int=float)
-        self.assertTrue(isinstance(rval, float))
-        self.assertEqual(rval, 1.0)
-
-    def test_decoder_optimizations(self):
-        # Several optimizations were made that skip over calls to
-        # the whitespace regex, so this test is designed to try and
-        # exercise the uncommon cases. The array cases are already covered.
-        rval = json.loads('{   "key"    :    "value"    ,  "k":"v"    }')
-        self.assertEqual(rval, {"key":"value", "k":"v"})
-
-    def test_empty_objects(self):
-        s = '{}'
-        self.assertEqual(json.loads(s), eval(s))
-        s = '[]'
-        self.assertEqual(json.loads(s), eval(s))
-        s = '""'
-        self.assertEqual(json.loads(s), eval(s))
-
-    def test_object_pairs_hook(self):
-        s = '{"xkd":1, "kcw":2, "art":3, "hxm":4, "qrt":5, "pad":6, "hoy":7}'
-        p = [("xkd", 1), ("kcw", 2), ("art", 3), ("hxm", 4),
-             ("qrt", 5), ("pad", 6), ("hoy", 7)]
-        self.assertEqual(json.loads(s), eval(s))
-        self.assertEqual(json.loads(s, object_pairs_hook=lambda x: x), p)
-        self.assertEqual(json.load(StringIO(s),
-                                   object_pairs_hook=lambda x: x), p)
-        od = json.loads(s, object_pairs_hook=OrderedDict)
-        self.assertEqual(od, OrderedDict(p))
-        self.assertEqual(type(od), OrderedDict)
-        # the object_pairs_hook takes priority over the object_hook
-        self.assertEqual(json.loads(s,
-                                    object_pairs_hook=OrderedDict,
-                                    object_hook=lambda x: None),
-                         OrderedDict(p))
-
-    def check_keys_reuse(self, source, loads):
-        rval = loads(source)
-        (a, b), (c, d) = sorted(rval[0]), sorted(rval[1])
-        self.assertIs(a, c)
-        self.assertIs(b, d)
-
-    def test_keys_reuse_str(self):
-        s = u'[{"a_key": 1, "b_\xe9": 2}, {"a_key": 3, "b_\xe9": 4}]'.encode('utf8')
-        self.check_keys_reuse(s, json.loads)
-
-    def test_keys_reuse_unicode(self):
-        s = u'[{"a_key": 1, "b_\xe9": 2}, {"a_key": 3, "b_\xe9": 4}]'
-        self.check_keys_reuse(s, json.loads)
-
-    def test_empty_strings(self):
-        self.assertEqual(json.loads('""'), "")
-        self.assertEqual(json.loads(u'""'), u"")
-        self.assertEqual(json.loads('[""]'), [""])
-        self.assertEqual(json.loads(u'[""]'), [u""])
-
-    def test_raw_decode(self):
-        cls = json.decoder.JSONDecoder
-        self.assertEqual(
-            ({'a': {}}, 9),
-            cls().raw_decode("{\"a\": {}}"))
-        # http://code.google.com/p/simplejson/issues/detail?id=85
-        self.assertEqual(
-            ({'a': {}}, 9),
-            cls(object_pairs_hook=dict).raw_decode("{\"a\": {}}"))
-        # https://github.com/simplejson/simplejson/pull/38
-        self.assertEqual(
-            ({'a': {}}, 11),
-            cls().raw_decode(" \n{\"a\": {}}"))

simplejson/tests/test_default.py

@@ -1,9 +0,0 @@
-from unittest import TestCase
-
-import simplejson as json
-
-class TestDefault(TestCase):
-    def test_default(self):
-        self.assertEqual(
-            json.dumps(type, default=repr),
-            json.dumps(repr(type)))

simplejson/tests/test_dump.py

@@ -1,121 +0,0 @@
-from unittest import TestCase
-from simplejson.compat import StringIO, long_type, b, binary_type, PY3
-import simplejson as json
-
-def as_text_type(s):
-    if PY3 and isinstance(s, binary_type):
-        return s.decode('ascii')
-    return s
-
-class TestDump(TestCase):
-    def test_dump(self):
-        sio = StringIO()
-        json.dump({}, sio)
-        self.assertEqual(sio.getvalue(), '{}')
-
-    def test_constants(self):
-        for c in [None, True, False]:
-            self.assertTrue(json.loads(json.dumps(c)) is c)
-            self.assertTrue(json.loads(json.dumps([c]))[0] is c)
-            self.assertTrue(json.loads(json.dumps({'a': c}))['a'] is c)
-
-    def test_stringify_key(self):
-        items = [(b('bytes'), 'bytes'),
-                 (1.0, '1.0'),
-                 (10, '10'),
-                 (True, 'true'),
-                 (False, 'false'),
-                 (None, 'null'),
-                 (long_type(100), '100')]
-        for k, expect in items:
-            self.assertEqual(
-                json.loads(json.dumps({k: expect})),
-                {expect: expect})
-            self.assertEqual(
-                json.loads(json.dumps({k: expect}, sort_keys=True)),
-                {expect: expect})
-        self.assertRaises(TypeError, json.dumps, {json: 1})
-        for v in [{}, {'other': 1}, {b('derp'): 1, 'herp': 2}]:
-            for sort_keys in [False, True]:
-                v0 = dict(v)
-                v0[json] = 1
-                v1 = dict((as_text_type(key), val) for (key, val) in v.items())
-                self.assertEqual(
-                    json.loads(json.dumps(v0, skipkeys=True, sort_keys=sort_keys)),
-                    v1)
-                self.assertEqual(
-                    json.loads(json.dumps({'': v0}, skipkeys=True, sort_keys=sort_keys)),
-                    {'': v1})
-                self.assertEqual(
-                    json.loads(json.dumps([v0], skipkeys=True, sort_keys=sort_keys)),
-                    [v1])
-
-    def test_dumps(self):
-        self.assertEqual(json.dumps({}), '{}')
-
-    def test_encode_truefalse(self):
-        self.assertEqual(json.dumps(
-                 {True: False, False: True}, sort_keys=True),
-                 '{"false": true, "true": false}')
-        self.assertEqual(
-            json.dumps(
-                {2: 3.0,
-                 4.0: long_type(5),
-                 False: 1,
-                 long_type(6): True,
-                 "7": 0},
-                sort_keys=True),
-            '{"2": 3.0, "4.0": 5, "6": true, "7": 0, "false": 1}')
-
-    def test_ordered_dict(self):
-        # http://bugs.python.org/issue6105
-        items = [('one', 1), ('two', 2), ('three', 3), ('four', 4), ('five', 5)]
-        s = json.dumps(json.OrderedDict(items))
-        self.assertEqual(
-            s,
-            '{"one": 1, "two": 2, "three": 3, "four": 4, "five": 5}')
-
-    def test_indent_unknown_type_acceptance(self):
-        """
-        A test against the regression mentioned at `github issue 29`_.
-
-        The indent parameter should accept any type which pretends to be
-        an instance of int or long when it comes to being multiplied by
-        strings, even if it is not actually an int or long, for
-        backwards compatibility.
-
-        .. _github issue 29:
-           http://github.com/simplejson/simplejson/issue/29
-        """
-
-        class AwesomeInt(object):
-            """An awesome reimplementation of integers"""
-
-            def __init__(self, *args, **kwargs):
-                if len(args) > 0:
-                    # [construct from literals, objects, etc.]
-                    # ...
-
-                    # Finally, if args[0] is an integer, store it
-                    if isinstance(args[0], int):
-                        self._int = args[0]
-
-            # [various methods]
-
-            def __mul__(self, other):
-                # [various ways to multiply AwesomeInt objects]
-                # ... finally, if the right-hand operand is not awesome enough,
-                # try to do a normal integer multiplication
-                if hasattr(self, '_int'):
-                    return self._int * other
-                else:
-                    raise NotImplementedError("To do non-awesome things with"
-                        " this object, please construct it from an integer!")
-
-        s = json.dumps([0, 1, 2], indent=AwesomeInt(3))
-        self.assertEqual(s, '[\n   0,\n   1,\n   2\n]')
-
-    def test_accumulator(self):
-        # the C API uses an accumulator that collects after 100,000 appends
-        lst = [0] * 100000
-        self.assertEqual(json.loads(json.dumps(lst)), lst)

simplejson/tests/test_encode_basestring_ascii.py

@@ -1,47 +0,0 @@
-from unittest import TestCase
-
-import simplejson.encoder
-from simplejson.compat import b
-
-CASES = [
-    (u'/\\"\ucafe\ubabe\uab98\ufcde\ubcda\uef4a\x08\x0c\n\r\t`1~!@#$%^&*()_+-=[]{}|;:\',./<>?', '"/\\\\\\"\\ucafe\\ubabe\\uab98\\ufcde\\ubcda\\uef4a\\b\\f\\n\\r\\t`1~!@#$%^&*()_+-=[]{}|;:\',./<>?"'),
-    (u'\u0123\u4567\u89ab\ucdef\uabcd\uef4a', '"\\u0123\\u4567\\u89ab\\ucdef\\uabcd\\uef4a"'),
-    (u'controls', '"controls"'),
-    (u'\x08\x0c\n\r\t', '"\\b\\f\\n\\r\\t"'),
-    (u'{"object with 1 member":["array with 1 element"]}', '"{\\"object with 1 member\\":[\\"array with 1 element\\"]}"'),
-    (u' s p a c e d ', '" s p a c e d "'),
-    (u'\U0001d120', '"\\ud834\\udd20"'),
-    (u'\u03b1\u03a9', '"\\u03b1\\u03a9"'),
-    (b('\xce\xb1\xce\xa9'), '"\\u03b1\\u03a9"'),
-    (u'\u03b1\u03a9', '"\\u03b1\\u03a9"'),
-    (b('\xce\xb1\xce\xa9'), '"\\u03b1\\u03a9"'),
-    (u'\u03b1\u03a9', '"\\u03b1\\u03a9"'),
-    (u'\u03b1\u03a9', '"\\u03b1\\u03a9"'),
-    (u"`1~!@#$%^&*()_+-={':[,]}|;.</>?", '"`1~!@#$%^&*()_+-={\':[,]}|;.</>?"'),
-    (u'\x08\x0c\n\r\t', '"\\b\\f\\n\\r\\t"'),
-    (u'\u0123\u4567\u89ab\ucdef\uabcd\uef4a', '"\\u0123\\u4567\\u89ab\\ucdef\\uabcd\\uef4a"'),
-]
-
-class TestEncodeBaseStringAscii(TestCase):
-    def test_py_encode_basestring_ascii(self):
-        self._test_encode_basestring_ascii(simplejson.encoder.py_encode_basestring_ascii)
-
-    def test_c_encode_basestring_ascii(self):
-        if not simplejson.encoder.c_encode_basestring_ascii:
-            return
-        self._test_encode_basestring_ascii(simplejson.encoder.c_encode_basestring_ascii)
-
-    def _test_encode_basestring_ascii(self, encode_basestring_ascii):
-        fname = encode_basestring_ascii.__name__
-        for input_string, expect in CASES:
-            result = encode_basestring_ascii(input_string)
-            #self.assertEqual(result, expect,
-            #    '{0!r} != {1!r} for {2}({3!r})'.format(
-            #        result, expect, fname, input_string))
-            self.assertEqual(result, expect,
-                '%r != %r for %s(%r)' % (result, expect, fname, input_string))
-
-    def test_sorted_dict(self):
-        items = [('one', 1), ('two', 2), ('three', 3), ('four', 4), ('five', 5)]
-        s = simplejson.dumps(dict(items), sort_keys=True)
-        self.assertEqual(s, '{"five": 5, "four": 4, "one": 1, "three": 3, "two": 2}')

simplejson/tests/test_encode_for_html.py

@@ -1,30 +0,0 @@
-import unittest
-
-import simplejson as json
-
-class TestEncodeForHTML(unittest.TestCase):
-
-    def setUp(self):
-        self.decoder = json.JSONDecoder()
-        self.encoder = json.JSONEncoderForHTML()
-
-    def test_basic_encode(self):
-        self.assertEqual(r'"\u0026"', self.encoder.encode('&'))
-        self.assertEqual(r'"\u003c"', self.encoder.encode('<'))
-        self.assertEqual(r'"\u003e"', self.encoder.encode('>'))
-
-    def test_basic_roundtrip(self):
-        for char in '&<>':
-            self.assertEqual(
-                char, self.decoder.decode(
-                    self.encoder.encode(char)))
-
-    def test_prevent_script_breakout(self):
-        bad_string = '</script><script>alert("gotcha")</script>'
-        self.assertEqual(
-            r'"\u003c/script\u003e\u003cscript\u003e'
-            r'alert(\"gotcha\")\u003c/script\u003e"',
-            self.encoder.encode(bad_string))
-        self.assertEqual(
-            bad_string, self.decoder.decode(
-                self.encoder.encode(bad_string)))

simplejson/tests/test_errors.py

@@ -1,35 +0,0 @@
-import sys
-from unittest import TestCase
-
-import simplejson as json
-from simplejson.compat import u, b
-
-class TestErrors(TestCase):
-    def test_string_keys_error(self):
-        data = [{'a': 'A', 'b': (2, 4), 'c': 3.0, ('d',): 'D tuple'}]
-        self.assertRaises(TypeError, json.dumps, data)
-
-    def test_decode_error(self):
-        err = None
-        try:
-            json.loads('{}\na\nb')
-        except json.JSONDecodeError:
-            err = sys.exc_info()[1]
-        else:
-            self.fail('Expected JSONDecodeError')
-        self.assertEqual(err.lineno, 2)
-        self.assertEqual(err.colno, 1)
-        self.assertEqual(err.endlineno, 3)
-        self.assertEqual(err.endcolno, 2)
-
-    def test_scan_error(self):
-        err = None
-        for t in (u, b):
-            try:
-                json.loads(t('{"asdf": "'))
-            except json.JSONDecodeError:
-                err = sys.exc_info()[1]
-            else:
-                self.fail('Expected JSONDecodeError')
-            self.assertEqual(err.lineno, 1)
-            self.assertEqual(err.colno, 9)

simplejson/tests/test_fail.py

@@ -1,119 +0,0 @@
-import sys
-from unittest import TestCase
-
-import simplejson as json
-
-# Fri Dec 30 18:57:26 2005
-JSONDOCS = [
-    # http://json.org/JSON_checker/test/fail1.json
-    '"A JSON payload should be an object or array, not a string."',
-    # http://json.org/JSON_checker/test/fail2.json
-    '["Unclosed array"',
-    # http://json.org/JSON_checker/test/fail3.json
-    '{unquoted_key: "keys must be quoted}',
-    # http://json.org/JSON_checker/test/fail4.json
-    '["extra comma",]',
-    # http://json.org/JSON_checker/test/fail5.json
-    '["double extra comma",,]',
-    # http://json.org/JSON_checker/test/fail6.json
-    '[   , "<-- missing value"]',
-    # http://json.org/JSON_checker/test/fail7.json
-    '["Comma after the close"],',
-    # http://json.org/JSON_checker/test/fail8.json
-    '["Extra close"]]',
-    # http://json.org/JSON_checker/test/fail9.json
-    '{"Extra comma": true,}',
-    # http://json.org/JSON_checker/test/fail10.json
-    '{"Extra value after close": true} "misplaced quoted value"',
-    # http://json.org/JSON_checker/test/fail11.json
-    '{"Illegal expression": 1 + 2}',
-    # http://json.org/JSON_checker/test/fail12.json
-    '{"Illegal invocation": alert()}',
-    # http://json.org/JSON_checker/test/fail13.json
-    '{"Numbers cannot have leading zeroes": 013}',
-    # http://json.org/JSON_checker/test/fail14.json
-    '{"Numbers cannot be hex": 0x14}',
-    # http://json.org/JSON_checker/test/fail15.json
-    '["Illegal backslash escape: \\x15"]',
-    # http://json.org/JSON_checker/test/fail16.json
-    '["Illegal backslash escape: \\\'"]',
-    # http://json.org/JSON_checker/test/fail17.json
-    '["Illegal backslash escape: \\017"]',
-    # http://json.org/JSON_checker/test/fail18.json
-    '[[[[[[[[[[[[[[[[[[[["Too deep"]]]]]]]]]]]]]]]]]]]]',
-    # http://json.org/JSON_checker/test/fail19.json
-    '{"Missing colon" null}',
-    # http://json.org/JSON_checker/test/fail20.json
-    '{"Double colon":: null}',
-    # http://json.org/JSON_checker/test/fail21.json
-    '{"Comma instead of colon", null}',
-    # http://json.org/JSON_checker/test/fail22.json
-    '["Colon instead of comma": false]',
-    # http://json.org/JSON_checker/test/fail23.json
-    '["Bad value", truth]',
-    # http://json.org/JSON_checker/test/fail24.json
-    "['single quote']",
-    # http://code.google.com/p/simplejson/issues/detail?id=3
-    u'["A\u001FZ control characters in string"]',
-    # misc based on coverage
-    '{',
-    '{]',
-    '{"foo": "bar"]',
-    '{"foo": "bar"',
-    'nul',
-    'nulx',
-    '-',
-    '-x',
-    '-e',
-    '-e0',
-    '-Infinite',
-    '-Inf',
-    'Infinit',
-    'Infinite',
-    'NaM',
-    'NuN',
-    'falsy',
-    'fal',
-    'trug',
-    'tru',
-    '1e',
-    '1ex',
-    '1e-',
-    '1e-x',
-]
-
-SKIPS = {
-    1: "why not have a string payload?",
-    18: "spec doesn't specify any nesting limitations",
-}
-
-class TestFail(TestCase):
-    def test_failures(self):
-        for idx, doc in enumerate(JSONDOCS):
-            idx = idx + 1
-            if idx in SKIPS:
-                json.loads(doc)
-                continue
-            try:
-                json.loads(doc)
-            except json.JSONDecodeError:
-                pass
-            else:
-                #self.fail("Expected failure for fail{0}.json: {1!r}".format(idx, doc))
-                self.fail("Expected failure for fail%d.json: %r" % (idx, doc))
-
-    def test_array_decoder_issue46(self):
-        # http://code.google.com/p/simplejson/issues/detail?id=46
-        for doc in [u'[,]', '[,]']:
-            try:
-                json.loads(doc)
-            except json.JSONDecodeError:
-                e = sys.exc_info()[1]
-                self.assertEqual(e.pos, 1)
-                self.assertEqual(e.lineno, 1)
-                self.assertEqual(e.colno, 1)
-            except Exception:
-                e = sys.exc_info()[1]
-                self.fail("Unexpected exception raised %r %s" % (e, e))
-            else:
-                self.fail("Unexpected success parsing '[,]'")

simplejson/tests/test_float.py

@@ -1,27 +0,0 @@
-import math
-from unittest import TestCase
-from simplejson.compat import long_type, text_type
-import simplejson as json
-from simplejson.decoder import NaN, PosInf, NegInf
-
-class TestFloat(TestCase):
-    def test_degenerates(self):
-        for inf in (PosInf, NegInf):
-            self.assertEqual(json.loads(json.dumps(inf)), inf)
-        # Python 2.5 doesn't have math.isnan
-        nan = json.loads(json.dumps(NaN))
-        self.assertTrue((0 + nan) != nan)
-
-    def test_floats(self):
-        for num in [1617161771.7650001, math.pi, math.pi**100,
-                    math.pi**-100, 3.1]:
-            self.assertEqual(float(json.dumps(num)), num)
-            self.assertEqual(json.loads(json.dumps(num)), num)
-            self.assertEqual(json.loads(text_type(json.dumps(num))), num)
-
-    def test_ints(self):
-        for num in [1, long_type(1), 1<<32, 1<<64]:
-            self.assertEqual(json.dumps(num), str(num))
-            self.assertEqual(int(json.dumps(num)), num)
-            self.assertEqual(json.loads(json.dumps(num)), num)
-            self.assertEqual(json.loads(text_type(json.dumps(num))), num)

simplejson/tests/test_indent.py

@@ -1,86 +0,0 @@
-from unittest import TestCase
-import textwrap
-
-import simplejson as json
-from simplejson.compat import StringIO
-
-class TestIndent(TestCase):
-    def test_indent(self):
-        h = [['blorpie'], ['whoops'], [], 'd-shtaeou', 'd-nthiouh',
-             'i-vhbjkhnth',
-             {'nifty': 87}, {'field': 'yes', 'morefield': False} ]
-
-        expect = textwrap.dedent("""\
-        [
-        \t[
-        \t\t"blorpie"
-        \t],
-        \t[
-        \t\t"whoops"
-        \t],
-        \t[],
-        \t"d-shtaeou",
-        \t"d-nthiouh",
-        \t"i-vhbjkhnth",
-        \t{
-        \t\t"nifty": 87
-        \t},
-        \t{
-        \t\t"field": "yes",
-        \t\t"morefield": false
-        \t}
-        ]""")
-
-
-        d1 = json.dumps(h)
-        d2 = json.dumps(h, indent='\t', sort_keys=True, separators=(',', ': '))
-        d3 = json.dumps(h, indent='  ', sort_keys=True, separators=(',', ': '))
-        d4 = json.dumps(h, indent=2, sort_keys=True, separators=(',', ': '))
-
-        h1 = json.loads(d1)
-        h2 = json.loads(d2)
-        h3 = json.loads(d3)
-        h4 = json.loads(d4)
-
-        self.assertEqual(h1, h)
-        self.assertEqual(h2, h)
-        self.assertEqual(h3, h)
-        self.assertEqual(h4, h)
-        self.assertEqual(d3, expect.replace('\t', '  '))
-        self.assertEqual(d4, expect.replace('\t', '  '))
-        # NOTE: Python 2.4 textwrap.dedent converts tabs to spaces,
-        #       so the following is expected to fail. Python 2.4 is not a
-        #       supported platform in simplejson 2.1.0+.
-        self.assertEqual(d2, expect)
-
-    def test_indent0(self):
-        h = {3: 1}
-        def check(indent, expected):
-            d1 = json.dumps(h, indent=indent)
-            self.assertEqual(d1, expected)
-
-            sio = StringIO()
-            json.dump(h, sio, indent=indent)
-            self.assertEqual(sio.getvalue(), expected)
-
-        # indent=0 should emit newlines
-        check(0, '{\n"3": 1\n}')
-        # indent=None is more compact
-        check(None, '{"3": 1}')
-
-    def test_separators(self):
-        lst = [1,2,3,4]
-        expect = '[\n1,\n2,\n3,\n4\n]'
-        expect_spaces = '[\n1, \n2, \n3, \n4\n]'
-        # Ensure that separators still works
-        self.assertEqual(
-            expect_spaces,
-            json.dumps(lst, indent=0, separators=(', ', ': ')))
-        # Force the new defaults
-        self.assertEqual(
-            expect,
-            json.dumps(lst, indent=0, separators=(',', ': ')))
-        # Added in 2.1.4
-        self.assertEqual(
-            expect,
-            json.dumps(lst, indent=0))

simplejson/tests/test_item_sort_key.py

@@ -1,20 +0,0 @@
-from unittest import TestCase
-
-import simplejson as json
-from operator import itemgetter
-
-class TestItemSortKey(TestCase):
-    def test_simple_first(self):
-        a = {'a': 1, 'c': 5, 'jack': 'jill', 'pick': 'axe', 'array': [1, 5, 6, 9], 'tuple': (83, 12, 3), 'crate': 'dog', 'zeak': 'oh'}
-        self.assertEqual(
-            '{"a": 1, "c": 5, "crate": "dog", "jack": "jill", "pick": "axe", "zeak": "oh", "array": [1, 5, 6, 9], "tuple": [83, 12, 3]}',
-            json.dumps(a, item_sort_key=json.simple_first))
-
-    def test_case(self):
-        a = {'a': 1, 'c': 5, 'Jack': 'jill', 'pick': 'axe', 'Array': [1, 5, 6, 9], 'tuple': (83, 12, 3), 'crate': 'dog', 'zeak': 'oh'}
-        self.assertEqual(
-            '{"Array": [1, 5, 6, 9], "Jack": "jill", "a": 1, "c": 5, "crate": "dog", "pick": "axe", "tuple": [83, 12, 3], "zeak": "oh"}',
-            json.dumps(a, item_sort_key=itemgetter(0)))
-        self.assertEqual(
-            '{"a": 1, "Array": [1, 5, 6, 9], "c": 5, "crate": "dog", "Jack": "jill", "pick": "axe", "tuple": [83, 12, 3], "zeak": "oh"}',
-            json.dumps(a, item_sort_key=lambda kv: kv[0].lower()))

simplejson/tests/test_namedtuple.py

@@ -1,122 +0,0 @@
-from __future__ import absolute_import
-import unittest
-import simplejson as json
-from simplejson.compat import StringIO
-
-try:
-    from collections import namedtuple
-except ImportError:
-    class Value(tuple):
-        def __new__(cls, *args):
-            return tuple.__new__(cls, args)
-
-        def _asdict(self):
-            return {'value': self[0]}
-    class Point(tuple):
-        def __new__(cls, *args):
-            return tuple.__new__(cls, args)
-
-        def _asdict(self):
-            return {'x': self[0], 'y': self[1]}
-else:
-    Value = namedtuple('Value', ['value'])
-    Point = namedtuple('Point', ['x', 'y'])
-
-class DuckValue(object):
-    def __init__(self, *args):
-        self.value = Value(*args)
-
-    def _asdict(self):
-        return self.value._asdict()
-
-class DuckPoint(object):
-    def __init__(self, *args):
-        self.point = Point(*args)
-
-    def _asdict(self):
-        return self.point._asdict()
-
-class DeadDuck(object):
-    _asdict = None
-
-class DeadDict(dict):
-    _asdict = None
-
-CONSTRUCTORS = [
-    lambda v: v,
-    lambda v: [v],
-    lambda v: [{'key': v}],
-]
-
-class TestNamedTuple(unittest.TestCase):
-    def test_namedtuple_dumps(self):
-        for v in [Value(1), Point(1, 2), DuckValue(1), DuckPoint(1, 2)]:
-            d = v._asdict()
-            self.assertEqual(d, json.loads(json.dumps(v)))
-            self.assertEqual(
-                d,
-                json.loads(json.dumps(v, namedtuple_as_object=True)))
-            self.assertEqual(d, json.loads(json.dumps(v, tuple_as_array=False)))
-            self.assertEqual(
-                d,
-                json.loads(json.dumps(v, namedtuple_as_object=True,
-                                      tuple_as_array=False)))
-
-    def test_namedtuple_dumps_false(self):
-        for v in [Value(1), Point(1, 2)]:
-            l = list(v)
-            self.assertEqual(
-                l,
-                json.loads(json.dumps(v, namedtuple_as_object=False)))
-            self.assertRaises(TypeError, json.dumps, v,
-                tuple_as_array=False, namedtuple_as_object=False)
-
-    def test_namedtuple_dump(self):
-        for v in [Value(1), Point(1, 2), DuckValue(1), DuckPoint(1, 2)]:
-            d = v._asdict()
-            sio = StringIO()
-            json.dump(v, sio)
-            self.assertEqual(d, json.loads(sio.getvalue()))
-            sio = StringIO()
-            json.dump(v, sio, namedtuple_as_object=True)
-            self.assertEqual(
-                d,
-                json.loads(sio.getvalue()))
-            sio = StringIO()
-            json.dump(v, sio, tuple_as_array=False)
-            self.assertEqual(d, json.loads(sio.getvalue()))
-            sio = StringIO()
-            json.dump(v, sio, namedtuple_as_object=True,
-                      tuple_as_array=False)
-            self.assertEqual(
-                d,
-                json.loads(sio.getvalue()))
-
-    def test_namedtuple_dump_false(self):
-        for v in [Value(1), Point(1, 2)]:
-            l = list(v)
-            sio = StringIO()
-            json.dump(v, sio, namedtuple_as_object=False)
-            self.assertEqual(
-                l,
-                json.loads(sio.getvalue()))
-            self.assertRaises(TypeError, json.dump, v, StringIO(),
-                tuple_as_array=False, namedtuple_as_object=False)
-
-    def test_asdict_not_callable_dump(self):
-        for f in CONSTRUCTORS:
-            self.assertRaises(TypeError,
-                json.dump, f(DeadDuck()), StringIO(), namedtuple_as_object=True)
-            sio = StringIO()
-            json.dump(f(DeadDict()), sio, namedtuple_as_object=True)
-            self.assertEqual(
-                json.dumps(f({})),
-                sio.getvalue())
-
-    def test_asdict_not_callable_dumps(self):
-        for f in CONSTRUCTORS:
-            self.assertRaises(TypeError,
-                json.dumps, f(DeadDuck()), namedtuple_as_object=True)
-            self.assertEqual(
-                json.dumps(f({})),
-                json.dumps(f(DeadDict()), namedtuple_as_object=True))

simplejson/tests/test_pass1.py

@@ -1,76 +0,0 @@
-from unittest import TestCase
-
-import simplejson as json
-
-# from http://json.org/JSON_checker/test/pass1.json
-JSON = r'''
-[
-    "JSON Test Pattern pass1",
-    {"object with 1 member":["array with 1 element"]},
-    {},
-    [],
-    -42,
-    true,
-    false,
-    null,
-    {
-        "integer": 1234567890,
-        "real": -9876.543210,
-        "e": 0.123456789e-12,
-        "E": 1.234567890E+34,
-        "":  23456789012E666,
-        "zero": 0,
-        "one": 1,
-        "space": " ",
-        "quote": "\"",
-        "backslash": "\\",
-        "controls": "\b\f\n\r\t",
-        "slash": "/ & \/",
-        "alpha": "abcdefghijklmnopqrstuvwyz",
-        "ALPHA": "ABCDEFGHIJKLMNOPQRSTUVWYZ",
-        "digit": "0123456789",
-        "special": "`1~!@#$%^&*()_+-={':[,]}|;.</>?",
-        "hex": "\u0123\u4567\u89AB\uCDEF\uabcd\uef4A",
-        "true": true,
-        "false": false,
-        "null": null,
-        "array":[  ],
-        "object":{  },
-        "address": "50 St. James Street",
-        "url": "http://www.JSON.org/",
-        "comment": "// /* <!-- --",
-        "# -- --> */": " ",
-        " s p a c e d " :[1,2 , 3
-
-,
-
-4 , 5        ,          6           ,7        ],
-        "compact": [1,2,3,4,5,6,7],
-        "jsontext": "{\"object with 1 member\":[\"array with 1 element\"]}",
-        "quotes": "&#34; \u0022 %22 0x22 034 &#x22;",
-        "\/\\\"\uCAFE\uBABE\uAB98\uFCDE\ubcda\uef4A\b\f\n\r\t`1~!@#$%^&*()_+-=[]{}|;:',./<>?"
-: "A key can be any string"
-    },
-    0.5 ,98.6
-,
-99.44
-,
-
-1066
-
-
-,"rosebud"]
-'''
-
-class TestPass1(TestCase):
-    def test_parse(self):
-        # test in/out equivalence and parsing
-        res = json.loads(JSON)
-        out = json.dumps(res)
-        self.assertEqual(res, json.loads(out))
-        try:
-            json.dumps(res, allow_nan=False)
-        except ValueError:
-            pass
-        else:
-            self.fail("23456789012E666 should be out of range")

simplejson/tests/test_pass2.py

@@ -1,14 +0,0 @@
-from unittest import TestCase
-import simplejson as json
-
-# from http://json.org/JSON_checker/test/pass2.json
-JSON = r'''
-[[[[[[[[[[[[[[[[[[["Not too deep"]]]]]]]]]]]]]]]]]]]
-'''
-
-class TestPass2(TestCase):
-    def test_parse(self):
-        # test in/out equivalence and parsing
-        res = json.loads(JSON)
-        out = json.dumps(res)
-        self.assertEqual(res, json.loads(out))

simplejson/tests/test_pass3.py

@@ -1,20 +0,0 @@
-from unittest import TestCase
-
-import simplejson as json
-
-# from http://json.org/JSON_checker/test/pass3.json
-JSON = r'''
-{
-    "JSON Test Pattern pass3": {
-        "The outermost value": "must be an object or array.",
-        "In this test": "It is an object."
-    }
-}
-'''
-
-class TestPass3(TestCase):
-    def test_parse(self):
-        # test in/out equivalence and parsing
-        res = json.loads(JSON)
-        out = json.dumps(res)
-        self.assertEqual(res, json.loads(out))

simplejson/tests/test_recursion.py

@@ -1,67 +0,0 @@
-from unittest import TestCase
-
-import simplejson as json
-
-class JSONTestObject:
-    pass
-
-
-class RecursiveJSONEncoder(json.JSONEncoder):
-    recurse = False
-    def default(self, o):
-        if o is JSONTestObject:
-            if self.recurse:
-                return [JSONTestObject]
-            else:
-                return 'JSONTestObject'
-        return json.JSONEncoder.default(o)
-
-
-class TestRecursion(TestCase):
-    def test_listrecursion(self):
-        x = []
-        x.append(x)
-        try:
-            json.dumps(x)
-        except ValueError:
-            pass
-        else:
-            self.fail("didn't raise ValueError on list recursion")
-        x = []
-        y = [x]
-        x.append(y)
-        try:
-            json.dumps(x)
-        except ValueError:
-            pass
-        else:
-            self.fail("didn't raise ValueError on alternating list recursion")
-        y = []
-        x = [y, y]
-        # ensure that the marker is cleared
-        json.dumps(x)
-
-    def test_dictrecursion(self):
-        x = {}
-        x["test"] = x
-        try:
-            json.dumps(x)
-        except ValueError:
-            pass
-        else:
-            self.fail("didn't raise ValueError on dict recursion")
-        x = {}
-        y = {"a": x, "b": x}
-        # ensure that the marker is cleared
-        json.dumps(y)
-
-    def test_defaultrecursion(self):
-        enc = RecursiveJSONEncoder()
-        self.assertEqual(enc.encode(JSONTestObject), '"JSONTestObject"')
-        enc.recurse = True
-        try:
-            enc.encode(JSONTestObject)
-        except ValueError:
-            pass
-        else:
-            self.fail("didn't raise ValueError on default recursion")

simplejson/tests/test_scanstring.py

@@ -1,147 +0,0 @@
-import sys
-from unittest import TestCase
-
-import simplejson as json
-import simplejson.decoder
-from simplejson.compat import b, PY3
-
-class TestScanString(TestCase):
-    # The bytes type is intentionally not used in most of these tests
-    # under Python 3 because the decoder immediately coerces to str before
-    # calling scanstring. In Python 2 we are testing the code paths
-    # for both unicode and str.
-    #
-    # The reason this is done is because Python 3 would require
-    # entirely different code paths for parsing bytes and str.
-    #
-    def test_py_scanstring(self):
-        self._test_scanstring(simplejson.decoder.py_scanstring)
-
-    def test_c_scanstring(self):
-        if not simplejson.decoder.c_scanstring:
-            return
-        self._test_scanstring(simplejson.decoder.c_scanstring)
-
-    def _test_scanstring(self, scanstring):
-        self.assertEqual(
-            scanstring('"z\\ud834\\udd20x"', 1, None, True),
-            (u'z\U0001d120x', 16))
-
-        if sys.maxunicode == 65535:
-            self.assertEqual(
-                scanstring(u'"z\U0001d120x"', 1, None, True),
-                (u'z\U0001d120x', 6))
-        else:
-            self.assertEqual(
-                scanstring(u'"z\U0001d120x"', 1, None, True),
-                (u'z\U0001d120x', 5))
-
-        self.assertEqual(
-            scanstring('"\\u007b"', 1, None, True),
-            (u'{', 8))
-
-        self.assertEqual(
-            scanstring('"A JSON payload should be an object or array, not a string."', 1, None, True),
-            (u'A JSON payload should be an object or array, not a string.', 60))
-
-        self.assertEqual(
-            scanstring('["Unclosed array"', 2, None, True),
-            (u'Unclosed array', 17))
-
-        self.assertEqual(
-            scanstring('["extra comma",]', 2, None, True),
-            (u'extra comma', 14))
-
-        self.assertEqual(
-            scanstring('["double extra comma",,]', 2, None, True),
-            (u'double extra comma', 21))
-
-        self.assertEqual(
-            scanstring('["Comma after the close"],', 2, None, True),
-            (u'Comma after the close', 24))
-
-        self.assertEqual(
-            scanstring('["Extra close"]]', 2, None, True),
-            (u'Extra close', 14))
-
-        self.assertEqual(
-            scanstring('{"Extra comma": true,}', 2, None, True),
-            (u'Extra comma', 14))
-
-        self.assertEqual(
-            scanstring('{"Extra value after close": true} "misplaced quoted value"', 2, None, True),
-            (u'Extra value after close', 26))
-
-        self.assertEqual(
-            scanstring('{"Illegal expression": 1 + 2}', 2, None, True),
-            (u'Illegal expression', 21))
-
-        self.assertEqual(
-            scanstring('{"Illegal invocation": alert()}', 2, None, True),
-            (u'Illegal invocation', 21))
-
-        self.assertEqual(
-            scanstring('{"Numbers cannot have leading zeroes": 013}', 2, None, True),
-            (u'Numbers cannot have leading zeroes', 37))
-
-        self.assertEqual(
-            scanstring('{"Numbers cannot be hex": 0x14}', 2, None, True),
-            (u'Numbers cannot be hex', 24))
-
-        self.assertEqual(
-            scanstring('[[[[[[[[[[[[[[[[[[[["Too deep"]]]]]]]]]]]]]]]]]]]]', 21, None, True),
-            (u'Too deep', 30))
-
-        self.assertEqual(
-            scanstring('{"Missing colon" null}', 2, None, True),
-            (u'Missing colon', 16))
-
-        self.assertEqual(
-            scanstring('{"Double colon":: null}', 2, None, True),
-            (u'Double colon', 15))
-
-        self.assertEqual(
-            scanstring('{"Comma instead of colon", null}', 2, None, True),
-            (u'Comma instead of colon', 25))
-
-        self.assertEqual(
-            scanstring('["Colon instead of comma": false]', 2, None, True),
-            (u'Colon instead of comma', 25))
-
-        self.assertEqual(
-            scanstring('["Bad value", truth]', 2, None, True),
-            (u'Bad value', 12))
-
-        for c in map(chr, range(0x00, 0x1f)):
-            self.assertEqual(
-                scanstring(c + '"', 0, None, False),
-                (c, 2))
-            self.assertRaises(
-                ValueError,
-                scanstring, c + '"', 0, None, True)
-
-        self.assertRaises(ValueError, scanstring, '', 0, None, True)
-        self.assertRaises(ValueError, scanstring, 'a', 0, None, True)
-        self.assertRaises(ValueError, scanstring, '\\', 0, None, True)
-        self.assertRaises(ValueError, scanstring, '\\u', 0, None, True)
-        self.assertRaises(ValueError, scanstring, '\\u0', 0, None, True)
-        self.assertRaises(ValueError, scanstring, '\\u01', 0, None, True)
-        self.assertRaises(ValueError, scanstring, '\\u012', 0, None, True)
-        self.assertRaises(ValueError, scanstring, '\\u0123', 0, None, True)
-        if sys.maxunicode > 65535:
-            self.assertRaises(ValueError, scanstring, '\\ud834"', 0, None, True),
-            self.assertRaises(ValueError, scanstring, '\\ud834\\u"', 0, None, True),
-            self.assertRaises(ValueError, scanstring, '\\ud834\\x0123"', 0, None, True),
-
-    def test_issue3623(self):
-        self.assertRaises(ValueError, json.decoder.scanstring, "xxx", 1,
-                          "xxx")
-        self.assertRaises(UnicodeDecodeError,
-                          json.encoder.encode_basestring_ascii, b("xx\xff"))
-
-    def test_overflow(self):
-        # Python 2.5 does not have maxsize, Python 3 does not have maxint
-        maxsize = getattr(sys, 'maxsize', getattr(sys, 'maxint', None))
-        assert maxsize is not None
-        self.assertRaises(OverflowError, json.decoder.scanstring, "xxx",
-                          maxsize + 1)

simplejson/tests/test_separators.py

@@ -1,42 +0,0 @@
-import textwrap
-from unittest import TestCase
-
-import simplejson as json
-
-
-class TestSeparators(TestCase):
-    def test_separators(self):
-        h = [['blorpie'], ['whoops'], [], 'd-shtaeou', 'd-nthiouh', 'i-vhbjkhnth',
-             {'nifty': 87}, {'field': 'yes', 'morefield': False} ]
-
-        expect = textwrap.dedent("""\
-        [
-          [
-            "blorpie"
-          ] ,
-          [
-            "whoops"
-          ] ,
-          [] ,
-          "d-shtaeou" ,
-          "d-nthiouh" ,
-          "i-vhbjkhnth" ,
-          {
-            "nifty" : 87
-          } ,
-          {
-            "field" : "yes" ,
-            "morefield" : false
-          }
-        ]""")
-
-
-        d1 = json.dumps(h)
-        d2 = json.dumps(h, indent='  ', sort_keys=True, separators=(' ,', ' : '))
-
-        h1 = json.loads(d1)
-        h2 = json.loads(d2)
-
-        self.assertEqual(h1, h)
-        self.assertEqual(h2, h)
-        self.assertEqual(d2, expect)

simplejson/tests/test_speedups.py

@@ -1,20 +0,0 @@
-from unittest import TestCase
-
-from simplejson import encoder, scanner
-
-def has_speedups():
-    return encoder.c_make_encoder is not None
-
-class TestDecode(TestCase):
-    def test_make_scanner(self):
-        if not has_speedups():
-            return
-        self.assertRaises(AttributeError, scanner.c_make_scanner, 1)
-
-    def test_make_encoder(self):
-        if not has_speedups():
-            return
-        self.assertRaises(TypeError, encoder.c_make_encoder,
-            None,
-            "\xCD\x7D\x3D\x4E\x12\x4C\xF9\x79\xD7\x52\xBA\x82\xF2\x27\x4A\x7D\xA0\xCA\x75",
-            None)

simplejson/tests/test_tuple.py

@@ -1,51 +0,0 @@
-import unittest
-
-from simplejson.compat import StringIO
-import simplejson as json
-
-class TestTuples(unittest.TestCase):
-    def test_tuple_array_dumps(self):
-        t = (1, 2, 3)
-        expect = json.dumps(list(t))
-        # Default is True
-        self.assertEqual(expect, json.dumps(t))
-        self.assertEqual(expect, json.dumps(t, tuple_as_array=True))
-        self.assertRaises(TypeError, json.dumps, t, tuple_as_array=False)
-        # Ensure that the "default" does not get called
-        self.assertEqual(expect, json.dumps(t, default=repr))
-        self.assertEqual(expect, json.dumps(t, tuple_as_array=True,
-                                            default=repr))
-        # Ensure that the "default" gets called
-        self.assertEqual(
-            json.dumps(repr(t)),
-            json.dumps(t, tuple_as_array=False, default=repr))
-
-    def test_tuple_array_dump(self):
-        t = (1, 2, 3)
-        expect = json.dumps(list(t))
-        # Default is True
-        sio = StringIO()
-        json.dump(t, sio)
-        self.assertEqual(expect, sio.getvalue())
-        sio = StringIO()
-        json.dump(t, sio, tuple_as_array=True)
-        self.assertEqual(expect, sio.getvalue())
-        self.assertRaises(TypeError, json.dump, t, StringIO(),
-                          tuple_as_array=False)
-        # Ensure that the "default" does not get called
-        sio = StringIO()
-        json.dump(t, sio, default=repr)
-        self.assertEqual(expect, sio.getvalue())
-        sio = StringIO()
-        json.dump(t, sio, tuple_as_array=True, default=repr)
-        self.assertEqual(expect, sio.getvalue())
-        # Ensure that the "default" gets called
-        sio = StringIO()
-        json.dump(t, sio, tuple_as_array=False, default=repr)
-        self.assertEqual(
-            json.dumps(repr(t)),
-            sio.getvalue())
-
-class TestNamedTuple(unittest.TestCase):
-    def test_namedtuple_dump(self):
-        pass

simplejson/tests/test_unicode.py

@@ -1,156 +0,0 @@
-import sys
-from unittest import TestCase
-
-import simplejson as json
-from simplejson.compat import unichr, text_type, b, u
-
-class TestUnicode(TestCase):
-    def test_encoding1(self):
-        encoder = json.JSONEncoder(encoding='utf-8')
-        u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
-        s = u.encode('utf-8')
-        ju = encoder.encode(u)
-        js = encoder.encode(s)
-        self.assertEqual(ju, js)
-
-    def test_encoding2(self):
-        u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
-        s = u.encode('utf-8')
-        ju = json.dumps(u, encoding='utf-8')
-        js = json.dumps(s, encoding='utf-8')
-        self.assertEqual(ju, js)
-
-    def test_encoding3(self):
-        u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
-        j = json.dumps(u)
-        self.assertEqual(j, '"\\u03b1\\u03a9"')
-
-    def test_encoding4(self):
-        u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
-        j = json.dumps([u])
-        self.assertEqual(j, '["\\u03b1\\u03a9"]')
-
-    def test_encoding5(self):
-        u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
-        j = json.dumps(u, ensure_ascii=False)
-        self.assertEqual(j, u'"' + u + u'"')
-
-    def test_encoding6(self):
-        u = u'\N{GREEK SMALL LETTER ALPHA}\N{GREEK CAPITAL LETTER OMEGA}'
-        j = json.dumps([u], ensure_ascii=False)
-        self.assertEqual(j, u'["' + u + u'"]')
-
-    def test_big_unicode_encode(self):
-        u = u'\U0001d120'
-        self.assertEqual(json.dumps(u), '"\\ud834\\udd20"')
-        self.assertEqual(json.dumps(u, ensure_ascii=False), u'"\U0001d120"')
-
-    def test_big_unicode_decode(self):
-        u = u'z\U0001d120x'
-        self.assertEqual(json.loads('"' + u + '"'), u)
-        self.assertEqual(json.loads('"z\\ud834\\udd20x"'), u)
-
-    def test_unicode_decode(self):
-        for i in range(0, 0xd7ff):
-            u = unichr(i)
-            #s = '"\\u{0:04x}"'.format(i)
-            s = '"\\u%04x"' % (i,)
-            self.assertEqual(json.loads(s), u)
-
-    def test_object_pairs_hook_with_unicode(self):
-        s = u'{"xkd":1, "kcw":2, "art":3, "hxm":4, "qrt":5, "pad":6, "hoy":7}'
-        p = [(u"xkd", 1), (u"kcw", 2), (u"art", 3), (u"hxm", 4),
-             (u"qrt", 5), (u"pad", 6), (u"hoy", 7)]
-        self.assertEqual(json.loads(s), eval(s))
-        self.assertEqual(json.loads(s, object_pairs_hook=lambda x: x), p)
-        od = json.loads(s, object_pairs_hook=json.OrderedDict)
-        self.assertEqual(od, json.OrderedDict(p))
-        self.assertEqual(type(od), json.OrderedDict)
-        # the object_pairs_hook takes priority over the object_hook
-        self.assertEqual(json.loads(s,
-                                    object_pairs_hook=json.OrderedDict,
-                                    object_hook=lambda x: None),
-                         json.OrderedDict(p))
-
-
-    def test_default_encoding(self):
-        self.assertEqual(json.loads(u'{"a": "\xe9"}'.encode('utf-8')),
-            {'a': u'\xe9'})
-
-    def test_unicode_preservation(self):
-        self.assertEqual(type(json.loads(u'""')), text_type)
-        self.assertEqual(type(json.loads(u'"a"')), text_type)
-        self.assertEqual(type(json.loads(u'["a"]')[0]), text_type)
-
-    def test_ensure_ascii_false_returns_unicode(self):
-        # http://code.google.com/p/simplejson/issues/detail?id=48
-        self.assertEqual(type(json.dumps([], ensure_ascii=False)), text_type)
-        self.assertEqual(type(json.dumps(0, ensure_ascii=False)), text_type)
-        self.assertEqual(type(json.dumps({}, ensure_ascii=False)), text_type)
-        self.assertEqual(type(json.dumps("", ensure_ascii=False)), text_type)
-
-    def test_ensure_ascii_false_bytestring_encoding(self):
-        # http://code.google.com/p/simplejson/issues/detail?id=48
-        doc1 = {u'quux': b('Arr\xc3\xaat sur images')}
-        doc2 = {u'quux': u('Arr\xeat sur images')}
-        doc_ascii = '{"quux": "Arr\\u00eat sur images"}'
-        doc_unicode = u'{"quux": "Arr\xeat sur images"}'
-        self.assertEqual(json.dumps(doc1), doc_ascii)
-        self.assertEqual(json.dumps(doc2), doc_ascii)
-        self.assertEqual(json.dumps(doc1, ensure_ascii=False), doc_unicode)
-        self.assertEqual(json.dumps(doc2, ensure_ascii=False), doc_unicode)
-
-    def test_ensure_ascii_linebreak_encoding(self):
-        # http://timelessrepo.com/json-isnt-a-javascript-subset
-        s1 = u'\u2029\u2028'
-        s2 = s1.encode('utf8')
-        expect = '"\\u2029\\u2028"'
-        self.assertEqual(json.dumps(s1), expect)
-        self.assertEqual(json.dumps(s2), expect)
-        self.assertEqual(json.dumps(s1, ensure_ascii=False), expect)
-        self.assertEqual(json.dumps(s2, ensure_ascii=False), expect)
-
-    def test_invalid_escape_sequences(self):
-        # incomplete escape sequence
-        self.assertRaises(json.JSONDecodeError, json.loads, '"\\u')
-        self.assertRaises(json.JSONDecodeError, json.loads, '"\\u1')
-        self.assertRaises(json.JSONDecodeError, json.loads, '"\\u12')
-        self.assertRaises(json.JSONDecodeError, json.loads, '"\\u123')
-        self.assertRaises(json.JSONDecodeError, json.loads, '"\\u1234')
-        # invalid escape sequence
-        self.assertRaises(json.JSONDecodeError, json.loads, '"\\u123x"')
-        self.assertRaises(json.JSONDecodeError, json.loads, '"\\u12x4"')
-        self.assertRaises(json.JSONDecodeError, json.loads, '"\\u1x34"')
-        self.assertRaises(json.JSONDecodeError, json.loads, '"\\ux234"')
-        if sys.maxunicode > 65535:
-            # unpaired low surrogate
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\udc00"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\udcff"')
-            # unpaired high surrogate
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800x"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800xx"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800xxxxxx"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u0"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u00"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u000"')
-            # invalid escape sequence for low surrogate
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u000x"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u00x0"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u0x00"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\ux000"')
-            # invalid value for low surrogate
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\u0000"')
-            self.assertRaises(json.JSONDecodeError, json.loads, '"\\ud800\\ufc00"')
-
-    def test_ensure_ascii_still_works(self):
-        # in the ascii range, ensure that everything is the same
-        for c in map(unichr, range(0, 127)):
-            self.assertEqual(
-                json.dumps(c, ensure_ascii=False),
-                json.dumps(c))
-        snowman = u'\N{SNOWMAN}'
-        self.assertEqual(
-            json.dumps(c, ensure_ascii=False),
-            '"' + c + '"')

simplejson/tool.py

@@ -1,42 +0,0 @@
-r"""Command-line tool to validate and pretty-print JSON
-
-Usage::
-
-    $ echo '{"json":"obj"}' | python -m simplejson.tool
-    {
-        "json": "obj"
-    }
-    $ echo '{ 1.2:3.4}' | python -m simplejson.tool
-    Expecting property name: line 1 column 2 (char 2)
-
-"""
-from __future__ import with_statement
-import sys
-import simplejson as json
-
-def main():
-    if len(sys.argv) == 1:
-        infile = sys.stdin
-        outfile = sys.stdout
-    elif len(sys.argv) == 2:
-        infile = open(sys.argv[1], 'r')
-        outfile = sys.stdout
-    elif len(sys.argv) == 3:
-        infile = open(sys.argv[1], 'r')
-        outfile = open(sys.argv[2], 'w')
-    else:
-        raise SystemExit(sys.argv[0] + " [infile [outfile]]")
-    with infile:
-        try:
-            obj = json.load(infile,
-                            object_pairs_hook=json.OrderedDict,
-                            use_decimal=True)
-        except ValueError:
-            raise SystemExit(sys.exc_info()[1])
-    with outfile:
-        json.dump(obj, outfile, sort_keys=True, indent='    ', use_decimal=True)
-        outfile.write('\n')
-
-
-if __name__ == '__main__':
-    main()

six.py

@@ -0,0 +1,838 @@
+"""Utilities for writing code that runs on Python 2 and 3"""
+
+# Copyright (c) 2010-2015 Benjamin Peterson
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from __future__ import absolute_import
+
+import functools
+import itertools
+import operator
+import sys
+import types
+
+__author__ = "Benjamin Peterson <benjamin@python.org>"
+__version__ = "1.9.0"
+
+
+# Useful for very coarse version differentiation.
+PY2 = sys.version_info[0] == 2
+PY3 = sys.version_info[0] == 3
+
+if PY3:
+    string_types = str,
+    integer_types = int,
+    class_types = type,
+    text_type = str
+    binary_type = bytes
+
+    MAXSIZE = sys.maxsize
+else:
+    string_types = basestring,
+    integer_types = (int, long)
+    class_types = (type, types.ClassType)
+    text_type = unicode
+    binary_type = str
+
+    if sys.platform.startswith("java"):
+        # Jython always uses 32 bits.
+        MAXSIZE = int((1 << 31) - 1)
+    else:
+        # It's possible to have sizeof(long) != sizeof(Py_ssize_t).
+        class X(object):
+            def __len__(self):
+                return 1 << 31
+        try:
+            len(X())
+        except OverflowError:
+            # 32-bit
+            MAXSIZE = int((1 << 31) - 1)
+        else:
+            # 64-bit
+            MAXSIZE = int((1 << 63) - 1)
+        del X
+
+
+def _add_doc(func, doc):
+    """Add documentation to a function."""
+    func.__doc__ = doc
+
+
+def _import_module(name):
+    """Import module, returning the module after the last dot."""
+    __import__(name)
+    return sys.modules[name]
+
+
+class _LazyDescr(object):
+
+    def __init__(self, name):
+        self.name = name
+
+    def __get__(self, obj, tp):
+        result = self._resolve()
+        setattr(obj, self.name, result) # Invokes __set__.
+        try:
+            # This is a bit ugly, but it avoids running this again by
+            # removing this descriptor.
+            delattr(obj.__class__, self.name)
+        except AttributeError:
+            pass
+        return result
+
+
+class MovedModule(_LazyDescr):
+
+    def __init__(self, name, old, new=None):
+        super(MovedModule, self).__init__(name)
+        if PY3:
+            if new is None:
+                new = name
+            self.mod = new
+        else:
+            self.mod = old
+
+    def _resolve(self):
+        return _import_module(self.mod)
+
+    def __getattr__(self, attr):
+        _module = self._resolve()
+        value = getattr(_module, attr)
+        setattr(self, attr, value)
+        return value
+
+
+class _LazyModule(types.ModuleType):
+
+    def __init__(self, name):
+        super(_LazyModule, self).__init__(name)
+        self.__doc__ = self.__class__.__doc__
+
+    def __dir__(self):
+        attrs = ["__doc__", "__name__"]
+        attrs += [attr.name for attr in self._moved_attributes]
+        return attrs
+
+    # Subclasses should override this
+    _moved_attributes = []
+
+
+class MovedAttribute(_LazyDescr):
+
+    def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None):
+        super(MovedAttribute, self).__init__(name)
+        if PY3:
+            if new_mod is None:
+                new_mod = name
+            self.mod = new_mod
+            if new_attr is None:
+                if old_attr is None:
+                    new_attr = name
+                else:
+                    new_attr = old_attr
+            self.attr = new_attr
+        else:
+            self.mod = old_mod
+            if old_attr is None:
+                old_attr = name
+            self.attr = old_attr
+
+    def _resolve(self):
+        module = _import_module(self.mod)
+        return getattr(module, self.attr)
+
+
+class _SixMetaPathImporter(object):
+    """
+    A meta path importer to import six.moves and its submodules.
+
+    This class implements a PEP302 finder and loader. It should be compatible
+    with Python 2.5 and all existing versions of Python3
+    """
+    def __init__(self, six_module_name):
+        self.name = six_module_name
+        self.known_modules = {}
+
+    def _add_module(self, mod, *fullnames):
+        for fullname in fullnames:
+            self.known_modules[self.name + "." + fullname] = mod
+
+    def _get_module(self, fullname):
+        return self.known_modules[self.name + "." + fullname]
+
+    def find_module(self, fullname, path=None):
+        if fullname in self.known_modules:
+            return self
+        return None
+
+    def __get_module(self, fullname):
+        try:
+            return self.known_modules[fullname]
+        except KeyError:
+            raise ImportError("This loader does not know module " + fullname)
+
+    def load_module(self, fullname):
+        try:
+            # in case of a reload
+            return sys.modules[fullname]
+        except KeyError:
+            pass
+        mod = self.__get_module(fullname)
+        if isinstance(mod, MovedModule):
+            mod = mod._resolve()
+        else:
+            mod.__loader__ = self
+        sys.modules[fullname] = mod
+        return mod
+
+    def is_package(self, fullname):
+        """
+        Return true, if the named module is a package.
+
+        We need this method to get correct spec objects with
+        Python 3.4 (see PEP451)
+        """
+        return hasattr(self.__get_module(fullname), "__path__")
+
+    def get_code(self, fullname):
+        """Return None
+
+        Required, if is_package is implemented"""
+        self.__get_module(fullname)  # eventually raises ImportError
+        return None
+    get_source = get_code  # same as get_code
+
+_importer = _SixMetaPathImporter(__name__)
+
+
+class _MovedItems(_LazyModule):
+    """Lazy loading of moved objects"""
+    __path__ = []  # mark as package
+
+
+_moved_attributes = [
+    MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"),
+    MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"),
+    MovedAttribute("filterfalse", "itertools", "itertools", "ifilterfalse", "filterfalse"),
+    MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"),
+    MovedAttribute("intern", "__builtin__", "sys"),
+    MovedAttribute("map", "itertools", "builtins", "imap", "map"),
+    MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"),
+    MovedAttribute("reload_module", "__builtin__", "imp", "reload"),
+    MovedAttribute("reduce", "__builtin__", "functools"),
+    MovedAttribute("shlex_quote", "pipes", "shlex", "quote"),
+    MovedAttribute("StringIO", "StringIO", "io"),
+    MovedAttribute("UserDict", "UserDict", "collections"),
+    MovedAttribute("UserList", "UserList", "collections"),
+    MovedAttribute("UserString", "UserString", "collections"),
+    MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"),
+    MovedAttribute("zip", "itertools", "builtins", "izip", "zip"),
+    MovedAttribute("zip_longest", "itertools", "itertools", "izip_longest", "zip_longest"),
+
+    MovedModule("builtins", "__builtin__"),
+    MovedModule("configparser", "ConfigParser"),
+    MovedModule("copyreg", "copy_reg"),
+    MovedModule("dbm_gnu", "gdbm", "dbm.gnu"),
+    MovedModule("_dummy_thread", "dummy_thread", "_dummy_thread"),
+    MovedModule("http_cookiejar", "cookielib", "http.cookiejar"),
+    MovedModule("http_cookies", "Cookie", "http.cookies"),
+    MovedModule("html_entities", "htmlentitydefs", "html.entities"),
+    MovedModule("html_parser", "HTMLParser", "html.parser"),
+    MovedModule("http_client", "httplib", "http.client"),
+    MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"),
+    MovedModule("email_mime_nonmultipart", "email.MIMENonMultipart", "email.mime.nonmultipart"),
+    MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"),
+    MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"),
+    MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"),
+    MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"),
+    MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"),
+    MovedModule("cPickle", "cPickle", "pickle"),
+    MovedModule("queue", "Queue"),
+    MovedModule("reprlib", "repr"),
+    MovedModule("socketserver", "SocketServer"),
+    MovedModule("_thread", "thread", "_thread"),
+    MovedModule("tkinter", "Tkinter"),
+    MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"),
+    MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"),
+    MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"),
+    MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"),
+    MovedModule("tkinter_tix", "Tix", "tkinter.tix"),
+    MovedModule("tkinter_ttk", "ttk", "tkinter.ttk"),
+    MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"),
+    MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"),
+    MovedModule("tkinter_colorchooser", "tkColorChooser",
+                "tkinter.colorchooser"),
+    MovedModule("tkinter_commondialog", "tkCommonDialog",
+                "tkinter.commondialog"),
+    MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"),
+    MovedModule("tkinter_font", "tkFont", "tkinter.font"),
+    MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"),
+    MovedModule("tkinter_tksimpledialog", "tkSimpleDialog",
+                "tkinter.simpledialog"),
+    MovedModule("urllib_parse", __name__ + ".moves.urllib_parse", "urllib.parse"),
+    MovedModule("urllib_error", __name__ + ".moves.urllib_error", "urllib.error"),
+    MovedModule("urllib", __name__ + ".moves.urllib", __name__ + ".moves.urllib"),
+    MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
+    MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"),
+    MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"),
+    MovedModule("winreg", "_winreg"),
+]
+for attr in _moved_attributes:
+    setattr(_MovedItems, attr.name, attr)
+    if isinstance(attr, MovedModule):
+        _importer._add_module(attr, "moves." + attr.name)
+del attr
+
+_MovedItems._moved_attributes = _moved_attributes
+
+moves = _MovedItems(__name__ + ".moves")
+_importer._add_module(moves, "moves")
+
+
+class Module_six_moves_urllib_parse(_LazyModule):
+    """Lazy loading of moved objects in six.moves.urllib_parse"""
+
+
+_urllib_parse_moved_attributes = [
+    MovedAttribute("ParseResult", "urlparse", "urllib.parse"),
+    MovedAttribute("SplitResult", "urlparse", "urllib.parse"),
+    MovedAttribute("parse_qs", "urlparse", "urllib.parse"),
+    MovedAttribute("parse_qsl", "urlparse", "urllib.parse"),
+    MovedAttribute("urldefrag", "urlparse", "urllib.parse"),
+    MovedAttribute("urljoin", "urlparse", "urllib.parse"),
+    MovedAttribute("urlparse", "urlparse", "urllib.parse"),
+    MovedAttribute("urlsplit", "urlparse", "urllib.parse"),
+    MovedAttribute("urlunparse", "urlparse", "urllib.parse"),
+    MovedAttribute("urlunsplit", "urlparse", "urllib.parse"),
+    MovedAttribute("quote", "urllib", "urllib.parse"),
+    MovedAttribute("quote_plus", "urllib", "urllib.parse"),
+    MovedAttribute("unquote", "urllib", "urllib.parse"),
+    MovedAttribute("unquote_plus", "urllib", "urllib.parse"),
+    MovedAttribute("urlencode", "urllib", "urllib.parse"),
+    MovedAttribute("splitquery", "urllib", "urllib.parse"),
+    MovedAttribute("splittag", "urllib", "urllib.parse"),
+    MovedAttribute("splituser", "urllib", "urllib.parse"),
+    MovedAttribute("uses_fragment", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_netloc", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_params", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_query", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_relative", "urlparse", "urllib.parse"),
+]
+for attr in _urllib_parse_moved_attributes:
+    setattr(Module_six_moves_urllib_parse, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_parse._moved_attributes = _urllib_parse_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_parse(__name__ + ".moves.urllib_parse"),
+                      "moves.urllib_parse", "moves.urllib.parse")
+
+
+class Module_six_moves_urllib_error(_LazyModule):
+    """Lazy loading of moved objects in six.moves.urllib_error"""
+
+
+_urllib_error_moved_attributes = [
+    MovedAttribute("URLError", "urllib2", "urllib.error"),
+    MovedAttribute("HTTPError", "urllib2", "urllib.error"),
+    MovedAttribute("ContentTooShortError", "urllib", "urllib.error"),
+]
+for attr in _urllib_error_moved_attributes:
+    setattr(Module_six_moves_urllib_error, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_error._moved_attributes = _urllib_error_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_error(__name__ + ".moves.urllib.error"),
+                      "moves.urllib_error", "moves.urllib.error")
+
+
+class Module_six_moves_urllib_request(_LazyModule):
+    """Lazy loading of moved objects in six.moves.urllib_request"""
+
+
+_urllib_request_moved_attributes = [
+    MovedAttribute("urlopen", "urllib2", "urllib.request"),
+    MovedAttribute("install_opener", "urllib2", "urllib.request"),
+    MovedAttribute("build_opener", "urllib2", "urllib.request"),
+    MovedAttribute("pathname2url", "urllib", "urllib.request"),
+    MovedAttribute("url2pathname", "urllib", "urllib.request"),
+    MovedAttribute("getproxies", "urllib", "urllib.request"),
+    MovedAttribute("Request", "urllib2", "urllib.request"),
+    MovedAttribute("OpenerDirector", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPDefaultErrorHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPRedirectHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPCookieProcessor", "urllib2", "urllib.request"),
+    MovedAttribute("ProxyHandler", "urllib2", "urllib.request"),
+    MovedAttribute("BaseHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPPasswordMgr", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPPasswordMgrWithDefaultRealm", "urllib2", "urllib.request"),
+    MovedAttribute("AbstractBasicAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPBasicAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("ProxyBasicAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("AbstractDigestAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPDigestAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("ProxyDigestAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPSHandler", "urllib2", "urllib.request"),
+    MovedAttribute("FileHandler", "urllib2", "urllib.request"),
+    MovedAttribute("FTPHandler", "urllib2", "urllib.request"),
+    MovedAttribute("CacheFTPHandler", "urllib2", "urllib.request"),
+    MovedAttribute("UnknownHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPErrorProcessor", "urllib2", "urllib.request"),
+    MovedAttribute("urlretrieve", "urllib", "urllib.request"),
+    MovedAttribute("urlcleanup", "urllib", "urllib.request"),
+    MovedAttribute("URLopener", "urllib", "urllib.request"),
+    MovedAttribute("FancyURLopener", "urllib", "urllib.request"),
+    MovedAttribute("proxy_bypass", "urllib", "urllib.request"),
+]
+for attr in _urllib_request_moved_attributes:
+    setattr(Module_six_moves_urllib_request, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_request._moved_attributes = _urllib_request_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_request(__name__ + ".moves.urllib.request"),
+                      "moves.urllib_request", "moves.urllib.request")
+
+
+class Module_six_moves_urllib_response(_LazyModule):
+    """Lazy loading of moved objects in six.moves.urllib_response"""
+
+
+_urllib_response_moved_attributes = [
+    MovedAttribute("addbase", "urllib", "urllib.response"),
+    MovedAttribute("addclosehook", "urllib", "urllib.response"),
+    MovedAttribute("addinfo", "urllib", "urllib.response"),
+    MovedAttribute("addinfourl", "urllib", "urllib.response"),
+]
+for attr in _urllib_response_moved_attributes:
+    setattr(Module_six_moves_urllib_response, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_response._moved_attributes = _urllib_response_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_response(__name__ + ".moves.urllib.response"),
+                      "moves.urllib_response", "moves.urllib.response")
+
+
+class Module_six_moves_urllib_robotparser(_LazyModule):
+    """Lazy loading of moved objects in six.moves.urllib_robotparser"""
+
+
+_urllib_robotparser_moved_attributes = [
+    MovedAttribute("RobotFileParser", "robotparser", "urllib.robotparser"),
+]
+for attr in _urllib_robotparser_moved_attributes:
+    setattr(Module_six_moves_urllib_robotparser, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_robotparser._moved_attributes = _urllib_robotparser_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_robotparser(__name__ + ".moves.urllib.robotparser"),
+                      "moves.urllib_robotparser", "moves.urllib.robotparser")
+
+
+class Module_six_moves_urllib(types.ModuleType):
+    """Create a six.moves.urllib namespace that resembles the Python 3 namespace"""
+    __path__ = []  # mark as package
+    parse = _importer._get_module("moves.urllib_parse")
+    error = _importer._get_module("moves.urllib_error")
+    request = _importer._get_module("moves.urllib_request")
+    response = _importer._get_module("moves.urllib_response")
+    robotparser = _importer._get_module("moves.urllib_robotparser")
+
+    def __dir__(self):
+        return ['parse', 'error', 'request', 'response', 'robotparser']
+
+_importer._add_module(Module_six_moves_urllib(__name__ + ".moves.urllib"),
+                      "moves.urllib")
+
+
+def add_move(move):
+    """Add an item to six.moves."""
+    setattr(_MovedItems, move.name, move)
+
+
+def remove_move(name):
+    """Remove item from six.moves."""
+    try:
+        delattr(_MovedItems, name)
+    except AttributeError:
+        try:
+            del moves.__dict__[name]
+        except KeyError:
+            raise AttributeError("no such move, %r" % (name,))
+
+
+if PY3:
+    _meth_func = "__func__"
+    _meth_self = "__self__"
+
+    _func_closure = "__closure__"
+    _func_code = "__code__"
+    _func_defaults = "__defaults__"
+    _func_globals = "__globals__"
+else:
+    _meth_func = "im_func"
+    _meth_self = "im_self"
+
+    _func_closure = "func_closure"
+    _func_code = "func_code"
+    _func_defaults = "func_defaults"
+    _func_globals = "func_globals"
+
+
+try:
+    advance_iterator = next
+except NameError:
+    def advance_iterator(it):
+        return it.next()
+next = advance_iterator
+
+
+try:
+    callable = callable
+except NameError:
+    def callable(obj):
+        return any("__call__" in klass.__dict__ for klass in type(obj).__mro__)
+
+
+if PY3:
+    def get_unbound_function(unbound):
+        return unbound
+
+    create_bound_method = types.MethodType
+
+    Iterator = object
+else:
+    def get_unbound_function(unbound):
+        return unbound.im_func
+
+    def create_bound_method(func, obj):
+        return types.MethodType(func, obj, obj.__class__)
+
+    class Iterator(object):
+
+        def next(self):
+            return type(self).__next__(self)
+
+    callable = callable
+_add_doc(get_unbound_function,
+         """Get the function out of a possibly unbound function""")
+
+
+get_method_function = operator.attrgetter(_meth_func)
+get_method_self = operator.attrgetter(_meth_self)
+get_function_closure = operator.attrgetter(_func_closure)
+get_function_code = operator.attrgetter(_func_code)
+get_function_defaults = operator.attrgetter(_func_defaults)
+get_function_globals = operator.attrgetter(_func_globals)
+
+
+if PY3:
+    def iterkeys(d, **kw):
+        return iter(d.keys(**kw))
+
+    def itervalues(d, **kw):
+        return iter(d.values(**kw))
+
+    def iteritems(d, **kw):
+        return iter(d.items(**kw))
+
+    def iterlists(d, **kw):
+        return iter(d.lists(**kw))
+
+    viewkeys = operator.methodcaller("keys")
+
+    viewvalues = operator.methodcaller("values")
+
+    viewitems = operator.methodcaller("items")
+else:
+    def iterkeys(d, **kw):
+        return iter(d.iterkeys(**kw))
+
+    def itervalues(d, **kw):
+        return iter(d.itervalues(**kw))
+
+    def iteritems(d, **kw):
+        return iter(d.iteritems(**kw))
+
+    def iterlists(d, **kw):
+        return iter(d.iterlists(**kw))
+
+    viewkeys = operator.methodcaller("viewkeys")
+
+    viewvalues = operator.methodcaller("viewvalues")
+
+    viewitems = operator.methodcaller("viewitems")
+
+_add_doc(iterkeys, "Return an iterator over the keys of a dictionary.")
+_add_doc(itervalues, "Return an iterator over the values of a dictionary.")
+_add_doc(iteritems,
+         "Return an iterator over the (key, value) pairs of a dictionary.")
+_add_doc(iterlists,
+         "Return an iterator over the (key, [values]) pairs of a dictionary.")
+
+
+if PY3:
+    def b(s):
+        return s.encode("latin-1")
+    def u(s):
+        return s
+    unichr = chr
+    if sys.version_info[1] <= 1:
+        def int2byte(i):
+            return bytes((i,))
+    else:
+        # This is about 2x faster than the implementation above on 3.2+
+        int2byte = operator.methodcaller("to_bytes", 1, "big")
+    byte2int = operator.itemgetter(0)
+    indexbytes = operator.getitem
+    iterbytes = iter
+    import io
+    StringIO = io.StringIO
+    BytesIO = io.BytesIO
+    _assertCountEqual = "assertCountEqual"
+    _assertRaisesRegex = "assertRaisesRegex"
+    _assertRegex = "assertRegex"
+else:
+    def b(s):
+        return s
+    # Workaround for standalone backslash
+    def u(s):
+        return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape")
+    unichr = unichr
+    int2byte = chr
+    def byte2int(bs):
+        return ord(bs[0])
+    def indexbytes(buf, i):
+        return ord(buf[i])
+    iterbytes = functools.partial(itertools.imap, ord)
+    import StringIO
+    StringIO = BytesIO = StringIO.StringIO
+    _assertCountEqual = "assertItemsEqual"
+    _assertRaisesRegex = "assertRaisesRegexp"
+    _assertRegex = "assertRegexpMatches"
+_add_doc(b, """Byte literal""")
+_add_doc(u, """Text literal""")
+
+
+def assertCountEqual(self, *args, **kwargs):
+    return getattr(self, _assertCountEqual)(*args, **kwargs)
+
+
+def assertRaisesRegex(self, *args, **kwargs):
+    return getattr(self, _assertRaisesRegex)(*args, **kwargs)
+
+
+def assertRegex(self, *args, **kwargs):
+    return getattr(self, _assertRegex)(*args, **kwargs)
+
+
+if PY3:
+    exec_ = getattr(moves.builtins, "exec")
+
+
+    def reraise(tp, value, tb=None):
+        if value is None:
+            value = tp()
+        if value.__traceback__ is not tb:
+            raise value.with_traceback(tb)
+        raise value
+
+else:
+    def exec_(_code_, _globs_=None, _locs_=None):
+        """Execute code in a namespace."""
+        if _globs_ is None:
+            frame = sys._getframe(1)
+            _globs_ = frame.f_globals
+            if _locs_ is None:
+                _locs_ = frame.f_locals
+            del frame
+        elif _locs_ is None:
+            _locs_ = _globs_
+        exec("""exec _code_ in _globs_, _locs_""")
+
+
+    exec_("""def reraise(tp, value, tb=None):
+    raise tp, value, tb
+""")
+
+
+if sys.version_info[:2] == (3, 2):
+    exec_("""def raise_from(value, from_value):
+    if from_value is None:
+        raise value
+    raise value from from_value
+""")
+elif sys.version_info[:2] > (3, 2):
+    exec_("""def raise_from(value, from_value):
+    raise value from from_value
+""")
+else:
+    def raise_from(value, from_value):
+        raise value
+
+
+print_ = getattr(moves.builtins, "print", None)
+if print_ is None:
+    def print_(*args, **kwargs):
+        """The new-style print function for Python 2.4 and 2.5."""
+        fp = kwargs.pop("file", sys.stdout)
+        if fp is None:
+            return
+        def write(data):
+            if not isinstance(data, basestring):
+                data = str(data)
+            # If the file has an encoding, encode unicode with it.
+            if (isinstance(fp, file) and
+                isinstance(data, unicode) and
+                fp.encoding is not None):
+                errors = getattr(fp, "errors", None)
+                if errors is None:
+                    errors = "strict"
+                data = data.encode(fp.encoding, errors)
+            fp.write(data)
+        want_unicode = False
+        sep = kwargs.pop("sep", None)
+        if sep is not None:
+            if isinstance(sep, unicode):
+                want_unicode = True
+            elif not isinstance(sep, str):
+                raise TypeError("sep must be None or a string")
+        end = kwargs.pop("end", None)
+        if end is not None:
+            if isinstance(end, unicode):
+                want_unicode = True
+            elif not isinstance(end, str):
+                raise TypeError("end must be None or a string")
+        if kwargs:
+            raise TypeError("invalid keyword arguments to print()")
+        if not want_unicode:
+            for arg in args:
+                if isinstance(arg, unicode):
+                    want_unicode = True
+                    break
+        if want_unicode:
+            newline = unicode("\n")
+            space = unicode(" ")
+        else:
+            newline = "\n"
+            space = " "
+        if sep is None:
+            sep = space
+        if end is None:
+            end = newline
+        for i, arg in enumerate(args):
+            if i:
+                write(sep)
+            write(arg)
+        write(end)
+if sys.version_info[:2] < (3, 3):
+    _print = print_
+    def print_(*args, **kwargs):
+        fp = kwargs.get("file", sys.stdout)
+        flush = kwargs.pop("flush", False)
+        _print(*args, **kwargs)
+        if flush and fp is not None:
+            fp.flush()
+
+_add_doc(reraise, """Reraise an exception.""")
+
+if sys.version_info[0:2] < (3, 4):
+    def wraps(wrapped, assigned=functools.WRAPPER_ASSIGNMENTS,
+              updated=functools.WRAPPER_UPDATES):
+        def wrapper(f):
+            f = functools.wraps(wrapped, assigned, updated)(f)
+            f.__wrapped__ = wrapped
+            return f
+        return wrapper
+else:
+    wraps = functools.wraps
+
+def with_metaclass(meta, *bases):
+    """Create a base class with a metaclass."""
+    # This requires a bit of explanation: the basic idea is to make a dummy
+    # metaclass for one level of class instantiation that replaces itself with
+    # the actual metaclass.
+    class metaclass(meta):
+        def __new__(cls, name, this_bases, d):
+            return meta(name, bases, d)
+    return type.__new__(metaclass, 'temporary_class', (), {})
+
+
+def add_metaclass(metaclass):
+    """Class decorator for creating a class with a metaclass."""
+    def wrapper(cls):
+        orig_vars = cls.__dict__.copy()
+        slots = orig_vars.get('__slots__')
+        if slots is not None:
+            if isinstance(slots, str):
+                slots = [slots]
+            for slots_var in slots:
+                orig_vars.pop(slots_var)
+        orig_vars.pop('__dict__', None)
+        orig_vars.pop('__weakref__', None)
+        return metaclass(cls.__name__, cls.__bases__, orig_vars)
+    return wrapper
+
+
+def python_2_unicode_compatible(klass):
+    """
+    A decorator that defines __unicode__ and __str__ methods under Python 2.
+    Under Python 3 it does nothing.
+
+    To support Python 2 and 3 with a single code base, define a __str__ method
+    returning text and apply this decorator to the class.
+    """
+    if PY2:
+        if '__str__' not in klass.__dict__:
+            raise ValueError("@python_2_unicode_compatible cannot be applied "
+                             "to %s because it doesn't define __str__()." %
+                             klass.__name__)
+        klass.__unicode__ = klass.__str__
+        klass.__str__ = lambda self: self.__unicode__().encode('utf-8')
+    return klass
+
+
+# Complete the moves implementation.
+# This code is at the end of this module to speed up module loading.
+# Turn this module into a package.
+__path__ = []  # required for PEP 302 and PEP 451
+__package__ = __name__  # see PEP 366 @ReservedAssignment
+if globals().get("__spec__") is not None:
+    __spec__.submodule_search_locations = []  # PEP 451 @UndefinedVariable
+# Remove other six meta path importers, since they cause problems. This can
+# happen if six is removed from sys.modules and then reloaded. (Setuptools does
+# this for some reason.)
+if sys.meta_path:
+    for i, importer in enumerate(sys.meta_path):
+        # Here's some real nastiness: Another "instance" of the six module might
+        # be floating around. Therefore, we can't use isinstance() to check for
+        # the six meta path importer, since the other six instance will have
+        # inserted an importer with different class.
+        if (type(importer).__name__ == "_SixMetaPathImporter" and
+            importer.name == __name__):
+            del sys.meta_path[i]
+            break
+    del i, importer
+# Finally, add the importer to the meta path import hook.
+sys.meta_path.append(_importer)

uritemplate/__init__.py

@@ -1,147 +1,265 @@
-# Early, and incomplete implementation of -04.
-#
+#!/usr/bin/env python
+
+"""
+URI Template (RFC6570) Processor
+"""
+
+__copyright__ = """\
+Copyright 2011-2013 Joe Gregorio
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
 import re
-import urllib
+try:
+   from urllib.parse import quote
+except ImportError:
+   from urllib import quote
+
+
+
+__version__ = "0.6"
 
 RESERVED = ":/?#[]@!$&'()*+,;="
-OPERATOR = "+./;?|!@"
-EXPLODE = "*+"
+OPERATOR = "+#./;?&|!@"
 MODIFIER = ":^"
-TEMPLATE = re.compile(r"{(?P<operator>[\+\./;\?|!@])?(?P<varlist>[^}]+)}", re.UNICODE)
-VAR = re.compile(r"^(?P<varname>[^=\+\*:\^]+)((?P<explode>[\+\*])|(?P<partial>[:\^]-?[0-9]+))?(=(?P<default>.*))?$", re.UNICODE)
-
-def _tostring(varname, value, explode, operator, safe=""):
-  if type(value) == type([]):
-    if explode == "+":
-      return ",".join([varname + "." + urllib.quote(x, safe) for x in value])
-    else:
-      return ",".join([urllib.quote(x, safe) for x in value])
-  if type(value) == type({}):
-    keys = value.keys()
-    keys.sort()
-    if explode == "+":
-      return ",".join([varname + "." + urllib.quote(key, safe) + "," + urllib.quote(value[key], safe) for key in keys])
-    else:
-      return ",".join([urllib.quote(key, safe) + "," + urllib.quote(value[key], safe) for key in keys])
-  else:
-    return urllib.quote(value, safe)
+TEMPLATE = re.compile("{([^\}]+)}")
 
 
-def _tostring_path(varname, value, explode, operator, safe=""):
-  joiner = operator
-  if type(value) == type([]):
-    if explode == "+":
-      return joiner.join([varname + "." + urllib.quote(x, safe) for x in value])
-    elif explode == "*":
-      return joiner.join([urllib.quote(x, safe) for x in value])
-    else:
-      return ",".join([urllib.quote(x, safe) for x in value])
-  elif type(value) == type({}):
-    keys = value.keys()
-    keys.sort()
-    if explode == "+":
-      return joiner.join([varname + "." + urllib.quote(key, safe) + joiner + urllib.quote(value[key], safe) for key in keys])
-    elif explode == "*":
-      return joiner.join([urllib.quote(key, safe) + joiner + urllib.quote(value[key], safe) for key in keys])
-    else:
-      return ",".join([urllib.quote(key, safe) + "," + urllib.quote(value[key], safe) for key in keys])
-  else:
-    if value:
-      return urllib.quote(value, safe)
-    else:
-      return ""
+def variables(template):
+    '''Returns the set of keywords in a uri template'''
+    vars = set()
+    for varlist in TEMPLATE.findall(template):
+        if varlist[0] in OPERATOR:
+            varlist = varlist[1:]
+        varspecs = varlist.split(',')
+        for var in varspecs:
+            # handle prefix values
+            var = var.split(':')[0]
+            # handle composite values
+            if var.endswith('*'):
+                var = var[:-1]
+            vars.add(var)
+    return vars
 
-def _tostring_query(varname, value, explode, operator, safe=""):
-  joiner = operator
-  varprefix = ""
-  if operator == "?":
-    joiner = "&"
-    varprefix = varname + "="
-  if type(value) == type([]):
-    if 0 == len(value):
-      return ""
-    if explode == "+":
-      return joiner.join([varname + "=" + urllib.quote(x, safe) for x in value])
-    elif explode == "*":
-      return joiner.join([urllib.quote(x, safe) for x in value])
+
+def _quote(value, safe, prefix=None):
+    if prefix is not None:
+        return quote(str(value)[:prefix], safe)
+    return quote(str(value), safe)
+
+
+def _tostring(varname, value, explode, prefix, operator, safe=""):
+    if isinstance(value, list):
+        return ",".join([_quote(x, safe) for x in value])
+    if isinstance(value, dict):
+        keys = sorted(value.keys())
+        if explode:
+            return ",".join([_quote(key, safe) + "=" + \
+                             _quote(value[key], safe) for key in keys])
+        else:
+            return ",".join([_quote(key, safe) + "," + \
+                             _quote(value[key], safe) for key in keys])
+    elif value is None:
+        return
     else:
-      return varprefix + ",".join([urllib.quote(x, safe) for x in value])
-  elif type(value) == type({}):
-    if 0 == len(value):
-      return ""
-    keys = value.keys()
-    keys.sort()
-    if explode == "+":
-      return joiner.join([varname + "." + urllib.quote(key, safe) + "=" + urllib.quote(value[key], safe) for key in keys])
-    elif explode == "*":
-      return joiner.join([urllib.quote(key, safe) + "=" + urllib.quote(value[key], safe) for key in keys])
+        return _quote(value, safe, prefix)
+
+
+def _tostring_path(varname, value, explode, prefix, operator, safe=""):
+    joiner = operator
+    if isinstance(value, list):
+        if explode:
+            out = [_quote(x, safe) for x in value if value is not None]
+        else:
+            joiner = ","
+            out = [_quote(x, safe) for x in value if value is not None]
+        if out:
+            return joiner.join(out)
+        else:
+            return
+    elif isinstance(value, dict):
+        keys = sorted(value.keys())
+        if explode:
+            out = [_quote(key, safe) + "=" + \
+                   _quote(value[key], safe) for key in keys \
+                   if value[key] is not None]
+        else:
+            joiner = ","
+            out = [_quote(key, safe) + "," + \
+                   _quote(value[key], safe) \
+                   for key in keys if value[key] is not None]
+        if out:
+            return joiner.join(out)
+        else:
+            return
+    elif value is None:
+        return
     else:
-      return varprefix + ",".join([urllib.quote(key, safe) + "," + urllib.quote(value[key], safe) for key in keys])
-  else:
-    if value:
-      return varname + "=" + urllib.quote(value, safe)
+        return _quote(value, safe, prefix)
+
+
+def _tostring_semi(varname, value, explode, prefix, operator, safe=""):
+    joiner = operator
+    if operator == "?":
+        joiner = "&"
+    if isinstance(value, list):
+        if explode:
+            out = [varname + "=" + _quote(x, safe) \
+                   for x in value if x is not None]
+            if out:
+                return joiner.join(out)
+            else:
+                return
+        else:
+            return varname + "=" + ",".join([_quote(x, safe) \
+                                             for x in value])
+    elif isinstance(value, dict):
+        keys = sorted(value.keys())
+        if explode:
+            return joiner.join([_quote(key, safe) + "=" + \
+                                _quote(value[key], safe) \
+                                for key in keys if key is not None])
+        else:
+            return varname + "=" + ",".join([_quote(key, safe) + "," + \
+                             _quote(value[key], safe) for key in keys \
+                             if key is not None])
     else:
-      return varname 
+        if value is None:
+            return
+        elif value:
+            return (varname + "=" + _quote(value, safe, prefix))
+        else:
+            return varname
+
+
+def _tostring_query(varname, value, explode, prefix, operator, safe=""):
+    joiner = operator
+    if operator in ["?", "&"]:
+        joiner = "&"
+    if isinstance(value, list):
+        if 0 == len(value):
+            return None
+        if explode:
+            return joiner.join([varname + "=" + _quote(x, safe) \
+                                for x in value])
+        else:
+            return (varname + "=" + ",".join([_quote(x, safe) \
+                                             for x in value]))
+    elif isinstance(value, dict):
+        if 0 == len(value):
+            return None
+        keys = sorted(value.keys())
+        if explode:
+            return joiner.join([_quote(key, safe) + "=" + \
+                                _quote(value[key], safe) \
+                                for key in keys])
+        else:
+            return varname + "=" + \
+                   ",".join([_quote(key, safe) + "," + \
+                             _quote(value[key], safe) for key in keys])
+    else:
+        if value is None:
+            return
+        elif value:
+            return (varname + "=" + _quote(value, safe, prefix))
+        else:
+            return (varname  + "=")
+
 
 TOSTRING = {
     "" : _tostring,
     "+": _tostring,
-    ";": _tostring_query,
+    "#": _tostring,
+    ";": _tostring_semi,
     "?": _tostring_query,
+    "&": _tostring_query,
     "/": _tostring_path,
     ".": _tostring_path,
     }
 
 
-def expand(template, vars):
-  def _sub(match):
-    groupdict = match.groupdict()
-    operator = groupdict.get('operator')
-    if operator is None:
-      operator = ''
-    varlist = groupdict.get('varlist')
+def expand(template, variables):
+    """
+    Expand template as a URI Template using variables.
+    """
+    def _sub(match):
+        expression = match.group(1)
+        operator = ""
+        if expression[0] in OPERATOR:
+            operator = expression[0]
+            varlist = expression[1:]
+        else:
+            varlist = expression
 
-    safe = "@"
-    if operator == '+':
-      safe = RESERVED
-    varspecs = varlist.split(",")
-    varnames = []
-    defaults = {}
-    for varspec in varspecs:
-      m = VAR.search(varspec)
-      groupdict = m.groupdict()
-      varname = groupdict.get('varname')
-      explode = groupdict.get('explode')
-      partial = groupdict.get('partial')
-      default = groupdict.get('default')
-      if default:
-        defaults[varname] = default
-      varnames.append((varname, explode, partial))
+        safe = ""
+        if operator in ["+", "#"]:
+            safe = RESERVED
+        varspecs = varlist.split(",")
+        varnames = []
+        defaults = {}
+        for varspec in varspecs:
+            default = None
+            explode = False
+            prefix = None
+            if "=" in varspec:
+                varname, default = tuple(varspec.split("=", 1))
+            else:
+                varname = varspec
+            if varname[-1] == "*":
+                explode = True
+                varname = varname[:-1]
+            elif ":" in varname:
+                try:
+                    prefix = int(varname[varname.index(":")+1:])
+                except ValueError:
+                    raise ValueError("non-integer prefix '{0}'".format(
+                       varname[varname.index(":")+1:]))
+                varname = varname[:varname.index(":")]
+            if default:
+                defaults[varname] = default
+            varnames.append((varname, explode, prefix))
 
-    retval = []
-    joiner = operator
-    prefix = operator
-    if operator == "+":
-      prefix = ""
-      joiner = ","
-    if operator == "?":
-      joiner = "&"
-    if operator == "":
-      joiner = ","
-    for varname, explode, partial in varnames:
-      if varname in vars:
-        value = vars[varname]
-        #if not value and (type(value) == type({}) or type(value) == type([])) and varname in defaults:
-        if not value and value != "" and varname in defaults:
-          value = defaults[varname]
-      elif varname in defaults:
-        value = defaults[varname]
-      else:
-        continue
-      retval.append(TOSTRING[operator](varname, value, explode, operator, safe=safe))
-    if "".join(retval):
-      return prefix + joiner.join(retval)
-    else:
-      return ""
+        retval = []
+        joiner = operator
+        start = operator
+        if operator == "+":
+            start = ""
+            joiner = ","
+        if operator == "#":
+            joiner = ","
+        if operator == "?":
+            joiner = "&"
+        if operator == "&":
+            start = "&"
+        if operator == "":
+            joiner = ","
+        for varname, explode, prefix in varnames:
+            if varname in variables:
+                value = variables[varname]
+                if not value and value != "" and varname in defaults:
+                    value = defaults[varname]
+            elif varname in defaults:
+                value = defaults[varname]
+            else:
+                continue
+            expanded = TOSTRING[operator](
+              varname, value, explode, prefix, operator, safe=safe)
+            if expanded is not None:
+                retval.append(expanded)
+        if len(retval) > 0:
+            return start + joiner.join(retval)
+        else:
+            return ""
 
-  return TEMPLATE.sub(_sub, template)
+    return TEMPLATE.sub(_sub, template)

whatsnew.txt

@@ -1,3 +1,80 @@
+GAM 3.5
+  -Support for the new Google Classroom API.
+   -create, update, info and delete courses
+   -add, remove and sync course teachers and students
+   -print courses and course participants
+  -Google CloudPrint API Support
+   -update, info, delete and report printers
+   -share, unshare and get ACLs for printers
+   -submit, cancel, report and delete print jobs
+  -Bug fixes and improvements to GAM batch commands
+
+GAM 3.45
+  -add six.py to solve compatability issues on OS X and Linux
+  -be conservative with password hashing to prevent timeouts
+  
+  If you see issues setting user passwords with GAM 3.44 or older, please upgrade to 3.45.
+
+GAM 3.44
+  -"gam update cros <id> assetid <asset>" allows updating of Chrome OS device Asset ID field. Thanks Erik Pitti!
+  -Windows versions of GAM now use pyinstaller instead of py2exe.
+  -upgraded versions of the googleapiclient and oauth2client libraries.
+  -"gam print cros" should produce a cleaner CSV now.
+
+GAM 3.43
+  -Fix crash when authenticating GAM related to Short URLs
+  -minor fixes to Drive and Calendar commands
+  -catch unauthorized service account errors and offer nice instructions.
+  -execute bit for gam.py (Thanks daethnir)
+
+GAM 3.42
+  -"gam <users> show driveactivity" displays Drive activity for user
+  -"gam report tokens" displays user OAuth activity report
+  -"gam mobile <id> action accountwipe" wipes account only on Android devices
+  -"gam <users> update labels" removes Inbox/ prefix from label names
+  -upgrades to oauth2client and googleapiclient libraries which GAM depends on
+  -"gam <users> show gmailprofile" shows Gmail mailbox details
+  -"gam license <sku>" commands to perform actions only on users with a given license
+  -bug fixes, lots of bug fixes
+
+GAM 3.41
+  -fix Google servers not returning file size on audit export download
+  -soft fail on license change errors
+  -fix for gam info domain errors
+
+GAM 3.4 "Oktoberfest"
+  -Support for creating and setting custom user schemas http://goo.gl/M9rQrI
+  -End user view of print users and user info commands.
+  -fix updating name/description for groups
+  -fix groups sync commands
+  -gam print groups members no longer fails on zero member groups
+  -make sure downloaded Drive file names are safe for OS filenames.
+  -gam info domain should no longer crash on getting customer ID
+  -other minor bug fixes
+
+GAM 3.32
+  -fix service account json files downloaded from new cloud console don't work with GAM
+  -use the new Gmail API for label command. Offers ability to rename labels and show/hide labels from the label and message list.
+  -copy Google drive files with commands like gam user <email> update drivefile id <fileid> copy
+  -print only one SKU for licenses with commands like gam print licenses sku vault
+  -short license names for Google Apps Message Security (gams), Google Apps Unlimtied (gau) and Google Vault Former Employee (vfe)
+
+GAM 3.31
+  -New command "gam user delete aliases" clears all aliases for user.
+  -"gam update user email vfe" will rename user to vfe.oldname.XXXXX@olddomain.com
+  -fix delete photo command
+  -fix password update/create for Windows builds
+
+GAM 3.3
+  -Major rewrites of "gam report" and "gam print users". Note that CSV headers have changed. Better performance and print users now supports "rich" fields like organization, phone, relations, etc.
+  -"gam report drive" now works for Google Apps Unlimited domains.
+  -"gam info user" now shows the user's licenses. Use "nolicenses" to prevent showing licenses.
+  -fix "gam print licenses" fails in large domains by reducing page size from 1000 to 100
+  -GAM now sends a sha-512 salted hashed password on user create and update for better security.
+  -cleanup unused features of old GData library.
+  -fix for Drive file downloading default format.
+  -upgrade to httplib v0.9 which may help with httplib.BadStatus errors.
+
 GAM 3.21
   -Fix crash when attempting to perform Drive operations for users with Drive service disabled.
   -Fix "gam info org" only prints first 100 users in org.
@@ -56,7 +133,7 @@ GAM 2.992
 
 GAM 2.991
   -gam print commands now support a "todrive" argument. When specified, instead of displaying the CSV output locally (or piping it to a file), GAM will upload the CSV data to a Google Docs Spreadsheet owned by the admin you've authenticated as. The spreadsheet will be opened automatically or, if you've created nobrowser.txt, a URL will be shown.
-  -GAM should handle non-English input characters better. Commands like: "gam.py update user rpinaya lastname Piñaya" should work without issue on Windows.
+  -GAM should handle non-English input characters better. Commands like: "gam.py update user rpinaya lastname Piñaya" should work without issue on Windows.
   -Improved errors in various places (less "explosions" more meaningful instructions.
   -gam undelete user is fixed to always use the user's id rather than email address. If an email address is supplied, GAM converts it to a id before attempting to delete. If more than 1 deleted user exists with that address, GAM displays the options.
 
@@ -222,4 +299,4 @@ GAM 1.8
 
 -The settings filter command http://code.google.com/p/google-apps-manager/wiki/ExamplesEmailSettings#Create_a_Filter now has more actions including forward, star, trash and never send to spam.
 
--Downloading Audit Exports now has partial resume support. GAM will not re-download files that already exist on the local drive. If a large export download fails you should delete the last file GAM was working on since it's incomplete and then restart the process, GAM will pick up with the last file.
+-Downloading Audit Exports now has partial resume support. GAM will not re-download files that already exist on the local drive. If a large export download fails you should delete the last file GAM was working on since it's incomplete and then restart the process, GAM will pick up with the last file.