google.appengine.ext.deferred.deferred module

Summary

A module that handles deferred execution of callables via the task queue.

Tasks consist of a callable and arguments to pass to it. The callable and its arguments are serialized and put on the task queue, which deserializes and executes them. The following callables can be used as tasks:

  1. Functions defined in the top level of a module

  2. Classes defined in the top level of a module

  3. Instances of classes in (2) that implement __call__

  4. Instance methods of objects of classes in (2)

  5. Class methods of classes in (2)

  6. Built-in functions

  7. Built-in methods

The following callables can NOT be used as tasks: 1) Nested functions or closures 2) Nested classes or objects of them 3) Lambda functions 4) Static methods

The arguments to the callable, and the object (in the case of method or object calls) must all be pickleable.

If you want your tasks to execute reliably, don’t use mutable global variables; they are not serialized with the task and may not be the same when your task executes as they were when it was enqueued (in fact, they will almost certainly be different).

If your app relies on manipulating the import path, make sure that the function you are deferring is defined in a module that can be found without import path manipulation. Alternately, you can include deferred.TaskHandler in your own webapp application instead of using the easy-install method detailed below.

When you create a deferred task using deferred.defer, the task is serialized, and an attempt is made to add it directly to the task queue. If the task is too big (larger than about 10 kilobytes when serialized), a datastore entry will be created for the task, and a new task will be enqueued, which will fetch the original task from the datastore and execute it. This is much less efficient than the direct execution model, so it’s a good idea to minimize the size of your tasks when possible.

In order for tasks to be processed, you need to set up the handler. Add the following to your app.yaml handlers section:

handlers: - url: /_ah/queue/deferred

script: $PYTHON_LIB/google/appengine/ext/deferred/handler.py login: admin

By default, the deferred module uses the URL above, and the default queue.

Example usage:

def do_something_later(key, amount):
  entity = MyModel.get(key)
  entity.total += amount
  entity.put()

# Use default URL and queue name, no task name, execute ASAP.
deferred.defer(do_something_later, my_key, 20)

# Providing non-default task queue arguments
deferred.defer(do_something_later, my_key, 20, _queue="foo", _countdown=60)

Contents

exception google.appengine.ext.deferred.deferred.Errorsource

Bases: exceptions.Exception

Base class for exceptions in this module.

exception google.appengine.ext.deferred.deferred.PermanentTaskFailuresource

Bases: google.appengine.ext.deferred.deferred.Error

Indicates that a task failed, and will never succeed.

exception google.appengine.ext.deferred.deferred.SingularTaskFailuresource

Bases: google.appengine.ext.deferred.deferred.Error

Indicates that a task failed once.

class google.appengine.ext.deferred.deferred.TaskHandlersource

Bases: google.appengine.ext.webapp._webapp25.RequestHandler

A webapp handler class that processes deferred invocations.

post()source
run_from_request()source

Default behavior for POST requests to deferred handler.

google.appengine.ext.deferred.deferred.defer(obj, *args, **kwargs)source

Defers a callable for execution later.

The default deferred URL of /_ah/queue/deferred will be used unless an alternate URL is explicitly specified. If you want to use the default URL for a queue, specify _url=None. If you specify a different URL, you will need to install the handler on that URL (see the module docstring for details).

Parameters

  • obj – The callable to execute. See module docstring for restrictions. _countdown, _eta, _headers, _name, _target, _transactional, _url, _retry_options, _queue: Passed through to the task queue - see the task queue documentation for details.

  • args – Positional arguments to call the callable with.

  • kwargs – Any other keyword arguments are passed through to the callable.

Returns

A taskqueue.Task object which represents an enqueued callable.

google.appengine.ext.deferred.deferred.invoke_member(obj, membername, *args, **kwargs)source

Retrieves a member of an object, then calls it with the provided arguments.

Parameters

  • obj – The object to operate on.

  • membername – The name of the member to retrieve from ojb.

  • args – Positional arguments to pass to the method.

  • kwargs – Keyword arguments to pass to the method.

Returns

The return value of the method invocation.

google.appengine.ext.deferred.deferred.main()source
google.appengine.ext.deferred.deferred.run(data)source

Unpickles and executes a task.

Parameters

data – A pickled tuple of (function, args, kwargs) to execute.

Returns

The return value of the function invocation.

google.appengine.ext.deferred.deferred.run_from_datastore(key)source

Retrieves a task from the datastore and executes it.

Parameters

key – The datastore key of a _DeferredTaskEntity storing the task.

Returns

The return value of the function invocation.

google.appengine.ext.deferred.deferred.serialize(obj, *args, **kwargs)source

Serializes a callable into a format recognized by the deferred executor.

Parameters

  • obj – The callable to serialize. See module docstring for restrictions.

  • args – Positional arguments to call the callable with.

  • kwargs – Keyword arguments to call the callable with.

Returns

A serialized representation of the callable.

google.appengine.ext.deferred.deferred.set_log_level(log_level)source

Sets the log level deferred will log to in normal circumstances.

Parameters

log_level – one of logging log levels, e.g. logging.DEBUG, logging.INFO, etc.