Types and Property Classes

Note: Developers building new applications are strongly encouraged to use the NDB Client Library, which has several benefits compared to this client library, such as automatic entity caching via the Memcache API. If you are currently using the older DB Client Library, read the DB to NDB Migration Guide

The App Engine datastore supports a fixed set of value types for properties on data entities. Property classes can define new types that are converted to and from the underlying value types, and the value types can be used directly with Expando dynamic properties and ListProperty aggregate property models.

The following table describes the Property classes whose values correspond directly with the underlying data types. Any of these value types can be used in an Expando dynamic property or ListProperty aggregate type.

Datastore Value Types

Datastore entity property values can be of one of the following types. See above for a list of corresponding Property classes to use with Model definitions.

Other than the Python standard types and users.User, all classes described in this section are provided by the google.appengine.ext.db module.

str or unicode

A short string (1500 bytes or less).

A str value is assumed to be text encoded with the ascii codec, and is converted to a unicode value before being stored. The value is returned by the datastore as a unicode value. For short strings using other codecs, use a unicode value.

Short strings are indexed by the datastore, and can be used in filters and sort orders. For text strings longer than 1500 bytes (which are not indexed), use a Text instance. For unencoded byte strings longer than 1500 bytes (also not indexed), use a Blob instance. For non-textual unencoded byte strings up to 1500 bytes (not characters) that should be indexed, use a ByteString instance.

Model property: StringProperty

bool

A Boolean value (True or False).

Model property: BooleanProperty

int or long

An integer value, up to 64 bits.

Python int values are converted to Python long values prior to storage. A value stored as an int will be returned as a long.

If a long larger than 64 bits is assigned, only the least significant 64 bits are stored.

Model property: IntegerProperty

float

A floating-point number.

Model property: FloatProperty

datetime.datetime

A date and time. See the datetime module documentation.

If the datetime value has a tzinfo attribute, it will be converted to the UTC time zone for storage. Values come back from the datastore as UTC, with a tzinfo of None. An application that needs date and time values to be in a particular time zone must set tzinfo correctly when updating the value, and convert values to the timezone when accessing the value.

Some libraries use the TZ environment variable to control the time zone applied to date-time values. App Engine sets this environment variable to "UTC". Note that changing this variable in an application will not change the behavior of some datetime functions, because changes to environment variables are not visible outside of the Python code.

If you only convert values to and from a particular time zone, you can implement a custom datetime.tzinfo to convert values from the datastore:

import datetime
import time

class Pacific_tzinfo(datetime.tzinfo):
    """Implementation of the Pacific timezone."""
    def utcoffset(self, dt):
        return datetime.timedelta(hours=-8) + self.dst(dt)

    def _FirstSunday(self, dt):
        """First Sunday on or after dt."""
        return dt + datetime.timedelta(days=(6-dt.weekday()))

    def dst(self, dt):
        # 2 am on the second Sunday in March
        dst_start = self._FirstSunday(datetime.datetime(dt.year, 3, 8, 2))
        # 1 am on the first Sunday in November
        dst_end = self._FirstSunday(datetime.datetime(dt.year, 11, 1, 1))

        if dst_start <= dt.replace(tzinfo=None) < dst_end:
            return datetime.timedelta(hours=1)
        else:
            return datetime.timedelta(hours=0)
    def tzname(self, dt):
        if self.dst(dt) == datetime.timedelta(hours=0):
            return "PST"
        else:
            return "PDT"

pacific_time = datetime.datetime.fromtimestamp(time.mktime(utc_time.timetuple()), Pacific_tzinfo())

See the datetime module documentation (including datetime.tzinfo). See also the third-party module pytz, though note that the pytz distribution has many files.

The DateTimeProperty model property class includes features such as the ability to automatically use the date and time a model instance is stored. These are features of the model, and are not available on the raw datastore value (such as in an Expando dynamic property).

Model properties: DateTimeProperty, DateProperty, TimeProperty

list

A list of values, each of which is of one of the supported data types.

When a list is used as the value of an Expando dynamic property, it cannot be an empty list. This is due to how list values are stored: When a list property has no items, it has no representation in the datastore. You can use a static property and the ListProperty class to represent an empty list value for a property.

Model property: ListProperty

db.Key

The key for another datastore entity.

Note: Key strings are limited to 1500 bytes or less.

m = Employee(name="Susan", key_name="susan5")
m.put()
e = Employee(name="Bob", manager=m.key())
e.put()

m_key = db.Key.from_path("Employee", "susan5")
e = Employee(name="Jennifer", manager=m_key)

Model properties: ReferenceProperty, SelfReferenceProperty

blobstore.BlobKey

The key for a Blobstore value, generated by the Blobstore when the value is uploaded.

Model properties: blobstore.BlobReferenceProperty

users.User

A user with a Google account.

A user.User value in the datastore does not get updated if the user changes their email address. For this reason, we strongly recommend that you avoid storing a users.User as a UserProperty value, because this includes the email address along with the unique ID. If a user changes the email address and you then compare the old, stored user.User to the new user.User value, they won't match. Instead, use the user.User value's user_id() as the user's stable unique identifier.

Model property: UserProperty

class Blob(arg=None)

Binary data, as a byte string. This is a subclass of the built-in str type.

Blob properties are not indexed, and cannot be used in filters or sort orders.

Blob is for binary data, such as images. It takes a str value, but this value is stored as a byte string and is not encoded as text. Use a Text instance for large text data.

Model property: BlobProperty

class MyModel(db.Model):
    blob = db.BlobProperty()

m = MyModel()
m.blob = db.Blob(open("image.png", "rb").read())

In XML, blobs are base-64 encoded whether or not they contain binary data.

class ByteString(arg)

A short blob value (a "byte string") of 1500 bytes or less. ByteString is a subclass of str, and takes an unencoded str value as an argument to its constructor.

ByteStrings are indexed by the datastore, and can be used in filters and sort orders. For byte strings longer than 1500 bytes (which are not indexed), use a Blob instance. For encoded text data, use str (short, indexed) or Text (long, not indexed).

Model property: ByteStringProperty

class Text(arg=None, encoding=None)

A long string. This is a subclass of the built-in unicode type.

arg a unicode or str value. If arg is a str, then it is parsed with the encoding specified by encoding, or ascii if no encoding is specified. See the list of standard encodings for possible values for encoding.

Unlike an entity property whose value is a simple str or unicode, a Text property can be more than 1500 bytes long. However, Text properties are not indexed, and cannot be used in filters or sort orders.

Model property: TextProperty

class MyModel(db.Model):
    text = db.TextProperty()

m = MyModel()
m.text = db.Text(u"kittens")

m.text = db.Text("kittens", encoding="latin-1")

class Category(tag)

A category or "tag". This is a subclass of the built-in unicode type.

Model property: CategoryProperty

class MyModel(db.Model):
    category = db.CategoryProperty()

m = MyModel()
m.category = db.Category("kittens")

In XML, this is an Atom Category element.

class Email(email)

An email address. This is a subclass of the built-in unicode type.

Neither the property class nor the value class perform validation of email addresses, they just store the value.

Model property: EmailProperty

class MyModel(db.Model):
    email_address = db.EmailProperty()

m = MyModel()
m.email_address = db.Email("larry@example.com")

In XML, this is a gd:email element.

class GeoPt(lat, lon=None)

A geographical point represented by floating-point latitude and longitude coordinates.

Model property: GeoPtProperty

In XML, this is a georss:point element.

class IM(protocol, address=None)

An instant messaging handle.

protocol is the canonical URL of the instant messaging service. Some possible values:

Protocol	Description
sip	SIP/SIMPLE
xmpp	XMPP/Jabber
http://aim.com/	AIM
http://icq.com/	ICQ
http://messenger.msn.com/	MSN Messenger
http://messenger.yahoo.com/	Yahoo Messenger
http://sametime.com/	Lotus Sametime
http://gadu-gadu.pl/	Gadu-Gadu
unknown	Unknown or unspecified

address is the handle's address.

Model property: IMProperty

class MyModel(db.Model):
    im = db.IMProperty()

m = MyModel()
m.im = db.IM("http://example.com/", "Larry97")

In XML, this is a gd:im element.

class Link(link)

A fully qualified URL. This is a subclass of the built-in unicode type.

Model property: LinkProperty

class MyModel(db.Model):
    link = db.LinkProperty()

m = MyModel()
m.link = db.Link("http://www.google.com/")

In XML, this is an Atom Link element.

class PhoneNumber(phone)

A human-readable telephone number. This is a subclass of the built-in unicode type.

Model property: PhoneNumberProperty

class MyModel(db.Model):
    phone = db.PhoneNumberProperty()

m = MyModel()
m.phone = db.PhoneNumber("1 (206) 555-1212")

In XML, this is a gd.phoneNumber element.

class PostalAddress(address)

A postal address. This is a subclass of the built-in unicode type.

Model property: PostalAddressProperty

class MyModel(db.Model):
    address = db.PostalAddressProperty()

m = MyModel()
m.address = db.PostalAddress("1600 Ampitheater Pkwy., Mountain View, CA")

In XML, this is a gd:postalAddress element.

class Rating(rating)

A user-provided rating for a piece of content, as an integer between 0 and 100. This is a subclass of the built-in long type. The class validates that the value is an integer between 0 and 100, and raises a BadValueError if the value is invalid.

Model property: RatingProperty

class MyModel(db.Model):
    rating = db.RatingProperty()

m = MyModel()
m.rating = db.Rating(97)

In XML, this is a gd:rating element.

Property Classes

All model property classes provided by google.appengine.ext.db are subclasses of the base class Property, and support all of the base constructor's arguments. See the base class documentation for information about those arguments.

The google.appengine.ext.db package provides the following model property classes:

class BlobProperty(...)

An uninterpreted collection of binary data.

Blob data is a byte string. For text data, which may involve encoding, use TextProperty.

Value type: Blob

class BooleanProperty(...)

A Boolean value (True or False).

Value type: bool

class ByteStringProperty(verbose_name=None, ...)

A short blob value (a "byte string") of 1500 bytes or less.

ByteStringProperty values are indexed, and can be used in filters and sort orders.

Like StringProperty, except that the value is not encoded in any way. The bytes are stored literally.

If the ByteStringProperty is required, the value cannot be an empty string.

Value type: ByteString

class CategoryProperty(...)

A category or "tag," a descriptive word or phrase.

Value type: Category

class DateProperty(verbose_name=None, auto_now=False, auto_now_add=False, ...)

A date without a time of day; see DateTimeProperty for more information.

Value type: datetime.date; converted internally to datetime.datetime

class DateTimeProperty(verbose_name=None, auto_now=False, auto_now_add=False, ...)

A date and time.

If auto_now is True, the property value is set to the current time whenever the model instance is stored in the datastore, overwriting the property's previous value. This is useful for tracking a "last modified" date and time for a model instance.

If auto_now_add is True, the property value is set to the current time the first time the model instance is stored in the datastore, unless the property has already been assigned a value. This is useful for storing a "created" date and time for a model instance.

Date-time values are stored as and returned using the UTC time zone. See datetime.datetime for a discussion of how to manage time zones.

Value type: datetime.datetime

class EmailProperty(...)

An email address.

Neither the property class nor the value class perform validation of email addresses, they just store the value.

Value type: Email

class FloatProperty(...)

A floating-point number.

Value type: float

class GeoPtProperty(...)

A geographical point represented by floating-point latitude and longitude coordinates.

Value type: GeoPt

class IMProperty(...)

An instant messaging handle.

Value type: IM

class IntegerProperty(...)

An integer value, up to 64 bits.

Python int values are converted to Python long values prior to storage. A value stored as an int will be returned as a long.

If a long larger than 64 bits is assigned, only the least significant 64 bits are stored.

Value type: int or long

class LinkProperty(...)

A fully qualified URL.

Value type: Link

class ListProperty(item_type, verbose_name=None, default=None, ...)

A list of values of the type specified by item_type.

In a query, comparing a list property to a value performs the test against the members of the list: list_property = value tests whether the value appears anywhere in the list, list_property < value tests whether any of the list members are less than the given value, and so forth.

A query cannot compare two list values. There is no way to test two lists for equality without testing each element for membership separately.

item_type is the type of the items in the list, as a Python type or class. All items in the list value must be of the given type. item_type must be one of the datastore value types, and cannot be list.

The value of a ListProperty cannot be None. It can, however, be an empty list. When None is specified for the default argument (or when the default argument is not specified), the default value of the property is the empty list.

Tip: Because ListProperty aggregate types do not use the Property classes, Property class features such as automatic values and validation are not applied automatically to members of the list value. If you want to validate a member value using a Property class, you can instantiate the class and call its validate() method on the value.

default is the default value for the list property. If None, the default is an empty list. A list property can define a custom validator to disallow the empty list.

See the Data Modeling page for more information on list properties and values.

Value type: a Python list of values of the specified type

class PhoneNumberProperty(...)

A human-readable telephone number.

Value type: PhoneNumber

class PostalAddressProperty(...)

A postal address.

Value type: PostalAddress

class RatingProperty()

A user-provided rating for a piece of content, as an integer between 0 and 100.

Value type: Rating

class ReferenceProperty(reference_class=None, verbose_name=None, collection_name=None, ...)

A reference to another model instance. For example, a reference may indicate a many-to-one relationship between the model with the property and the model referenced by the property.

reference_class is the model class of the model instance being referenced. If specified, only model instances of the class can be assigned to this property. If None, any model instance can be the value of this property.

collection_name is the name of the property to give to the referenced model class. The value of the property is a Query for all entities that reference the entity. If no collection_name is set, then modelname_set (with the name of the referenced model in lowercase letters and _set added) is used.

Note: collection_name must be set if there are multiple properties within the same model referencing the same model class. Otherwise, a DuplicatePropertyError will be raised when the default names are generated.

ReferenceProperty automatically references and dereferences model instances as property values: a model instance can be assigned to a reference property directly, and its key will be used. The ReferenceProperty value can be used as if it were a model instance, and the datastore entity will be fetched and the model instance created when it is first used in this way. Untouched reference properties do not query for unneeded data.

class Author(db.Model):
    name = db.StringProperty()

class Story(db.Model):
    author = db.ReferenceProperty(Author)

story = db.get(story_key)
author_name = story.author.name

author = db.get(author_key)
stories_by_author = author.story_set.get()

As with a Key value, it is possible for a reference property value to refer to a data entity that does not exist. If a referenced entity is deleted from the datastore, references to the entity are not updated. Accessing an entity that does not exist raises a ReferencePropertyResolveError.

Deleting an entity does not delete entities referred to by a reference property.

Value type: db.Key

class SelfReferenceProperty(verbose_name=None, collection_name=None, ...)

A reference to another model instance of the same class (see ReferenceProperty).

Value type: db.Key

class StringListProperty(verbose_name=None, default=None, ...)

Similar to a list property of Python str or unicode (basestring) values.

Value type: a Python list of str or unicode values

class StringProperty(verbose_name=None, multiline=False, ...)

A short string. Takes a Python str or unicode (basestring) value of 1500 bytes or less.

StringProperty values are indexed, and can be used in filters and sort orders.

If multiline is False, the value cannot include linefeed characters. The djangoforms library uses this to enforce a difference between text fields and textarea fields in the data model, and others can use it for a similar purpose.

If the string property is required, its value cannot be an empty string.

Value type: str or unicode

class TextProperty()

A long string.

Unlike StringProperty, a TextProperty value can be more than 1500 bytes long. However, TextProperty values are not indexed and cannot be used in filters or sort orders.

TextProperty values store text with a text encoding. For binary data, use BlobProperty.

If the text property is required, its value cannot be an empty string.

Value type: Text

class TimeProperty(verbose_name=None, auto_now=False, auto_now_add=False, ...)

A time of day without a date. Takes a Python standard library datetime.time value; see DateTimeProperty for more information.

Value type: datetime.time; converted internally to datetime.datetime

class UserProperty(verbose_name=None, auto_current_user=False, auto_current_user_add=False, ...)

Important: We strongly recommend that you do not store a UserProperty, since it includes the email address and the user's unique ID. If a user changes their email address and you compare their old, stored User to the new User value, they won't match.

A user with a Google account.

If auto_current_user is True, the property value is set to the currently signed-in user whenever the model instance is stored in the datastore, overwriting the property's previous value. This is useful for tracking which user modifies a model instance.

If auto_current_user_add is True, the property value is set to the currently signed-in user the first time the model instance is stored in the datastore, unless the property has already been assigned a value. This is useful for tracking which user creates a model instance, which may not be the same user that modifies it later.

UserProperty does not accept a default value. Default values are set when the model class is first imported, and with import caching may not be the currently signed-in user.

Value type: users.User