google.appengine.ext.bulkload.connector_interface module

Summary

Bulkloader interfaces for the format reader/writers.

Contents

class google.appengine.ext.bulkload.connector_interface.ConnectorInterfacesource

Bases: object

Abstract base class describing the external Connector interface.

The External Connector interface describes the transition between an external data source, e.g. CSV file, XML file, or some sort of database interface, and the intermediate bulkloader format, which is a dictionary or similar structure representing the external transformation of the data.

On import, the generate_import_record generator is the only method called.

On export, the initialize_export method is called once, followed by a call to write_dict for each record, followed by a call to finalize_export.

The bulkload_state is a BulkloadState object from google.appengine.ext.bulkload.bulkload_config. The interesting properties to a Connector object are the loader_opts and exporter_opts, which are strings passed in from the bulkloader command line.

finalize_export()source

Performs finalization actions after every record is written.

generate_import_record(filename, bulkload_state)source

A function which returns an iterator over dictionaries.

This is the only method used on import.

Parameters

  • filename – The –filename argument passed in on the bulkloader command line. This value is opaque to the bulkloader and thus could specify any sort of descriptor for your generator.

  • bulkload_state – Passed in BulkloadConfig.BulkloadState object.

Returns

An iterator describing an individual record. Typically a dictionary, to be used with dict_to_model. Typically implemented as a generator.

initialize_export(filename, bulkload_state)source

Initialize the output file.

Parameters

  • filename – The string given as the –filename flag argument.

  • bulkload_state – Passed in BulkloadConfig.BulkloadState object.

These values are opaque to the bulkloader and thus could specify any sort of descriptor for your exporter.

write_dict(dictionary)source

Write one record for the specified entity.

Parameters

dictionary – A post-transform dictionary.

google.appengine.ext.bulkload.connector_interface.create_from_options(options, name='unknown')source

Factory using an options dictionary.

This is frequently implemented as the constructor on the connector class, or a static or class method on the connector class.

Parameters

  • options – Parsed dictionary from yaml file, interpretation is up to the implementor of this class.

  • name – Identifier of this transform to be used in error messages.

Returns

An object which implements the ConnectorInterface interface.