Data ingestion and data schemas

This page describes the types of data ingestion and the data schemas available for Telecom Subscriber Insights.

Telecom Subscriber Insights offers two types of data ingestion and several data schemas. You present datasets that conform to the data schemas, and then Telecom Subscriber Insights uses the data that you provide to create offers for your customers.

Data ingestion types

Telecom Subscriber Insights offers the following types of data ingestion:

Usage and account summary data. Telecom Subscriber Insights requires regular updates to usage data and account summary data. Usage and account summary data can be ingested in real time, near real time, or periodically. Real-time data or near real-time data are published to a Pub/Sub topic that is configured during the initial setup steps. The usage and account summary data can also be pushed periodically, usually once a day, and this is accomplished by writing the data to a Cloud Storage bucket that is configured during the initial setup.
Data plan status. Telecom Subscriber Insights also requires information about each subscriber's current data plan status. This data is shared with Telecom Subscriber Insights by using the Google Mobile Data Plan Sharing API.

Data tables

The following section describes Telecom Subscriber Insights tables and schemas that you can use to push data to Telecom Subscriber Insights. Before pushing data, ensure that your datasets conform to the schemas described in the following sections.

Available datasets

The following table describes the dataset tables available within Telecom Subscriber Insights.

Dataset name and schema link	Definition
Usage records	A snapshot of subscriber usage records and the balance history of a given plan.
Account summary	A summary of subscriber accounts and the plans that they have purchased.
CPID events	Events logged per carrier plan identifier (CPID) if you have a configured CPID server.

Dataset schema definitions

The following sections describe the schema definitions for each of the available tables for Telecom Subscriber Insights.

Usage records

[
  {
    "name": "sub_id",
    "type": "STRING",
    "mode": "REQUIRED",
    "description": "Subscriber identification. A privacy preserving subscriber identity for the subscriber."
  },
  {
    "name": "usage",
    "type": "RECORD",
    "mode": "REPEATED",
    "description": "It represents the current usage of the subscriber for a given plan.",
    "fields": [
      {
        "name": "plan_id",
        "type": "STRING",
        "mode": "REQUIRED",
        "description": "Unique plan identifier."
      },
      {
        "name": "current_usage",
        "type": "FLOAT",
        "mode": "NULLABLE",
        "description": "Describes the current usage of the plan. The unit is specified in usage_unit."
      },
      {
        "name": "usage_pct",
        "type": "FLOAT",
        "mode": "NULLABLE",
        "description": "Describes the percentage used of the allotment. Range: [0, 100]"
      },
      {
        "name": "allotted_usage",
        "type": "FLOAT",
        "mode": "NULLABLE",
        "description": "Describes the allotted usage of the plan. The unit is specified in usage_unit."
      },
      {
        "name": "start_date",
        "type": "TIMESTAMP",
        "mode": "NULLABLE",
        "description": "Plan start date."
      },
      {
        "name": "end_date",
        "type": "TIMESTAMP",
        "mode": "NULLABLE",
        "description": "Plan end date."
      },
      {
        "name": "usage_unit",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "Describes the unit of measure: messages, minutes, bytes."
      }
    ]
  },
  {
    "name": "qoe_metrics",
    "type": "RECORD",
    "mode": "NULLABLE",
    "description": "Describes the quality of experience of the subscriber.",
    "fields": [
      {
        "name": "uptime",
        "type": "INTEGER",
        "mode": "NULLABLE",
        "description": "How long the user has been connected to the network without interruption in seconds."
      },
      {
        "name": "network_score",
        "type": "FLOAT",
        "mode": "NULLABLE",
        "description": "The quality of the network network_score out of 100."
      }
    ]
  },
  {
    "name": "update_time",
    "type": "TIMESTAMP",
    "mode": "REQUIRED",
    "description": "Timestamp when this record was updated."
  }
]

Account summary

[
  {
    "name": "sub_id",
    "type": "STRING",
    "mode": "REQUIRED",
    "description": "Subscriber identification. A privacy preserving subscriber identity for the subscriber."
  },
  {
    "name": "purchased_plans",
    "type": "RECORD",
    "mode": "REPEATED",
    "description": "List of all the purchased plans for the subscriber.",
    "fields": [
      {
        "name": "plan_name",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "Plan name"
      },
      {
        "name": "plan_id",
        "type": "STRING",
        "mode": "REQUIRED",
        "description": "Unique identifier for the plan"
      },
      {
        "name": "plan_description",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "promo_message",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "Additional promotional message for the user."
      },
      {
        "name": "language_code",
        "type": "STRING",
        "mode": "NULLABLE",
        "description": "The BCP-47 language code, such as en-US or sr-Latn."
      },
      {
        "name": "max_rate_kbps",
        "type": "INTEGER",
        "mode": "NULLABLE",
        "description": "The maximum bandwidth that the plan offers to the user."
      },
      {
        "name": "cost",
        "type": "RECORD",
        "mode": "NULLABLE",
        "description": "Cost of the purchased plans.",
        "fields": [
          {
            "name": "currency_code",
            "type": "STRING",
            "mode": "NULLABLE",
            "description": "The three-letter currency code defined in ISO 4217."
          },
          {
            "name": "units",
            "type": "FLOAT",
            "mode": "NULLABLE",
            "description": "The value of the amount."
          }
        ]
      },
      {
        "name": "duration_minutes",
        "type": "INTEGER",
        "mode": "NULLABLE"
      },
      {
        "name": "quota_bytes",
        "type": "INTEGER",
        "mode": "NULLABLE",
        "description": "Package quota in bytes."
      },
      {
        "name": "quota_minutes",
        "type": "INTEGER",
        "mode": "NULLABLE",
        "description": "Package quota in minutes."
      },
      {
        "name": "buy_time",
        "type": "TIMESTAMP",
        "mode": "NULLABLE",
        "description": "Timestamp when the user bought this plan."
      }
    ]
  },
  {
    "name": "update_time",
    "type": "TIMESTAMP",
    "mode": "REQUIRED",
    "description": "Updated timestamp."
  }
]

CPID events

message CpidGenEvent {
  optional string cpid = 1;
  optional string sub_id = 2;
  optional string timestamp = 3;
}