Class PslScanBuilder (0.4.5) 

public class PslScanBuilder implements ScanBuilder, Scan, SupportsReportStatistics

Both Scan and ScanBuilder implementation, otherwise estimateStatistics() is not called due to bug in DataSourceV2Relation.

Information from https://github.com/GoogleCloudDataproc/spark-bigquery-connector/blob/b0b2a8add37bd044e5a9976ce635bb4230957f29/spark-bigquery-dsv2/spark-3.1-bigquery/src/main/java/com/google/cloud/spark/bigquery/v2/BigQueryScanBuilder.java#L30

Inheritance

java.lang.Object > PslScanBuilder

Implements

org.apache.spark.sql.connector.read.ScanBuilder, org.apache.spark.sql.connector.read.Scan, org.apache.spark.sql.connector.read.SupportsReportStatistics

Constructors

PslScanBuilder(PslReadDataSourceOptions options)

public PslScanBuilder(PslReadDataSourceOptions options)
Parameter
Name Description
options PslReadDataSourceOptions

Methods

build()

public Scan build()
Returns
Type Description
org.apache.spark.sql.connector.read.Scan

description()

public String description()
Returns
Type Description
String

estimateStatistics()

public Statistics estimateStatistics()
Returns
Type Description
org.apache.spark.sql.connector.read.Statistics

readSchema()

public StructType readSchema()
Returns
Type Description
org.apache.spark.sql.types.StructType

toContinuousStream(String checkpointLocation)

public ContinuousStream toContinuousStream(String checkpointLocation)
Parameter
Name Description
checkpointLocation String
Returns
Type Description
org.apache.spark.sql.connector.read.streaming.ContinuousStream

toMicroBatchStream(String checkpointLocation)

public MicroBatchStream toMicroBatchStream(String checkpointLocation)
Parameter
Name Description
checkpointLocation String
Returns
Type Description
org.apache.spark.sql.connector.read.streaming.MicroBatchStream