Some or all of the information on this page might not apply to Cloud de Confiance by S3NS. See Differences from Google Cloud for more details.

Package google.cloud.bigquery.storage.v1alpha

Index

MetastorePartitionService (interface)
BatchCreateMetastorePartitionsRequest (message)
BatchCreateMetastorePartitionsResponse (message)
BatchDeleteMetastorePartitionsRequest (message)
BatchSizeTooLargeError (message)
BatchUpdateMetastorePartitionsRequest (message)
BatchUpdateMetastorePartitionsResponse (message)
CreateMetastorePartitionRequest (message)
FieldSchema (message)
ListMetastorePartitionsRequest (message)
ListMetastorePartitionsResponse (message)
MetastorePartition (message)
MetastorePartitionList (message)
MetastorePartitionValues (message)
ReadStream (message)
SerDeInfo (message)
StorageDescriptor (message)
StreamList (message)
StreamMetastorePartitionsRequest (message)
StreamMetastorePartitionsResponse (message)
UpdateMetastorePartitionRequest (message)

MetastorePartitionService

BigQuery Metastore Partition Service API. This service is used for managing metastore partitions in BigQuery metastore. The service supports only batch operations for write.

BatchCreateMetastorePartitions

BatchDeleteMetastorePartitions

BatchUpdateMetastorePartitions

ListMetastorePartitions

StreamMetastorePartitions

BatchCreateMetastorePartitionsRequest

Request message for BatchCreateMetastorePartitions.

Fields

Fields
`parent`	`string` Required. Reference to the table to where the metastore partitions to be added, in the format of projects/{project}/locations/{location}/datasets/{dataset}/tables/{table}.
`requests[]`	`CreateMetastorePartitionRequest` Required. Requests to add metastore partitions to the table.
`skip_existing_partitions`	`bool` Optional. Mimics the ifNotExists flag in IMetaStoreClient add_partitions(..). If the flag is set to false, the server will return ALREADY_EXISTS if any partition already exists. If the flag is set to true, the server will skip existing partitions and insert only the non-existing partitions. A maximum of 900 partitions can be inserted in a batch.

parent

string

Required. Reference to the table to where the metastore partitions to be added, in the format of projects/{project}/locations/{location}/datasets/{dataset}/tables/{table}.

requests[]

CreateMetastorePartitionRequest

Required. Requests to add metastore partitions to the table.

skip_existing_partitions

bool

Optional. Mimics the ifNotExists flag in IMetaStoreClient add_partitions(..). If the flag is set to false, the server will return ALREADY_EXISTS if any partition already exists. If the flag is set to true, the server will skip existing partitions and insert only the non-existing partitions. A maximum of 900 partitions can be inserted in a batch.

BatchCreateMetastorePartitionsResponse

Response message for BatchCreateMetastorePartitions.

Fields

Fields
`partitions[]`	`MetastorePartition` The list of metastore partitions that have been created.

partitions[]

MetastorePartition

The list of metastore partitions that have been created.

BatchDeleteMetastorePartitionsRequest

Request message for BatchDeleteMetastorePartitions. The MetastorePartition is uniquely identified by values, which is an ordered list. Hence, there is no separate name or partition id field.

Fields

Fields
`parent`	`string` Required. Reference to the table to which these metastore partitions belong, in the format of projects/{project}/locations/{location}/datasets/{dataset}/tables/{table}.
`partition_values[]`	`MetastorePartitionValues` Required. The list of metastore partitions (identified by its values) to be deleted. A maximum of 900 partitions can be deleted in a batch.

parent

string

Required. Reference to the table to which these metastore partitions belong, in the format of projects/{project}/locations/{location}/datasets/{dataset}/tables/{table}.

partition_values[]

MetastorePartitionValues

Required. The list of metastore partitions (identified by its values) to be deleted. A maximum of 900 partitions can be deleted in a batch.

BatchSizeTooLargeError

Structured custom error message for batch size too large error. The error can be attached as error details in the returned rpc Status for more structured error handling in the client.

Fields

Fields
`max_batch_size`	`int64` The maximum number of items that are supported in a single batch. This is returned as a hint to the client to adjust the batch size.
`error_message`	`string` Optional. The error message that is returned to the client.

max_batch_size

int64

The maximum number of items that are supported in a single batch. This is returned as a hint to the client to adjust the batch size.

error_message

string

Optional. The error message that is returned to the client.

BatchUpdateMetastorePartitionsRequest

Request message for BatchUpdateMetastorePartitions.

Fields

Fields
`parent`	`string` Required. Reference to the table to which these metastore partitions belong, in the format of projects/{project}/locations/{location}/datasets/{dataset}/tables/{table}.
`requests[]`	`UpdateMetastorePartitionRequest` Required. Requests to update metastore partitions in the table.

parent

string

Required. Reference to the table to which these metastore partitions belong, in the format of projects/{project}/locations/{location}/datasets/{dataset}/tables/{table}.

requests[]

UpdateMetastorePartitionRequest

Required. Requests to update metastore partitions in the table.

BatchUpdateMetastorePartitionsResponse

Response message for BatchUpdateMetastorePartitions.

Fields

Fields
`partitions[]`	`MetastorePartition` The list of metastore partitions that have been updated. A maximum of 900 partitions can be updated in a batch.

partitions[]

MetastorePartition

The list of metastore partitions that have been updated. A maximum of 900 partitions can be updated in a batch.

CreateMetastorePartitionRequest

Request message for CreateMetastorePartition. The MetastorePartition is uniquely identified by values, which is an ordered list. Hence, there is no separate name or partition id field.

Fields

Fields
`parent`	`string` Required. Reference to the table to where the metastore partition to be added, in the format of projects/{project}/databases/{databases}/tables/{table}.
`metastore_partition`	`MetastorePartition` Required. The metastore partition to be added.

parent

string

Required. Reference to the table to where the metastore partition to be added, in the format of projects/{project}/databases/{databases}/tables/{table}.

metastore_partition

MetastorePartition

Required. The metastore partition to be added.

FieldSchema

Schema description of a metastore partition column.

Fields

Fields
`name`	`string` Required. The name of the column. The maximum length of the name is 1024 characters
`type`	`string` Required. The type of the metastore partition column. Maximum allowed length is 1024 characters.

name

string

Required. The name of the column. The maximum length of the name is 1024 characters

type

string

Required. The type of the metastore partition column. Maximum allowed length is 1024 characters.

ListMetastorePartitionsRequest

Request message for ListMetastorePartitions.

Fields

Fields
`parent`	`string` Required. Reference to the table to which these metastore partitions belong, in the format of projects/{project}/locations/{location}/datasets/{dataset}/tables/{table}.
`filter`	`string` Optional. SQL text filtering statement, similar to a WHERE clause in a query. Only supports single-row expressions. Aggregate functions are not supported. Examples: "int_field > 5" "date_field = CAST('2014-9-27' as DATE)" "nullable_field is not NULL" "st_equals(geo_field, st_geofromtext("POINT(2, 2)"))" "numeric_field BETWEEN 1.0 AND 5.0" Restricted to a maximum length for 1 MB.

parent

string

Required. Reference to the table to which these metastore partitions belong, in the format of projects/{project}/locations/{location}/datasets/{dataset}/tables/{table}.

filter

string

Optional. SQL text filtering statement, similar to a WHERE clause in a query. Only supports single-row expressions. Aggregate functions are not supported.

Examples: "int_field > 5" "date_field = CAST('2014-9-27' as DATE)" "nullable_field is not NULL" "st_equals(geo_field, st_geofromtext("POINT(2, 2)"))" "numeric_field BETWEEN 1.0 AND 5.0" Restricted to a maximum length for 1 MB.

ListMetastorePartitionsResponse

Response message for ListMetastorePartitions.

Fields

Fields
Union field `response`. The response depends on the number of metastore partitions to be returned; it can be a list of partitions or a list of ReadStream objects. For the second situation, the BigQuery Read API ReadRows method must be used to stream the data and convert it into a list of partitions. `response` can be only one of the following:
`partitions`	`MetastorePartitionList` The list of partitions.
`streams`	`StreamList` The list of streams.

Union field response. The response depends on the number of metastore partitions to be returned; it can be a list of partitions or a list of ReadStream objects. For the second situation, the BigQuery Read API ReadRows method must be used to stream the data and convert it into a list of partitions. response can be only one of the following:

partitions

MetastorePartitionList

The list of partitions.

streams

StreamList

The list of streams.

MetastorePartition

Information about a Hive partition.

Fields
`values[]`	`string` Required. Represents the values of the partition keys, where each value corresponds to a specific partition key in the order in which the keys are defined. Each value is limited to 1024 characters.
`create_time`	`Timestamp` Output only. The creation time of the partition.
`storage_descriptor`	`StorageDescriptor` Optional. Contains information about the physical storage of the data in the partition.
`parameters`	`map<string, string>` Optional. Additional parameters or metadata associated with the partition. Maximum size 10 KiB.
`fields[]`	`FieldSchema` Optional. List of columns.

MetastorePartitionList

List of metastore partitions.

Fields

Fields
`partitions[]`	`MetastorePartition` Required. List of partitions.

partitions[]

MetastorePartition

Required. List of partitions.

MetastorePartitionValues

Represents the values of a metastore partition.

Fields

Fields
`values[]`	`string` Required. The values of the partition keys, where each value corresponds to a specific partition key in the order in which the keys are defined.

values[]

string

Required. The values of the partition keys, where each value corresponds to a specific partition key in the order in which the keys are defined.

ReadStream

Information about a single stream that is used to read partitions.

Fields

Fields
`name`	`string` Output only. Identifier. Name of the stream, in the form `projects/{project_id}/locations/{location}/sessions/{session_id}/streams/{stream_id}`.

name

string

Output only. Identifier. Name of the stream, in the form projects/{project_id}/locations/{location}/sessions/{session_id}/streams/{stream_id}.

SerDeInfo

Serializer and deserializer information.

Fields

Fields
`name`	`string` Optional. Name of the SerDe. The maximum length is 256 characters.
`serialization_library`	`string` Required. Specifies a fully-qualified class name of the serialization library that is responsible for the translation of data between table representation and the underlying low-level input and output format structures. The maximum length is 256 characters.
`parameters`	`map<string, string>` Optional. Key-value pairs that define the initialization parameters for the serialization library. Maximum size 10 Kib.

name

string

Optional. Name of the SerDe. The maximum length is 256 characters.

serialization_library

string

Required. Specifies a fully-qualified class name of the serialization library that is responsible for the translation of data between table representation and the underlying low-level input and output format structures. The maximum length is 256 characters.

parameters

map<string, string>

Optional. Key-value pairs that define the initialization parameters for the serialization library. Maximum size 10 Kib.

StorageDescriptor

Contains information about the physical storage of the data in the metastore partition.

Fields
`location_uri`	`string` Optional. The physical location of the metastore partition (e.g. `gs://spark-dataproc-data/pangea-data/case_sensitive/` or `gs://spark-dataproc-data/pangea-data/*`).
`input_format`	`string` Optional. Specifies the fully qualified class name of the InputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat"). The maximum length is 128 characters.
`output_format`	`string` Optional. Specifies the fully qualified class name of the OutputFormat (e.g. "org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat"). The maximum length is 128 characters.
`serde_info`	`SerDeInfo` Optional. Serializer and deserializer information.

StreamList

List of streams.

Fields

Fields
`streams[]`	`ReadStream` Output only. List of streams.

streams[]

ReadStream

Output only. List of streams.

StreamMetastorePartitionsRequest

The top-level message sent by the client to the [Partitions.StreamMetastorePartitions][] method. Follows the default gRPC streaming maximum size of 4 MB.

Fields

Fields
`parent`	`string` Required. Reference to the table to where the partition to be added, in the format of projects/{project}/locations/{location}/datasets/{dataset}/tables/{table}.
`metastore_partitions[]`	`MetastorePartition` Optional. A list of metastore partitions to be added to the table.
`skip_existing_partitions`	`bool` Optional. Mimics the ifNotExists flag in IMetaStoreClient add_partitions(..). If the flag is set to false, the server will return ALREADY_EXISTS on commit if any partition already exists. If the flag is set to true: 1) the server will skip existing partitions insert only the non-existing partitions as part of the commit. 2) The client must set the `skip_existing_partitions` field to true for all requests in the stream.

parent

string

Required. Reference to the table to where the partition to be added, in the format of projects/{project}/locations/{location}/datasets/{dataset}/tables/{table}.

metastore_partitions[]

MetastorePartition

Optional. A list of metastore partitions to be added to the table.

skip_existing_partitions

bool

Optional. Mimics the ifNotExists flag in IMetaStoreClient add_partitions(..). If the flag is set to false, the server will return ALREADY_EXISTS on commit if any partition already exists. If the flag is set to true: 1) the server will skip existing partitions insert only the non-existing partitions as part of the commit. 2) The client must set the skip_existing_partitions field to true for all requests in the stream.

StreamMetastorePartitionsResponse

This is the response message sent by the server to the client for the [Partitions.StreamMetastorePartitions][] method when the commit is successful. Server will close the stream after sending this message.

Fields

Fields
`total_partitions_streamed_count`	`int64` Total count of partitions streamed by the client during the lifetime of the stream. This is only set in the final response message before closing the stream.
`total_partitions_inserted_count`	`int64` Total count of partitions inserted by the server during the lifetime of the stream. This is only set in the final response message before closing the stream.

total_partitions_streamed_count

int64

Total count of partitions streamed by the client during the lifetime of the stream. This is only set in the final response message before closing the stream.

total_partitions_inserted_count

int64

Total count of partitions inserted by the server during the lifetime of the stream. This is only set in the final response message before closing the stream.

UpdateMetastorePartitionRequest

Request message for UpdateMetastorePartition.

Fields

Fields
`metastore_partition`	`MetastorePartition` Required. The metastore partition to be updated.
`update_mask`	`FieldMask` Optional. The list of fields to update.

metastore_partition

MetastorePartition

Required. The metastore partition to be updated.

update_mask

FieldMask

Optional. The list of fields to update.