Class CloudStorage (2.32.0) 

CloudStorage(mapping=None, *, ignore_unknown_fields=False, **kwargs)

Ingestion settings for Cloud Storage.

This message has oneof_ fields (mutually exclusive fields). For each oneof, at most one member field can be set at the same time. Setting any member of the oneof automatically clears all other members.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields

Attributes
Name Description
state :noindex: google.pubsub_v1.types.IngestionDataSourceSettings.CloudStorage.State
Output only. An output-only field that indicates the state of the Cloud Storage ingestion source.
bucket :noindex: str
Optional. Cloud Storage bucket. The bucket name must be without any prefix like "gs://". See the [bucket naming requirements] (https://cloud.google.com/storage/docs/buckets#naming).
text_format :noindex: google.pubsub_v1.types.IngestionDataSourceSettings.CloudStorage.TextFormat
Optional. Data from Cloud Storage will be interpreted as text. This field is a member of oneof_ input_format.
avro_format :noindex: google.pubsub_v1.types.IngestionDataSourceSettings.CloudStorage.AvroFormat
Optional. Data from Cloud Storage will be interpreted in Avro format. This field is a member of oneof_ input_format.
pubsub_avro_format :noindex: google.pubsub_v1.types.IngestionDataSourceSettings.CloudStorage.PubSubAvroFormat
Optional. It will be assumed data from Cloud Storage was written via `Cloud Storage subscriptions
minimum_object_create_time :noindex: google.protobuf.timestamp_pb2.Timestamp
Optional. Only objects with a larger or equal creation timestamp will be ingested.
match_glob :noindex: str
Optional. Glob pattern used to match objects that will be ingested. If unset, all objects will be ingested. See the `supported patterns

Classes

AvroFormat

AvroFormat(mapping=None, *, ignore_unknown_fields=False, **kwargs)

Configuration for reading Cloud Storage data in Avro binary format. The bytes of each object will be set to the data field of a Pub/Sub message.

PubSubAvroFormat

PubSubAvroFormat(mapping=None, *, ignore_unknown_fields=False, **kwargs)

Configuration for reading Cloud Storage data written via Cloud Storage subscriptions <https://cloud.google.com/pubsub/docs/cloudstorage>__. The data and attributes fields of the originally exported Pub/Sub message will be restored when publishing.

State

State(value)

Possible states for ingestion from Cloud Storage.

    - storage.objects.list: to list the objects in a bucket.
    - storage.objects.get: to read the objects in a bucket.
    - storage.buckets.get: to verify the bucket exists.
PUBLISH_PERMISSION_DENIED (3):
    Permission denied encountered while publishing to the topic.
    This can happen if the Pub/Sub SA has not been granted the
    `appropriate publish
    permissions <https://cloud.google.com/pubsub/docs/access-control#pubsub.publisher>`__
BUCKET_NOT_FOUND (4):
    The provided Cloud Storage bucket doesn't
    exist.
TOO_MANY_OBJECTS (5):
    The Cloud Storage bucket has too many
    objects, ingestion will be paused.

TextFormat

TextFormat(mapping=None, *, ignore_unknown_fields=False, **kwargs)

Configuration for reading Cloud Storage data in text format. Each line of text as specified by the delimiter will be set to the data field of a Pub/Sub message.

.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields