Cloud Dataproc API C++ Client Library

An idiomatic C++ client library for the Cloud Dataproc API, a managed Apache Spark and Apache Hadoop service that lets you take advantage of open source data tools for batch processing, querying, streaming, and machine learning. This library allows you to manage Cloud Dataproc resources, but it does not provide APIs to run C++ applications in Cloud Dataproc.

While this library is GA, please note Google Cloud C++ client libraries do not follow Semantic Versioning.

Quickstart

The following shows the code that you'll run in the google/cloud/dataproc/quickstart/ directory, which should give you a taste of the Cloud Dataproc API C++ client library API.

#include "google/cloud/dataproc/v1/cluster_controller_client.h"
#include "google/cloud/common_options.h"
#include <iostream>

int main(int argc, char* argv[]) try {
  if (argc != 3) {
    std::cerr << "Usage: " << argv[0] << " project-id region\n";
    return 1;
  }
  std::string const project_id = argv[1];
  std::string const region = argv[2];

  namespace dataproc = ::google::cloud::dataproc_v1;

  auto client = dataproc::ClusterControllerClient(
      dataproc::MakeClusterControllerConnection(region == "global" ? ""
                                                                   : region));

  for (auto c : client.ListClusters(project_id, region)) {
    if (!c) throw std::move(c).status();
    std::cout << c->cluster_name() << "\n";
  }

  return 0;
} catch (google::cloud::Status const& status) {
  std::cerr << "google::cloud::Status thrown: " << status << "\n";
  return 1;
}

Main classes

This library offers multiple *Client classes, which are listed below. Each one of these classes exposes all the RPCs for a gRPC service as member functions of the class. This library groups multiple gRPC services because they are part of the same product or are often used together. A typical example may be the administrative and data plane operations for a single product.

The library also has other classes that provide helpers, configuration parameters, and infrastructure to mock the *Client classes when testing your application.

Retry, Backoff, and Idempotency Policies.

The library automatically retries requests that fail with transient errors, and uses exponential backoff to backoff between retries. Application developers can override the default policies.

More Information