This guide provides an overview of using Cloud Run to host apps, run inference, and build AI workflows.
Cloud Run for hosting AI applications, agents, and scalable API endpoints
Cloud Run provides a fully managed platform that scales your AI apps and workloads.
When you host AI apps on Cloud Run, you typically have the following architectural components:
- Serving and orchestration: You deploy your application code or container to Cloud Run.
- AI models: You use Google's AI models, open-source models, or custom models with your app.
- Integrations: You can connect to Cloud de Confiance services or third-party services for memory, databases, storage, security, and more.
- Tools: You can connect to tools for other tasks and operations.
The following diagram shows a high-level overview of using Cloud Run as a hosting platform for AI apps:
As shown in the diagram:
Within the serving and orchestration layer, a Cloud Run service acts as a scalable API endpoint for your application's core logic. It efficiently manages multiple concurrent users through automatic, on-demand, and rapid scaling of instances.
You bring your container to deploy to Cloud Run by packaging your application and its dependency into a container.
Your AI app acts as a scalable API endpoint that handles incoming requests and sends data to a pre-trained AI model for processing, then returns the results.
If you have a custom model that you've trained yourself, you can also use that model with your Cloud Run resource.
Cloud de Confiance by S3NS offers a wide variety of solutions to support your AI application's infrastructure.You can Also integrate with third-party solutions.
Tools let your AI apps and models interact with services, APIs, or websites that run externally or on Cloud Run.
For example, if your AI app is an AI agent, your agent might send a request to an MCP server to execute an external tool, or use tools running in your container, like code execution, computer use, information retrieval, and so forth.