Reference documentation and code samples for the Gemini Enterprise for Customer Experience V1BETA API class Google::Cloud::Ces::V1beta::Evaluation::GoldenExpectation.
Represents a single, checkable requirement.
Inherits
- Object
Extended By
- Google::Protobuf::MessageExts::ClassMethods
Includes
- Google::Protobuf::MessageExts
Methods
#agent_response
def agent_response() -> ::Google::Cloud::Ces::V1beta::Message-
(::Google::Cloud::Ces::V1beta::Message) — Optional. Check that the agent responded with the correct response. The
role "agent" is implied.
Note: The following fields are mutually exclusive:
agent_response,tool_call,tool_response,agent_transfer,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
#agent_response=
def agent_response=(value) -> ::Google::Cloud::Ces::V1beta::Message-
value (::Google::Cloud::Ces::V1beta::Message) — Optional. Check that the agent responded with the correct response. The
role "agent" is implied.
Note: The following fields are mutually exclusive:
agent_response,tool_call,tool_response,agent_transfer,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
-
(::Google::Cloud::Ces::V1beta::Message) — Optional. Check that the agent responded with the correct response. The
role "agent" is implied.
Note: The following fields are mutually exclusive:
agent_response,tool_call,tool_response,agent_transfer,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
#agent_response_hallucination_metrics_config_override
def agent_response_hallucination_metrics_config_override() -> ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::HallucinationMetricsConfig- (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::HallucinationMetricsConfig) — Optional. Overrides for agent_response hallucination metrics.
#agent_response_hallucination_metrics_config_override=
def agent_response_hallucination_metrics_config_override=(value) -> ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::HallucinationMetricsConfig- value (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::HallucinationMetricsConfig) — Optional. Overrides for agent_response hallucination metrics.
- (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::HallucinationMetricsConfig) — Optional. Overrides for agent_response hallucination metrics.
#agent_response_semantic_similarity_metrics_config_override
def agent_response_semantic_similarity_metrics_config_override() -> ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::SemanticSimilarityMetricsConfig- (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::SemanticSimilarityMetricsConfig) — Optional. Overrides for agent_response semantic similarity metrics.
#agent_response_semantic_similarity_metrics_config_override=
def agent_response_semantic_similarity_metrics_config_override=(value) -> ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::SemanticSimilarityMetricsConfig- value (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::SemanticSimilarityMetricsConfig) — Optional. Overrides for agent_response semantic similarity metrics.
- (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::SemanticSimilarityMetricsConfig) — Optional. Overrides for agent_response semantic similarity metrics.
#agent_transfer
def agent_transfer() -> ::Google::Cloud::Ces::V1beta::AgentTransfer-
(::Google::Cloud::Ces::V1beta::AgentTransfer) — Optional. Check that the agent transferred the conversation to a
different agent.
Note: The following fields are mutually exclusive:
agent_transfer,tool_call,tool_response,agent_response,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
#agent_transfer=
def agent_transfer=(value) -> ::Google::Cloud::Ces::V1beta::AgentTransfer-
value (::Google::Cloud::Ces::V1beta::AgentTransfer) — Optional. Check that the agent transferred the conversation to a
different agent.
Note: The following fields are mutually exclusive:
agent_transfer,tool_call,tool_response,agent_response,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
-
(::Google::Cloud::Ces::V1beta::AgentTransfer) — Optional. Check that the agent transferred the conversation to a
different agent.
Note: The following fields are mutually exclusive:
agent_transfer,tool_call,tool_response,agent_response,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
#comparison_type
def comparison_type() -> ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::ComparisonType- (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::ComparisonType) — Optional. The comparison type to use for the expectation check.
#comparison_type=
def comparison_type=(value) -> ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::ComparisonType- value (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::ComparisonType) — Optional. The comparison type to use for the expectation check.
- (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::ComparisonType) — Optional. The comparison type to use for the expectation check.
#expectation_level_metrics_thresholds_override
def expectation_level_metrics_thresholds_override() -> ::Google::Cloud::Ces::V1beta::EvaluationMetricsThresholds::GoldenEvaluationMetricsThresholds::ExpectationLevelMetricsThresholds- (::Google::Cloud::Ces::V1beta::EvaluationMetricsThresholds::GoldenEvaluationMetricsThresholds::ExpectationLevelMetricsThresholds) — Optional. Overrides metrics at the step level.
#expectation_level_metrics_thresholds_override=
def expectation_level_metrics_thresholds_override=(value) -> ::Google::Cloud::Ces::V1beta::EvaluationMetricsThresholds::GoldenEvaluationMetricsThresholds::ExpectationLevelMetricsThresholds- value (::Google::Cloud::Ces::V1beta::EvaluationMetricsThresholds::GoldenEvaluationMetricsThresholds::ExpectationLevelMetricsThresholds) — Optional. Overrides metrics at the step level.
- (::Google::Cloud::Ces::V1beta::EvaluationMetricsThresholds::GoldenEvaluationMetricsThresholds::ExpectationLevelMetricsThresholds) — Optional. Overrides metrics at the step level.
#mock_tool_response
def mock_tool_response() -> ::Google::Cloud::Ces::V1beta::ToolResponse-
(::Google::Cloud::Ces::V1beta::ToolResponse) — Optional. The tool response to mock, with the parameters of interest
specified. Any parameters not specified will be hallucinated by the
LLM.
Note: The following fields are mutually exclusive:
mock_tool_response,tool_call,tool_response,agent_response,agent_transfer,updated_variables,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
#mock_tool_response=
def mock_tool_response=(value) -> ::Google::Cloud::Ces::V1beta::ToolResponse-
value (::Google::Cloud::Ces::V1beta::ToolResponse) — Optional. The tool response to mock, with the parameters of interest
specified. Any parameters not specified will be hallucinated by the
LLM.
Note: The following fields are mutually exclusive:
mock_tool_response,tool_call,tool_response,agent_response,agent_transfer,updated_variables,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
-
(::Google::Cloud::Ces::V1beta::ToolResponse) — Optional. The tool response to mock, with the parameters of interest
specified. Any parameters not specified will be hallucinated by the
LLM.
Note: The following fields are mutually exclusive:
mock_tool_response,tool_call,tool_response,agent_response,agent_transfer,updated_variables,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
#no_tool_calls
def no_tool_calls() -> ::Boolean-
(::Boolean) — Optional. Check that no tools were called during this turn.
Note: The following fields are mutually exclusive:
no_tool_calls,tool_call,tool_response,agent_response,agent_transfer,updated_variables,mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.
#no_tool_calls=
def no_tool_calls=(value) -> ::Boolean-
value (::Boolean) — Optional. Check that no tools were called during this turn.
Note: The following fields are mutually exclusive:
no_tool_calls,tool_call,tool_response,agent_response,agent_transfer,updated_variables,mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.
-
(::Boolean) — Optional. Check that no tools were called during this turn.
Note: The following fields are mutually exclusive:
no_tool_calls,tool_call,tool_response,agent_response,agent_transfer,updated_variables,mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.
#note
def note() -> ::String- (::String) — Optional. A note for this requirement, useful in reporting when specific checks fail. E.g., "Check_Payment_Tool_Called".
#note=
def note=(value) -> ::String- value (::String) — Optional. A note for this requirement, useful in reporting when specific checks fail. E.g., "Check_Payment_Tool_Called".
- (::String) — Optional. A note for this requirement, useful in reporting when specific checks fail. E.g., "Check_Payment_Tool_Called".
#skip_evaluation
def skip_evaluation() -> ::Boolean- (::Boolean) — Optional. If set to true, this specific expectation will not be evaluated.
#skip_evaluation=
def skip_evaluation=(value) -> ::Boolean- value (::Boolean) — Optional. If set to true, this specific expectation will not be evaluated.
- (::Boolean) — Optional. If set to true, this specific expectation will not be evaluated.
#tool_call
def tool_call() -> ::Google::Cloud::Ces::V1beta::ToolCall-
(::Google::Cloud::Ces::V1beta::ToolCall) — Optional. Check that a specific tool was called with the parameters.
Note: The following fields are mutually exclusive:
tool_call,tool_response,agent_response,agent_transfer,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
#tool_call=
def tool_call=(value) -> ::Google::Cloud::Ces::V1beta::ToolCall-
value (::Google::Cloud::Ces::V1beta::ToolCall) — Optional. Check that a specific tool was called with the parameters.
Note: The following fields are mutually exclusive:
tool_call,tool_response,agent_response,agent_transfer,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
-
(::Google::Cloud::Ces::V1beta::ToolCall) — Optional. Check that a specific tool was called with the parameters.
Note: The following fields are mutually exclusive:
tool_call,tool_response,agent_response,agent_transfer,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
#tool_response
def tool_response() -> ::Google::Cloud::Ces::V1beta::ToolResponse-
(::Google::Cloud::Ces::V1beta::ToolResponse) — Optional. Check that a specific tool had the expected response.
Note: The following fields are mutually exclusive:
tool_response,tool_call,agent_response,agent_transfer,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
#tool_response=
def tool_response=(value) -> ::Google::Cloud::Ces::V1beta::ToolResponse-
value (::Google::Cloud::Ces::V1beta::ToolResponse) — Optional. Check that a specific tool had the expected response.
Note: The following fields are mutually exclusive:
tool_response,tool_call,agent_response,agent_transfer,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
-
(::Google::Cloud::Ces::V1beta::ToolResponse) — Optional. Check that a specific tool had the expected response.
Note: The following fields are mutually exclusive:
tool_response,tool_call,agent_response,agent_transfer,updated_variables,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
#updated_variables
def updated_variables() -> ::Google::Protobuf::Struct-
(::Google::Protobuf::Struct) — Optional. Check that the agent updated the session variables to the
expected values. Used to also capture agent variable updates for golden
evals.
Note: The following fields are mutually exclusive:
updated_variables,tool_call,tool_response,agent_response,agent_transfer,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
#updated_variables=
def updated_variables=(value) -> ::Google::Protobuf::Struct-
value (::Google::Protobuf::Struct) — Optional. Check that the agent updated the session variables to the
expected values. Used to also capture agent variable updates for golden
evals.
Note: The following fields are mutually exclusive:
updated_variables,tool_call,tool_response,agent_response,agent_transfer,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.
-
(::Google::Protobuf::Struct) — Optional. Check that the agent updated the session variables to the
expected values. Used to also capture agent variable updates for golden
evals.
Note: The following fields are mutually exclusive:
updated_variables,tool_call,tool_response,agent_response,agent_transfer,mock_tool_response,no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.