Class EvaluationParserConfig.CustomCodeParserConfig.Builder (3.100.0) 

public static final class EvaluationParserConfig.CustomCodeParserConfig.Builder extends GeneratedMessage.Builder<EvaluationParserConfig.CustomCodeParserConfig.Builder> implements EvaluationParserConfig.CustomCodeParserConfigOrBuilder

Configuration for parsing the LLM response using custom code.

Protobuf type google.cloud.aiplatform.v1beta1.EvaluationParserConfig.CustomCodeParserConfig

Inheritance

java.lang.Object > AbstractMessageLite.Builder<MessageType,BuilderType> > AbstractMessage.Builder<BuilderType> > GeneratedMessage.Builder > EvaluationParserConfig.CustomCodeParserConfig.Builder

Static Methods

getDescriptor()

public static final Descriptors.Descriptor getDescriptor()
Returns
Type Description
Descriptor

Methods

build()

public EvaluationParserConfig.CustomCodeParserConfig build()
Returns
Type Description
EvaluationParserConfig.CustomCodeParserConfig

buildPartial()

public EvaluationParserConfig.CustomCodeParserConfig buildPartial()
Returns
Type Description
EvaluationParserConfig.CustomCodeParserConfig

clear()

public EvaluationParserConfig.CustomCodeParserConfig.Builder clear()
Returns
Type Description
EvaluationParserConfig.CustomCodeParserConfig.Builder
Overrides

clearParsingFunction()

public EvaluationParserConfig.CustomCodeParserConfig.Builder clearParsingFunction()

Required. Python function for parsing results. The function should be defined within this string.

The function takes a list of strings (LLM responses) and should return either a list of dictionaries (for rubrics) or a single dictionary (for a metric result).

Example function signature: def parse(responses: list[str]) -> list[dict[str, Any]] | dict[str, Any]:

When parsing rubrics, return a list of dictionaries, where each dictionary represents a Rubric. Example for rubrics: [ { "content": {"property": {"description": "The response is factual."}}, "type": "FACTUALITY", "importance": "HIGH" }, { "content": {"property": {"description": "The response is fluent."}}, "type": "FLUENCY", "importance": "MEDIUM" } ]

When parsing critique results, return a dictionary representing a MetricResult. Example for a metric result: { "score": 0.8, "explanation": "The model followed most instructions.", "rubric_verdicts": [...] }

... code for result extraction and aggregation

optional string parsing_function = 1 [(.google.api.field_behavior) = REQUIRED];

Returns
Type Description
EvaluationParserConfig.CustomCodeParserConfig.Builder

This builder for chaining.

getDefaultInstanceForType()

public EvaluationParserConfig.CustomCodeParserConfig getDefaultInstanceForType()
Returns
Type Description
EvaluationParserConfig.CustomCodeParserConfig

getDescriptorForType()

public Descriptors.Descriptor getDescriptorForType()
Returns
Type Description
Descriptor
Overrides

getParsingFunction()

public String getParsingFunction()

Required. Python function for parsing results. The function should be defined within this string.

The function takes a list of strings (LLM responses) and should return either a list of dictionaries (for rubrics) or a single dictionary (for a metric result).

Example function signature: def parse(responses: list[str]) -> list[dict[str, Any]] | dict[str, Any]:

When parsing rubrics, return a list of dictionaries, where each dictionary represents a Rubric. Example for rubrics: [ { "content": {"property": {"description": "The response is factual."}}, "type": "FACTUALITY", "importance": "HIGH" }, { "content": {"property": {"description": "The response is fluent."}}, "type": "FLUENCY", "importance": "MEDIUM" } ]

When parsing critique results, return a dictionary representing a MetricResult. Example for a metric result: { "score": 0.8, "explanation": "The model followed most instructions.", "rubric_verdicts": [...] }

... code for result extraction and aggregation

optional string parsing_function = 1 [(.google.api.field_behavior) = REQUIRED];

Returns
Type Description
String

The parsingFunction.

getParsingFunctionBytes()

public ByteString getParsingFunctionBytes()

Required. Python function for parsing results. The function should be defined within this string.

The function takes a list of strings (LLM responses) and should return either a list of dictionaries (for rubrics) or a single dictionary (for a metric result).

Example function signature: def parse(responses: list[str]) -> list[dict[str, Any]] | dict[str, Any]:

When parsing rubrics, return a list of dictionaries, where each dictionary represents a Rubric. Example for rubrics: [ { "content": {"property": {"description": "The response is factual."}}, "type": "FACTUALITY", "importance": "HIGH" }, { "content": {"property": {"description": "The response is fluent."}}, "type": "FLUENCY", "importance": "MEDIUM" } ]

When parsing critique results, return a dictionary representing a MetricResult. Example for a metric result: { "score": 0.8, "explanation": "The model followed most instructions.", "rubric_verdicts": [...] }

... code for result extraction and aggregation

optional string parsing_function = 1 [(.google.api.field_behavior) = REQUIRED];

Returns
Type Description
ByteString

The bytes for parsingFunction.

hasParsingFunction()

public boolean hasParsingFunction()

Required. Python function for parsing results. The function should be defined within this string.

The function takes a list of strings (LLM responses) and should return either a list of dictionaries (for rubrics) or a single dictionary (for a metric result).

Example function signature: def parse(responses: list[str]) -> list[dict[str, Any]] | dict[str, Any]:

When parsing rubrics, return a list of dictionaries, where each dictionary represents a Rubric. Example for rubrics: [ { "content": {"property": {"description": "The response is factual."}}, "type": "FACTUALITY", "importance": "HIGH" }, { "content": {"property": {"description": "The response is fluent."}}, "type": "FLUENCY", "importance": "MEDIUM" } ]

When parsing critique results, return a dictionary representing a MetricResult. Example for a metric result: { "score": 0.8, "explanation": "The model followed most instructions.", "rubric_verdicts": [...] }

... code for result extraction and aggregation

optional string parsing_function = 1 [(.google.api.field_behavior) = REQUIRED];

Returns
Type Description
boolean

Whether the parsingFunction field is set.

internalGetFieldAccessorTable()

protected GeneratedMessage.FieldAccessorTable internalGetFieldAccessorTable()
Returns
Type Description
FieldAccessorTable
Overrides

isInitialized()

public final boolean isInitialized()
Returns
Type Description
boolean
Overrides

mergeFrom(EvaluationParserConfig.CustomCodeParserConfig other)

public EvaluationParserConfig.CustomCodeParserConfig.Builder mergeFrom(EvaluationParserConfig.CustomCodeParserConfig other)
Parameter
Name Description
other EvaluationParserConfig.CustomCodeParserConfig
Returns
Type Description
EvaluationParserConfig.CustomCodeParserConfig.Builder

mergeFrom(CodedInputStream input, ExtensionRegistryLite extensionRegistry)

public EvaluationParserConfig.CustomCodeParserConfig.Builder mergeFrom(CodedInputStream input, ExtensionRegistryLite extensionRegistry)
Parameters
Name Description
input CodedInputStream
extensionRegistry ExtensionRegistryLite
Returns
Type Description
EvaluationParserConfig.CustomCodeParserConfig.Builder
Overrides
Exceptions
Type Description
IOException

mergeFrom(Message other)

public EvaluationParserConfig.CustomCodeParserConfig.Builder mergeFrom(Message other)
Parameter
Name Description
other Message
Returns
Type Description
EvaluationParserConfig.CustomCodeParserConfig.Builder
Overrides

setParsingFunction(String value)

public EvaluationParserConfig.CustomCodeParserConfig.Builder setParsingFunction(String value)

Required. Python function for parsing results. The function should be defined within this string.

The function takes a list of strings (LLM responses) and should return either a list of dictionaries (for rubrics) or a single dictionary (for a metric result).

Example function signature: def parse(responses: list[str]) -> list[dict[str, Any]] | dict[str, Any]:

When parsing rubrics, return a list of dictionaries, where each dictionary represents a Rubric. Example for rubrics: [ { "content": {"property": {"description": "The response is factual."}}, "type": "FACTUALITY", "importance": "HIGH" }, { "content": {"property": {"description": "The response is fluent."}}, "type": "FLUENCY", "importance": "MEDIUM" } ]

When parsing critique results, return a dictionary representing a MetricResult. Example for a metric result: { "score": 0.8, "explanation": "The model followed most instructions.", "rubric_verdicts": [...] }

... code for result extraction and aggregation

optional string parsing_function = 1 [(.google.api.field_behavior) = REQUIRED];

Parameter
Name Description
value String

The parsingFunction to set.
Returns
Type Description
EvaluationParserConfig.CustomCodeParserConfig.Builder

This builder for chaining.

setParsingFunctionBytes(ByteString value)

public EvaluationParserConfig.CustomCodeParserConfig.Builder setParsingFunctionBytes(ByteString value)

Required. Python function for parsing results. The function should be defined within this string.

The function takes a list of strings (LLM responses) and should return either a list of dictionaries (for rubrics) or a single dictionary (for a metric result).

Example function signature: def parse(responses: list[str]) -> list[dict[str, Any]] | dict[str, Any]:

When parsing rubrics, return a list of dictionaries, where each dictionary represents a Rubric. Example for rubrics: [ { "content": {"property": {"description": "The response is factual."}}, "type": "FACTUALITY", "importance": "HIGH" }, { "content": {"property": {"description": "The response is fluent."}}, "type": "FLUENCY", "importance": "MEDIUM" } ]

When parsing critique results, return a dictionary representing a MetricResult. Example for a metric result: { "score": 0.8, "explanation": "The model followed most instructions.", "rubric_verdicts": [...] }

... code for result extraction and aggregation

optional string parsing_function = 1 [(.google.api.field_behavior) = REQUIRED];

Parameter
Name Description
value ByteString

The bytes for parsingFunction to set.
Returns
Type Description
EvaluationParserConfig.CustomCodeParserConfig.Builder

This builder for chaining.