Vertex AI v1 API - Class SpeculativeDecodingSpec.Types.NgramSpeculation (3.25.0) 

public sealed class SpeculativeDecodingSpec.Types.NgramSpeculation : IMessage<SpeculativeDecodingSpec.Types.NgramSpeculation>, IEquatable<SpeculativeDecodingSpec.Types.NgramSpeculation>, IDeepCloneable<SpeculativeDecodingSpec.Types.NgramSpeculation>, IBufferMessage, IMessage

Reference documentation and code samples for the Vertex AI v1 API class SpeculativeDecodingSpec.Types.NgramSpeculation.

N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.

Inheritance

object > SpeculativeDecodingSpec.Types.NgramSpeculation

Namespace

Google.Cloud.AIPlatform.V1

Assembly

Google.Cloud.AIPlatform.V1.dll

Constructors

NgramSpeculation()

public NgramSpeculation()

NgramSpeculation(NgramSpeculation)

public NgramSpeculation(SpeculativeDecodingSpec.Types.NgramSpeculation other)
Parameter
Name Description
other SpeculativeDecodingSpecTypesNgramSpeculation

Properties

NgramSize

public int NgramSize { get; set; }

The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified.

Property Value
Type Description
int