Optional apiThe version of the API functions. Part of the path.
Optional authOptional customIf you are planning to connect to a model that lives under a custom endpoint provide the "customModelURL" which will override the automatic URL building
This is necessary in cases when you want to point to a fine-tuned model or a model that has been hidden under VertexAI Endpoints.
In those cases, specifying the GoogleVertexAIModelParams.model param
will not be necessary and will be ignored.
GoogleVertexAILLMConnection.buildUrl
Optional endpointHostname for the API call
Optional locationRegion where the LLM is stored
Optional maxMaximum number of tokens to generate in the completion.
Optional modelModel to use
Optional temperatureSampling temperature to use
Optional topKTop-k changes how the model selects tokens for output.
A top-k of 1 means the selected token is the most probable among all tokens in the model’s vocabulary (also called greedy decoding), while a top-k of 3 means that the next token is selected from among the 3 most probable tokens (using temperature).
Optional topPTop-p changes how the model selects tokens for output.
Tokens are selected from most probable to least until the sum of their probabilities equals the top-p value.
For example, if tokens A, B, and C have a probability of .3, .2, and .1 and the top-p value is .5, then the model will select either A or B as the next token (using temperature).
Generated using TypeDoc
Interface representing the input to the Google Vertex AI model.