Generative AI
Last updated
Was this helpful?
Last updated
Was this helpful?
Components to leverage native Generative AI capabilities on Data Warehouses.
Description
This components calls function in BigQuery for each row on the input table.
Inputs
Model [FQN]: The path for the model to be used in the format project_id.dataset.model
Prompt column: The column in the input source used to generate the prompt.
Max Output Tokens: an INT64 value in the range [1,1024] that sets the maximum number of tokens that the model outputs. Specify a lower value for shorter responses and a higher value for longer responses. The default is 50.
Temperature: a FLOAT64 value in the range [0.0,1.0] that is used for sampling during the response generation, which occurs when top_k and top_p are applied. It controls the degree of randomness in token selection. Lower temperature values are good for prompts that require a more deterministic and less open-ended or creative response, while higher temperature values can lead to more diverse or creative results. A temperature value of 0 is deterministic, meaning that the highest probability response is always selected. The default is 1.0.
Top P: an INT64 value in the range [1,40] that changes how the model selects tokens for output. Specify a lower value for less random responses and a higher value for more random responses. The default is 40.
Top K: a FLOAT64 value in the range [0.0,1.0] that changes how the model selects tokens for output. Specify a lower value for less random responses and a higher value for more random responses. The default is 1.0.
Tokens are selected from the most (based on the top_k value) to least probable until the sum of their probabilities equals the top_p value. For example, if tokens A, B, and C have a probability of 0.3, 0.2, and 0.1 and the top_p value is 0.5, then the model selects either A or B as the next token by using the temperature value and doesn't consider C.
Outputs
Result table [Table]
External links