MaxQuestionSize
The maximum number of tokens that the question to Answer Server can contain. If a user question contains more than this number of tokens, Answer Server rejects the question and returns an error.
The token count for a question depends on the model that you use. To obtain an accurate token count for your model, you must provide a get_token_count
function in the script that you use to call the LLM. See Create a RAG Script.
Type: | Integer |
Default: | 50 |
Required: | No |
Configuration Section: | MySystem |
Example: | MaxQuestionSize=70
|
See Also: |