Skip to main content
[] Rariny February 21, 2023, 5:31am 1

For any service, it is not a rare case that some users ask non-sense questions or send a message that is not even related to the business purpose. For that case I want to use a default reply for example to say “Sorry, this is not part of our services.” if the user input does not match any answer from the dataset.

So how to add a default reply to the model to avoid it from sending inappropriate answer that does not match the user prompt?

sps February 21, 2023, 5:46am 2

What existing approach are you taking @Rariny ? Embeddings or fine-tune?

Rariny February 21, 2023, 6:04am 3

I use fine-tune to achieve this. I collected all the users input and add completion to any specific prompt.

sps February 21, 2023, 9:05am 4

Embedding is a better way to achieve this functionality. But if you’re using fine-tune, consider adding prompt completion pairs for the ‘not’ cases. Basically, these should include things thst you don’t do and the completion would have the message “Sorry, this is not part of our services”.

Kind of like how we train image classification model and include the ‘not’ cases.

1 Like

I might have an approach that might help solve this for you using text-embedding-ada-002. After reading a previous comment it seems that it might actually be useful.

Basically, you can use the text-embedding-ada-002 model to get embeddings for the input string against the training dataset prompts that you used and compute the similarity. If the similarity metric passes above a certain threshold, you can request your model to generate a response, else simply return a generic statement. Here is the code that can be used for this approach.

import json import numpy as np def get_similarity(user_prompt): # Initialize the string which will contain the strings for similarity test. Includes prompt. list_strings = [user_prompt] with open(“.jsonl”, “r”) as f: for line in f: data = json.loads(line) list_strings.append(data[“prompt”]) # assuming prompt-completion pairs. To pick only prompts. resp = openai.Embedding.create( input=list_strings, engine=”text-embedding-ada-002″) similarity_scores = [] embedding_a = resp[‘data’][0][’embedding’] # embedding for the user’s prompt for i in range(1,len(list_strings)): embedding_b = resp[‘data’][i][’embedding’] similarity_score = np.dot(embedding_a, embedding_b) # simple similarity metric similarity_scores.append(similarity_score) threshold = 0.87 if(max(similarity_scores) >= threshold): return gpt_response(user_prompt)[‘choices’][0][‘text’] #get response from your model else: return “Some Generic Output”

A couple of things to note:

  1. This uses the most basic ways to get the similarities, there are other ways to calculate similarities from embeddings so you might find one of the other methods more suitable. For simplicity, I’ve used the simple dot product.
  2. This clearly increases the cost incurred as each input is going to use some OpenAI resources for calculating the embeddings. So if the prompt is very long or if you have a large amount of training data, cost is definitely going to increase. Though it uses a better model which results in good results with lesser cost incurred.
  3. Make sure to mess around with the threshold value as per your wish and use-case. Also, if you’re using a different method for calculating similarity.

Hope this helps!

1 Like

Rariny February 23, 2023, 12:59pm 7

This is really helpful, thank you so much for this.

SDX

About SDX

Leave a Reply

%d bloggers like this: