help me, i am doing review summarisation, using a transformer, i have finetuned it…
- User
help me, i am doing review summarisation, using a transformer, i have finetuned it (instruction fine-tuning on 50 cars), using pegasus large, then using in my application via pipeline,
problem is my model is hallucinating , creating details on it's own which were not mentioned, what to do here?
Assistant
Okay, let's break down some code examples and practical steps you can take right now to address the hallucination issues in your Pegasus-based summarization pipeline. Given your setup (instruction fine-tuning on Pegasus-large and deployment via
pipeline), I'll focus on techniques that are relatively easy to implement and don't require retraining the model from scratch immediately (though retraining with better data is important long-term).Assumptions:
- You're using the
transformerslibrary from Hugging Face. - You have your fine-tuned Pegasus model loaded or accessible through the
pipeline.
1. Adjust Decoding Parameters in the Pipeline:
This is the easiest and quickest thing to try. Within your pipeline, you can modify the generation parameters to influence the summarization process. Lowering the temperature and increasing the repetition penalty are good starting points.
from transformers import pipeline # Assuming 'your_finetuned_model' is the path or identifier for your fine-tuned model summarizer = pipeline("summarization", model="your_finetuned_model", device=0) # Use device=0 for GPU if available review_text = "This car has excellent fuel economy and a comfortable ride. However, the infotainment system is clunky." summary = summarizer(review_text, max_length=130, # Adjust as needed min_length=30, # Adjust as needed do_sample=True, # Enable sampling temperature=0.7, # Lower temperature for less randomness (default is 1.0) top_p=0.92, # alternative to temperature for controlling sampling repetition_penalty=1.2, # Increase penalty to prevent repetition (and hallucination) length_penalty = 1.0, # penalize longer sentences to have a more concise output num_beams=4, # Use beam search for better quality no_repeat_ngram_size=3 #prevent trigrams from repeating ) print(summary[0]['summary_text'])Explanation:
temperature: Controls the randomness of token selection. Lower values (e.g., 0.7, 0.5) make the model more conservative and less likely to generate unusual or hallucinated tokens.top_p: Also influences the sampling. Smaller values mean that the model only considers the top p probability values. Recommended along with temperature.repetition_penalty: Penalizes the model for repeating words or phrases. A value greater than 1.0 (e.g., 1.1, 1.2) discourages repetition. This can indirectly help with hallucination, as hallucinated details are often repeated to reinforce them.length_penalty: Penalizes longer generated sentences to have a more concise summarynum_beams: Use Beam search for better generationno_repeat_ngram_size: Specify the n-gram size to avoid repeating.
Experiment: Try different combinations of these parameters. Start by lowering the temperature and increasing the repetition penalty, then adjust them incrementally to find the best balance.
2. Simple Keyword Filtering Post-Processing:
A quick and dirty way to reduce obvious hallucination is to check if the generated summary contains any keywords or phrases from the original review. If it doesn't, you can flag it for human review or discard it. This is not a perfect solution, but it can catch some cases.
def contains_keywords(summary, review): """Checks if the summary contains keywords from the review.""" summary_lower = summary.lower() review_lower = review.lower() keywords = review_lower.split() # Simple keyword extraction - improve if needed for keyword in keywords: if keyword in summary_lower: return True return False summary_text = summary[0]['summary_text'] if not contains_keywords(summary_text, review_text): print("Warning: Summary may contain hallucinations. Requires review.") # Handle the case where the summary is flagged (e.g., send to human review) else: print(summary_text)Improvements to Keyword Filtering:
- Use more sophisticated keyword extraction (e.g., TF-IDF, RAKE).
- Use stemming/lemmatization to match keywords even if they have slightly different forms.
- Consider using a list of important keywords instead of all words.
3. Fact-Checking with a Pre-trained Model (Entailment/NLI):
This is more involved but can be very effective. Use a Natural Language Inference (NLI) model to determine if the summary is logically entailed by the review. If the NLI model says the review does not entail the summary, it's a sign of hallucination. Hugging Face makes this fairly easy.
from transformers import pipeline # Load a pre-trained NLI model nli_model = pipeline("text-entailment", model="facebook/bart-large-mnli", device=0) # Ensure GPU is available def is_entailed(review, summary): """Checks if the review entails the summary using an NLI model.""" result = nli_model(review, summary, candidate_labels=["entailment", "neutral", "contradiction"]) print(result) #debug return result['labels'][0] == 'entailment' summary_text = summary[0]['summary_text'] if not is_entailed(review_text, summary_text): print("Warning: Summary may contain hallucinations according to NLI model.") # Handle the case where the summary is flagged else: print(summary_text)Explanation:
- We use the
text-entailmentpipeline with a pre-trained NLI model (in this case,facebook/bart-large-mnli). There are other choices and models too. - The
is_entailedfunction feeds the review and summary to the NLI model and checks if the predicted label is "entailment". - If the model predicts "contradiction" or "neutral", it suggests the summary contains information not supported by the review.
Important Considerations for NLI:
- Model Choice:
bart-large-mnliis a good starting point, but you might need to experiment with other NLI models to find one that works best for your specific domain and review style. - Thresholding: The NLI model outputs probabilities. Instead of a strict "entailment/not entailment" decision, you can use a probability threshold. For example, only flag summaries where the entailment probability is below a certain value (e.g., 0.7). This allows for more nuanced control.
- Computational Cost: NLI models can be computationally expensive, especially for long reviews. Consider using a smaller NLI model or truncating the review if necessary.
4. Longer-Term Solutions (Retraining):
The above techniques are good for mitigating the problem right now. However, the best long-term solution is to retrain your Pegasus model with much more data:
- Gather More Data: Aim for hundreds or even thousands of car reviews with high-quality, faithful summaries. Scrape websites, use public datasets, or create your own.
- Data Augmentation: Use techniques like back-translation (translate the reviews to another language and back) to generate more training examples. Be careful to ensure the back-translated data is still accurate.
- Focus on Faithfulness during Fine-Tuning:
- Consider using a loss function that explicitly penalizes hallucination during fine-tuning (this is more advanced). One example is to use a pre-trained NLI model to evaluate the entailment between the review and the generated summary during training and add a loss term based on the NLI score.
Prioritization:
- Implement decoding parameter adjustments immediately. This is the easiest and quickest win.
- Implement the keyword filtering as a simple safeguard.
- Evaluate and implement the NLI-based fact-checking. This is the most promising technique for improving faithfulness.
- Start gathering more data and plan for retraining your model. This is the long-term solution.
Remember to test these techniques thoroughly on a held-out set of reviews to evaluate their effectiveness before deploying them to your production system. Good luck! Let me know if you have specific questions about any of these steps.
- You're using the
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.