Chatbot from distilabel.embeddings import SentenceTransformerEmbeddings from…
- User
Chatbot
from distilabel.embeddings import SentenceTransformerEmbeddings from distilabel.llms import InferenceEndpointsLLM from distilabel.mixins.runtime_parameters import RuntimeParameter from distilabel.pipeline import Pipeline from distilabel.steps import EmbeddingGeneration, ConversationTemplate, FaissNearestNeighbour, StepInput, StepOutput, step from distilabel.steps.tasks import MagpieGenerator, TextGeneration
REASONING_PROMPT = """ "You are an AI assistant specialized in logical thinking and problem-solving. Your purpose is to help users work through complex ideas, analyze situations, and draw conclusions based on given information. Approach each query with structured thinking, break down problems into manageable parts, and guide users through the reasoning process step-by-step." """
REFLECTION_PROMPT = """ You're an AI assistant that responds to the user with maximum accuracy. To do so, your first task is to think what the user is asking for, thinking step by step. During this thinking phase, you will have reflections that will help you clarifying ambiguities. In each reflection you will list the possibilities and finally choose one. Between reflections, you can think again. At the end of the thinking, you must draw a conclusion. You only need to generate the minimum text that will help you generating a better output, don't be verbose while thinking. Finally, you will generate an output based on the previous thinking.
This is the output format you have to follow:
@step(inputs=["instruction", "generation"], step_type="global") def FilteringStep(inputs: StepInput) -> StepOutput: yield [ input for input in inputs
if instruction and generation are not null or empty
if (input["instruction"] is not None and input["instruction"] != '') and (input["generation"] is not None and input["generation"] != '') ]
with Pipeline("Reflection Dataset") as pipeline: magpie = MagpieGenerator( only_instruction=True, num_rows=20000, batch_size=5, system_prompt=REASONING_PROMPT, llm=InferenceEndpointsLLM( model_id="meta-llama/Meta-Llama-3.1-70B-Instruct", tokenizer_id="meta-llama/Meta-Llama-3.1-70B-Instruct", magpie_pre_query_template="llama3", generation_kwargs={"temperature": 0.8, "max_new_tokens": 2048}, ) )
response = TextGeneration( input_batch_size=5, system_prompt=REFLECTION_PROMPT, llm=InferenceEndpointsLLM( model_id="meta-llama/Meta-Llama-3.1-70B-Instruct", tokenizer_id="meta-llama/Meta-Llama-3.1-70B-Instruct", generation_kwargs={"temperature": 0.8, "max_new_tokens": 2048}, ), )
conversation = ConversationTemplate( input_mappings={ "instruction": "instruction", "response": "generation", }, output_mappings={ "conversation": "text", }, )
filter_null = FilteringStep()
embedding_generation = EmbeddingGeneration( embeddings=SentenceTransformerEmbeddings( model="sentence-transformers/all-MiniLM-L6-v2", trust_remote_code=True, device="cpu", ), )
faiss = FaissNearestNeighbour()
magpie >> response >> conversation >> filter_null >> embedding_generation >> faiss
if name == "main": distiset = pipeline.run( use_cache=False, )
distiset.push_to_hub( repo_id="thesven/Reflective-MAGLLAMA-v0.1", private=True, include_script=True )
- Assistant
Hello! It looks like you're working on a Python script that constructs a pipeline using the
distilabellibrary to generate a reflection dataset with the Meta LLaMA 3.1 70B Instruct model. I'd be happy to help review your code and provide any assistance you might need.Potential Issues and Suggestions:
-
if __name__ == "__main__":Block: In your script, you have the following line:if name == "main":In Python, to check if the script is being run as the main program, you should use double underscores (
__) aroundnameandmain, like this:if __name__ == "__main__":This ensures that the code block only executes when the script is run directly, not when it's imported as a module.
-
Hidden Characters in the Code: It appears there might be some non-standard spaces or hidden characters in your code. For example, the spaces around
nameand==might be non-breaking spaces or other Unicode characters. This can cause syntax errors. To fix this:- Replace the problematic line manually to eliminate any hidden characters.
- Ensure you're using a proper code editor that highlights or removes invisible characters.
-
Model Availability: You're using the model
"meta-llama/Meta-Llama-3.1-70B-Instruct". Make sure:- You have access to this model and the necessary permissions to use it.
- The model is correctly installed or accessible in your environment.
-
Library Imports: Ensure that all the libraries and modules you're importing are installed in your environment. You might need to install
distilabeland its dependencies:pip install distilabelIf
distilabelis not available viapip, make sure to install it from the appropriate source or repository. -
GPU/Device Configuration: In the
EmbeddingGenerationstep, you're setting the device to"cpu". If you have access to a GPU and your models support it, you might want to setdevice="cuda"for better performance.embeddings=SentenceTransformerEmbeddings( model="sentence-transformers/all-MiniLM-L6-v2", trust_remote_code=True, device="cuda", # Use 'cuda' if a GPU is available ), -
Token Generation Parameters: In your
InferenceEndpointsLLM, you're settingmax_new_tokensto2048. Depending on the model and the context length it supports, this might be too large. Ensure that themax_new_tokensparameter does not exceed the model's maximum context length. -
Error Handling: Consider adding try-except blocks around your pipeline execution to catch and handle any runtime errors gracefully. This can help in debugging issues when running the script.
if __name__ == "__main__": try: distiset = pipeline.run(use_cache=False) distiset.push_to_hub( repo_id="thesven/Reflective-MAGLLAMA-v0.1", private=True, include_script=True ) except Exception as e: print(f"An error occurred: {e}")
Next Steps:
- Run the Script: After making the above adjustments, try running your script again to see if the issues are resolved.
- Provide Error Messages: If you encounter any specific errors or unexpected behavior, please share the error messages. This will make it easier to diagnose and address the problem.
- Check Dependencies: Verify that all dependencies are up-to-date and compatible with your version of Python.
Additional Assistance:
If you have any questions about specific parts of the code or need further clarification on how to implement any of the suggestions, feel free to ask. I'm here to help!
-