I have a prompt that has 4 steps. step 1 is it formats some lesson observation notes,…
- User
I have a prompt that has 4 steps. step 1 is it formats some lesson observation notes, step 2 is it creates a report/written feedback for a teacher based on the observation, and expands on some points and includes advice etc, step 3 is it asks questions during the verbal feedback, what will make that teacher think critically and thoughtfully on their lesson based on what was identified, step 4 is to again write a report but this time matching all the observations to the teachers standards.
Now, when I'm testing the best ai models to help me with this prompt/goal/custom ai etc, which parameters do I need to be looking at to see which model is best in creating the best outcomes for me. Maths, instruction following, multi-turn, creative writing, coding, hard prompts, longer quiery? If you need to clarify anything or ask any more questions before choosing the best, ask me. Always justify your reasoning.
Assistant
This is a fantastic and complex prompt! To determine the best AI model for your needs, you're right to think about specific parameters. Let's break down each of your suggested parameters and see how they relate to your 4-step prompt, and then I'll suggest the most crucial ones and why.
Analyzing Your Suggested Parameters:
-
Maths: Not directly relevant. Unless your lesson observations involve numerical data analysis or calculations (which is unlikely in a standard lesson observation context), mathematical ability isn't a key factor. You're dealing with language, pedagogy, and feedback, not numerical problem-solving.
-
Instruction Following: Extremely Crucial. Your prompt is highly structured with four distinct steps. The AI must be able to accurately follow these instructions in the correct order and understand the nuances of each step (formatting, expanding, questioning, matching to standards). If the AI struggles with instruction following, the entire process will fall apart.
-
Multi-turn: Moderately Relevant, but not in the traditional conversational sense. While your prompt isn't a back-and-forth conversation, it is a multi-stage process where the output of one step informs the next. A model good at "multi-turn" in a broader sense might be better at maintaining context across these steps and ensuring consistency and coherence throughout the entire process. However, it's less about conversational memory and more about handling a complex, multi-part instruction set.
-
Creative Writing: Highly Relevant, but in a specific way. You don't need purely imaginative creative writing. Instead, you need:
- Clear and Professional Writing: For the reports (steps 2 and 4), the writing needs to be professional, well-structured, and easy to understand for a teacher.
- Constructive and Insightful Feedback: Step 2 requires expanding on points and giving advice. This needs to be more than just summarizing notes; it needs to be thoughtful, insightful, and genuinely helpful.
- Thought-Provoking Question Generation: Step 3 requires crafting questions that stimulate critical thinking. This is a form of "creative" question design, not just simple factual recall questions.
-
Coding: Not relevant. Unless you intend to integrate the AI output into a coding environment or automate the process through code (which is a separate consideration), coding ability is not directly needed for the core task of generating lesson observation reports and feedback.
-
Hard Prompts: Extremely Crucial. Your prompt is a hard prompt. It's multi-faceted, requires nuanced understanding of pedagogy and feedback, and demands different types of outputs (formatted notes, reports, questions). Testing on "hard prompts" is essential to see how well the AI handles complexity, ambiguity, and the need for sophisticated reasoning. Models that excel at simple prompts might fail on this type of task.
-
Longer Query: Moderately Relevant. Your prompt itself might be relatively long, especially when you include example observation notes and teacher standards in your testing. The AI needs to be able to handle longer input contexts without losing coherence or accuracy. However, the complexity of the prompt is more important than just its length.
Justification and Prioritization: The Most Important Parameters
Based on the analysis, here's a prioritized list of parameters you should focus on when testing AI models for your lesson observation prompt, along with justifications:
-
Instruction Following (Crucial):
- Justification: Your prompt is explicitly structured in 4 steps. If the AI can't accurately follow these steps, the entire process will be flawed. You need to test if the AI correctly identifies each step, executes them in the right order, and understands the specific requirements of each step (formatting, expanding, questioning, matching).
- How to Test: Give the AI your full 4-step prompt with clear instructions for each step. Evaluate if the output for each step aligns with your expectations and the instructions provided. Look for errors in order, misinterpretations of instructions, or steps being skipped.
-
Hard Prompts (Crucial):
- Justification: Your prompt is inherently complex due to its multi-stage nature, the need for pedagogical understanding, and the requirement for nuanced outputs (feedback, critical questions). Testing on "hard prompts" like yours will reveal which models can handle complex instructions, reason effectively, and generate high-quality outputs in challenging scenarios.
- How to Test: Use your actual 4-step prompt as a "hard prompt." Compare the outputs of different models on this prompt. Look for models that produce comprehensive, insightful, and well-structured outputs across all four steps, demonstrating an understanding of the underlying task.
-
Creative Writing (Specifically: Professional, Constructive, Insightful Writing & Question Generation) (Highly Relevant):
- Justification: Steps 2 and 3 require more than just factual summarization. You need the AI to generate:
- Professional and Clear Reports: The reports need to be well-written, grammatically correct, and use appropriate professional language for teacher feedback.
- Constructive and Actionable Feedback: Step 2 needs to provide advice and expand on points in a way that is genuinely helpful for the teacher's development. It shouldn't be generic or superficial.
- Thought-Provoking and Critical Questions: Step 3 requires questions that go beyond simple recall and encourage the teacher to reflect deeply on their practice.
- How to Test: Evaluate the quality of the written feedback and questions generated by the AI. Ask yourself:
- Is the writing clear, professional, and easy to understand?
- Is the feedback constructive and actionable? Does it offer specific suggestions for improvement?
- Are the questions genuinely thought-provoking and likely to stimulate critical reflection?
- Is the tone appropriate and supportive?
- Justification: Steps 2 and 3 require more than just factual summarization. You need the AI to generate:
-
Multi-turn (Contextual Awareness for Multi-Step Prompts) (Moderately Relevant):
- Justification: While not conversational multi-turn, the AI needs to maintain context across the four steps to ensure coherence and consistency. For example, the questions in step 3 should be directly related to the observations and feedback generated in steps 1 and 2. A model good at handling multi-turn scenarios might be better at managing this multi-stage process.
- How to Test: Examine if the output of each step logically flows from the previous steps. Are the questions in step 3 directly relevant to the observations and feedback in steps 1 and 2? Does the final report in step 4 accurately reflect the observations and link them to teacher standards in a consistent manner?
-
Longer Query (Moderately Relevant):
- Justification: Your prompt, especially with example observation notes and teacher standards, might be relatively long. Testing with longer queries ensures the AI can handle the input length without losing information or coherence.
- How to Test: Test with your full, detailed prompt, including realistic lengths of observation notes and teacher standard lists. See if the AI can process the entire input effectively and generate outputs that are comprehensive and accurate, even with a longer input.
In Summary:
For your specific goal of creating an AI for lesson observation reports and feedback, the most crucial parameters to focus on when testing AI models are Instruction Following, Hard Prompts, and Creative Writing (specifically for professional, constructive, and insightful writing and question generation). Multi-turn (contextual awareness for multi-step prompts) and Longer Query are also relevant but less critical than the first three.
By focusing your testing on these parameters, you'll be able to effectively evaluate which AI models are best suited to handle the complexity and nuances of your 4-step lesson observation prompt and deliver the high-quality outputs you need. Good luck with your testing!
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.