I want some help brainstorming ideas for research questions about how generative AI can…
- User
I want some help brainstorming ideas for research questions about how generative AI can affect university students' writing processes, specifically in terms of students' metacognition. I have a background in cognitive science, and I prefer to work with quantitative experimental methods. Give me 5 very different suggestions for research questions that I could investigate within this topic, keep it specific by including independent and dependent variables + how I would measure them.
- Assistant
Here are five distinct research questions you could investigate:
1. Does AI feedback timing affect metacognitive monitoring accuracy?
- IV: Timing of AI feedback (immediate during drafting vs. delayed after completion vs. no AI feedback control)
- DV: Metacognitive monitoring accuracy measured by comparing students' confidence ratings for text quality against actual expert evaluations of their drafts
- Measurement: Students rate confidence in paragraph/section quality on 7-point scales; blind expert raters score the same sections; calculate correlation coefficients between confidence and actual quality
2. How does AI assistance level influence students' metacognitive strategy selection?
- IV: Level of AI assistance (outline generation vs. sentence-level suggestions vs. full paragraph generation vs. control)
- DV: Frequency and type of metacognitive strategies used during writing
- Measurement: Think-aloud protocols coded for strategy types (planning, monitoring, evaluating, revising); keystroke logging to capture revision patterns; post-task strategy questionnaires
3. Does prior AI writing experience moderate the relationship between AI use and metacognitive awareness?
- IV: AI writing experience level (novice vs. intermediate vs. experienced users)
- DV: Metacognitive awareness of writing processes measured through knowledge of cognition and regulation of cognition
- Measurement: Pre/post Metacognitive Awareness of Writing Strategies Inventory; writing process questionnaires; behavioral measures of self-regulation during timed writing tasks
4. How does AI transparency about its limitations affect students' metacognitive calibration?
- IV: AI transparency condition (explicit limitation warnings vs. confidence scores displayed vs. black-box AI vs. control)
- DV: Metacognitive calibration measured as alignment between predicted and actual writing performance
- Measurement: Students predict their essay scores before submission; compare predictions to standardized rubric scores; calculate calibration accuracy using Brier scores or bias/discrimination indices
5. Does collaborative AI writing versus individual AI writing differentially impact metacognitive regulation strategies?
- IV: Writing context (individual with AI vs. peer collaboration with AI vs. individual without AI)
- DV: Use of metacognitive regulation strategies during writing tasks
- Measurement: Screen recordings coded for regulation behaviors (goal-setting, progress monitoring, strategy switching); self-report regulation strategy checklists; analysis of draft revision patterns and planning documents
Each study could use randomized controlled designs with undergraduate participants across different writing tasks to establish causal relationships between AI interaction patterns and metacognitive processes.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.