Skip to content
TheChatArchiveThe Chat Archive
GeminiGemini (model unspecified)· GoogleHugging Face ShareLM (browser-extension contributions)

Give me an honest and critical peer review

  1. User

    Give me an honest and critical peer review

  2. Gemini Assistant

    This is a comprehensive peer review of the manuscript "Character Development in Oral Testimonies: Computational Modeling of Religiosity in Holocaust Narratives."Recommendation: Major Revisions RequiredSummaryThis paper presents a computational analysis of religious trajectories (specifically Belief and Practice) in 1,000 Holocaust survivor testimonies. The authors propose a pipeline involving segmentation, filtering (RoBERTa), and classification (Mistral-7B) to map these trajectories. They offer a taxonomy of trajectory shapes (e.g., Oscillating, Constant-Positive) and provide quantitative analyses regarding gender and geography.General AssessmentThe paper is well-written and tackles a high-value, sensitive topic within Digital Humanities (DH). The distinction between religious practice and religious belief is a strong theoretical contribution that yields nuanced results. However, the paper relies heavily on technical methods "adopted with minimal changes", raising questions about technical novelty. Furthermore, the validation methods—specifically the reliance on "silver standards" derived from coarser methods—weakens the claims of accuracy.Major Critiques1. Technical Novelty vs. ApplicationThe authors state in a footnote: "We note that the technical and algorithmic part of the paper was adopted with minimal changes".Critique: If the algorithmic contribution is minimal, the paper must stand entirely on its application and historical insights. However, the "Experiments" and "Validation" sections take up a significant portion of the paper (pages 8–10).Recommendation: You must explicitly cite the original work from which the algorithm was adopted in the main text, not just a footnote. Clarify exactly what is new here. Is it just the prompt engineering for the religious domain? If so, the "Contributions" section should be revised to de-emphasize the "computational method" and emphasize the adaptation of the method for religious analysis.2. Validation Logic and the "Silver Standard"The validation relies on comparing the model's output to "silver-standard reference trajectories" generated via topic modeling and keyword lookups (thesaurus).+1Critique: This is methodologically circular and potentially flawed. You are using coarser, bag-of-words approaches (topics/keywords) to validate a context-aware LLM approach.The authors admit the reference data is sparse and lacks valence.If the LLM disagrees with the topic model, the LLM might actually be right (contextual understanding) while the topic model is wrong. Using the topic model as a ground truth penalizes the LLM for being more sophisticated.Recommendation: While the manual evaluation (§7.3) is strong, the automatic validation using topic modeling is weak. It serves better as a comparison of methods rather than a validation of the LLM. I suggest reframing this section: "Comparison with Keyword-Based Approaches" rather than "Validation."3. The "Christian Bias" ProblemIn the Limitations section, the authors note: "the models frequently assigned positive valence to acts such as praying to Jesus or attending church".Critique: This is a critical threat to validity, not just a limitation. The study focuses on Jewish survivors. If a survivor mentions a "righteous gentile" taking them to church to hide, and the model flags this as "Positive Religious Practice," it conflates survival strategy with religious identity.Recommendation: You need to quantify this error rate. A manual audit of 50-100 "Church/Jesus" mentions is required to see how often they are misclassified as Jewish religious practice. If the rate is high, the "Oscillating Practice" findings might be contaminated by these false positives.4. Interpretation of "Oscillating" TrajectoriesThe paper claims that 68% of practice trajectories are "Oscillating" and interprets this as a "lived form of traditional Judaism" reflecting "situational negotiations".+2Critique: "Oscillation" in a computational signal can also mean noise. Given the moderate F1 scores for the model (0.43 for practice), how do you distinguish between genuine narrative oscillation and model instability?Recommendation: Provide a "smoothness" analysis or a confidence interval. Did you interpret rapid flipping between Active/Inactive as meaningful oscillation, or is it an artifact of the segmentation?Specific Observations & Minor IssuesFigure 2 (Distribution): The distribution of segments shows a U-shape (high at start/end). The text attributes this to SF interview guidelines.+1Comment: This is a very helpful insight. It validates the assumption that narrative time ≈ chronological time for this specific corpus.Gender Analysis: The finding that female segments are 1.4x more likely to express absence of practice (p<10−12) is fascinating.Comment: The interpretation that women had "fewer formal rituals" is sound. However, consider addressing domestic rituals (lighting candles, kashrut) which are female-centric. Did the model fail to capture domestic labor as "practice"?Citation Formatting:There are placeholders in the text, e.g., "10.48694/jcls.xx X".In the reference list, standard citation formats are sometimes broken or repetitive (e.g., repetitive URLs in ).+1Baseline Definitions: The definition of "Original" in Table 1 is "Randomly select points from the original distribution of the predicted label".Comment: This phrasing is confusing. Does it mean a shuffled version of the predicted sequence? Clarify if this baseline preserves the count of labels but randomizes their position.ConclusionThe paper makes a compelling case for the complexity of religious identity in Holocaust testimonies. The separation of Belief (Constant-Positive) and Practice (Oscillating) is a significant finding that challenges the "crisis of faith" narrative. However, the paper must be more transparent about the origin of its algorithms, more rigorous in its validation (moving away from weak silver standards), and must quantitatively address the "Christian bias" in the LLM rather than relegating it to the discussion.Decision: Revise and Resubmit.

  3. User

    fix: @book{culture,  ISBN = {9780231180283},  URL = {http://www.jstor.org/stable/10.7312/mohr18028},  author = {JOHN W. Mohr and CHRISTOPHER A. BAIL and MARGARET FRYE and JENNIFER C. LENA and OMAR LIZARDO and TERENCE E. McDONNELL and ANN MISCHE and IDDO TAVORY and FREDERICK F. WHERRY},  publisher = {Columbia University Press},  title = {Measuring Culture},  urldate = {2025-11-13},  year = {2020} }

  4. Gemini Assistant

    Here is the fixed BibTeX entry. The primary correction was adjusting the author names from uppercase to standard capitalization, which ensures they render correctly in your bibliography.Code snippet@book{culture, title = {Measuring Culture}, author = {Mohr, John W. and Bail, Christopher A. and Frye, Margaret and Lena, Jennifer C. and Lizardo, Omar and McDonnell, Terence E. and Mische, Ann and Tavory, Iddo and Wherry, Frederick F.}, year = {2020}, publisher = {Columbia University Press}, isbn = {9780231180283}, url = {http://www.jstor.org/stable/10.7312/mohr18028}, urldate = {2025-11-13} }

  5. User

    are the emdash es used right here? Belief is annotated using a parallel schema consisting of the classes \textit{positive}, \textit{negative}, and \textit{other}. Examples include sincere praying (\textit{positive}) -- What did you do? ... Nothing. I just prayed. I just prayed.'' -- and rejection of God (\textit{negative}) -- Why didn’t he help us? I don’t believe... There is no God''. The \textit{other} category captures ambivalent or complex positions, where belief is expressed with uncertainty, conditionality, or internal tension rather than clear affirmation or denial.

  6. Gemini Assistant

    Technically, no, they are not implemented correctly for a standard LaTeX document, although the grammatical intention is fine.Here are the specific issues and how to fix them:1. The Code (-- vs ---)In LaTeX, -- renders as an en dash (–), which is typically used for number ranges (e.g., 1990–2000). For an em dash (—), which is used for breaks in thought or setting off appositives (as you are doing here), you must use three hyphens: ---.2. SpacingIn standard academic typesetting (and LaTeX defaults), em dashes are usually placed without spaces around them. Using -- (space, en dash, space) is a valid stylistic choice in some web writing, but in formal LaTeX papers, --- (no spaces) is preferred.3. Structural ClarityUsing dashes to set off quotes twice in one sentence can be visually choppy. Colons are often cleaner for introducing examples like this.Option A: Fix the dashes (Keep your current structure)Use --- and remove the spaces.Examples include sincere praying (\textit{positive})---What did you do? ... Nothing. I just prayed. I just prayed.''---and rejection of God (\textit{negative})---Why didn’t he help us? I don’t believe... There is no God''.Option B: Use Colons (Recommended for flow)This removes the visual clutter of the dashes and makes the sentence easier to parse.Examples include sincere praying (\textit{positive}): What did you do? ... Nothing. I just prayed. I just prayed.'' and rejection of God (\textit{negative}): Why didn’t he help us? I don’t believe... There is no God''.Option C: If you really prefer the spaced lookIf you prefer the visual style of a dash with spaces, standard LaTeX practice is to use an em dash with thin spaces, or stick to the en dash with spaces (though some editors may flag this).Examples include sincere praying (\textit{positive})--- ``What did you do? ...''--- and ...

  7. User

    """ Belief is annotated using a parallel schema consisting of the classes \textit{positive}, \textit{negative}, and \textit{other}. Examples include sincere praying (\textit{positive})---What did you do? ... Nothing. I just prayed. I just prayed.''---and rejection of God (\textit{negative})---Why didn’t he help us? I don’t believe... There is no God''. The \textit{other} category captures ambivalent or complex positions, where belief is expressed with uncertainty, conditionality, or internal tension rather than clear affirmation or denial. """

  8. Gemini Assistant

    Yes, this is now correct for standard LaTeX typesetting.Here is why:The Code: You are using --- (three hyphens), which LaTeX correctly renders as a solid em dash (—).The Spacing: You have removed the spaces around the dashes (e.g., (\textit{positive})---``What). This is the standard convention in professional typesetting; the em dash sits flush against the surrounding text.The Grammar: You are using the dashes to insert a break or a parenthetical example, which is the correct grammatical function of an em dash.This version will compile perfectly.

  9. User

    Should Large Language models be capitalized? The subsequent classification of belief and practice is carried out by prompting and fine-tuning Large Language Models. Specifically, we fine-tune Mistral-7B-instruct (\cite{jiang2023clip}), on data from the second annotation task. Before reaching this configuration, we compared multiple setups to the annotated test-set, including prompting a series of GPT-4.1 models\footnote{\url{https://platform.openai.com/docs/models/gpt-4.1}, version: 2025-04-14} on multiple settings. For training, we use Mistralai's mistral-finetune repository\footnote{\url{https://github.com/mistralai/mistral-finetune}} with their default hyperparameters and test multiple seeds.\footnote{Hyperparameters: Lora Rank=64, sequence length=16,384, batch size=1, learning rate=1e-4, number of micro-batches=1. Number of training steps=100, weight decay=0.1, pct-start=0.05. Seed for belief=1; for practice=5} The best performing model across all practice and belief classes is GPT4.1 with an average F1 of 0.55 for practice and 0.64 for belief. The fine-tuned Mistral-7B scores 0.43 and 0.45 for practice and belief, respectively.\footnote{Owing to limitations with large-scale API access, we use Mistral-7B to generate the full dataset of trajectories. The prompts we run follow the format of Self-Consistency \cite{Wang2022SelfConsistencyIC} and are carefully designed with guidance from Anthropic's prompt generator (\url{https://console.anthropic.com/login)}. Considering the complicated annotation process and IAA, along with the models' bias toward stereotypes in the context of religion \cite{plaza-del-arco-etal-2024-divine}, these moderate F1 scores are unsurprising.}

  10. Gemini Assistant

    No, "large language models" should generally not be capitalized unless you are using the acronym (LLMs) or it appears at the start of a sentence. It is a common noun phrase, not a proper noun.In your text, you currently have:"...prompting and fine-tuning Large Language Models."This should be corrected to lowercase.Here is the corrected paragraph with that fix, as well as a few other consistency and style adjustments (bolded for visibility):The subsequent classification of belief and practice is carried out by prompting and fine-tuning large language models. Specifically, we fine-tune Mistral-7B-instruct (\cite{jiang2023clip}) on data from the second annotation task. Before reaching this configuration, we compared multiple setups to the annotated test-set, including prompting a series of GPT-4.1 models\footnote{\url{https://platform.openai.com/docs/models/gpt-4.1}, version: 2025-04-14} on multiple settings. For training, we use Mistralai's mistral-finetune repository\footnote{\url{https://github.com/mistralai/mistral-finetune}} with their default hyperparameters and test multiple seeds.\footnote{Hyperparameters: Lora Rank=64, sequence length=16,384, batch size=1, learning rate=1e-4, number of micro-batches=1. Number of training steps=100, weight decay=0.1, pct-start=0.05. Seed for belief=1; for practice=5} The best performing model across all practice and belief classes is GPT-4.1 with an average F1 of 0.55 for practice and 0.64 for belief. The fine-tuned Mistral-7B scores 0.43 and 0.45 for practice and belief, respectively.\footnote{Owing to limitations with large-scale API access, we use Mistral-7B to generate the full dataset of trajectories. The prompts we run follow the format of Self-Consistency \cite{Wang2022SelfConsistencyIC} and are carefully designed with guidance from Anthropic's prompt generator (\url{https://console.anthropic.com/login)}. Considering the complicated annotation process and IAA, along with the models' bias toward stereotypes in the context of religion \cite{plaza-del-arco-etal-2024-divine}, these moderate F1 scores are unsurprising.}Specific Fixes:Capitalization: Changed "Large Language Models" to "large language models".Consistency: Changed "GPT4.1" to "GPT-4.1" to match the usage earlier in the paragraph ("GPT-4.1 models").Punctuation: Removed the comma after (\cite{jiang2023clip})—it isn't grammatically necessary there ("Specifically, we fine-tune X on data...").

  11. User

    Whats better? 1. The trajectories are mapped by converting the belief and practice labels into numerical values, based on the structure and according to valence: \textit{constant-positive/negative, ascending/descending, oscillating}, and the degree of coverage, relative to the testimony storyline: tmax−tmint_{max} - t_{min} (\autoref{structures}). 2. The trajectories are mapped by converting the belief and practice labels into numerical values, based on structure and valence: \textit{constant-positive/negative, ascending/descending, oscillating}, and the degree of coverage, relative to the testimony storyline: tmax−tmint_{max} - t_{min} (\autoref{structures}).

  12. Gemini Assistant

    Option 2 is better.It is more concise and flows better. "Based on structure and valence" is a cleaner parallel construction than "based on the structure and according to valence."Here is a slightly polished version of Option 2 that improves the punctuation for smoother reading:The trajectories are mapped by converting the belief and practice labels into numerical values based on structure and valence (\textit{constant-positive/negative, ascending/descending, oscillating}) and the degree of coverage relative to the testimony storyline (tmax​−tmin​) (\autoref{structures}).

  13. User

    Is this grammatically correct? The trajectories are mapped by converting the belief and practice labels into numerical values, based on structure and valence: \textit{constant-positive/negative, ascending/descending, oscillating}, and the degree of coverage, relative to the testimony storyline: tmax−tmint_{max} - t_{min} (\autoref{structures}). This is a simplified taxonomy designed to reduce complexity as much as possible – ignoring: the density or frequency of tags and the segments labeled as \textit{other}. For this analysis, each sequence is simplified by removing any neutral values it contains and then collapsing consecutive identical into a single instance. For example, [-1,1,0,1,1] becomes [-1,1].

  14. Gemini Assistant

    It is not quite grammatically correct. There are two specific errors that need fixing:Sentence 2 (Punctuation): You should not use a colon after "ignoring" because it separates the verb from its direct object.Sentence 3 (Missing Word): The phrase "collapsing consecutive identical into" is missing a noun. You likely mean "consecutive identical values" or "elements."Here is the corrected version with the fixes in bold:"The trajectories are mapped by converting the belief and practice labels into numerical values based on structure and valence (\textit{constant-positive/negative, ascending/descending, oscillating}) and the degree of coverage relative to the testimony storyline (tmax​−tmin​) (\autoref{structures}). This is a simplified taxonomy designed to reduce complexity as much as possible, ignoring the density or frequency of tags and the segments labeled as \textit{other}. For this analysis, each sequence is simplified by removing any neutral values it contains and then collapsing consecutive identical values into a single instance. For example, [-1,1,0,1,1] becomes [-1,1]."Key Changes Made:Sentence 1: Replaced the colon/list structure with parentheses for better flow. This clarifies that "degree of coverage" is a separate criteria from the "valence" types.Sentence 2: Removed the colon after "ignoring" and replaced the dash with a comma (smoother).Sentence 3: Added the word "values" after "consecutive identical."

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.