Skip to content
TheChatArchiveThe Chat Archive
gpt-4o-search-preview-high· OpenAIHugging Face LMArena Search Arena 24K

Please browse the web to collect the list of post-2021 works on long content generation…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Perplexitysonar’s answer.
  1. User

    Please browse the web to collect the list of post-2021 works on long content generation with LLMs. You can cite up to 15 works.

  2. Assistant

    Here is a list of notable works published after 2021 focusing on long content generation using Large Language Models (LLMs):

    1. LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs (August 2024)

      • Authors: Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li
      • Summary: This paper introduces AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling LLMs to generate coherent outputs exceeding 20,000 words. The authors also present LongWriter-6k, a dataset containing 6,000 supervised fine-tuning data with output lengths ranging from 2,000 to 32,000 words, and develop LongBench-Write, a benchmark for evaluating ultra-long generation capabilities[1].
    2. RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text (May 2023)

      • Authors: Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui, Tiannan Wang, Zhenxin Xiao, Yifan Hou, Ryan Cotterell, Mrinmaya Sachan
      • Summary: RecurrentGPT introduces a language-based simulation of the recurrence mechanism in RNNs, enabling GPT models to generate texts of arbitrary length without forgetting. The approach involves generating a paragraph of text at each timestep and updating its language-based long-short term memory stored on the hard drive and the prompt, respectively[2].
    3. Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach (July 2024)

      • Authors: Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Michael Bendersky
      • Summary: This study compares Retrieval Augmented Generation (RAG) and long-context LLMs, aiming to leverage the strengths of both. The authors propose Self-Route, a method that routes queries to RAG or long-context models based on model self-reflection, significantly reducing computation cost while maintaining comparable performance to long-context models[3].
    4. LongGenBench: Benchmarking Long-Form Generation in Long Context LLMs (September 2024)

      • Authors: Yuhao Wu, Ming Shan Hee, Zhiqing Hu, Roy Ka-Wei Lee
      • Summary: LongGenBench is a benchmark designed to evaluate the long-form text generation capabilities of long-context LLMs. The benchmark tests models' ability to identify specific events within generated long text sequences, highlighting challenges in generating coherent long-form text that follows instructions[4].

    These works contribute significantly to the field of long content generation with LLMs, addressing challenges related to output length limitations, coherence, and instruction adherence.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.

Identical first prompt