Please browse the web to collect the list of post-2021 works on long content generation…
- User
Please browse the web to collect the list of post-2021 works on long content generation with LLMs. You can cite up to 15 works.
- Assistant
Here is a list of notable works published after 2021 focusing on long content generation using Large Language Models (LLMs):
-
LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs (August 2024)
- Authors: Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li
- Summary: This paper introduces AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling LLMs to generate coherent outputs exceeding 20,000 words. The authors also present LongWriter-6k, a dataset containing 6,000 supervised fine-tuning data with output lengths ranging from 2,000 to 32,000 words, and develop LongBench-Write, a benchmark for evaluating ultra-long generation capabilities[1].
-
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text (May 2023)
- Authors: Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui, Tiannan Wang, Zhenxin Xiao, Yifan Hou, Ryan Cotterell, Mrinmaya Sachan
- Summary: RecurrentGPT introduces a language-based simulation of the recurrence mechanism in RNNs, enabling GPT models to generate texts of arbitrary length without forgetting. The approach involves generating a paragraph of text at each timestep and updating its language-based long-short term memory stored on the hard drive and the prompt, respectively[2].
-
Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach (July 2024)
- Authors: Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Michael Bendersky
- Summary: This study compares Retrieval Augmented Generation (RAG) and long-context LLMs, aiming to leverage the strengths of both. The authors propose Self-Route, a method that routes queries to RAG or long-context models based on model self-reflection, significantly reducing computation cost while maintaining comparable performance to long-context models[3].
-
LongGenBench: Benchmarking Long-Form Generation in Long Context LLMs (September 2024)
- Authors: Yuhao Wu, Ming Shan Hee, Zhiqing Hu, Roy Ka-Wei Lee
- Summary: LongGenBench is a benchmark designed to evaluate the long-form text generation capabilities of long-context LLMs. The benchmark tests models' ability to identify specific events within generated long text sequences, highlighting challenges in generating coherent long-form text that follows instructions[4].
These works contribute significantly to the field of long content generation with LLMs, addressing challenges related to output length limitations, coherence, and instruction adherence.
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.