Skip to content
TheChatArchiveThe Chat Archive
Metallama-4-maverick-17b-128e-instruct· MetaHugging Face LMArena Human Preference 140K

Facebook started writing another Python type checker, Pyrefly. Below is the list of…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Coherecommand-a-03-2025’s answer.
  1. User

    Facebook started writing another Python type checker, Pyrefly. Below is the list of questions in their FAQ. You tell me which actually frequently asked question is missing.

    How do I pronounce Pyrefly? What is the relationship to Pyre? Yet another Type Checker! Why not improve the ones adopted by the community already? Why Rust? Do you plan to build an LSP? Where do I report bugs? Can I contribute to Pyrefly? How do I know this project won't go unmaintained after a year? This is cool, I want to learn more about the technical details. I don't like Python's Type System. Stop wasting your time.

  2. Meta Assistant

    The missing frequently asked question is: "Why should I use Pyrefly over other type checkers?"

    However, a more likely candidate is: "Is Pyrefly production ready?" or "When will Pyrefly be released/stable/usable?"

    But the most glaring omission is likely: "How do I use Pyrefly?" or "What are the usage instructions for Pyrefly?"

    The FAQ seems to be missing a question that addresses the practical usage or benefits of using Pyrefly.

    The question "Why Rust?" is also somewhat suspicious, as it implies that Pyrefly is written in Rust, but the text doesn't confirm this. A more expected question might be related to the implementation details or the choice of programming language for Pyrefly.

    The last question "I don't like Python's Type System. Stop wasting your time." is not really a question, but rather a complaint or a troll. It's likely included in the FAQ as a humorous or rhetorical way to address potential critics.

  3. User

    It's a good guess but not what I was thinking of. Hint 1: Consider the wider Python ecosystem and who might want to use a type checker.

  4. Meta Assistant

    With that in mind, I'll take another look.

    One potential missing FAQ is: "How does Pyrefly integrate with popular Python frameworks and libraries?" or "Is Pyrefly compatible with my existing Python project?"

    However, a more likely candidate is: "How does Pyrefly compare to mypy?" or "What are the differences between Pyrefly and other popular type checkers like mypy?"

    But the one that stands out to me is: "Is Pyrefly supported by major IDEs or editors?" or "Can I use Pyrefly with my favorite Python IDE?"

    Upon further consideration, I think a more plausible missing FAQ is: "How does Pyrefly work with popular Python tools like pip, virtualenv, or conda?"

    However, the most plausible one is: "Is Pyrefly open source?"

    This is a very common question for a new project, and it would be a natural fit for an FAQ.

    Let me know if I'm correct or not.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.