Skip to content
TheChatArchiveThe Chat Archive
gpt-4.1-2025-04-14· OpenAIHugging Face LMArena Human Preference 140K

parsing as AST doesn't do what I want. I want to check the input string interpreted as a…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Geminigemini-2.0-flash-001’s answer.
  1. User

    parsing as AST doesn't do what I want. I want to check the input string interpreted as a python program. Running it in a subprocess also doesn't really fix the issue of exec being dangerous.

  2. Assistant

    You're absolutely right:

    • Parsing as AST only checks syntax, not runtime behavior.
    • Running in a subprocess still executes arbitrary code, which is dangerous if the input is untrusted.

    If you want to "interpret the input string as a Python program" but avoid the dangers of exec or subprocess, there is no way to safely do this for arbitrary code.
    Python does not provide a built-in, secure sandbox for executing untrusted code. Any method that actually "interprets" or "runs" the code (not just parsing) will have the same risks as exec.

    What are your options?

    1. If you trust the input:

    You can use exec or subprocess as before.

    2. If you do NOT trust the input:

    • You cannot safely interpret arbitrary Python code in the same process.
    • Even restricted environments like restrictedpython or sandboxed interpreters are not fully secure.
    • The only truly safe way is to run the code in a completely isolated environment (e.g., a Docker container, a VM, or a separate process with OS-level sandboxing and resource limits).

    3. If you only want to check for exceptions in a "safe" subset of code:

    • You could write a custom parser/interpreter for a very limited subset of Python, but this is a non-trivial task.

    Summary

    • There is no way to safely interpret arbitrary Python code from a string without the risks of exec or subprocess.
    • If you must do this, only do so with trusted input, and be aware of the risks.
    • If you want to check for exceptions without running the code, it's not possible.

    Example (with trusted input, for completeness):

    import sys
    
    def main():
        input_code = sys.stdin.read()
        try:
            exec(input_code)
            print("ok")
        except Exception:
            print("exception detected", file=sys.stderr)
    
    if __name__ == "__main__":
        main()
    

    But again: this is dangerous for untrusted input.


    If you want to discuss safe alternatives or clarify your use case, let me know!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.