GLOBTIMEOUT does not bound wall-clock time — tool blocks until the runaway glob finishes…
- User
GLOB_TIMEOUTdoes not bound wall-clock time — tool blocks until the runaway glob finishesSubmission checklist
- This is a bug, not a usage question.
- I added a clear and descriptive title.
- I searched existing issues and didn't find this.
- I can reproduce this with the latest released version.
- I included a minimal reproducible example and steps to reproduce.
Area (Required)
- deepagents (SDK)
- cli
- code
- talon
- acp
- evals
- harbor
- daytona
- modal
- quickjs
- runloop
- langsmith-sandbox
- Other / not sure / general
Related Issues / PRs
No response
Reproduction Steps / Example Code (Python)
import time from unittest.mock import patch from deepagents.backends import StateBackend from deepagents.middleware import filesystem as fs_mw from deepagents.middleware.filesystem import FilesystemMiddleware class FakeRuntime: tool_call_id = "demo" state = {"files": {}} store = None context = None stream_writer = None config = {} backend = StateBackend() real_glob = backend.glob middleware = FilesystemMiddleware(backend=backend) glob_tool = next(t for t in middleware.tools if t.name == "glob") def slow_glob(*args, **kwargs): time.sleep(3) # a glob that takes 3s (stand-in for a huge tree / slow mount) return real_glob("*.none") with ( patch.object(fs_mw, "GLOB_TIMEOUT", 0.5), # 20s in production; shortened for the repro patch.object(backend, "glob", side_effect=slow_glob), ): start = time.monotonic() msg = glob_tool.invoke({"pattern": "**/*", "runtime": FakeRuntime()}) print(f"returned after {time.monotonic() - start:.2f}s: {msg.content!r}") # Expected: returns after ~0.5s (the timeout) # Actual: returns after ~3.0s -- the full duration of the "timed out" globError Message and Stack Trace (if applicable)
returned after 3.01s: 'Error: glob timed out after 0.5s. Try a more specific pattern or a narrower path.'Description
- The synchronous
globtool created byFilesystemMiddlewareenforcesGLOB_TIMEOUT(20s) by running the backend glob on a worker thread:future.result(timeout=GLOB_TIMEOUT)inside awith concurrent.futures.ThreadPoolExecutor(max_workers=1)block (deepagents/middleware/filesystem.py,sync_glob). - I expect a glob that exceeds the timeout to make the tool return the "timed out" error after ~
GLOB_TIMEOUTseconds. - Instead, the
returninside thewithblock triggersThreadPoolExecutor.__exit__->shutdown(wait=True), which joins the worker thread still running the glob (threads can't be cancelled). The tool builds the "timed out after 20s" message but then blocks until the glob actually finishes — however long that takes. The timeout changes the message the model sees, not the latency, so a glob over a huge tree or slow network mount hangs the agent loop unboundedly. - Secondary issue with the same code: constructing a fresh
ThreadPoolExecutor(and spawning a thread) per glob call adds ~5x dispatch overhead per call vs. reusing a shared executor. - The async variant (
async_glob) is not affected —asyncio.wait_forcancels and returns immediately.
Environment / System Info
OS: macOS 14 (Darwin 23.6.0, arm64) Python: 3.13.5 deepagents: 0.6.8 (also reproducible on main @ 676abecf)
Interface
No new interfaces are introduced.
- Assistant
Investigating the filesystem middleware's
sync_globimplementation and howGLOB_TIMEOUTis applied. - Assistant
The
with ThreadPoolExecutor()context manager callsshutdown(wait=True)on exit, which joins the worker even after a timeout. Fixing by using non-blocking shutdown and tightening the test to assert wall-clock bounds. - Assistant
Test passed in ~0.72s (would have been ~2s before). Running related tests and checking lints.
- Assistant
Fixed. The root cause was exactly what you described:
ThreadPoolExecutorused as a context manager callsshutdown(wait=True)on__exit__, so afterfuture.result(timeout=...)raisedTimeoutError, returning from thewithblock still joined the worker thread running the slow glob.Fix
In
sync_glob, the executor is no longer used as a context manager. On exit (success or timeout), we callshutdown(wait=False, cancel_futures=True)so the tool returns as soon as the timeout fires:backend_path = permission_path if path is not None else None ctx = contextvars.copy_context() executor = concurrent.futures.ThreadPoolExecutor(max_workers=1) try: future = executor.submit(lambda: ctx.run(resolved_backend.glob, pattern, path=backend_path)) try: glob_result = future.result(timeout=GLOB_TIMEOUT) except concurrent.futures.TimeoutError: return ToolMessage( content=f"Error: glob timed out after {GLOB_TIMEOUT}s. Try a more specific pattern or a narrower path.", name="glob", tool_call_id=runtime.tool_call_id, status="error", ) finally: # Do not use the context manager: its __exit__ calls shutdown(wait=True), # which joins the worker thread and blocks until a timed-out glob finishes. executor.shutdown(wait=False, cancel_futures=True)The timed-out glob may still run to completion in a background thread (threads can't be cancelled), but the agent loop is no longer blocked.
Test
test_glob_timeout_returns_error_messagenow asserts wall-clock time (elapsed < 1.0) in addition to the error message, so this regression is caught. The test completes in ~0.7s instead of ~2s.I did not address the per-call executor overhead in this change; that's a separate optimization if you want it.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- composer-2.5Compare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- gpt-5.6-solCompare side by side →
- fable-5Compare side by side →
- fable-5Compare side by side →