Skip to content
TheChatArchiveThe Chat Archive
composer-2.5· unknownHugging Face ibragim-bad/swe_rebench_07_2026_trajectories

MDC context no longer applied consistently in task/flow logs after upgrade to v1.0…

  1. User

    MDC context no longer applied consistently in task/flow logs after upgrade to v1.0

    Describe the issue

    Hi,

    We recently upgraded from kestra v0.19.5 to v1.0.3 and noticed a change in the logging behavior.

    when using a logback config as such

    <?xml version="1.0" encoding="UTF-8"?>
    <configuration debug="false">
        <include resource="logback/base.xml" />
        <include resource="logback/gcp.xml" />
    
        <root level="WARN">
            <appender-ref ref="CONSOLE_JSON_OUT" />
            <appender-ref ref="CONSOLE_JSON_ERR" />
        </root>
    </configuration>
    

    the additionnal info from the task/flow/execution are not always logged.

    example from a Hello-world flow run:

    {"timestampSeconds":1761224032,"timestampNanos":565000000,"severity":"INFO","thread":"jdbc-execution-executor_0","logger":"execution.gnu_170208","message":"[tenant: main] [namespace: company.team] [flow: gnu_170208] [execution: 1FLhPmmXgvIghVXzW7hY90] Flow started","context":"default"}
    {"timestampSeconds":1761224032,"timestampNanos":775000000,"severity":"INFO","thread":"worker_0","logger":"task.gnu_170208.hello","message":"[tenant: main] [namespace: company.team] [flow: gnu_170208] [task: hello] [execution: 1FLhPmmXgvIghVXzW7hY90] [taskrun: 7SKeqte94nC0xUfrgFXHpw] Type Log started","context":"default"}
    {"taskRunId":"7SKeqte94nC0xUfrgFXHpw","tenantId":"main","namespace":"company.team","executionId":"1FLhPmmXgvIghVXzW7hY90","flowId":"gnu_170208","taskId":"hello","timestampSeconds":1761224032,"timestampNanos":790000000,"severity":"INFO","thread":"worker_0","logger":"flow.gnu_170208.hello","message":"Hello World! 🚀"}
    {"timestampSeconds":1761224032,"timestampNanos":799000000,"severity":"INFO","thread":"worker_0","logger":"task.gnu_170208.hello","message":"[tenant: main] [namespace: company.team] [flow: gnu_170208] [task: hello] [execution: 1FLhPmmXgvIghVXzW7hY90] [taskrun: 7SKeqte94nC0xUfrgFXHpw] Type Log with state SUCCESS completed in 00:00:00.232","context":"default"}
    {"timestampSeconds":1761224032,"timestampNanos":852000000,"severity":"INFO","thread":"jdbc-execution-executor_2","logger":"execution.gnu_170208","message":"[tenant: main] [namespace: company.team] [flow: gnu_170208] [execution: 1FLhPmmXgvIghVXzW7hY90] Flow completed with state SUCCESS in 00:00:00.405","context":"default"}
    
    

    Only the hello world log itself gets the MDC context.

    It was different in v0.19.5 we relied on this behavior to monitor flow executions.

    I've debug kestra locally and i've been able to add that context by manually setting the contextMap in this method https://github.com/kestra-io/kestra/blob/eb22d3f6ee5b6b92f8a12f03948e7517a02e39cc/core/src/main/java/io/kestra/core/services/LogService.java#L45

    such as :

    public void logExecution(Execution execution, Level level, String message, Object... args) {
        Logger logger = logger(execution);
        MDC.setContextMap(Map.of("tenantId", execution.getTenantId(), "namespace", execution.getNamespace(), "flowId", execution.getFlowId(), "executionId", execution.getId()));
    
        logExecution(execution, logger, level, message, args);
    }
    

    Environment

    • Kestra Version: develop, v1.0

    Interface

    No new interfaces are introduced.

  2. Assistant

    Investigating how MDC context is applied in task and flow logging after the v1.0 upgrade.

  3. Assistant

    Implementing MDC context in Logs.java so framework logs include structured fields in JSON layouts, matching RunContextLogger behavior.

  4. Assistant

    Root cause

    In v1.0, framework logs (execution/task lifecycle messages) go through Logs.java, which only embeds context in the message string:

    [tenant: main] [namespace: company.team] [flow: gnu_170208] ...
    

    Task content logs (e.g. "Hello World!") go through RunContextLogger, which sets MDC via an isolated LoggerContext using LogEntry.toMap(). Your GCP JSON layout (StackdriverJsonLayout) reads MDC fields, so only those task logs got structured fields like tenantId, executionId, etc.

    Fix

    Updated Logs.java to set SLF4J MDC before each log call, using the same field names as LogEntry.toMap():

    • Execution logs: tenantId, namespace, flowId, executionId
    • Task run logs: above + taskId, taskRunId
    • Trigger logs: above + triggerId
    • Flow logs: tenantId, namespace, flowId

    A withMdc() helper saves and restores the previous MDC context, which matters because these methods run on shared thread pools (jdbc-execution-executor_*, worker_*).

    Tests

    Added MDC assertions to LogsTest and a test verifying MDC is restored after logging. All 5 tests pass.

    After this change, executor/worker framework logs will include the same structured MDC fields as task content logs in your JSON output, restoring the monitoring behavior you had in v0.19.5.The MDC fix is in place and verified: Logs.java now sets SLF4J MDC for execution, task, trigger, and flow framework logs, and all 5 LogsTest tests pass. Framework logs should now include structured fields (tenantId, executionId, etc.) in your GCP JSON output, matching task content logs.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.