Design a computational graph for "Recurrent networks with recurrent connections between…
- User
Design a computational graph for "Recurrent networks with recurrent connections between hidden units, that read an entire sequence and then produce a single output" and develop the forward propagation equations for the same.
- Assistant
Here’s a simple unrolled computational graph for an RNN that reads a sequence
({x_1,\dots,x_T}) and—after the last time step—produces one output (y).flowchart LR direction LR subgraph Unrolled_RNN h0(["h₀"]) x1(["x₁"]) h1(["h₁"]) x2(["x₂"]) h2(["h₂"]) x3(["x₃"]) h3(["h₃"]) XT(["..."]) xT(["x_T"]) hT(["h_T"]) y (["y"]) h0 --> h1 x1 --> h1 h1 --> h2 x2 --> h2 h2 --> h3 x3 --> h3 h3 --> XT XT --> xT xT --> hT h(T-1)(["h_{T-1}"]) --> hT hT --> y endLegend:
- (x_t\in\R^{d_x}) is the input at time (t).
- (h_t\in\R^{d_h}) is the hidden state after seeing (x_t).
- (y\in\R^{d_y}) is the final output (only after time (T)).
Forward-Propagation Equations
-
Initialization
[ h_0 = \mathbf{0}\quad(\text{often a zero‐vector}) ] -
Recurrence (for (t=1,\dots,T))
[ h_t = \sigma_h!\bigl(W_{xh},x_t ;+; W_{hh},h_{t-1} ;+; b_h\bigr) ] • (W_{xh}\in\R^{d_h\times d_x})
• (W_{hh}\in\R^{d_h\times d_h})
• (b_h\in\R^{d_h})
• (\sigma_h(\cdot)) = hidden activation (e.g.\ (\tanh) or ReLU) -
Output (after (t=T))
[ y = \sigma_y!\bigl(W_{hy},h_T ;+; b_y\bigr) ] • (W_{hy}\in\R^{d_y\times d_h})
• (b_y\in\R^{d_y})
• (\sigma_y(\cdot)) = output activation
(e.g.\ softmax for classification, identity for regression)
Walk-Through (toy example, (T=3))
- (h_1 = \sigma_h(W_{xh}x_1 + W_{hh}h_0 + b_h))
- (h_2 = \sigma_h(W_{xh}x_2 + W_{hh}h_1 + b_h))
- (h_3 = \sigma_h(W_{xh}x_3 + W_{hh}h_2 + b_h))
- (y = \sigma_y(W_{hy}h_3 + b_y))
That’s it—by unrolling the recurrence you get a directed acyclic “computational graph,”
and the above equations perform one forward pass from ({x_t}) → ({h_t}) → (y).
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.