import numpy as np import matplotlib.pyplot as plt def f(x): return x - x*3 def…
- User
import numpy as np import matplotlib.pyplot as plt
def f(x): return x - x**3
def fdash(x): return 1 - 3*x**2
t = np.linspace(-2,2,100) plt.plot(t, f(t)) plt.show()
point = -1 learning_rate = 0.01 for k in range(1000): update = learning_rate * fdash(point) point -= update
point
-np.sqrt(1/3)
import matplotlib.animation as animation from matplotlib.animation import PillowWriter
sgd_points = [] point = -5 learning_rate = 0.01 for k in range(1000): update = learning_rate * fdash(point) point -= update sgd_points.append(point)
fig = plt.figure() ax = plt.axes()
plt.xlim(-5,5) plt.ylim(-5,5) ax.grid(alpha=0.5) ax.scatter(-1, f(-1), color='red') x_ = np.linspace(-5, 5, 1000) y_ = f(x_) ax.plot(x_, y_) plt.show()
def animate(i): ax.clear() ax.plot(x_, y_) plt.xlim(-5,5) plt.ylim(-5,5) ax.grid(alpha=0.5) ax.scatter(sgd_points[i], f(sgd_points[i]), color='red')
anim = animation.FuncAnimation(fig, animate, frames=200, interval=20) anim.save('gd.mp4')
Przypadek 2D
def f(x, y): return 2*(x2) + 2xy + 2*(y2) - 6*x
def fdash_x(x, y): return 4x + 2y - 6
def fdash_y(x, y): return 2x + 4y
def gradient(point): _x = point[0][0] _y = point[1][0] return np.array([[ fdash_x(_x, _y) ], [ fdash_y(_x, _y) ]])
point = np.array([[-1], [1]], dtype=np.float64)
for k in range(1000): update = learning_rate * gradient(point) point -= update
point
Problemy
Overshooting (przestrzelenie)
"""
"""
def fdash(x): return 1 - 3*x**2
point = -100 learning_rate = 0.01 for k in range(1000): update = learning_rate * fdash(point) point -= update print(point)
Oscylacja
"""
"""
def fdash(x): return 2*x + 1
point = -10 learning_rate = 1 for k in range(10): update = learning_rate * fdash(point) point -= update print(point)
Rozbieżność (divergence)
def fdash(x): return 2*x + 1
point = -100 learning_rate = 2 for k in range(10): update = learning_rate * fdash(point) point -= update print(point)
Minimum zależy od punktu początkowe
"""
"""
def f(x): return x4 + 3*x3 + 2*x**2
x_ = np.linspace(-3, 2, 100)
plt.plot(x_, f(x_)) plt.ylim(-1,20)
def fdash(x): return 4x**3 + 9x**2 + 4*x
point = -0.6 learning_rate = 0.01 for k in range(1000): update = learning_rate * fdash(point) point -= update print(point)
xp = point
def fdash(x): return 4x**3 + 9x**2 + 4*x
point = -0.61 learning_rate = 0.01 for k in range(1000): update = learning_rate * fdash(point) point -= update print(point)
xs = point
plt.plot(x_, f(x_)) plt.ylim(-1,20) plt.scatter(xp, f(xp), color='red') plt.scatter(xs, f(xs), color='green')
Pochodna może być bardzo mała
"""
"""
def fdash(x): return 10*x**9
point = -1 learning_rate = 0.01 for k in range(100000000): update = learning_rate * fdash(point) point -= update print(point)
Proszę rozszerzyć kod, tak aby móc wizualizować minimalizację funkcji .
- Assistant
To visualize the minimization of a function ( f \colon \mathbb{R}^2 \to \mathbb{R} ), we can extend the existing code by plotting the contours of the function and overlaying the path taken by the gradient descent algorithm. Here's how you can do it:
import numpy as np import matplotlib.pyplot as plt # Define the function f(x, y) and its gradients def f(x, y): return 2*(x**2) + 2*x*y + 2*(y**2) - 6*x def fdash_x(x, y): return 4*x + 2*y - 6 def fdash_y(x, y): return 2*x + 4*y # Gradient function def gradient(point): _x = point[0] _y = point[1] return np.array([fdash_x(_x, _y), fdash_y(_x, _y)]) # Gradient descent parameters point = np.array([-1.0, 1.0]) # Starting point learning_rate = 0.01 iterations = 1000 # Store the points for visualization trajectory = [point.copy()] # Perform gradient descent for k in range(iterations): update = learning_rate * gradient(point) point -= update trajectory.append(point.copy()) # Convert trajectory to a NumPy array for easy indexing trajectory = np.array(trajectory) # Create a meshgrid for plotting the contours x = np.linspace(-3, 3, 400) y = np.linspace(-3, 3, 400) X, Y = np.meshgrid(x, y) Z = f(X, Y) # Plot the contours of the function plt.figure(figsize=(10, 6)) plt.contour(X, Y, Z, levels=50, cmap='jet') plt.colorbar(label='f(x, y)') # Plot the trajectory of the gradient descent plt.plot(trajectory[:, 0], trajectory[:, 1], 'o-', color='red', markersize=3, label='Gradient Descent Path') plt.scatter(trajectory[0, 0], trajectory[0, 1], color='green', s=100, label='Start') plt.scatter(trajectory[-1, 0], trajectory[-1, 1], color='blue', s=100, label='End') # Set plot details plt.title('Gradient Descent Minimization of f(x, y)') plt.xlabel('x') plt.ylabel('y') plt.legend() plt.grid(True) plt.show()Explanation:
-
Function Definition:
- We define the function ( f(x, y) = 2x^2 + 2xy + 2y^2 - 6x ) and its partial derivatives with respect to ( x ) and ( y ).
-
Gradient Calculation:
- The
gradientfunction computes the gradient vector (\nabla f(x, y)).
- The
-
Gradient Descent Algorithm:
- We initialize a starting point and perform gradient descent for a specified number of iterations.
- After each update, we store the current point to visualize the path.
-
Visualization:
- We create a grid of ( x ) and ( y ) values to compute ( Z = f(X, Y) ) for contour plotting.
- We use
plt.contourto plot the contour lines of the function. - We overlay the path of the gradient descent by plotting the stored trajectory.
- We mark the starting point in green and the ending point in blue for clarity.
-
Final Plot:
- The plot displays the contours of the function with the gradient descent path overlaid.
- This visualization helps to understand how the algorithm navigates the function's landscape towards the minimum.
Resulting Plot:
The resulting plot will show contour lines representing the levels of the function ( f(x, y) ). The red line with dots represents the path taken by the gradient descent algorithm from the starting point to the minimum. The starting and ending points are highlighted for reference.
Adjustments:
-
Learning Rate and Iterations:
- You can adjust the
learning_rateanditerationsvariables to see how the algorithm's convergence changes.
- You can adjust the
-
Starting Point:
- Changing the initial
pointcan demonstrate how different starting positions affect the path to the minimum.
- Changing the initial
-
Function Complexity:
- To explore more complex functions, redefine
f,fdash_x, andfdash_yaccordingly.
- To explore more complex functions, redefine
Note:
- Ensure that you have the necessary libraries installed (
numpyandmatplotlib). - This code provides a visual insight into the optimization process in two dimensions, which can be extended to more complex scenarios or functions as needed.
-