Skip to content
TheChatArchiveThe Chat Archive
o1-preview-2024-09-12· OpenAIAi2 WildChat-4.8M (AI2)

import numpy as np import matplotlib.pyplot as plt def f(x): return x - x*3 def…

  1. User

    import numpy as np import matplotlib.pyplot as plt

    def f(x): return x - x**3

    def fdash(x): return 1 - 3*x**2

    t = np.linspace(-2,2,100) plt.plot(t, f(t)) plt.show()

    point = -1 learning_rate = 0.01 for k in range(1000): update = learning_rate * fdash(point) point -= update

    point

    -np.sqrt(1/3)

    import matplotlib.animation as animation from matplotlib.animation import PillowWriter

    sgd_points = [] point = -5 learning_rate = 0.01 for k in range(1000): update = learning_rate * fdash(point) point -= update sgd_points.append(point)

    fig = plt.figure() ax = plt.axes()

    plt.xlim(-5,5) plt.ylim(-5,5) ax.grid(alpha=0.5) ax.scatter(-1, f(-1), color='red') x_ = np.linspace(-5, 5, 1000) y_ = f(x_) ax.plot(x_, y_) plt.show()

    def animate(i): ax.clear() ax.plot(x_, y_) plt.xlim(-5,5) plt.ylim(-5,5) ax.grid(alpha=0.5) ax.scatter(sgd_points[i], f(sgd_points[i]), color='red')

    anim = animation.FuncAnimation(fig, animate, frames=200, interval=20) anim.save('gd.mp4')

    Przypadek 2D

    def f(x, y): return 2*(x2) + 2xy + 2*(y2) - 6*x

    def fdash_x(x, y): return 4x + 2y - 6

    def fdash_y(x, y): return 2x + 4y

    def gradient(point): _x = point[0][0] _y = point[1][0] return np.array([[ fdash_x(_x, _y) ], [ fdash_y(_x, _y) ]])

    point = np.array([[-1], [1]], dtype=np.float64)

    for k in range(1000): update = learning_rate * gradient(point) point -= update

    point

    Problemy

    Overshooting (przestrzelenie)

    """

    f(x)=x−x3f(x) = x - x^3

    """

    def fdash(x): return 1 - 3*x**2

    point = -100 learning_rate = 0.01 for k in range(1000): update = learning_rate * fdash(point) point -= update print(point)

    Oscylacja

    """

    f(x)=x2+x+1f(x) = x^2 + x + 1

    """

    def fdash(x): return 2*x + 1

    point = -10 learning_rate = 1 for k in range(10): update = learning_rate * fdash(point) point -= update print(point)

    Rozbieżność (divergence)

    def fdash(x): return 2*x + 1

    point = -100 learning_rate = 2 for k in range(10): update = learning_rate * fdash(point) point -= update print(point)

    Minimum zależy od punktu początkowe

    """

    f(x)=x4+3x3+2x2f(x) = x^4 + 3x^3 + 2x^2 f′(x)=4x3+9x2+4xf'(x) = 4x^3 + 9x^2 + 4x

    """

    def f(x): return x4 + 3*x3 + 2*x**2

    x_ = np.linspace(-3, 2, 100)

    plt.plot(x_, f(x_)) plt.ylim(-1,20)

    def fdash(x): return 4x**3 + 9x**2 + 4*x

    point = -0.6 learning_rate = 0.01 for k in range(1000): update = learning_rate * fdash(point) point -= update print(point)

    xp = point

    def fdash(x): return 4x**3 + 9x**2 + 4*x

    point = -0.61 learning_rate = 0.01 for k in range(1000): update = learning_rate * fdash(point) point -= update print(point)

    xs = point

    plt.plot(x_, f(x_)) plt.ylim(-1,20) plt.scatter(xp, f(xp), color='red') plt.scatter(xs, f(xs), color='green')

    Pochodna może być bardzo mała

    """

    f(x)=x10f(x) = x^{10}

    """

    def fdash(x): return 10*x**9

    point = -1 learning_rate = 0.01 for k in range(100000000): update = learning_rate * fdash(point) point -= update print(point)

    Proszę rozszerzyć kod, tak aby móc wizualizować minimalizację funkcji f ⁣:R2→Rf \colon \mathbb{R}^2 \to \mathbb{R}.

  2. Assistant

    To visualize the minimization of a function ( f \colon \mathbb{R}^2 \to \mathbb{R} ), we can extend the existing code by plotting the contours of the function and overlaying the path taken by the gradient descent algorithm. Here's how you can do it:

    import numpy as np
    import matplotlib.pyplot as plt
    
    # Define the function f(x, y) and its gradients
    def f(x, y):
        return 2*(x**2) + 2*x*y + 2*(y**2) - 6*x
    
    def fdash_x(x, y):
        return 4*x + 2*y - 6
    
    def fdash_y(x, y):
        return 2*x + 4*y
    
    # Gradient function
    def gradient(point):
        _x = point[0]
        _y = point[1]
        return np.array([fdash_x(_x, _y), fdash_y(_x, _y)])
    
    # Gradient descent parameters
    point = np.array([-1.0, 1.0])  # Starting point
    learning_rate = 0.01
    iterations = 1000
    
    # Store the points for visualization
    trajectory = [point.copy()]
    
    # Perform gradient descent
    for k in range(iterations):
        update = learning_rate * gradient(point)
        point -= update
        trajectory.append(point.copy())
    
    # Convert trajectory to a NumPy array for easy indexing
    trajectory = np.array(trajectory)
    
    # Create a meshgrid for plotting the contours
    x = np.linspace(-3, 3, 400)
    y = np.linspace(-3, 3, 400)
    X, Y = np.meshgrid(x, y)
    Z = f(X, Y)
    
    # Plot the contours of the function
    plt.figure(figsize=(10, 6))
    plt.contour(X, Y, Z, levels=50, cmap='jet')
    plt.colorbar(label='f(x, y)')
    
    # Plot the trajectory of the gradient descent
    plt.plot(trajectory[:, 0], trajectory[:, 1], 'o-', color='red', markersize=3, label='Gradient Descent Path')
    plt.scatter(trajectory[0, 0], trajectory[0, 1], color='green', s=100, label='Start')
    plt.scatter(trajectory[-1, 0], trajectory[-1, 1], color='blue', s=100, label='End')
    
    # Set plot details
    plt.title('Gradient Descent Minimization of f(x, y)')
    plt.xlabel('x')
    plt.ylabel('y')
    plt.legend()
    plt.grid(True)
    plt.show()
    

    Explanation:

    • Function Definition:

      • We define the function ( f(x, y) = 2x^2 + 2xy + 2y^2 - 6x ) and its partial derivatives with respect to ( x ) and ( y ).
    • Gradient Calculation:

      • The gradient function computes the gradient vector (\nabla f(x, y)).
    • Gradient Descent Algorithm:

      • We initialize a starting point and perform gradient descent for a specified number of iterations.
      • After each update, we store the current point to visualize the path.
    • Visualization:

      • We create a grid of ( x ) and ( y ) values to compute ( Z = f(X, Y) ) for contour plotting.
      • We use plt.contour to plot the contour lines of the function.
      • We overlay the path of the gradient descent by plotting the stored trajectory.
      • We mark the starting point in green and the ending point in blue for clarity.
    • Final Plot:

      • The plot displays the contours of the function with the gradient descent path overlaid.
      • This visualization helps to understand how the algorithm navigates the function's landscape towards the minimum.

    Resulting Plot:

    The resulting plot will show contour lines representing the levels of the function ( f(x, y) ). The red line with dots represents the path taken by the gradient descent algorithm from the starting point to the minimum. The starting and ending points are highlighted for reference.

    Adjustments:

    • Learning Rate and Iterations:

      • You can adjust the learning_rate and iterations variables to see how the algorithm's convergence changes.
    • Starting Point:

      • Changing the initial point can demonstrate how different starting positions affect the path to the minimum.
    • Function Complexity:

      • To explore more complex functions, redefine f, fdash_x, and fdash_y accordingly.

    Note:

    • Ensure that you have the necessary libraries installed (numpy and matplotlib).
    • This code provides a visual insight into the optimization process in two dimensions, which can be extended to more complex scenarios or functions as needed.