Skip to content
TheChatArchiveThe Chat Archive
gpt-4-turbo-2024-04-09· OpenAIAi2 WildChat-4.8M (AI2)

Please summarize the following: Glaze and Nightshade Q & A Downloads Feedback…

  1. User

    Please summarize the following: Glaze and Nightshade Q & A Downloads Feedback Papers/Media

    Glaze and Nightshade Q & A Downloads Feedback Papers/Media A Note on the new attack paper: "Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI" This is a note discussing the results of the new paper "Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI," by Hönig, Rando, Carlini, and Tramèr.

    Update: please note (as we say below) that the image examples below are obtained from our own implementation of noisy upscaler. The authors of the attack have shared their code with us, and we are running tests on their version. There will be updates to this webpage as our tests progress.

    We would first like to thank the authors for rigorously testing Glaze against the strong attacks proposed in their paper, and for responsibly disclosing it to us before making it public. The authors introduced a method called "noisy upscaling," which utilizes diffusion models to iteratively remove artifacts from images, drawing on similar principles as DiffPure. They found this technique to be reasonably effective at bypassing the protection offered by Glaze V1.1 (and similar tools Mist and Anti-dreambooth), and also includes results of its performance on Glaze V2.0.

    Intuitively, the noisy upscaler attack adds a large volume of randomized noise to the image, then runs it through an upscaler with diffusion fill-in. The significant addition of noise forces the diffusion model to replace large portions of the image's pixels with its best guesses as to what should be there. The effect is somewhat analogous to firing many many small pellets at an image (shotgun style), and then using diffusion to fill in the holes, effectively inpainting lots and lots of small regions of the image. What this attack is able to do is to replace many pixels in the image with diffusion inpainting. But in removing Glaze effects, the attack is also throwing away the original art's pixels.

    As one might expect, this means that the more Glaze you remove, the more details you are removing of the original style that the model is trying to learn in its mimicry attempt. Not surprisingly, this affects different art styles in different ways. Smooth/flat color styles with less texture will be more vulnerable, while more textured styles will see their styles degrade much faster along with Glaze under the attack.

    After the authors shared their work with us, we were able to implement their noisy upscaler for local testing. In our tests, we verified some of the authors' claims: the method did have significant impact in removing Glaze V1 protection, and had some impact on Glaze V2.0 protection. However, like many prior removal techniques, this attack often pays a price of significantly degraded image quality. The degradation is particularly problematic for faces, textured art, and high-contrast artwork.

    Key Difference of Opinion. There is one important difference in how we view the issue of protective tools and the opinions expressed in the paper. The paper's key message is that we should not develop or release protective tools that can be imperfect or broken in the future, and that imperfect protection is worse than no protection at all. We disagree. Security is an ongoing battle, and protective tools like anti-virus scanners, network firewalls, and email spam filters are examples of valuable tools that have great utility despite the lack of future proof guarantees.

    More importantly, there are real harms done to artists on a daily basis, and a protective tool can have a huge impact even if it eventually is broken in the future (and is then repaired/updated). Most/all artists today understand this is an ongoing battle, that the protection of their artistic creations requires more than a one-pass solution. So as attacks come, we will be here to analyze them, understand them, and work to improve Glaze (and other tools) to overcome them. And artists will update their tools and re-glaze their images as needed. Will it be perfect? Probably not. But we will do our best to be transparent with artists and help them understand risks and limitations of these tools. For our part, the effort is worth it. Respectfully, we believe the authors perhaps underestimate artists' resilience and the steps they are willing to take in order to protect their art.

    Next steps. Having analyzed this new attack on both v1.1 and v2.0, we have added some short term tweaks to the v2.0 algorithm to provide additional robustness against noisy upscaling. The new update Glaze v2.1 is ready for download later today, and will be integrated into WebGlaze later this week. Longer term, we are planning to develop an even stronger update against similar types of attacks. Below, we included some relevant examples of images that illustrate a) the noisy upscaler's limitations on certain image types, and b) some samples of the performance of Glaze 2.1 under the noisy upscaler attack. While we cannot guarantee our implementation of the attack is identical to that described in the paper, it is accurate to the best of our knowledge, and in our tests, it successfully replicates the paper's results on v1.1 samples.

    A sample before and after (Our implementation of) noisy upscaler. This illustrates the noisy upscaler's impact on images with detailed textures and those with lower contrast. We will add more image samples shortly.

    Glaze 2.1 Results under (our implementation of) noisy upscaler. These images from artists capture three different types of images, detailed photographs, high textured art, and art with more smooth gradients/flat color elements.

    ALL ARTWORKS AND IMAGES GLAZED AND POSTED WITH PERMISSION. ALL RIGHTS RESERVED BY RESPECTIVE ARTISTS. SAND LAB, UNIVERSITY OF CHICAGO Copyright ©2024 . All Rights Reserved. — Designed with love by Untree.co Distributed by ThemeWagon

  2. Assistant

    The document discusses a new attack paper titled "Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI" by Hönig, Rando, Carlini, and Tramèr, which explores the limitations of the protection tool called Glaze against generative AI. The authors of the paper introduced an attack method termed "noisy upscaling" that involves adding noise to images and using diffusion models to iteratively fill in details. This method has proven somewhat effective in bypassing Glaze's protections.

    Glaze had to adapt its technology in response to the noisy upscaler; updates to Glaze V2.1 have been made to improve robustness against this type of attack. The document emphasizes that while security tools like Glaze may not provide unbreakable protections, they are crucial as they can still offer substantial protection against many types of attacks. It underscores the importance of continuing to update and improve tools like Glaze, as it aligns with how real-world security measures must evolve over time.

    In essence, despite the challenges presented by innovative attacks, protecting artistic creations remains a dynamic and ongoing effort. The Glaze team is committed to enhancing their tool's effectiveness in the face of new threats. Samples of affected images and tweaks to responses are mentioned as ways to demonstrate adjustments made by Glaze in response to the noisy upscaler attack.

    This upscaler's limitations and impacts on different image styles have also been discussed, showing how it variably affects images with different characteristics. Finally, the advancements in version Glaze 2.1, aiming to provide increased security against adversarial perturbations, are highlighted with the promise of further improvements.