The AnimatedDiff extension for Automatic1111 needs some work, but the results are promising. See replies for more examples.
(right-click loop and fullscreen for the best effect)
Here's what I learned from testing:
* You need enough VRAM to render all the frames at once, in a single batch.
* 16 frames at 512x512 uses 8.9 GB, 512x768 uses 11.2 GB for me.
* Use the 1.4 motion module. 1.5 almost always has watermarks, and often very little motion.
* Crank the CFG up to 10-25, otherwise the images are faded and lack detail. This seems like a bug in the extension because the original implementation works at normal CFG.
* Going over 75 tokens will change the image halfway through because of the way A1111 handles long prompts, so keep it less than 75.
* Face restore seems less stable between frames, try turning it off.
* Dynamic prompts should be removed, or they'll change for each frame (which might be neat if done intentionally).
* The motion modules were trained on 16 images. You can do more or less but don't expect good results. Using 24 frames often has too much change between frames but can still produce good results. 8 frames is good enough for an image to look "alive" but not long enough to show much motion.
* Remember to edit ui-config.json after running the plugin to set your defaults. Search for "animatediff" and set the motion module and output formats.
AnimateDiff combines well with One Button Prompt, but be careful not to go over 75 tokens. Try turning down the complexity to 3 or 4.
#stablediffusion #aiart #animatediff #grownostr