Basic Training Background color issue

[IsaacAkintaro]

Describe the bug

Hello!
Thanks so much for the tutorial. Here is what I have noticed:
There is a color background issue with the huggingface diffusion tutorial: https://huggingface.co/docs/diffusers/tutorials/basic_training

This was the output from the tutorial, you can see the background colour slightly in each of the images after 50 epochs:

Copying the unet structure and learning rate I get similar results after 10 epochs:

However, running 10 epochs, with a smaller unet and bigger learning rate (5e-4), after 10 epochs, the coloration gets even more pronounced:
The background colour seems to correspond to the colour of the butterflies.

I noticed this issue when I work with my own data see original and generated output, when I use a bigger unet and 1e-4 learning rate.

Reproduction

Basic Training PyTorch Implementation:
https://huggingface.co/docs/diffusers/tutorials/basic_training

Logs

No response

System Info

Tutorial available via Google Colab.

Who can help?

@DN6 @sayakpaul

[sayakpaul]

This is a training dynamics related question and not a library issue per se. So, converting this into a discussion.