Dog-Breed Text-to-image Flow Matching from Pretraining to Fine-Tuning on 4 Consumer GPUs

TL;DR

The MiniDog framework enables efficient training of text-to-image diffusion models on consumer GPUs. It achieves pretraining in 55 minutes and fine-tuning in 15 minutes using four 3090 GPUs. This framework makes training such models more accessible.

Why It Matters

The development of MiniDog has practical significance as it lowers the barrier to entry for training text-to-image models, allowing more researchers and developers to explore this field. This can lead to more innovative applications and advancements in areas like image generation and multimodal learning. It also demonstrates the potential for efficient use of consumer-grade hardware in AI research.

Key Findings

• Efficient pretraining and fine-tuning of diffusion models on consumer GPUs
• Use of E2E-INVAE for semantic latent space
• Achievement of fast training times with precomputed and cached image and text features