LLMs, GenAI, RAG, agents, evaluation · insight-0573

Sweep guidance strength as a fidelity-versus-diversity knob, like temperature

Situation
A team generating synthetic training images sets a high guidance scale because individual samples look sharpest.
Action
Sweep the guidance scale, measure fidelity and diversity on the sweep, and pick the setting by the downstream goal, higher for single showcase images, lower for training-data variety.
Reason
Classifier-free guidance was shown to trade sample diversity for fidelity, similar in spirit to low-temperature sampling in language models.
Boundary
Conditional diffusion models for images, audio and other modalities.

Symptom or risk

Samples become prompt-faithful but near-identical, so the augmented dataset adds little variety and downstream models do not improve.

Models or products that fix guidance internally leave only prompt and seed variation.

Higher guidance improves prompt adherence but reduces variety.

Choosing settings from a few attractive samples.

Ho and Salimans show that guidance trades sample diversity for fidelity, in the spirit of low-temperature or truncated sampling.
Classifier-Free Diffusion Guidance