mirrors/stable-diffusion-webui
1
Fork 0
mirror of https://github.com/AUTOMATIC1111/stable-diffusion-webui.git synced 2025-12-21 09:30:30 -08:00
Code Issues Projects Releases Packages Wiki Activity

Add CPU fp8 support

Browse source 
Since norm layer need fp32, I only convert the linear operation layer(conv2d/linear)

And TE have some pytorch function not support bf16 amp in CPU. I add a condition to indicate if the autocast is for unet.
This commit is contained in:
Kohaku-Blueleaf 2023-10-24 01:49:05 +08:00
parent 5f9ddfa46f
commit eaa9f5162f
3 changed files with 22 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

2
modules/processing.py
View file

@ -865,7 +865,7 @@ def process_images_inner(p: StableDiffusionProcessing) -> Processed:
if p.n_iter > 1:
shared.state.job = f"Batch {n+1} out of {p.n_iter}"
with devices.without_autocast() if devices.unet_needs_upcast else devices.autocast():
with devices.without_autocast() if devices.unet_needs_upcast else devices.autocast(unet=True):
samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
if getattr(samples_ddim, 'already_decoded', False):