Lower VRAM usage by only having one model loaded at a time

[bolshoytoster]

This fork changes Interrogator to only load BLIP to VRAM on init, and leave CLIP in RAM until it's needed.

When interrogate is first called, it does BLIP inference, unloads it, loads CLIP, then does CLIP inference. 'Unloaded' in this case just means 'in RAM'.

Using this, I can run classic/fast interrogation on 4GB of VRAM, 'best' is still a little too big however.

This could cause performance issues when processing a large number of images, so I also added interrogate_batch and corresponding interrogate_{classic,fast}_batch. There was no need to add a batch function for negative interrogation since it only uses CLIP.

This also includes automatic black formatting and extra type hints, which can be removed if you want.

I'd like to contribute my changes, since this makes clip-interrogator more accessible.

It could maybe be disabled by default due to slight performance issues for code that doesn't use one of the batch functions and could just be enabled through Config.

[pharmapsychotic]

Thanks bolshoytoster! That's really cool idea to do full pass over batch with BLIP then second pass over it with CLIP! I'm looking at VRAM usage right now

[bolshoytoster]

I also found another way to reduce VRAM usage to <4 GB on the standard interrogation, but it reduces accuracy and involves editing the open_clip package. I'll try to find a way to avoid that or open a PR to there to allow it.

Edit: Nevermind, my latest commit fixes a bug that meant it was using float32 instead of float16 while using GPU, it currently runs on my 4GB GPU, but takes ~10 minutes.