Breakdown of Models and Workflows Platform agnostic

What are these?

INFO

What we call Models are referred to as Concepts over at Graydient and its native ecosystem.

We chose to stick with the term "Models" because this is what you'll see these referred to as in the general Stable Diffusion ecosystem.

Here's a secret, promise not to tell anyone! I was already familiar with Stable Diffusion and the community around it before I was familiar with Graydient and their ecosystem. Thus, when I see "Concepts" in my head I automatically read it as "Models". We're a little too far in to change course now!

Glad you asked! These will be the heart of your generative journey with Graydient, and thus here at BitVector! (Checkpoint) Models & Workflows act as "Engines" in a manner of speaking, they ultimately influence what sort of results (Renders) you get back from your requests (Jobs).

At BitVector, we support the usage of the majority of Models and Workflows that Graydient has to offer - there are some exceptions, but the ones that most people will end up using are supported on the BitVector platform.

TIP

There are some differences to be aware of between the (Checkpoint) Models system and the Workflows system - but every now and then we'll use the term Renderer. We use this to indicate "Any Workflow or Model can be used here". At the end of the day, they are both used to "create something".

Saying "Both Models and Workflows" is a bit of a mouthful, so we condense it down to Renderer for the sake of brevity.

Checkpoint Models vs Workflows

So, what is the difference?

I'll bet that is your next question, naturally! There are quite a few technical distinctions between these, but explaining all of the technical differences would require a wiki of its own, and would probably be quite boring to most people!

WARNING

There are actually multiple types/categories of Models, we'll touch on that a bit more later, but the below note is regarding Checkpoints ("Full Models") in specific.

At the risk of making an overgeneralization, most of the time Models use older generation architectures, while Workflows typically use newer generation architectures. If you're by chance familiar with the Stable Diffusion ecosystem, models are generally SDXL (plus its sub-archetypes such as "Pony" and "Illustrious" models) and SD1.5 based. There was also SD3, but we don't talk about that.

TIP

Coming from PirateDiffusion? Here's the quicker breakdown:

Checkpoint/Model-based requests are the equivalent of /render Your fancy prompt here <some-checkpoint-here>

Workflow based requests are the equivalent of /workflow /run:some-workflow-here Your fancy prompt here

Now, we could talk about the technical differences between those architectures, but for the sake of keeping things a bit more brief I'll leave that part out. Lets keep it simple instead:

Checkpoint Models (SD1.5 + SDXL):

• Prompting is typically "keyword" based, with those keywords being separated by commas
• Tends to have more limits on resolutions it can create at
• Lighter in terms of required resources, but is also faster due to this
• Can only generate images (technically, there was "SVD" - Stable Video Diffusion, but similar to SD3 we don't really talk about this one - it was superseded fairly quickly)
• Has a lot of flexibility due to how long they've been around for
  • For example, if you're wanting to generate "anime-style" images, Pony/Illustrious models are still "the king" of this style of images.
  • Has a LOT of LoRAs (more on this later) available

Workflows

• Generally works with natural language processing
  • The "prompt reading" mechanism, so to speak, uses tech that is far closer to an LLM (and is literally an LLM in some cases) so can understand nuances like "A zombie carrying a lunchbox in its left hand" far better than SD1.5/SDXL would. SD1.5/SDXL would probably give you a zombie and a lunchbox, and possibly in one of it's hands.
• Are not limited to images - some can generate videos, some can generate audio, and now we're even getting some that can generate videos with audio (which sounds like a "simple" thing but is definitely not from a tech perspective!)
• Tends to have an easier time with more distinct characteristics. For example, SD1.5/SDXL heavily struggles with text. A lot of the newer generation tech can handle text with relative ease.

Well actually...

Workflows are more akin to a "runnable recipe". They do use models "under the hood" - but due to how these newer models are architected, using the "raw model" would not be as simple as "Load model -> Enter prompt -> Click run". There is a lot that goes on behind-the-scenes when you run a Workflow - so from your perspective, the process looks the same (Choose a renderer, enter a prompt, and click run) but it is why there is a need for separate systems.

If you want to know more of what I mean by this, look up "ComfyUI Workflows" in a Google Image search, and behold how crazy it gets!

WARNING

To keep things fair and to ensure compute resources can be used by all customers, Graydient enforces a three minute limit on Workflow runs. The policy is basically "You can use as insane of settings (such as resolution) that you want, so long as the combination of settings you set can run in three minutes or less". Once a request has hit three minutes, the request is forcefully stopped and will result in a timeout error.

Technically, this applies to the older model render style system, but since the archetypes of those models are so much lighter, it is far more difficult to hit the three minute limit. Workflows however typically have heavier models used behind-the-scenes, and thus if you crank the settings up too high, you may hit this limit. Try it anyways though, as there's no actual consequence other than needing to retry the request with potentially lower settings!