workflow ?

#1
by System36 - opened

It doesn't work with the standard ComfyUI workflow node for LTX2.

It doesn't work with the standard ComfyUI workflow node for LTX2.

You can find the workflows here:

https://github.com/Lightricks/ComfyUI-LTXVideo/tree/master/example_workflows

It doesn't work with the standard ComfyUI workflow node for LTX2.

You can find the workflows here:

https://github.com/Lightricks/ComfyUI-LTXVideo/tree/master/example_workflows

LTXVGemmaCLIPModelLoader
No files matching pattern 'tokenizer.model' found under C:\Users\Intersurf\Documents\ComfyUI\models not working with the official workflow

got prompt
!!! Exception during processing !!! No files matching pattern 'tokenizer.model' found under C:\ComfyUI\models
Traceback (most recent call last):
File "C:\ComfyUI\execution.py", line 518, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\execution.py", line 329, in get_output_data
return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\execution.py", line 303, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "C:\ComfyUI\execution.py", line 291, in process_inputs
result = f(**inputs)
^^^^^^^^^^^
File "C:\ComfyUI\custom_nodes\ComfyUI-LTXVideo\gemma_encoder.py", line 571, in load_model
tokenizer_path = Path(find_matching_dir(model_root, "tokenizer.model"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\custom_nodes\ComfyUI-LTXVideo\gemma_encoder.py", line 534, in find_matching_dir
raise FileNotFoundError(
FileNotFoundError: No files matching pattern 'tokenizer.model' found under C:\ComfyUI\models

Prompt executed in 0.42 seconds

You can get the missing files from here:
https://huggingface.co/google/gemma-3-12b-it-qat-q4_0-unquantized/tree/main

or, i took the time to zip all the small files, but you still need the big files from the link above.

https://github.com/LockMan007/PixelSwirl/tree/main/ComfyUI-misc/Gemma-LTX2
"Gemma-missing the 32mb Tokenizer.json and large model files.zip"
my zip file includes:

README.md
added_tokens.json
chat_template.json
config.json
generation_config.json
model.safetensors.index.json
preprocessor_config.json
processor_config.json
special_tokens_map.json
tokenizer.model
tokenizer_config.json

DOES NOT INCLUDE the larger files, this is just the small files!

If this is too confusing, you can manually just download all the files from the original source I zipped from:
https://huggingface.co/google/gemma-3-12b-it-qat-q4_0-unquantized/tree/main

However, I still get ComfyUI crashing on the Gemma loader and I'd rather not download all those 5 shards unless I absolutely have to,
Kijai seems to claim the single "gemma_3_12B_it_fp8_e4m3fn.safetensors" is all you need, but it doesn't work for me.
image

You guys are using the wrong workflow.
Use the native ones: https://blog.comfy.org/p/ltx-2-open-source-audio-video-ai

Unpack the subgraph and replace 'gemma_3_12B_it.safetensors' with this one on its loader node

Yeah, this is confusing that ComfyUI templates have quite different workflows from the one in LTX own github.
One major difference is this node - LTX one has πŸ…›πŸ…£πŸ…§ Gemma 3 Model Loader which needs a folder with shards and tokenizer config, ComfyUI one has LTXV Audio Text Encoder Loader which expects a merged Gemma model (also a simple merge with usual Python merging scripts does not work, there's some trick needed which, as I understand, has been applied to this model here.
Another major difference is the resolution split - LTX workflow is much faster, maybe it generates initial video in different resolution before upscaling, which somehow makes it 2x faster than the ComfyUI template.
And ComfyUI template has a LoRA in the upscaler that messes up the final video, it gets patchy. The LTX workflow does not use that LoRA.

You guys are using the wrong workflow.
Use the native ones: https://blog.comfy.org/p/ltx-2-open-source-audio-video-ai

Unpack the subgraph and replace 'gemma_3_12B_it.safetensors' with this one on its loader node

thx! this wf worked its even slightly different from the new comfy ui templates

You guys are using the wrong workflow.
Use the native ones: https://blog.comfy.org/p/ltx-2-open-source-audio-video-ai

Unpack the subgraph and replace 'gemma_3_12B_it.safetensors' with this one on its loader node

thx! this wf worked its even slightly different from the new comfy ui templates

Im new to comfyui, can you share your workflow json or a screenshot of the workflow? The readme says "--lowvram isn't needed if using the DualClipLoader, as it can be set to use cpu only." - not sure what to connect since the workflow from comfy doesn't include dualcliploader

The Link I shared: https://blog.comfy.org/p/ltx-2-open-source-audio-video-ai contains a few blue buttons for the workflows, T2V, I2V and a few others. Just scroll down a little and you won't miss them.
Click on them and a new tab will open in your browser then select that tab, right click -> save as -> it will save the workflow as a .json file.
Put it where you usually save your comfy workflows and boot it up.

The native workflows use a subgraph - its basically a single node that contains many nodes inside it. Select it and a tooltip will show up above that node with some buttons. One of those buttons will display a text tooltip (when you place the mouse over it and wait a second or two) saying 'Unpack'. Click on that button and now you have the full workflow visible. You will see the 'notes' nodes and the 'Save Video' node are not properly positioned so you need to drag them to a position where they won't obstruct the other nodes.

Find the node that loads the 'gemma_3_12B_it.safetensors' model and replace it with the one from here. Should be named 'LTXAVTextEncoderLoader' or something like that.

ATTENTION

ComfyUI very recently pushed an update that adds the ability to set the device on the 'LTXAVTextEncoderLoader' node: https://github.com/Comfy-Org/ComfyUI/commit/1c705f7bfb0fb59f6213dfb85ec5d5dc2ce4300e#diff-a57f6e0ff3b8633183b3eabfa31261f195ad582b920419694abd068b39874bbd.
Unless you have a BEEFY GPU - set it to 'CPU'. You need to update ComfyUI to the very latest branch (not release) in order to get this. Setting the device to CPU will solve most out of memory problems related to this text encoder - even at FP8 you would still have those -.-
Alternatively, you could boot up ComfyUI with '--novram --reserve-vram 10' - this will force everything you load to use the CPU as the main device by default and only sends it to GPU when that's possible - while reserving 10% of you VRAM at all times just to be safe. You probably don't need this anymore if you are using the latest branch.

EDIT

Some additional tips:

  • Replace the main model to the FP8 distilled one and disable the node that is loading the distilled lora (because the model itself is already distilled).
  • Number of steps and CFG value are still the same even with the distilled model.
  • For I2V workflows you need to replace the node that is doing the resizing with a custom one that does a better job based on your image input dimensions. Otherwise that node will not preserve the aspect ratio of your input unless you manually change its values every time and on top of that - it will more often than not CROP your image by the center -.- WAN 2.2 I2V native workflows also do this crap so its not something new. But we can do better than this. It seems however, its not very good with Tall aspect ratios - but it can do them nonetheless.

Sign up or log in to comment