Torchao SD3 int8wo

[WasamiKirua]

Describe the bug

Exception when click Run, SD3 quantization

Reproduction

• Mellon + ModularDiffuser setup (https://github.com/cubiq/Mellon/blob/main/modules/ModularDiffusers/README.md)
• uv pip install -U torch torchao
• python main.py

SD3 Text Encoder Loader: Dtype bfloat16, Quantization: TorchAO, Quant Type: int8wo

Logs

20260314 21.08.15 [ERR] An unexpected error occurred during exec: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)
20260314 21.08.15 [ERR] Error executing node 50pz3Zy-IBsosDgI_t6gC (modules.Experiments.SD3TextEncodersLoader)
20260314 21.08.15 [ERR] Traceback (most recent call last):
  File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 216, in __call__
    output = getattr(self, self.CALLBACK)(**self.params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in execute
    t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 465, in mm_exec
    return memory_manager.exec(func, device, models, exclude, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 188, in exec
    raise e
  File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 172, in exec
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in <lambda>
    t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 52, in quantize
    torchao(model, quant_type=torchao_quant_type)
  File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 70, in torchao
    dtype = get_torchao_quant_method(quant_type)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 96, in get_torchao_quant_method
    from torchao.quantization import (
ImportError: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/media/wasami/comfyui/Mellon/mellon/server.py", line 1015, in execute_node
    self.node_cache[id](**args)
  File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 220, in __call__
    raise RuntimeError(f"Error executing {self.module_name}.{self.class_name}: {e}")
RuntimeError: Error executing modules.Experiments.SD3TextEncodersLoader: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)

20260314 21.08.15 [ERR] Error occurred in Traceback (most recent call last):
  File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 216, in __call__
    output = getattr(self, self.CALLBACK)(**self.params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in execute
    t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 465, in mm_exec
    return memory_manager.exec(func, device, models, exclude, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 188, in exec
    raise e
  File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 172, in exec
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in <lambda>
    t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 52, in quantize
    torchao(model, quant_type=torchao_quant_type)
  File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 70, in torchao
    dtype = get_torchao_quant_method(quant_type)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 96, in get_torchao_quant_method
    from torchao.quantization import (
ImportError: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/media/wasami/comfyui/Mellon/mellon/server.py", line 318, in _main_worker
    result = await self.loop.run_in_executor(None, partial(task, *args))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wasami/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/wasami/comfyui/Mellon/mellon/server.py", line 916, in execute_graph
    self.execute_node(id, nodes[id], sid)
  File "/media/wasami/comfyui/Mellon/mellon/server.py", line 1019, in execute_node
    raise e
  File "/media/wasami/comfyui/Mellon/mellon/server.py", line 1015, in execute_node
    self.node_cache[id](**args)

System Info

Ubuntu 24.04
RTX 5090

Who can help?

No response

[tdvipp]

This is because Mellon is out of date. I suggest using ComfyUI as an alternative.

[asomoza]

I think this issue should be opened in Mellon though, and probably OP is aware the they can do this with ComfyUI since it has Mellon inside a directory named comfyui.

If you want to test the part of Mellon that is not modular diffusers you should install it with the normal method, you're pointing to the readme for modular diffusers.

The nodes and the examples have a clear distinction of what is from normal Mellon and what is modular diffusers