ONNXConfig: Add a configuration for all available models #16308

chainyo · 2022-03-21T18:15:54Z

chainyo · 2022-03-21T18:20:03Z

GPT-J: add GPT-J ONNX config to Transformers #16274
FlauBERT: Add Flaubert OnnxConfig to Transformers #16279

chainyo · 2022-03-21T19:17:00Z

LayoutLMv2: Add LayoutLMv2 OnnxConfig #16309

vumichien · 2022-03-22T06:14:53Z

Let me try with BigBird

vumichien · 2022-03-26T16:09:44Z

Bigbird add Bigbird ONNX config #16427

LysandreJik · 2022-03-28T09:59:12Z

Love the initiative here, thanks for opening an issue! Added the Good First Issue label so that it's more visible :)

chainyo · 2022-03-29T12:33:49Z

Love the initiative here, thanks for opening an issue! Added the Good First Issue label so that it's more visible :)

Thanks for the label. I don't know if it's easy to begin, but it's cool if more people see this and can contribute!

aakashb95 · 2022-03-31T07:27:39Z

I would like to try with Luke. However, Luke doesn't support any features apart from default AutoModel. It's main feature is LukeForEntityPairClassification for relation extraction. Should I convert luke-base to Onnx or LukeForEntityPairClassification which has a classifier head?

xiadingZ · 2022-03-31T12:09:01Z

Data2vecAudio doesn't have ONNXConfig yet. I write its ONNXConfig according to Data2VecTextOnnxConfig but it throws error. Can anyone help me?

from typing import Mapping, OrderedDict
from transformers.onnx import OnnxConfig

from transformers import AutoConfig
from pathlib import Path
from transformers.onnx import export
from transformers import AutoTokenizer, AutoModel

class Data2VecAudioOnnxConfig(OnnxConfig):
    @property
    def inputs(self):
        return OrderedDict(
            [
                ("input_values", {0: "batch", 1: "sequence"}),
                ("attention_mask", {0: "batch", 1: "sequence"}),
            ]
        )
    
config = AutoConfig.from_pretrained("facebook/data2vec-audio-base-960h")
onnx_config = Data2VecAudioOnnxConfig(config)



onnx_path = Path("facebook/data2vec-audio-base-960h")
model_ckpt = "facebook/data2vec-audio-base-960h"
base_model = AutoModel.from_pretrained(model_ckpt)
tokenizer = AutoTokenizer.from_pretrained(model_ckpt)

onnx_inputs, onnx_outputs = export(tokenizer, base_model, onnx_config, onnx_config.default_onnx_opset, onnx_path)

errors

ValueError                                Traceback (most recent call last)
/var/folders/2t/0w65vdjs2m32w5mmzzgtqrhw0000gn/T/ipykernel_59977/667985886.py in <module>
     27 tokenizer = AutoTokenizer.from_pretrained(model_ckpt)
     28 
---> 29 onnx_inputs, onnx_outputs = export(tokenizer, base_model, onnx_config, onnx_config.default_onnx_opset, onnx_path)

~/miniconda3/lib/python3.9/site-packages/transformers/onnx/convert.py in export(tokenizer, model, config, opset, output)
    255 
    256     if is_torch_available() and issubclass(type(model), PreTrainedModel):
--> 257         return export_pytorch(tokenizer, model, config, opset, output)
    258     elif is_tf_available() and issubclass(type(model), TFPreTrainedModel):
    259         return export_tensorflow(tokenizer, model, config, opset, output)

~/miniconda3/lib/python3.9/site-packages/transformers/onnx/convert.py in export_pytorch(tokenizer, model, config, opset, output)
    112 
    113             if not inputs_match:
--> 114                 raise ValueError("Model and config inputs doesn't match")
    115 
    116             config.patch_ops()

ValueError: Model and config inputs doesn't match

chainyo · 2022-04-01T17:05:35Z

I would like to try with Luke. However, Luke doesn't support any features apart from default AutoModel. It's main feature is LukeForEntityPairClassification for relation extraction. Should I convert luke-base to Onnx or LukeForEntityPairClassification which has a classifier head?

When you implement the ONNX Config for a model it's working for all kind of task, because the base model and the ones pre-packaged for fine-tuning have the same inputs.

So you can base your implementation on the base model and other tasks will work too.

chainyo · 2022-04-02T17:54:38Z

LUKE Add Luke to Onnx #16562 from @aakashb95 👍

jorabara · 2022-04-11T16:23:38Z

Still learning

pikaqqqqqq · 2022-04-14T08:31:53Z

Issue description

Hello, thank you for supporting GPTJ with ONNX. But when I exported an ONNX checkpoint using transformers-4.18.0, I got the issue like below.

(venv) root@V100:~# python -m transformers.onnx --model=gpt-j-6B/ onnx/
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/data/lvenv/lib/python3.8/site-packages/transformers/onnx/__main__.py", line 99, in <module>
    main()
  File "/data/venv/lib/python3.8/site-packages/transformers/onnx/__main__.py", line 62, in main
    raise ValueError(f"Unsupported model type: {config.model_type}")
ValueError: Unsupported model type: gptj

I found GPTJ with ONNX seems supported when I checked your document transformers-4.18.0 [https://huggingface.co/docs/transformers/serialization#exporting-a-model-for-an-unsupported-architecture] and code [src/transformers/onnx/features.py etc.]. But I still got this issue. And then, I checked the parameter of config.model_type in File "/data/venv/lib/python3.8/site-packages/transformers/onnx/main.py", which is related to two parameters [from ..models.auto.feature_extraction_auto import FEATURE_EXTRACTOR_MAPPING_NAMES, from ..models.auto.tokenization_auto import TOKENIZER_MAPPING_NAMES]. I did not find GPTJ's config in these configs. It seems not sensible.

Environment info

Platform: Ubuntu 20.04.2
python: 3.8.10
PyTorch: 1.10.0+cu113
transformers: 4.18.0
GPU: V100

chainyo · 2022-04-14T15:10:59Z

Hello, thank you for supporting GPTJ with ONNX. But when I exported an ONNX checkpoint using transformers-4.18.0, I got the issue like below.

Hello @pikaqqqqqq, thanks for reporting the problem. I opened a PR with a quick fix to avoid this problem, check #16780

chainyo · 2022-04-20T17:13:01Z

ConvBERT: Add OnnxConfig for ConvBERT #16859

skrsna · 2022-04-20T18:14:46Z

Hello 👋🏽, I added RoFormer onnx config here #16861, I'm not 100% sure who to ask for review so I'm posting this here. Thanks 🙏🏽

Tanmay06 · 2022-04-21T13:16:05Z

Hi! I would like try building the ONNX config for Reformer.

chainyo · 2022-04-21T17:40:49Z

Hi! I would like try building the ONNX config for Reformer.

Hi @Tanmay06 that would be awesome. Don't hesitate to open a PR with your work when you feel it's quite good. You can ping me anytime if you need help!

chamidullinr · 2022-04-26T09:52:00Z

Hello! I would like to work on ONNX config for ResNet.

chainyo · 2022-04-26T11:22:40Z

Hello! I would like to work on ONNX config for ResNet.

Nice, don't hesitate to ping me if help is needed 🤗

nandwalritik · 2022-04-28T10:37:25Z

Hi! I would like to work on ONNX config for BigBirdPegasus.

chainyo · 2022-04-28T11:37:37Z

Hi! I would like to work on ONNX config for BigBirdPegasus.

Hi, nice! If you need help you can tag me.

sijunhe · 2022-04-30T14:57:23Z

#17027 Here is one for XLNet!

WaterKnight1998 · 2022-10-07T09:15:09Z

DonutSwin: Adds DonutSwin to models exportable with ONNX #19401

RaghavPrabhakar66 · 2022-10-24T12:17:08Z

@chainyo Hi, I would like to work on TrOCR.

NielsRogge · 2022-10-24T13:14:29Z

TrOCR and Donut are now supported per #19254

chainyo · 2022-10-24T15:28:30Z

@chainyo Hi, I would like to work on TrOCR.

TrOCR and Donut are now supported per #19254

@RaghavPrabhakar66 Maybe there is another model you could implement?

RaghavPrabhakar66 · 2022-10-24T15:56:14Z

Sure. I can work on ImageGPT.

chainyo · 2022-10-28T15:37:16Z

Can we re-open this? Please @sgugger 🤗

RaghavPrabhakar66 · 2022-10-28T18:59:43Z

@chainyo After gaining some experience with ImageGPT, I would like to work on CANINE and DecisionTransformer (if working on more than one model is allowed.)

BakingBrains · 2022-10-29T04:34:23Z

@chainyo would love to take up PoolFormer if there's no one working on it yet?

chainyo · 2022-10-29T12:59:27Z

@chainyo After gaining some experience with ImageGPT, I would like to work on CANINE and DecisionTransformer (if working on more than one model is allowed.)

@RaghavPrabhakar66 Yes of course! 👍

@chainyo would love to take up PoolFormer if there's no one working on it yet?

I don't think so, it's open! 🤗 @BakingBrains

RaghavPrabhakar66 · 2022-10-31T09:41:38Z

@chainyo I was working on Canine and was facing some errors while running the following command:

python -m transformers.onnx onnx --model="google/canine-s"

CanineOnnxConfig:

class CanineOnnxConfig(OnnxConfig):
    @property
    def inputs(self) -> Mapping[str, Mapping[int, str]]:
        if self.task == "multiple-choice":
            dynamic_axis = {0: "batch", 1: "choice", 2: "sequence"}
        else:
            dynamic_axis = {0: "batch", 1: "sequence"}
        return OrderedDict(
            [
                ("input_ids", dynamic_axis),
                ("token_type_ids", dynamic_axis),
                ("attention_mask", dynamic_axis),
            ]
        )
    
    @property
    def default_onnx_opset(self) -> int:
        return 13
    
    def generate_dummy_inputs(
        self,
        preprocessor: "PreTrainedTokenizerBase",
        batch_size: int = 1,
        seq_length: int = 6,
        num_choices: int = -1,
        is_pair: bool = False,
        framework: Optional[TensorType] = None,
        tokenizer: "PreTrainedTokenizerBase" = None,
    ) -> Mapping[str, Any]:

        batch_size = compute_effective_axis_dimension(
                batch_size, fixed_dimension=OnnxConfig.default_fixed_batch, num_token_to_add=0
            )
        token_to_add = preprocessor.num_special_tokens_to_add(is_pair)
        seq_length = compute_effective_axis_dimension(
                seq_length, fixed_dimension=OnnxConfig.default_fixed_sequence, num_token_to_add=token_to_add
            )
        
        dummy_inputs = [" ".join(["<unk>"]) * seq_length, " ".join(["<unk>"]) * (seq_length+3)] * batch_size
        inputs = dict(preprocessor(dummy_inputs, padding="longest", truncation=True, return_tensors=framework))

        return inputs

Error:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│                                                                                                  │
│ /usr/lib/python3.10/runpy.py:196 in _run_module_as_main                                          │
│                                                                                                  │
│   193 │   main_globals = sys.modules["__main__"].__dict__                                        │
│   194 │   if alter_argv:                                                                         │
│   195 │   │   sys.argv[0] = mod_spec.origin                                                      │
│ ❱ 196 │   return _run_code(code, main_globals, None,                                             │
│   197 │   │   │   │   │    "__main__", mod_spec)                                                 │
│   198                                                                                            │
│   199 def run_module(mod_name, init_globals=None,                                                │
│ /usr/lib/python3.10/runpy.py:86 in _run_code                                                     │
│                                                                                                  │
│    83 │   │   │   │   │      __loader__ = loader,                                                │
│    84 │   │   │   │   │      __package__ = pkg_name,                                             │
│    85 │   │   │   │   │      __spec__ = mod_spec)                                                │
│ ❱  86 │   exec(code, run_globals)                                                                │
│    87 │   return run_globals                                                                     │
│    88                                                                                            │
│    89 def _run_module_code(code, init_globals=None,                                              │
│                                                                                                  │
│ /home/luke/dev/huggingface/transformers/src/transformers/onnx/__main__.py:180 in <module>        │
│                                                                                                  │
│   177 if __name__ == "__main__":                                                                 │
│   178 │   logger = logging.get_logger("transformers.onnx")  # pylint: disable=invalid-name       │
│   179 │   logger.setLevel(logging.INFO)                                                          │
│ ❱ 180 │   main()                                                                                 │
│   181                                                                                            │
│                                                                                                  │
│ /home/luke/dev/huggingface/transformers/src/transformers/onnx/__main__.py:173 in main            │
│                                                                                                  │
│   170 │   │   if args.atol is None:                                                              │
│   171 │   │   │   args.atol = onnx_config.atol_for_validation                                    │
│   172 │   │                                                                                      │
│ ❱ 173 │   │   validate_model_outputs(onnx_config, preprocessor, model, args.output, onnx_outpu   │
│   174 │   │   logger.info(f"All good, model saved at: {args.output.as_posix()}")                 │
│   175                                                                                            │
│   176                                                                                            │
│                                                                                                  │
│ /home/luke/dev/huggingface/transformers/src/transformers/onnx/convert.py:417 in                  │
│ validate_model_outputs                                                                           │
│                                                                                                  │
│   414 │   │   │   onnx_inputs[name] = value.numpy()                                              │
│   415 │                                                                                          │
│   416 │   # Compute outputs from the ONNX model                                                  │
│ ❱ 417 │   onnx_outputs = session.run(onnx_named_outputs, onnx_inputs)                            │
│   418 │                                                                                          │
│   419 │   # Check we have a subset of the keys into onnx_outputs against ref_outputs             │
│   420 │   ref_outputs_set, onnx_outputs_set = set(ref_outputs_dict.keys()), set(onnx_named_out   │
│                                                                                                  │
│ /home/luke/dev/huggingface/transformers/venv/lib/python3.10/site-packages/onnxruntime/capi/onnxr │
│ untime_inference_collection.py:200 in run                                                        │
│                                                                                                  │
│   197 │   │   if not output_names:                                                               │
│   198 │   │   │   output_names = [output.name for output in self._outputs_meta]                  │
│   199 │   │   try:                                                                               │
│ ❱ 200 │   │   │   return self._sess.run(output_names, input_feed, run_options)                   │
│   201 │   │   except C.EPFail as err:                                                            │
│   202 │   │   │   if self._enable_fallback:                                                      │
│   203 │   │   │   │   print("EP Error: {} using {}".format(str(err), self._providers))           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Concat node. Name:'Concat_1713' Status Message: concat.cc:159 PrepareForCompute Non concat axis dimensions
must match: Axis 2 has mismatched dimensions of 5 and 4

chainyo · 2022-10-31T11:15:22Z

@chainyo I was working on Canine and was facing some errors while running the following command

Hey @RaghavPrabhakar66, it comes from how you preprocess the dummy_inputs.
Before returning it print the shape of the dummy_inputs and check if they look like the expected inputs, you defined in the config.

blakechi · 2022-11-06T21:25:56Z

Hi @chainyo, I would like to take LED and CvT if there aren't folks working on them. 😃

chainyo · 2022-11-23T09:53:13Z

Hi @chainyo, I would like to take LED and CvT if there aren't folks working on them. smiley

Go for it. Feel free to open a PR (one per architecture) once you are done with your implementation!

hchings · 2022-12-01T08:27:21Z

Hi @chainyo, I added ONNX config for RemBERT in this PR. Please take a look and appreciate any guidance.

sgugger · 2022-12-07T19:33:39Z

The ONNX export is now part of the optimum library. For backward compatibility, we will keep what is inside Transformers for now but we won't add any new configs. We will just merge the PRs currently opened once all comments have been addressed, but we won't accept new ones in the Transformers code base.

Closing this issue here, if you want to work on ONNX export, I invite you to go on the optimum repo :-)

someshfengde · 2023-02-18T11:37:37Z

hi I'm working on Swin Transformer

NielsRogge · 2023-02-18T11:41:04Z

Hi,

Swin is already supported as can be seen here. Also, all ONNX exports are now being discussed here: huggingface/optimum#555

jorabara · 2023-02-18T11:48:08Z

Please unsubscibe

…

On Sat, Feb 18, 2023, 7:41 PM NielsRogge ***@***.***> wrote: Hi, Swin is already supported as can be seen here <https://github.com/huggingface/transformers/blob/7f1cdf18958efef6339040ba91edb32ae7377720/src/transformers/models/swin/configuration_swin.py#L166>. Also, all ONNX export is now being discussed here: huggingface/optimum#555 <huggingface/optimum#555> — Reply to this email directly, view it on GitHub <#16308 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AL5G67G7JJJMJ6NUBGDMSNLWYCYN3ANCNFSM5RIQTGKA> . You are receiving this because you commented.Message ID: ***@***.***>

someshfengde · 2023-02-18T12:36:33Z

Thanks @NielsRogge I'm newcomer and about to start contributing to this repo :)

LysandreJik added the Good First Issue label Mar 28, 2022

chainyo mentioned this issue Apr 2, 2022

Export longformer to ONNX #16463

Closed

echarlaix mentioned this issue Apr 15, 2022

support for Tapas model huggingface/optimum#141

Closed

skrsna mentioned this issue Apr 20, 2022

Add onnx config for RoFormer #16861

Merged

3 tasks

0xrushi mentioned this issue Apr 22, 2022

added deit onnx config #16887

Merged

5 tasks

chamidullinr mentioned this issue Apr 26, 2022

Add ResNet to models exportable with ONNX #16948

Closed

5 tasks

sijunhe mentioned this issue Apr 30, 2022

Add XLNet OnnxConfig #17027

Closed

5 tasks

manandey mentioned this issue May 1, 2022

add mobilebert onnx configs #17029

Merged

JingyaHuang mentioned this issue Sep 23, 2022

Add marian model type support for ORTOptimizer huggingface/optimum#392

Closed

bibhabasumohapatra mentioned this issue Oct 6, 2022

add ONNX support for swin transformer #19390

Merged

4 tasks

WaterKnight1998 mentioned this issue Oct 7, 2022

Adds DonutSwin to models exportable with ONNX #19401

Closed

4 tasks

RaghavPrabhakar66 mentioned this issue Oct 25, 2022

Add Onnx Config for ImageGPT #19868

Merged

sgugger closed this as completed in #19868 Oct 28, 2022

sgugger reopened this Oct 28, 2022

BakingBrains mentioned this issue Oct 29, 2022

Add Onnx Config for PoolFormer #19962

Closed

hchings mentioned this issue Dec 1, 2022

Add RemBERT ONNX config #20520

Merged

4 tasks

michaelbenayoun mentioned this issue Dec 7, 2022

Community contribution - optimum.exporters.onnx support for new models! huggingface/optimum#555

Open

sgugger closed this as completed Dec 7, 2022

BakingBrains mentioned this issue Dec 22, 2022

Add Onnx Config for PoolFormer #20868

Merged

ONNXConfig: Add a configuration for all available models #16308

ONNXConfig: Add a configuration for all available models #16308

Comments

chainyo commented Mar 21, 2022 • edited

chainyo commented Mar 21, 2022

chainyo commented Mar 21, 2022

vumichien commented Mar 22, 2022

vumichien commented Mar 26, 2022

LysandreJik commented Mar 28, 2022

chainyo commented Mar 29, 2022 • edited

aakashb95 commented Mar 31, 2022

xiadingZ commented Mar 31, 2022

chainyo commented Apr 1, 2022 • edited

chainyo commented Apr 2, 2022

jorabara commented Apr 11, 2022

pikaqqqqqq commented Apr 14, 2022

Issue description

Environment info

chainyo commented Apr 14, 2022 • edited

chainyo commented Apr 20, 2022

skrsna commented Apr 20, 2022

Tanmay06 commented Apr 21, 2022

chainyo commented Apr 21, 2022

chamidullinr commented Apr 26, 2022

chainyo commented Apr 26, 2022

nandwalritik commented Apr 28, 2022

chainyo commented Apr 28, 2022

sijunhe commented Apr 30, 2022

WaterKnight1998 commented Oct 7, 2022 • edited

RaghavPrabhakar66 commented Oct 24, 2022

NielsRogge commented Oct 24, 2022 • edited

chainyo commented Oct 24, 2022

RaghavPrabhakar66 commented Oct 24, 2022

chainyo commented Oct 28, 2022

RaghavPrabhakar66 commented Oct 28, 2022

BakingBrains commented Oct 29, 2022 • edited

chainyo commented Oct 29, 2022 • edited

RaghavPrabhakar66 commented Oct 31, 2022

chainyo commented Oct 31, 2022

blakechi commented Nov 6, 2022

chainyo commented Nov 23, 2022

hchings commented Dec 1, 2022

sgugger commented Dec 7, 2022

someshfengde commented Feb 18, 2023

NielsRogge commented Feb 18, 2023 • edited

jorabara commented Feb 18, 2023 via email

someshfengde commented Feb 18, 2023

chainyo commented Mar 21, 2022 •

edited

chainyo commented Mar 29, 2022 •

edited

chainyo commented Apr 1, 2022 •

edited

chainyo commented Apr 14, 2022 •

edited

WaterKnight1998 commented Oct 7, 2022 •

edited

NielsRogge commented Oct 24, 2022 •

edited

BakingBrains commented Oct 29, 2022 •

edited

chainyo commented Oct 29, 2022 •

edited

NielsRogge commented Feb 18, 2023 •

edited