diff --git a/iteration_3/README.md b/iteration_3/README.md index bc72bddf2bf0a59d3c7ef71037d12dcbc0c13b76..50a39266408a4780e26a02db57fe797e7a62ec6d 100644 --- a/iteration_3/README.md +++ b/iteration_3/README.md @@ -113,3 +113,112 @@ python -m coreml.inference --fp32 Other quantization tiers (int8 weight-only, int4 palettization) deferred to a future iteration — fp16 already pays for itself on disk and warm latency. + +## Token-axis buckets (Trial 11) + +The `bert` and `fused_diffusion_sampler` packages reject `ct.RangeDim` +on the token axis (HF Albert + cross-attn produce ops MIL refuses with +"data-dependent shapes were disabled"). The default packages above +hard-code T = 57, which caps prompts at ~37 chars. + +To support longer prompts without RangeDim, this iteration ships +**three additional fixed-T variants** of each constrained stage: + +| File | Compute | Size | +|---------------------------------------------------|--------------|-------| +| `bert_fp16_t64.mlpackage` | ALL | 12 MB | +| `bert_fp16_t128.mlpackage` | ALL | 12 MB | +| `bert_fp16_t256.mlpackage` | ALL | 12 MB | +| `fused_diffusion_sampler_fp16_t64.mlpackage` | ALL | 48 MB | +| `fused_diffusion_sampler_fp16_t128.mlpackage` | ALL | 48 MB | +| `fused_diffusion_sampler_fp16_t256.mlpackage` | ALL | 48 MB | +| **Sub-total (extra over the 8 defaults)** | | **180 MB** | + +The original `bert_fp16.mlpackage` / `fused_diffusion_sampler_fp16.mlpackage` +(T = 57) remain in the manifest as the default fast path — every +sentence that fits T = 57 should keep using them. The bucketed variants +are loaded on demand for longer prompts. + +Loader policy (Swift / Python): + +``` +real_n = #espeak tokens +if real_n <= 57: use *_fp16.mlpackage (default) +elif real_n <= 64: use *_fp16_t64.mlpackage +elif real_n <= 128: use *_fp16_t128.mlpackage +elif real_n <= 256: use *_fp16_t256.mlpackage +else: error (extend the bucket ladder) +``` + +Pad the token + attention_mask tensors with zeros to the chosen +bucket's T. `bert` honours `attention_mask`, so contamination at +padded positions is bounded; the sampler attends to bert output, so +it inherits the same masking. + +Per-bucket end-to-end inference verified by `coreml/inference_buckets.py +--all` (writes `coreml/out_t{64,128,256}.wav`): + +| Bucket | Prompt | Tokens | Audio | Pipeline | +|--------|--------------------------------------------|--------|--------|----------| +| 64 | "Hello there. How are you today?" | 36 | 2.42 s | 494 ms | +| 128 | "StyleTTS 2 is a text to speech model." | 57 | 3.60 s | 414 ms | +| 256 | longer paragraph (see `inference_buckets.py`) | 154 | 8.37 s | 4933 ms | + +T = 256 cost is dominated by `decoder_upsample` at 4.5 s / 4.9 s +(real-time-ish CPU_ONLY at 24 kHz × 8.4 s output). Bucket-swap cost +itself is a few ms; the rest of the pipeline scales with output +frame count, not bucket size. + +**Total iteration_3 footprint with buckets: 451 MB** (274 MB defaults ++ 180 MB buckets), or skip the T = 57 defaults entirely and ship only +buckets to save ~60 MB. + +### Build / refresh the bucketed packages + +```bash +cd models/tts/styletts2 + +# Build buckets (writes to coreml/packages/, run once) +uv run python coreml/build_buckets.py \ + --buckets 64,128,256 --stages bert,sampler --precision fp16 + +# Stage into iteration_3 + compile +for T in 64 128 256; do + for stage in bert fused_diffusion_sampler; do + cp -R "coreml/packages/${stage}_fp16_t${T}.mlpackage" \ + "iteration_3/packages/${stage}_fp16_t${T}.mlpackage" + xcrun coremlcompiler compile \ + "iteration_3/packages/${stage}_fp16_t${T}.mlpackage" \ + "iteration_3/compiled/" + done +done + +# Validate +uv run python coreml/inference_buckets.py --all --output-dir coreml +``` + +### HuggingFace upload manifest + +Upload the entire `iteration_3/packages/` tree (14 mlpackages): + +``` +iteration_3/packages/ +├── text_encoder_fp16.mlpackage +├── bert_fp16.mlpackage ← T=57 default +├── bert_fp16_t64.mlpackage ← bucket +├── bert_fp16_t128.mlpackage ← bucket +├── bert_fp16_t256.mlpackage ← bucket +├── ref_encoder_fp16.mlpackage +├── fused_diffusion_sampler_fp16.mlpackage ← T=57 default +├── fused_diffusion_sampler_fp16_t64.mlpackage ← bucket +├── fused_diffusion_sampler_fp16_t128.mlpackage ← bucket +├── fused_diffusion_sampler_fp16_t256.mlpackage ← bucket +├── duration_predictor_fp16.mlpackage +├── fused_f0n_har_source.mlpackage ← fp32 (cumsum drift) +├── decoder_pre_fp16.mlpackage +└── decoder_upsample_fp16.mlpackage +``` + +Total: **451 MB** (12 fp16 stages + 1 fp32 stage + 1 cumsum-sensitive +stage). Compiled `.mlmodelc` siblings live next to the packages in +`iteration_3/compiled/` — same file count, same total size. diff --git a/iteration_3/compiled/.DS_Store b/iteration_3/compiled/.DS_Store index 4f707caa4d6ed6ca6813175f1f98dea1e2d58547..dbd0a469c296afd3cfd0362a1abd5ece76c24d69 100644 Binary files a/iteration_3/compiled/.DS_Store and b/iteration_3/compiled/.DS_Store differ diff --git a/iteration_3/compiled/bert_fp16_t128.mlmodelc/analytics/coremldata.bin b/iteration_3/compiled/bert_fp16_t128.mlmodelc/analytics/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..1be3cc215709f6ac6a5e7a848256b4ce584bcfa7 --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t128.mlmodelc/analytics/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ffbc105f1a1ce78756729151d8f8d6669f0dc418d5146ea32f26c26bb6fb555 +size 243 diff --git a/iteration_3/compiled/bert_fp16_t128.mlmodelc/coremldata.bin b/iteration_3/compiled/bert_fp16_t128.mlmodelc/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..620c37913e3417ca982b011d90a74fbc000aeadb --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t128.mlmodelc/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:605c075757566cb93de9f4cb848a115ab2e586ab678d134e86fbb1d7646ea28b +size 441 diff --git a/iteration_3/compiled/bert_fp16_t128.mlmodelc/metadata.json b/iteration_3/compiled/bert_fp16_t128.mlmodelc/metadata.json new file mode 100644 index 0000000000000000000000000000000000000000..049c7bfba021d9d98d4db5165857ff62bb3dcbe1 --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t128.mlmodelc/metadata.json @@ -0,0 +1,94 @@ +[ + { + "metadataOutputVersion" : "3.0", + "storagePrecision" : "Float16", + "outputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float16", + "formattedType" : "MultiArray (Float16 1 × 128 × 768)", + "shortDescription" : "", + "shape" : "[1, 128, 768]", + "name" : "sequence_output", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float16", + "formattedType" : "MultiArray (Float16 1 × 512 × 128)", + "shortDescription" : "", + "shape" : "[1, 512, 128]", + "name" : "var_1030", + "type" : "MultiArray" + } + ], + "modelParameters" : [ + + ], + "specificationVersion" : 9, + "mlProgramOperationTypeHistogram" : { + "Ios18.linear" : 74, + "Ios18.scaledDotProductAttention" : 12, + "Ios18.sub" : 1, + "Select" : 2, + "Ios18.expandDims" : 2, + "Ios18.gelu" : 12, + "Ios18.gather" : 1, + "Ios18.add" : 27, + "Tile" : 1, + "Ios18.layerNorm" : 25, + "Ios18.transpose" : 49, + "Ios18.cast" : 5, + "Ios18.reshape" : 48, + "Ios18.greaterEqual" : 1 + }, + "computePrecision" : "Mixed (Float16, Int16, Int32)", + "isUpdatable" : "0", + "stateSchema" : [ + + ], + "availability" : { + "macOS" : "15.0", + "tvOS" : "18.0", + "visionOS" : "2.0", + "watchOS" : "11.0", + "iOS" : "18.0", + "macCatalyst" : "18.0" + }, + "modelType" : { + "name" : "MLModelType_mlProgram" + }, + "userDefinedMetadata" : { + "com.github.apple.coremltools.conversion_date" : "2026-05-08", + "com.github.apple.coremltools.source" : "torch==2.11.0", + "com.github.apple.coremltools.version" : "9.0", + "com.github.apple.coremltools.source_dialect" : "TorchScript" + }, + "inputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Int32", + "formattedType" : "MultiArray (Int32 1 × 128)", + "shortDescription" : "", + "shape" : "[1, 128]", + "name" : "tokens", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Int32", + "formattedType" : "MultiArray (Int32 1 × 128)", + "shortDescription" : "", + "shape" : "[1, 128]", + "name" : "attention_mask", + "type" : "MultiArray" + } + ], + "generatedClassName" : "bert_fp16_t128", + "method" : "predict" + } +] \ No newline at end of file diff --git a/iteration_3/compiled/bert_fp16_t128.mlmodelc/model.mil b/iteration_3/compiled/bert_fp16_t128.mlmodelc/model.mil new file mode 100644 index 0000000000000000000000000000000000000000..ff26586cfc9ec4a6a38e0e23aed51f37a28b1c30 --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t128.mlmodelc/model.mil @@ -0,0 +1,442 @@ +program(1.3) +[buildInfo = dict({{"coremlc-component-MIL", "3520.4.1"}, {"coremlc-version", "3520.5.1"}, {"coremltools-component-torch", "2.11.0"}, {"coremltools-source-dialect", "TorchScript"}, {"coremltools-version", "9.0"}})] +{ + func main(tensor attention_mask, tensor tokens) { + int32 inputs_embeds_batch_dims_0 = const()[name = string("inputs_embeds_batch_dims_0"), val = int32(0)]; + bool inputs_embeds_validate_indices_0 = const()[name = string("inputs_embeds_validate_indices_0"), val = bool(false)]; + tensor bert_embeddings_word_embeddings_weight_to_fp16 = const()[name = string("bert_embeddings_word_embeddings_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(64)))]; + string tokens_to_int16_dtype_0 = const()[name = string("tokens_to_int16_dtype_0"), val = string("int16")]; + string cast_53_dtype_0 = const()[name = string("cast_53_dtype_0"), val = string("int32")]; + int32 greater_equal_0_y_0 = const()[name = string("greater_equal_0_y_0"), val = int32(0)]; + tensor tokens_to_int16 = cast(dtype = tokens_to_int16_dtype_0, x = tokens)[name = string("cast_58")]; + tensor cast_53 = cast(dtype = cast_53_dtype_0, x = tokens_to_int16)[name = string("cast_57")]; + tensor greater_equal_0 = greater_equal(x = cast_53, y = greater_equal_0_y_0)[name = string("greater_equal_0")]; + int32 slice_by_index_0 = const()[name = string("slice_by_index_0"), val = int32(178)]; + tensor add_0 = add(x = cast_53, y = slice_by_index_0)[name = string("add_0")]; + tensor select_0 = select(a = cast_53, b = add_0, cond = greater_equal_0)[name = string("select_0")]; + int32 inputs_embeds_cast_fp16_cast_uint16_axis_0 = const()[name = string("inputs_embeds_cast_fp16_cast_uint16_axis_0"), val = int32(0)]; + string select_0_to_int16_dtype_0 = const()[name = string("select_0_to_int16_dtype_0"), val = string("int16")]; + tensor select_0_to_int16 = cast(dtype = select_0_to_int16_dtype_0, x = select_0)[name = string("cast_56")]; + tensor inputs_embeds_cast_fp16_cast_uint16_cast_uint16 = gather(axis = inputs_embeds_cast_fp16_cast_uint16_axis_0, batch_dims = inputs_embeds_batch_dims_0, indices = select_0_to_int16, validate_indices = inputs_embeds_validate_indices_0, x = bert_embeddings_word_embeddings_weight_to_fp16)[name = string("inputs_embeds_cast_fp16_cast_uint16_cast_uint16")]; + tensor token_type_embeddings_1_to_fp16 = const()[name = string("token_type_embeddings_1_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(45696)))]; + tensor embeddings_1_cast_fp16 = add(x = inputs_embeds_cast_fp16_cast_uint16_cast_uint16, y = token_type_embeddings_1_to_fp16)[name = string("embeddings_1_cast_fp16")]; + tensor position_embeddings_1_to_fp16 = const()[name = string("position_embeddings_1_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(78528)))]; + tensor input_5_cast_fp16 = add(x = embeddings_1_cast_fp16, y = position_embeddings_1_to_fp16)[name = string("input_5_cast_fp16")]; + tensor input_7_axes_0 = const()[name = string("input_7_axes_0"), val = tensor([-1])]; + tensor bert_embeddings_LayerNorm_weight_to_fp16 = const()[name = string("bert_embeddings_LayerNorm_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(111360)))]; + tensor bert_embeddings_LayerNorm_bias_to_fp16 = const()[name = string("bert_embeddings_LayerNorm_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(111680)))]; + fp16 var_34_to_fp16 = const()[name = string("op_34_to_fp16"), val = fp16(0x1p-24)]; + tensor input_7_cast_fp16 = layer_norm(axes = input_7_axes_0, beta = bert_embeddings_LayerNorm_bias_to_fp16, epsilon = var_34_to_fp16, gamma = bert_embeddings_LayerNorm_weight_to_fp16, x = input_5_cast_fp16)[name = string("input_7_cast_fp16")]; + tensor var_79_axes_0 = const()[name = string("op_79_axes_0"), val = tensor([1])]; + tensor var_79 = expand_dims(axes = var_79_axes_0, x = attention_mask)[name = string("op_79")]; + tensor var_81_axes_0 = const()[name = string("op_81_axes_0"), val = tensor([2])]; + tensor var_81 = expand_dims(axes = var_81_axes_0, x = var_79)[name = string("op_81")]; + tensor var_90_reps_0 = const()[name = string("op_90_reps_0"), val = tensor([1, 1, 128, 1])]; + tensor var_90 = tile(reps = var_90_reps_0, x = var_81)[name = string("op_90")]; + fp16 var_96_to_fp16 = const()[name = string("op_96_to_fp16"), val = fp16(0x1p+0)]; + string var_95_to_fp16_dtype_0 = const()[name = string("op_95_to_fp16_dtype_0"), val = string("fp16")]; + tensor var_90_to_fp16 = cast(dtype = var_95_to_fp16_dtype_0, x = var_90)[name = string("cast_55")]; + tensor inverted_mask_cast_fp16 = sub(x = var_96_to_fp16, y = var_90_to_fp16)[name = string("inverted_mask_cast_fp16")]; + string var_103_dtype_0 = const()[name = string("op_103_dtype_0"), val = string("bool")]; + fp16 var_104_to_fp16 = const()[name = string("op_104_to_fp16"), val = fp16(-inf)]; + tensor inverted_mask_cast_fp16_to_bool = cast(dtype = var_103_dtype_0, x = inverted_mask_cast_fp16)[name = string("cast_54")]; + tensor attention_mask_cast_fp16 = select(a = var_104_to_fp16, b = inverted_mask_cast_fp16, cond = inverted_mask_cast_fp16_to_bool)[name = string("attention_mask_cast_fp16")]; + tensor bert_encoder_embedding_hidden_mapping_in_weight_to_fp16 = const()[name = string("bert_encoder_embedding_hidden_mapping_in_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(112000)))]; + tensor bert_encoder_embedding_hidden_mapping_in_bias_to_fp16 = const()[name = string("bert_encoder_embedding_hidden_mapping_in_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(308672)))]; + tensor linear_0_cast_fp16 = linear(bias = bert_encoder_embedding_hidden_mapping_in_bias_to_fp16, weight = bert_encoder_embedding_hidden_mapping_in_weight_to_fp16, x = input_7_cast_fp16)[name = string("linear_0_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(310272)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(1489984)))]; + tensor linear_1_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = linear_0_cast_fp16)[name = string("linear_1_cast_fp16")]; + tensor var_143 = const()[name = string("op_143"), val = tensor([1, 128, 12, 64])]; + tensor x_3_cast_fp16 = reshape(shape = var_143, x = linear_1_cast_fp16)[name = string("x_3_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(1491584)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2671296)))]; + tensor linear_2_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = linear_0_cast_fp16)[name = string("linear_2_cast_fp16")]; + tensor var_152 = const()[name = string("op_152"), val = tensor([1, 128, 12, 64])]; + tensor x_7_cast_fp16 = reshape(shape = var_152, x = linear_2_cast_fp16)[name = string("x_7_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2672896)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(3852608)))]; + tensor linear_3_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = linear_0_cast_fp16)[name = string("linear_3_cast_fp16")]; + tensor var_161 = const()[name = string("op_161"), val = tensor([1, 128, 12, 64])]; + tensor x_11_cast_fp16 = reshape(shape = var_161, x = linear_3_cast_fp16)[name = string("x_11_cast_fp16")]; + tensor transpose_72_perm_0 = const()[name = string("transpose_72_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_73_perm_0 = const()[name = string("transpose_73_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_74_perm_0 = const()[name = string("transpose_74_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_74 = transpose(perm = transpose_74_perm_0, x = x_11_cast_fp16)[name = string("transpose_154")]; + tensor transpose_73 = transpose(perm = transpose_73_perm_0, x = x_7_cast_fp16)[name = string("transpose_155")]; + tensor transpose_72 = transpose(perm = transpose_72_perm_0, x = x_3_cast_fp16)[name = string("transpose_156")]; + tensor attention_output_1_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_73, query = transpose_72, value = transpose_74)[name = string("attention_output_1_cast_fp16")]; + tensor attention_output_3_perm_0 = const()[name = string("attention_output_3_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_167 = const()[name = string("op_167"), val = tensor([1, 128, 768])]; + tensor attention_output_3_cast_fp16 = transpose(perm = attention_output_3_perm_0, x = attention_output_1_cast_fp16)[name = string("transpose_153")]; + tensor input_9_cast_fp16 = reshape(shape = var_167, x = attention_output_3_cast_fp16)[name = string("input_9_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(3854208)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5033920)))]; + tensor linear_4_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_9_cast_fp16)[name = string("linear_4_cast_fp16")]; + tensor input_11_cast_fp16 = add(x = linear_0_cast_fp16, y = linear_4_cast_fp16)[name = string("input_11_cast_fp16")]; + tensor input_13_axes_0 = const()[name = string("input_13_axes_0"), val = tensor([-1])]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5035520)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5037120)))]; + fp16 var_118_to_fp16 = const()[name = string("op_118_to_fp16"), val = fp16(0x1p-24)]; + tensor input_13_cast_fp16 = layer_norm(axes = input_13_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_11_cast_fp16)[name = string("input_13_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5038720)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(8184512)))]; + tensor linear_5_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_13_cast_fp16)[name = string("linear_5_cast_fp16")]; + string input_17_mode_0 = const()[name = string("input_17_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_17_cast_fp16 = gelu(mode = input_17_mode_0, x = linear_5_cast_fp16)[name = string("input_17_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(8188672)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11334464)))]; + tensor linear_6_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_17_cast_fp16)[name = string("linear_6_cast_fp16")]; + tensor input_19_cast_fp16 = add(x = linear_6_cast_fp16, y = input_13_cast_fp16)[name = string("input_19_cast_fp16")]; + tensor hidden_states_3_axes_0 = const()[name = string("hidden_states_3_axes_0"), val = tensor([-1])]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11336064)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11337664)))]; + tensor hidden_states_3_cast_fp16 = layer_norm(axes = hidden_states_3_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_19_cast_fp16)[name = string("hidden_states_3_cast_fp16")]; + tensor linear_7_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_3_cast_fp16)[name = string("linear_7_cast_fp16")]; + tensor var_218 = const()[name = string("op_218"), val = tensor([1, 128, 12, 64])]; + tensor x_15_cast_fp16 = reshape(shape = var_218, x = linear_7_cast_fp16)[name = string("x_15_cast_fp16")]; + tensor linear_8_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_3_cast_fp16)[name = string("linear_8_cast_fp16")]; + tensor var_227 = const()[name = string("op_227"), val = tensor([1, 128, 12, 64])]; + tensor x_19_cast_fp16 = reshape(shape = var_227, x = linear_8_cast_fp16)[name = string("x_19_cast_fp16")]; + tensor linear_9_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_3_cast_fp16)[name = string("linear_9_cast_fp16")]; + tensor var_236 = const()[name = string("op_236"), val = tensor([1, 128, 12, 64])]; + tensor x_23_cast_fp16 = reshape(shape = var_236, x = linear_9_cast_fp16)[name = string("x_23_cast_fp16")]; + tensor transpose_75_perm_0 = const()[name = string("transpose_75_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_76_perm_0 = const()[name = string("transpose_76_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_77_perm_0 = const()[name = string("transpose_77_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_77 = transpose(perm = transpose_77_perm_0, x = x_23_cast_fp16)[name = string("transpose_150")]; + tensor transpose_76 = transpose(perm = transpose_76_perm_0, x = x_19_cast_fp16)[name = string("transpose_151")]; + tensor transpose_75 = transpose(perm = transpose_75_perm_0, x = x_15_cast_fp16)[name = string("transpose_152")]; + tensor attention_output_5_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_76, query = transpose_75, value = transpose_77)[name = string("attention_output_5_cast_fp16")]; + tensor attention_output_7_perm_0 = const()[name = string("attention_output_7_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_242 = const()[name = string("op_242"), val = tensor([1, 128, 768])]; + tensor attention_output_7_cast_fp16 = transpose(perm = attention_output_7_perm_0, x = attention_output_5_cast_fp16)[name = string("transpose_149")]; + tensor input_21_cast_fp16 = reshape(shape = var_242, x = attention_output_7_cast_fp16)[name = string("input_21_cast_fp16")]; + tensor linear_10_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_21_cast_fp16)[name = string("linear_10_cast_fp16")]; + tensor input_23_cast_fp16 = add(x = hidden_states_3_cast_fp16, y = linear_10_cast_fp16)[name = string("input_23_cast_fp16")]; + tensor input_25_axes_0 = const()[name = string("input_25_axes_0"), val = tensor([-1])]; + tensor input_25_cast_fp16 = layer_norm(axes = input_25_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_23_cast_fp16)[name = string("input_25_cast_fp16")]; + tensor linear_11_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_25_cast_fp16)[name = string("linear_11_cast_fp16")]; + string input_29_mode_0 = const()[name = string("input_29_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_29_cast_fp16 = gelu(mode = input_29_mode_0, x = linear_11_cast_fp16)[name = string("input_29_cast_fp16")]; + tensor linear_12_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_29_cast_fp16)[name = string("linear_12_cast_fp16")]; + tensor input_31_cast_fp16 = add(x = linear_12_cast_fp16, y = input_25_cast_fp16)[name = string("input_31_cast_fp16")]; + tensor hidden_states_5_axes_0 = const()[name = string("hidden_states_5_axes_0"), val = tensor([-1])]; + tensor hidden_states_5_cast_fp16 = layer_norm(axes = hidden_states_5_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_31_cast_fp16)[name = string("hidden_states_5_cast_fp16")]; + tensor linear_13_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_5_cast_fp16)[name = string("linear_13_cast_fp16")]; + tensor var_293 = const()[name = string("op_293"), val = tensor([1, 128, 12, 64])]; + tensor x_27_cast_fp16 = reshape(shape = var_293, x = linear_13_cast_fp16)[name = string("x_27_cast_fp16")]; + tensor linear_14_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_5_cast_fp16)[name = string("linear_14_cast_fp16")]; + tensor var_302 = const()[name = string("op_302"), val = tensor([1, 128, 12, 64])]; + tensor x_31_cast_fp16 = reshape(shape = var_302, x = linear_14_cast_fp16)[name = string("x_31_cast_fp16")]; + tensor linear_15_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_5_cast_fp16)[name = string("linear_15_cast_fp16")]; + tensor var_311 = const()[name = string("op_311"), val = tensor([1, 128, 12, 64])]; + tensor x_35_cast_fp16 = reshape(shape = var_311, x = linear_15_cast_fp16)[name = string("x_35_cast_fp16")]; + tensor transpose_78_perm_0 = const()[name = string("transpose_78_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_79_perm_0 = const()[name = string("transpose_79_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_80_perm_0 = const()[name = string("transpose_80_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_80 = transpose(perm = transpose_80_perm_0, x = x_35_cast_fp16)[name = string("transpose_146")]; + tensor transpose_79 = transpose(perm = transpose_79_perm_0, x = x_31_cast_fp16)[name = string("transpose_147")]; + tensor transpose_78 = transpose(perm = transpose_78_perm_0, x = x_27_cast_fp16)[name = string("transpose_148")]; + tensor attention_output_9_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_79, query = transpose_78, value = transpose_80)[name = string("attention_output_9_cast_fp16")]; + tensor attention_output_11_perm_0 = const()[name = string("attention_output_11_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_317 = const()[name = string("op_317"), val = tensor([1, 128, 768])]; + tensor attention_output_11_cast_fp16 = transpose(perm = attention_output_11_perm_0, x = attention_output_9_cast_fp16)[name = string("transpose_145")]; + tensor input_33_cast_fp16 = reshape(shape = var_317, x = attention_output_11_cast_fp16)[name = string("input_33_cast_fp16")]; + tensor linear_16_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_33_cast_fp16)[name = string("linear_16_cast_fp16")]; + tensor input_35_cast_fp16 = add(x = hidden_states_5_cast_fp16, y = linear_16_cast_fp16)[name = string("input_35_cast_fp16")]; + tensor input_37_axes_0 = const()[name = string("input_37_axes_0"), val = tensor([-1])]; + tensor input_37_cast_fp16 = layer_norm(axes = input_37_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_35_cast_fp16)[name = string("input_37_cast_fp16")]; + tensor linear_17_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_37_cast_fp16)[name = string("linear_17_cast_fp16")]; + string input_41_mode_0 = const()[name = string("input_41_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_41_cast_fp16 = gelu(mode = input_41_mode_0, x = linear_17_cast_fp16)[name = string("input_41_cast_fp16")]; + tensor linear_18_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_41_cast_fp16)[name = string("linear_18_cast_fp16")]; + tensor input_43_cast_fp16 = add(x = linear_18_cast_fp16, y = input_37_cast_fp16)[name = string("input_43_cast_fp16")]; + tensor hidden_states_7_axes_0 = const()[name = string("hidden_states_7_axes_0"), val = tensor([-1])]; + tensor hidden_states_7_cast_fp16 = layer_norm(axes = hidden_states_7_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_43_cast_fp16)[name = string("hidden_states_7_cast_fp16")]; + tensor linear_19_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_7_cast_fp16)[name = string("linear_19_cast_fp16")]; + tensor var_368 = const()[name = string("op_368"), val = tensor([1, 128, 12, 64])]; + tensor x_39_cast_fp16 = reshape(shape = var_368, x = linear_19_cast_fp16)[name = string("x_39_cast_fp16")]; + tensor linear_20_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_7_cast_fp16)[name = string("linear_20_cast_fp16")]; + tensor var_377 = const()[name = string("op_377"), val = tensor([1, 128, 12, 64])]; + tensor x_43_cast_fp16 = reshape(shape = var_377, x = linear_20_cast_fp16)[name = string("x_43_cast_fp16")]; + tensor linear_21_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_7_cast_fp16)[name = string("linear_21_cast_fp16")]; + tensor var_386 = const()[name = string("op_386"), val = tensor([1, 128, 12, 64])]; + tensor x_47_cast_fp16 = reshape(shape = var_386, x = linear_21_cast_fp16)[name = string("x_47_cast_fp16")]; + tensor transpose_81_perm_0 = const()[name = string("transpose_81_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_82_perm_0 = const()[name = string("transpose_82_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_83_perm_0 = const()[name = string("transpose_83_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_83 = transpose(perm = transpose_83_perm_0, x = x_47_cast_fp16)[name = string("transpose_142")]; + tensor transpose_82 = transpose(perm = transpose_82_perm_0, x = x_43_cast_fp16)[name = string("transpose_143")]; + tensor transpose_81 = transpose(perm = transpose_81_perm_0, x = x_39_cast_fp16)[name = string("transpose_144")]; + tensor attention_output_13_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_82, query = transpose_81, value = transpose_83)[name = string("attention_output_13_cast_fp16")]; + tensor attention_output_15_perm_0 = const()[name = string("attention_output_15_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_392 = const()[name = string("op_392"), val = tensor([1, 128, 768])]; + tensor attention_output_15_cast_fp16 = transpose(perm = attention_output_15_perm_0, x = attention_output_13_cast_fp16)[name = string("transpose_141")]; + tensor input_45_cast_fp16 = reshape(shape = var_392, x = attention_output_15_cast_fp16)[name = string("input_45_cast_fp16")]; + tensor linear_22_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_45_cast_fp16)[name = string("linear_22_cast_fp16")]; + tensor input_47_cast_fp16 = add(x = hidden_states_7_cast_fp16, y = linear_22_cast_fp16)[name = string("input_47_cast_fp16")]; + tensor input_49_axes_0 = const()[name = string("input_49_axes_0"), val = tensor([-1])]; + tensor input_49_cast_fp16 = layer_norm(axes = input_49_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_47_cast_fp16)[name = string("input_49_cast_fp16")]; + tensor linear_23_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_49_cast_fp16)[name = string("linear_23_cast_fp16")]; + string input_53_mode_0 = const()[name = string("input_53_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_53_cast_fp16 = gelu(mode = input_53_mode_0, x = linear_23_cast_fp16)[name = string("input_53_cast_fp16")]; + tensor linear_24_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_53_cast_fp16)[name = string("linear_24_cast_fp16")]; + tensor input_55_cast_fp16 = add(x = linear_24_cast_fp16, y = input_49_cast_fp16)[name = string("input_55_cast_fp16")]; + tensor hidden_states_9_axes_0 = const()[name = string("hidden_states_9_axes_0"), val = tensor([-1])]; + tensor hidden_states_9_cast_fp16 = layer_norm(axes = hidden_states_9_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_55_cast_fp16)[name = string("hidden_states_9_cast_fp16")]; + tensor linear_25_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_9_cast_fp16)[name = string("linear_25_cast_fp16")]; + tensor var_443 = const()[name = string("op_443"), val = tensor([1, 128, 12, 64])]; + tensor x_51_cast_fp16 = reshape(shape = var_443, x = linear_25_cast_fp16)[name = string("x_51_cast_fp16")]; + tensor linear_26_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_9_cast_fp16)[name = string("linear_26_cast_fp16")]; + tensor var_452 = const()[name = string("op_452"), val = tensor([1, 128, 12, 64])]; + tensor x_55_cast_fp16 = reshape(shape = var_452, x = linear_26_cast_fp16)[name = string("x_55_cast_fp16")]; + tensor linear_27_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_9_cast_fp16)[name = string("linear_27_cast_fp16")]; + tensor var_461 = const()[name = string("op_461"), val = tensor([1, 128, 12, 64])]; + tensor x_59_cast_fp16 = reshape(shape = var_461, x = linear_27_cast_fp16)[name = string("x_59_cast_fp16")]; + tensor transpose_84_perm_0 = const()[name = string("transpose_84_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_85_perm_0 = const()[name = string("transpose_85_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_86_perm_0 = const()[name = string("transpose_86_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_86 = transpose(perm = transpose_86_perm_0, x = x_59_cast_fp16)[name = string("transpose_138")]; + tensor transpose_85 = transpose(perm = transpose_85_perm_0, x = x_55_cast_fp16)[name = string("transpose_139")]; + tensor transpose_84 = transpose(perm = transpose_84_perm_0, x = x_51_cast_fp16)[name = string("transpose_140")]; + tensor attention_output_17_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_85, query = transpose_84, value = transpose_86)[name = string("attention_output_17_cast_fp16")]; + tensor attention_output_19_perm_0 = const()[name = string("attention_output_19_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_467 = const()[name = string("op_467"), val = tensor([1, 128, 768])]; + tensor attention_output_19_cast_fp16 = transpose(perm = attention_output_19_perm_0, x = attention_output_17_cast_fp16)[name = string("transpose_137")]; + tensor input_57_cast_fp16 = reshape(shape = var_467, x = attention_output_19_cast_fp16)[name = string("input_57_cast_fp16")]; + tensor linear_28_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_57_cast_fp16)[name = string("linear_28_cast_fp16")]; + tensor input_59_cast_fp16 = add(x = hidden_states_9_cast_fp16, y = linear_28_cast_fp16)[name = string("input_59_cast_fp16")]; + tensor input_61_axes_0 = const()[name = string("input_61_axes_0"), val = tensor([-1])]; + tensor input_61_cast_fp16 = layer_norm(axes = input_61_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_59_cast_fp16)[name = string("input_61_cast_fp16")]; + tensor linear_29_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_61_cast_fp16)[name = string("linear_29_cast_fp16")]; + string input_65_mode_0 = const()[name = string("input_65_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_65_cast_fp16 = gelu(mode = input_65_mode_0, x = linear_29_cast_fp16)[name = string("input_65_cast_fp16")]; + tensor linear_30_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_65_cast_fp16)[name = string("linear_30_cast_fp16")]; + tensor input_67_cast_fp16 = add(x = linear_30_cast_fp16, y = input_61_cast_fp16)[name = string("input_67_cast_fp16")]; + tensor hidden_states_11_axes_0 = const()[name = string("hidden_states_11_axes_0"), val = tensor([-1])]; + tensor hidden_states_11_cast_fp16 = layer_norm(axes = hidden_states_11_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_67_cast_fp16)[name = string("hidden_states_11_cast_fp16")]; + tensor linear_31_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_11_cast_fp16)[name = string("linear_31_cast_fp16")]; + tensor var_518 = const()[name = string("op_518"), val = tensor([1, 128, 12, 64])]; + tensor x_63_cast_fp16 = reshape(shape = var_518, x = linear_31_cast_fp16)[name = string("x_63_cast_fp16")]; + tensor linear_32_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_11_cast_fp16)[name = string("linear_32_cast_fp16")]; + tensor var_527 = const()[name = string("op_527"), val = tensor([1, 128, 12, 64])]; + tensor x_67_cast_fp16 = reshape(shape = var_527, x = linear_32_cast_fp16)[name = string("x_67_cast_fp16")]; + tensor linear_33_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_11_cast_fp16)[name = string("linear_33_cast_fp16")]; + tensor var_536 = const()[name = string("op_536"), val = tensor([1, 128, 12, 64])]; + tensor x_71_cast_fp16 = reshape(shape = var_536, x = linear_33_cast_fp16)[name = string("x_71_cast_fp16")]; + tensor transpose_87_perm_0 = const()[name = string("transpose_87_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_88_perm_0 = const()[name = string("transpose_88_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_89_perm_0 = const()[name = string("transpose_89_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_89 = transpose(perm = transpose_89_perm_0, x = x_71_cast_fp16)[name = string("transpose_134")]; + tensor transpose_88 = transpose(perm = transpose_88_perm_0, x = x_67_cast_fp16)[name = string("transpose_135")]; + tensor transpose_87 = transpose(perm = transpose_87_perm_0, x = x_63_cast_fp16)[name = string("transpose_136")]; + tensor attention_output_21_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_88, query = transpose_87, value = transpose_89)[name = string("attention_output_21_cast_fp16")]; + tensor attention_output_23_perm_0 = const()[name = string("attention_output_23_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_542 = const()[name = string("op_542"), val = tensor([1, 128, 768])]; + tensor attention_output_23_cast_fp16 = transpose(perm = attention_output_23_perm_0, x = attention_output_21_cast_fp16)[name = string("transpose_133")]; + tensor input_69_cast_fp16 = reshape(shape = var_542, x = attention_output_23_cast_fp16)[name = string("input_69_cast_fp16")]; + tensor linear_34_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_69_cast_fp16)[name = string("linear_34_cast_fp16")]; + tensor input_71_cast_fp16 = add(x = hidden_states_11_cast_fp16, y = linear_34_cast_fp16)[name = string("input_71_cast_fp16")]; + tensor input_73_axes_0 = const()[name = string("input_73_axes_0"), val = tensor([-1])]; + tensor input_73_cast_fp16 = layer_norm(axes = input_73_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_71_cast_fp16)[name = string("input_73_cast_fp16")]; + tensor linear_35_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_73_cast_fp16)[name = string("linear_35_cast_fp16")]; + string input_77_mode_0 = const()[name = string("input_77_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_77_cast_fp16 = gelu(mode = input_77_mode_0, x = linear_35_cast_fp16)[name = string("input_77_cast_fp16")]; + tensor linear_36_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_77_cast_fp16)[name = string("linear_36_cast_fp16")]; + tensor input_79_cast_fp16 = add(x = linear_36_cast_fp16, y = input_73_cast_fp16)[name = string("input_79_cast_fp16")]; + tensor hidden_states_13_axes_0 = const()[name = string("hidden_states_13_axes_0"), val = tensor([-1])]; + tensor hidden_states_13_cast_fp16 = layer_norm(axes = hidden_states_13_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_79_cast_fp16)[name = string("hidden_states_13_cast_fp16")]; + tensor linear_37_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_13_cast_fp16)[name = string("linear_37_cast_fp16")]; + tensor var_593 = const()[name = string("op_593"), val = tensor([1, 128, 12, 64])]; + tensor x_75_cast_fp16 = reshape(shape = var_593, x = linear_37_cast_fp16)[name = string("x_75_cast_fp16")]; + tensor linear_38_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_13_cast_fp16)[name = string("linear_38_cast_fp16")]; + tensor var_602 = const()[name = string("op_602"), val = tensor([1, 128, 12, 64])]; + tensor x_79_cast_fp16 = reshape(shape = var_602, x = linear_38_cast_fp16)[name = string("x_79_cast_fp16")]; + tensor linear_39_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_13_cast_fp16)[name = string("linear_39_cast_fp16")]; + tensor var_611 = const()[name = string("op_611"), val = tensor([1, 128, 12, 64])]; + tensor x_83_cast_fp16 = reshape(shape = var_611, x = linear_39_cast_fp16)[name = string("x_83_cast_fp16")]; + tensor transpose_90_perm_0 = const()[name = string("transpose_90_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_91_perm_0 = const()[name = string("transpose_91_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_92_perm_0 = const()[name = string("transpose_92_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_92 = transpose(perm = transpose_92_perm_0, x = x_83_cast_fp16)[name = string("transpose_130")]; + tensor transpose_91 = transpose(perm = transpose_91_perm_0, x = x_79_cast_fp16)[name = string("transpose_131")]; + tensor transpose_90 = transpose(perm = transpose_90_perm_0, x = x_75_cast_fp16)[name = string("transpose_132")]; + tensor attention_output_25_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_91, query = transpose_90, value = transpose_92)[name = string("attention_output_25_cast_fp16")]; + tensor attention_output_27_perm_0 = const()[name = string("attention_output_27_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_617 = const()[name = string("op_617"), val = tensor([1, 128, 768])]; + tensor attention_output_27_cast_fp16 = transpose(perm = attention_output_27_perm_0, x = attention_output_25_cast_fp16)[name = string("transpose_129")]; + tensor input_81_cast_fp16 = reshape(shape = var_617, x = attention_output_27_cast_fp16)[name = string("input_81_cast_fp16")]; + tensor linear_40_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_81_cast_fp16)[name = string("linear_40_cast_fp16")]; + tensor input_83_cast_fp16 = add(x = hidden_states_13_cast_fp16, y = linear_40_cast_fp16)[name = string("input_83_cast_fp16")]; + tensor input_85_axes_0 = const()[name = string("input_85_axes_0"), val = tensor([-1])]; + tensor input_85_cast_fp16 = layer_norm(axes = input_85_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_83_cast_fp16)[name = string("input_85_cast_fp16")]; + tensor linear_41_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_85_cast_fp16)[name = string("linear_41_cast_fp16")]; + string input_89_mode_0 = const()[name = string("input_89_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_89_cast_fp16 = gelu(mode = input_89_mode_0, x = linear_41_cast_fp16)[name = string("input_89_cast_fp16")]; + tensor linear_42_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_89_cast_fp16)[name = string("linear_42_cast_fp16")]; + tensor input_91_cast_fp16 = add(x = linear_42_cast_fp16, y = input_85_cast_fp16)[name = string("input_91_cast_fp16")]; + tensor hidden_states_15_axes_0 = const()[name = string("hidden_states_15_axes_0"), val = tensor([-1])]; + tensor hidden_states_15_cast_fp16 = layer_norm(axes = hidden_states_15_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_91_cast_fp16)[name = string("hidden_states_15_cast_fp16")]; + tensor linear_43_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_15_cast_fp16)[name = string("linear_43_cast_fp16")]; + tensor var_668 = const()[name = string("op_668"), val = tensor([1, 128, 12, 64])]; + tensor x_87_cast_fp16 = reshape(shape = var_668, x = linear_43_cast_fp16)[name = string("x_87_cast_fp16")]; + tensor linear_44_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_15_cast_fp16)[name = string("linear_44_cast_fp16")]; + tensor var_677 = const()[name = string("op_677"), val = tensor([1, 128, 12, 64])]; + tensor x_91_cast_fp16 = reshape(shape = var_677, x = linear_44_cast_fp16)[name = string("x_91_cast_fp16")]; + tensor linear_45_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_15_cast_fp16)[name = string("linear_45_cast_fp16")]; + tensor var_686 = const()[name = string("op_686"), val = tensor([1, 128, 12, 64])]; + tensor x_95_cast_fp16 = reshape(shape = var_686, x = linear_45_cast_fp16)[name = string("x_95_cast_fp16")]; + tensor transpose_93_perm_0 = const()[name = string("transpose_93_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_94_perm_0 = const()[name = string("transpose_94_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_95_perm_0 = const()[name = string("transpose_95_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_95 = transpose(perm = transpose_95_perm_0, x = x_95_cast_fp16)[name = string("transpose_126")]; + tensor transpose_94 = transpose(perm = transpose_94_perm_0, x = x_91_cast_fp16)[name = string("transpose_127")]; + tensor transpose_93 = transpose(perm = transpose_93_perm_0, x = x_87_cast_fp16)[name = string("transpose_128")]; + tensor attention_output_29_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_94, query = transpose_93, value = transpose_95)[name = string("attention_output_29_cast_fp16")]; + tensor attention_output_31_perm_0 = const()[name = string("attention_output_31_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_692 = const()[name = string("op_692"), val = tensor([1, 128, 768])]; + tensor attention_output_31_cast_fp16 = transpose(perm = attention_output_31_perm_0, x = attention_output_29_cast_fp16)[name = string("transpose_125")]; + tensor input_93_cast_fp16 = reshape(shape = var_692, x = attention_output_31_cast_fp16)[name = string("input_93_cast_fp16")]; + tensor linear_46_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_93_cast_fp16)[name = string("linear_46_cast_fp16")]; + tensor input_95_cast_fp16 = add(x = hidden_states_15_cast_fp16, y = linear_46_cast_fp16)[name = string("input_95_cast_fp16")]; + tensor input_97_axes_0 = const()[name = string("input_97_axes_0"), val = tensor([-1])]; + tensor input_97_cast_fp16 = layer_norm(axes = input_97_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_95_cast_fp16)[name = string("input_97_cast_fp16")]; + tensor linear_47_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_97_cast_fp16)[name = string("linear_47_cast_fp16")]; + string input_101_mode_0 = const()[name = string("input_101_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_101_cast_fp16 = gelu(mode = input_101_mode_0, x = linear_47_cast_fp16)[name = string("input_101_cast_fp16")]; + tensor linear_48_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_101_cast_fp16)[name = string("linear_48_cast_fp16")]; + tensor input_103_cast_fp16 = add(x = linear_48_cast_fp16, y = input_97_cast_fp16)[name = string("input_103_cast_fp16")]; + tensor hidden_states_17_axes_0 = const()[name = string("hidden_states_17_axes_0"), val = tensor([-1])]; + tensor hidden_states_17_cast_fp16 = layer_norm(axes = hidden_states_17_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_103_cast_fp16)[name = string("hidden_states_17_cast_fp16")]; + tensor linear_49_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_17_cast_fp16)[name = string("linear_49_cast_fp16")]; + tensor var_743 = const()[name = string("op_743"), val = tensor([1, 128, 12, 64])]; + tensor x_99_cast_fp16 = reshape(shape = var_743, x = linear_49_cast_fp16)[name = string("x_99_cast_fp16")]; + tensor linear_50_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_17_cast_fp16)[name = string("linear_50_cast_fp16")]; + tensor var_752 = const()[name = string("op_752"), val = tensor([1, 128, 12, 64])]; + tensor x_103_cast_fp16 = reshape(shape = var_752, x = linear_50_cast_fp16)[name = string("x_103_cast_fp16")]; + tensor linear_51_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_17_cast_fp16)[name = string("linear_51_cast_fp16")]; + tensor var_761 = const()[name = string("op_761"), val = tensor([1, 128, 12, 64])]; + tensor x_107_cast_fp16 = reshape(shape = var_761, x = linear_51_cast_fp16)[name = string("x_107_cast_fp16")]; + tensor transpose_96_perm_0 = const()[name = string("transpose_96_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_97_perm_0 = const()[name = string("transpose_97_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_98_perm_0 = const()[name = string("transpose_98_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_98 = transpose(perm = transpose_98_perm_0, x = x_107_cast_fp16)[name = string("transpose_122")]; + tensor transpose_97 = transpose(perm = transpose_97_perm_0, x = x_103_cast_fp16)[name = string("transpose_123")]; + tensor transpose_96 = transpose(perm = transpose_96_perm_0, x = x_99_cast_fp16)[name = string("transpose_124")]; + tensor attention_output_33_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_97, query = transpose_96, value = transpose_98)[name = string("attention_output_33_cast_fp16")]; + tensor attention_output_35_perm_0 = const()[name = string("attention_output_35_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_767 = const()[name = string("op_767"), val = tensor([1, 128, 768])]; + tensor attention_output_35_cast_fp16 = transpose(perm = attention_output_35_perm_0, x = attention_output_33_cast_fp16)[name = string("transpose_121")]; + tensor input_105_cast_fp16 = reshape(shape = var_767, x = attention_output_35_cast_fp16)[name = string("input_105_cast_fp16")]; + tensor linear_52_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_105_cast_fp16)[name = string("linear_52_cast_fp16")]; + tensor input_107_cast_fp16 = add(x = hidden_states_17_cast_fp16, y = linear_52_cast_fp16)[name = string("input_107_cast_fp16")]; + tensor input_109_axes_0 = const()[name = string("input_109_axes_0"), val = tensor([-1])]; + tensor input_109_cast_fp16 = layer_norm(axes = input_109_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_107_cast_fp16)[name = string("input_109_cast_fp16")]; + tensor linear_53_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_109_cast_fp16)[name = string("linear_53_cast_fp16")]; + string input_113_mode_0 = const()[name = string("input_113_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_113_cast_fp16 = gelu(mode = input_113_mode_0, x = linear_53_cast_fp16)[name = string("input_113_cast_fp16")]; + tensor linear_54_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_113_cast_fp16)[name = string("linear_54_cast_fp16")]; + tensor input_115_cast_fp16 = add(x = linear_54_cast_fp16, y = input_109_cast_fp16)[name = string("input_115_cast_fp16")]; + tensor hidden_states_19_axes_0 = const()[name = string("hidden_states_19_axes_0"), val = tensor([-1])]; + tensor hidden_states_19_cast_fp16 = layer_norm(axes = hidden_states_19_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_115_cast_fp16)[name = string("hidden_states_19_cast_fp16")]; + tensor linear_55_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_19_cast_fp16)[name = string("linear_55_cast_fp16")]; + tensor var_818 = const()[name = string("op_818"), val = tensor([1, 128, 12, 64])]; + tensor x_111_cast_fp16 = reshape(shape = var_818, x = linear_55_cast_fp16)[name = string("x_111_cast_fp16")]; + tensor linear_56_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_19_cast_fp16)[name = string("linear_56_cast_fp16")]; + tensor var_827 = const()[name = string("op_827"), val = tensor([1, 128, 12, 64])]; + tensor x_115_cast_fp16 = reshape(shape = var_827, x = linear_56_cast_fp16)[name = string("x_115_cast_fp16")]; + tensor linear_57_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_19_cast_fp16)[name = string("linear_57_cast_fp16")]; + tensor var_836 = const()[name = string("op_836"), val = tensor([1, 128, 12, 64])]; + tensor x_119_cast_fp16 = reshape(shape = var_836, x = linear_57_cast_fp16)[name = string("x_119_cast_fp16")]; + tensor transpose_99_perm_0 = const()[name = string("transpose_99_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_100_perm_0 = const()[name = string("transpose_100_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_101_perm_0 = const()[name = string("transpose_101_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_101 = transpose(perm = transpose_101_perm_0, x = x_119_cast_fp16)[name = string("transpose_118")]; + tensor transpose_100 = transpose(perm = transpose_100_perm_0, x = x_115_cast_fp16)[name = string("transpose_119")]; + tensor transpose_99 = transpose(perm = transpose_99_perm_0, x = x_111_cast_fp16)[name = string("transpose_120")]; + tensor attention_output_37_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_100, query = transpose_99, value = transpose_101)[name = string("attention_output_37_cast_fp16")]; + tensor attention_output_39_perm_0 = const()[name = string("attention_output_39_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_842 = const()[name = string("op_842"), val = tensor([1, 128, 768])]; + tensor attention_output_39_cast_fp16 = transpose(perm = attention_output_39_perm_0, x = attention_output_37_cast_fp16)[name = string("transpose_117")]; + tensor input_117_cast_fp16 = reshape(shape = var_842, x = attention_output_39_cast_fp16)[name = string("input_117_cast_fp16")]; + tensor linear_58_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_117_cast_fp16)[name = string("linear_58_cast_fp16")]; + tensor input_119_cast_fp16 = add(x = hidden_states_19_cast_fp16, y = linear_58_cast_fp16)[name = string("input_119_cast_fp16")]; + tensor input_121_axes_0 = const()[name = string("input_121_axes_0"), val = tensor([-1])]; + tensor input_121_cast_fp16 = layer_norm(axes = input_121_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_119_cast_fp16)[name = string("input_121_cast_fp16")]; + tensor linear_59_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_121_cast_fp16)[name = string("linear_59_cast_fp16")]; + string input_125_mode_0 = const()[name = string("input_125_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_125_cast_fp16 = gelu(mode = input_125_mode_0, x = linear_59_cast_fp16)[name = string("input_125_cast_fp16")]; + tensor linear_60_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_125_cast_fp16)[name = string("linear_60_cast_fp16")]; + tensor input_127_cast_fp16 = add(x = linear_60_cast_fp16, y = input_121_cast_fp16)[name = string("input_127_cast_fp16")]; + tensor hidden_states_21_axes_0 = const()[name = string("hidden_states_21_axes_0"), val = tensor([-1])]; + tensor hidden_states_21_cast_fp16 = layer_norm(axes = hidden_states_21_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_127_cast_fp16)[name = string("hidden_states_21_cast_fp16")]; + tensor linear_61_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_21_cast_fp16)[name = string("linear_61_cast_fp16")]; + tensor var_893 = const()[name = string("op_893"), val = tensor([1, 128, 12, 64])]; + tensor x_123_cast_fp16 = reshape(shape = var_893, x = linear_61_cast_fp16)[name = string("x_123_cast_fp16")]; + tensor linear_62_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_21_cast_fp16)[name = string("linear_62_cast_fp16")]; + tensor var_902 = const()[name = string("op_902"), val = tensor([1, 128, 12, 64])]; + tensor x_127_cast_fp16 = reshape(shape = var_902, x = linear_62_cast_fp16)[name = string("x_127_cast_fp16")]; + tensor linear_63_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_21_cast_fp16)[name = string("linear_63_cast_fp16")]; + tensor var_911 = const()[name = string("op_911"), val = tensor([1, 128, 12, 64])]; + tensor x_131_cast_fp16 = reshape(shape = var_911, x = linear_63_cast_fp16)[name = string("x_131_cast_fp16")]; + tensor transpose_102_perm_0 = const()[name = string("transpose_102_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_103_perm_0 = const()[name = string("transpose_103_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_104_perm_0 = const()[name = string("transpose_104_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_104 = transpose(perm = transpose_104_perm_0, x = x_131_cast_fp16)[name = string("transpose_114")]; + tensor transpose_103 = transpose(perm = transpose_103_perm_0, x = x_127_cast_fp16)[name = string("transpose_115")]; + tensor transpose_102 = transpose(perm = transpose_102_perm_0, x = x_123_cast_fp16)[name = string("transpose_116")]; + tensor attention_output_41_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_103, query = transpose_102, value = transpose_104)[name = string("attention_output_41_cast_fp16")]; + tensor attention_output_43_perm_0 = const()[name = string("attention_output_43_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_917 = const()[name = string("op_917"), val = tensor([1, 128, 768])]; + tensor attention_output_43_cast_fp16 = transpose(perm = attention_output_43_perm_0, x = attention_output_41_cast_fp16)[name = string("transpose_113")]; + tensor input_129_cast_fp16 = reshape(shape = var_917, x = attention_output_43_cast_fp16)[name = string("input_129_cast_fp16")]; + tensor linear_64_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_129_cast_fp16)[name = string("linear_64_cast_fp16")]; + tensor input_131_cast_fp16 = add(x = hidden_states_21_cast_fp16, y = linear_64_cast_fp16)[name = string("input_131_cast_fp16")]; + tensor input_133_axes_0 = const()[name = string("input_133_axes_0"), val = tensor([-1])]; + tensor input_133_cast_fp16 = layer_norm(axes = input_133_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_131_cast_fp16)[name = string("input_133_cast_fp16")]; + tensor linear_65_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_133_cast_fp16)[name = string("linear_65_cast_fp16")]; + string input_137_mode_0 = const()[name = string("input_137_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_137_cast_fp16 = gelu(mode = input_137_mode_0, x = linear_65_cast_fp16)[name = string("input_137_cast_fp16")]; + tensor linear_66_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_137_cast_fp16)[name = string("linear_66_cast_fp16")]; + tensor input_139_cast_fp16 = add(x = linear_66_cast_fp16, y = input_133_cast_fp16)[name = string("input_139_cast_fp16")]; + tensor hidden_states_axes_0 = const()[name = string("hidden_states_axes_0"), val = tensor([-1])]; + tensor hidden_states_cast_fp16 = layer_norm(axes = hidden_states_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_139_cast_fp16)[name = string("hidden_states_cast_fp16")]; + tensor linear_67_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_cast_fp16)[name = string("linear_67_cast_fp16")]; + tensor var_968 = const()[name = string("op_968"), val = tensor([1, 128, 12, 64])]; + tensor x_135_cast_fp16 = reshape(shape = var_968, x = linear_67_cast_fp16)[name = string("x_135_cast_fp16")]; + tensor linear_68_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_cast_fp16)[name = string("linear_68_cast_fp16")]; + tensor var_977 = const()[name = string("op_977"), val = tensor([1, 128, 12, 64])]; + tensor x_139_cast_fp16 = reshape(shape = var_977, x = linear_68_cast_fp16)[name = string("x_139_cast_fp16")]; + tensor linear_69_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_cast_fp16)[name = string("linear_69_cast_fp16")]; + tensor var_986 = const()[name = string("op_986"), val = tensor([1, 128, 12, 64])]; + tensor x_cast_fp16 = reshape(shape = var_986, x = linear_69_cast_fp16)[name = string("x_cast_fp16")]; + tensor transpose_105_perm_0 = const()[name = string("transpose_105_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_106_perm_0 = const()[name = string("transpose_106_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_107_perm_0 = const()[name = string("transpose_107_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_107 = transpose(perm = transpose_107_perm_0, x = x_cast_fp16)[name = string("transpose_110")]; + tensor transpose_106 = transpose(perm = transpose_106_perm_0, x = x_139_cast_fp16)[name = string("transpose_111")]; + tensor transpose_105 = transpose(perm = transpose_105_perm_0, x = x_135_cast_fp16)[name = string("transpose_112")]; + tensor attention_output_45_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_106, query = transpose_105, value = transpose_107)[name = string("attention_output_45_cast_fp16")]; + tensor attention_output_perm_0 = const()[name = string("attention_output_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_992 = const()[name = string("op_992"), val = tensor([1, 128, 768])]; + tensor attention_output_cast_fp16 = transpose(perm = attention_output_perm_0, x = attention_output_45_cast_fp16)[name = string("transpose_109")]; + tensor input_141_cast_fp16 = reshape(shape = var_992, x = attention_output_cast_fp16)[name = string("input_141_cast_fp16")]; + tensor linear_70_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_141_cast_fp16)[name = string("linear_70_cast_fp16")]; + tensor input_143_cast_fp16 = add(x = hidden_states_cast_fp16, y = linear_70_cast_fp16)[name = string("input_143_cast_fp16")]; + tensor input_145_axes_0 = const()[name = string("input_145_axes_0"), val = tensor([-1])]; + tensor input_145_cast_fp16 = layer_norm(axes = input_145_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_143_cast_fp16)[name = string("input_145_cast_fp16")]; + tensor linear_71_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_145_cast_fp16)[name = string("linear_71_cast_fp16")]; + string input_149_mode_0 = const()[name = string("input_149_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_149_cast_fp16 = gelu(mode = input_149_mode_0, x = linear_71_cast_fp16)[name = string("input_149_cast_fp16")]; + tensor linear_72_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_149_cast_fp16)[name = string("linear_72_cast_fp16")]; + tensor input_151_cast_fp16 = add(x = linear_72_cast_fp16, y = input_145_cast_fp16)[name = string("input_151_cast_fp16")]; + tensor sequence_output_axes_0 = const()[name = string("sequence_output_axes_0"), val = tensor([-1])]; + tensor sequence_output = layer_norm(axes = sequence_output_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_151_cast_fp16)[name = string("sequence_output_cast_fp16")]; + tensor bert_encoder_weight_to_fp16 = const()[name = string("bert_encoder_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11339264)))]; + tensor bert_encoder_bias_to_fp16 = const()[name = string("bert_encoder_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(12125760)))]; + tensor linear_73_cast_fp16 = linear(bias = bert_encoder_bias_to_fp16, weight = bert_encoder_weight_to_fp16, x = sequence_output)[name = string("linear_73_cast_fp16")]; + tensor var_1030_perm_0 = const()[name = string("op_1030_perm_0"), val = tensor([0, -1, -2])]; + tensor var_1030 = transpose(perm = var_1030_perm_0, x = linear_73_cast_fp16)[name = string("transpose_108")]; + } -> (sequence_output, var_1030); +} \ No newline at end of file diff --git a/iteration_3/compiled/bert_fp16_t128.mlmodelc/weights/weight.bin b/iteration_3/compiled/bert_fp16_t128.mlmodelc/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..df96cc13a5400d026fdae28981166d7389f06921 --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t128.mlmodelc/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ff3ca8fac0332427ddfe5e78954382359d26516284113001a7484b60455eb10 +size 12126848 diff --git a/iteration_3/compiled/bert_fp16_t256.mlmodelc/analytics/coremldata.bin b/iteration_3/compiled/bert_fp16_t256.mlmodelc/analytics/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..420a99d27a0512074508232a6d4b2f396e9c3b6e --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t256.mlmodelc/analytics/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cdd98e544fdd1d3002b708faa81092ba37b68369dfedee5cb0152a8340b68bdc +size 243 diff --git a/iteration_3/compiled/bert_fp16_t256.mlmodelc/coremldata.bin b/iteration_3/compiled/bert_fp16_t256.mlmodelc/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..2bd888af5b31623975411a01434af9a55156d5ce --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t256.mlmodelc/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2aba3b0f5fbc0614df555dcbfd939baa534dece2cf898b67db6a99169e55875a +size 441 diff --git a/iteration_3/compiled/bert_fp16_t256.mlmodelc/metadata.json b/iteration_3/compiled/bert_fp16_t256.mlmodelc/metadata.json new file mode 100644 index 0000000000000000000000000000000000000000..9f7001f3d670b920702d262f41b4d09fb8e3eb66 --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t256.mlmodelc/metadata.json @@ -0,0 +1,94 @@ +[ + { + "metadataOutputVersion" : "3.0", + "storagePrecision" : "Float16", + "outputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float16", + "formattedType" : "MultiArray (Float16 1 × 256 × 768)", + "shortDescription" : "", + "shape" : "[1, 256, 768]", + "name" : "sequence_output", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float16", + "formattedType" : "MultiArray (Float16 1 × 512 × 256)", + "shortDescription" : "", + "shape" : "[1, 512, 256]", + "name" : "var_1030", + "type" : "MultiArray" + } + ], + "modelParameters" : [ + + ], + "specificationVersion" : 9, + "mlProgramOperationTypeHistogram" : { + "Ios18.linear" : 74, + "Ios18.scaledDotProductAttention" : 12, + "Ios18.sub" : 1, + "Select" : 2, + "Ios18.expandDims" : 2, + "Ios18.gelu" : 12, + "Ios18.gather" : 1, + "Ios18.add" : 27, + "Tile" : 1, + "Ios18.layerNorm" : 25, + "Ios18.transpose" : 49, + "Ios18.cast" : 5, + "Ios18.reshape" : 48, + "Ios18.greaterEqual" : 1 + }, + "computePrecision" : "Mixed (Float16, Int16, Int32)", + "isUpdatable" : "0", + "stateSchema" : [ + + ], + "availability" : { + "macOS" : "15.0", + "tvOS" : "18.0", + "visionOS" : "2.0", + "watchOS" : "11.0", + "iOS" : "18.0", + "macCatalyst" : "18.0" + }, + "modelType" : { + "name" : "MLModelType_mlProgram" + }, + "userDefinedMetadata" : { + "com.github.apple.coremltools.conversion_date" : "2026-05-08", + "com.github.apple.coremltools.source" : "torch==2.11.0", + "com.github.apple.coremltools.version" : "9.0", + "com.github.apple.coremltools.source_dialect" : "TorchScript" + }, + "inputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Int32", + "formattedType" : "MultiArray (Int32 1 × 256)", + "shortDescription" : "", + "shape" : "[1, 256]", + "name" : "tokens", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Int32", + "formattedType" : "MultiArray (Int32 1 × 256)", + "shortDescription" : "", + "shape" : "[1, 256]", + "name" : "attention_mask", + "type" : "MultiArray" + } + ], + "generatedClassName" : "bert_fp16_t256", + "method" : "predict" + } +] \ No newline at end of file diff --git a/iteration_3/compiled/bert_fp16_t256.mlmodelc/model.mil b/iteration_3/compiled/bert_fp16_t256.mlmodelc/model.mil new file mode 100644 index 0000000000000000000000000000000000000000..fa808e2ee63bc2f0692aa9f7f790f6331c290b1b --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t256.mlmodelc/model.mil @@ -0,0 +1,442 @@ +program(1.3) +[buildInfo = dict({{"coremlc-component-MIL", "3520.4.1"}, {"coremlc-version", "3520.5.1"}, {"coremltools-component-torch", "2.11.0"}, {"coremltools-source-dialect", "TorchScript"}, {"coremltools-version", "9.0"}})] +{ + func main(tensor attention_mask, tensor tokens) { + int32 inputs_embeds_batch_dims_0 = const()[name = string("inputs_embeds_batch_dims_0"), val = int32(0)]; + bool inputs_embeds_validate_indices_0 = const()[name = string("inputs_embeds_validate_indices_0"), val = bool(false)]; + tensor bert_embeddings_word_embeddings_weight_to_fp16 = const()[name = string("bert_embeddings_word_embeddings_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(64)))]; + string tokens_to_int16_dtype_0 = const()[name = string("tokens_to_int16_dtype_0"), val = string("int16")]; + string cast_53_dtype_0 = const()[name = string("cast_53_dtype_0"), val = string("int32")]; + int32 greater_equal_0_y_0 = const()[name = string("greater_equal_0_y_0"), val = int32(0)]; + tensor tokens_to_int16 = cast(dtype = tokens_to_int16_dtype_0, x = tokens)[name = string("cast_58")]; + tensor cast_53 = cast(dtype = cast_53_dtype_0, x = tokens_to_int16)[name = string("cast_57")]; + tensor greater_equal_0 = greater_equal(x = cast_53, y = greater_equal_0_y_0)[name = string("greater_equal_0")]; + int32 slice_by_index_0 = const()[name = string("slice_by_index_0"), val = int32(178)]; + tensor add_0 = add(x = cast_53, y = slice_by_index_0)[name = string("add_0")]; + tensor select_0 = select(a = cast_53, b = add_0, cond = greater_equal_0)[name = string("select_0")]; + int32 inputs_embeds_cast_fp16_cast_uint16_axis_0 = const()[name = string("inputs_embeds_cast_fp16_cast_uint16_axis_0"), val = int32(0)]; + string select_0_to_int16_dtype_0 = const()[name = string("select_0_to_int16_dtype_0"), val = string("int16")]; + tensor select_0_to_int16 = cast(dtype = select_0_to_int16_dtype_0, x = select_0)[name = string("cast_56")]; + tensor inputs_embeds_cast_fp16_cast_uint16_cast_uint16 = gather(axis = inputs_embeds_cast_fp16_cast_uint16_axis_0, batch_dims = inputs_embeds_batch_dims_0, indices = select_0_to_int16, validate_indices = inputs_embeds_validate_indices_0, x = bert_embeddings_word_embeddings_weight_to_fp16)[name = string("inputs_embeds_cast_fp16_cast_uint16_cast_uint16")]; + tensor token_type_embeddings_1_to_fp16 = const()[name = string("token_type_embeddings_1_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(45696)))]; + tensor embeddings_1_cast_fp16 = add(x = inputs_embeds_cast_fp16_cast_uint16_cast_uint16, y = token_type_embeddings_1_to_fp16)[name = string("embeddings_1_cast_fp16")]; + tensor position_embeddings_1_to_fp16 = const()[name = string("position_embeddings_1_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(111296)))]; + tensor input_5_cast_fp16 = add(x = embeddings_1_cast_fp16, y = position_embeddings_1_to_fp16)[name = string("input_5_cast_fp16")]; + tensor input_7_axes_0 = const()[name = string("input_7_axes_0"), val = tensor([-1])]; + tensor bert_embeddings_LayerNorm_weight_to_fp16 = const()[name = string("bert_embeddings_LayerNorm_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(176896)))]; + tensor bert_embeddings_LayerNorm_bias_to_fp16 = const()[name = string("bert_embeddings_LayerNorm_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(177216)))]; + fp16 var_34_to_fp16 = const()[name = string("op_34_to_fp16"), val = fp16(0x1p-24)]; + tensor input_7_cast_fp16 = layer_norm(axes = input_7_axes_0, beta = bert_embeddings_LayerNorm_bias_to_fp16, epsilon = var_34_to_fp16, gamma = bert_embeddings_LayerNorm_weight_to_fp16, x = input_5_cast_fp16)[name = string("input_7_cast_fp16")]; + tensor var_79_axes_0 = const()[name = string("op_79_axes_0"), val = tensor([1])]; + tensor var_79 = expand_dims(axes = var_79_axes_0, x = attention_mask)[name = string("op_79")]; + tensor var_81_axes_0 = const()[name = string("op_81_axes_0"), val = tensor([2])]; + tensor var_81 = expand_dims(axes = var_81_axes_0, x = var_79)[name = string("op_81")]; + tensor var_90_reps_0 = const()[name = string("op_90_reps_0"), val = tensor([1, 1, 256, 1])]; + tensor var_90 = tile(reps = var_90_reps_0, x = var_81)[name = string("op_90")]; + fp16 var_96_to_fp16 = const()[name = string("op_96_to_fp16"), val = fp16(0x1p+0)]; + string var_95_to_fp16_dtype_0 = const()[name = string("op_95_to_fp16_dtype_0"), val = string("fp16")]; + tensor var_90_to_fp16 = cast(dtype = var_95_to_fp16_dtype_0, x = var_90)[name = string("cast_55")]; + tensor inverted_mask_cast_fp16 = sub(x = var_96_to_fp16, y = var_90_to_fp16)[name = string("inverted_mask_cast_fp16")]; + string var_103_dtype_0 = const()[name = string("op_103_dtype_0"), val = string("bool")]; + fp16 var_104_to_fp16 = const()[name = string("op_104_to_fp16"), val = fp16(-inf)]; + tensor inverted_mask_cast_fp16_to_bool = cast(dtype = var_103_dtype_0, x = inverted_mask_cast_fp16)[name = string("cast_54")]; + tensor attention_mask_cast_fp16 = select(a = var_104_to_fp16, b = inverted_mask_cast_fp16, cond = inverted_mask_cast_fp16_to_bool)[name = string("attention_mask_cast_fp16")]; + tensor bert_encoder_embedding_hidden_mapping_in_weight_to_fp16 = const()[name = string("bert_encoder_embedding_hidden_mapping_in_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(177536)))]; + tensor bert_encoder_embedding_hidden_mapping_in_bias_to_fp16 = const()[name = string("bert_encoder_embedding_hidden_mapping_in_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(374208)))]; + tensor linear_0_cast_fp16 = linear(bias = bert_encoder_embedding_hidden_mapping_in_bias_to_fp16, weight = bert_encoder_embedding_hidden_mapping_in_weight_to_fp16, x = input_7_cast_fp16)[name = string("linear_0_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(375808)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(1555520)))]; + tensor linear_1_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = linear_0_cast_fp16)[name = string("linear_1_cast_fp16")]; + tensor var_143 = const()[name = string("op_143"), val = tensor([1, 256, 12, 64])]; + tensor x_3_cast_fp16 = reshape(shape = var_143, x = linear_1_cast_fp16)[name = string("x_3_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(1557120)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2736832)))]; + tensor linear_2_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = linear_0_cast_fp16)[name = string("linear_2_cast_fp16")]; + tensor var_152 = const()[name = string("op_152"), val = tensor([1, 256, 12, 64])]; + tensor x_7_cast_fp16 = reshape(shape = var_152, x = linear_2_cast_fp16)[name = string("x_7_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2738432)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(3918144)))]; + tensor linear_3_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = linear_0_cast_fp16)[name = string("linear_3_cast_fp16")]; + tensor var_161 = const()[name = string("op_161"), val = tensor([1, 256, 12, 64])]; + tensor x_11_cast_fp16 = reshape(shape = var_161, x = linear_3_cast_fp16)[name = string("x_11_cast_fp16")]; + tensor transpose_72_perm_0 = const()[name = string("transpose_72_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_73_perm_0 = const()[name = string("transpose_73_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_74_perm_0 = const()[name = string("transpose_74_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_74 = transpose(perm = transpose_74_perm_0, x = x_11_cast_fp16)[name = string("transpose_154")]; + tensor transpose_73 = transpose(perm = transpose_73_perm_0, x = x_7_cast_fp16)[name = string("transpose_155")]; + tensor transpose_72 = transpose(perm = transpose_72_perm_0, x = x_3_cast_fp16)[name = string("transpose_156")]; + tensor attention_output_1_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_73, query = transpose_72, value = transpose_74)[name = string("attention_output_1_cast_fp16")]; + tensor attention_output_3_perm_0 = const()[name = string("attention_output_3_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_167 = const()[name = string("op_167"), val = tensor([1, 256, 768])]; + tensor attention_output_3_cast_fp16 = transpose(perm = attention_output_3_perm_0, x = attention_output_1_cast_fp16)[name = string("transpose_153")]; + tensor input_9_cast_fp16 = reshape(shape = var_167, x = attention_output_3_cast_fp16)[name = string("input_9_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(3919744)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5099456)))]; + tensor linear_4_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_9_cast_fp16)[name = string("linear_4_cast_fp16")]; + tensor input_11_cast_fp16 = add(x = linear_0_cast_fp16, y = linear_4_cast_fp16)[name = string("input_11_cast_fp16")]; + tensor input_13_axes_0 = const()[name = string("input_13_axes_0"), val = tensor([-1])]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5101056)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5102656)))]; + fp16 var_118_to_fp16 = const()[name = string("op_118_to_fp16"), val = fp16(0x1p-24)]; + tensor input_13_cast_fp16 = layer_norm(axes = input_13_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_11_cast_fp16)[name = string("input_13_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5104256)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(8250048)))]; + tensor linear_5_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_13_cast_fp16)[name = string("linear_5_cast_fp16")]; + string input_17_mode_0 = const()[name = string("input_17_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_17_cast_fp16 = gelu(mode = input_17_mode_0, x = linear_5_cast_fp16)[name = string("input_17_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(8254208)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11400000)))]; + tensor linear_6_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_17_cast_fp16)[name = string("linear_6_cast_fp16")]; + tensor input_19_cast_fp16 = add(x = linear_6_cast_fp16, y = input_13_cast_fp16)[name = string("input_19_cast_fp16")]; + tensor hidden_states_3_axes_0 = const()[name = string("hidden_states_3_axes_0"), val = tensor([-1])]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11401600)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11403200)))]; + tensor hidden_states_3_cast_fp16 = layer_norm(axes = hidden_states_3_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_19_cast_fp16)[name = string("hidden_states_3_cast_fp16")]; + tensor linear_7_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_3_cast_fp16)[name = string("linear_7_cast_fp16")]; + tensor var_218 = const()[name = string("op_218"), val = tensor([1, 256, 12, 64])]; + tensor x_15_cast_fp16 = reshape(shape = var_218, x = linear_7_cast_fp16)[name = string("x_15_cast_fp16")]; + tensor linear_8_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_3_cast_fp16)[name = string("linear_8_cast_fp16")]; + tensor var_227 = const()[name = string("op_227"), val = tensor([1, 256, 12, 64])]; + tensor x_19_cast_fp16 = reshape(shape = var_227, x = linear_8_cast_fp16)[name = string("x_19_cast_fp16")]; + tensor linear_9_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_3_cast_fp16)[name = string("linear_9_cast_fp16")]; + tensor var_236 = const()[name = string("op_236"), val = tensor([1, 256, 12, 64])]; + tensor x_23_cast_fp16 = reshape(shape = var_236, x = linear_9_cast_fp16)[name = string("x_23_cast_fp16")]; + tensor transpose_75_perm_0 = const()[name = string("transpose_75_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_76_perm_0 = const()[name = string("transpose_76_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_77_perm_0 = const()[name = string("transpose_77_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_77 = transpose(perm = transpose_77_perm_0, x = x_23_cast_fp16)[name = string("transpose_150")]; + tensor transpose_76 = transpose(perm = transpose_76_perm_0, x = x_19_cast_fp16)[name = string("transpose_151")]; + tensor transpose_75 = transpose(perm = transpose_75_perm_0, x = x_15_cast_fp16)[name = string("transpose_152")]; + tensor attention_output_5_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_76, query = transpose_75, value = transpose_77)[name = string("attention_output_5_cast_fp16")]; + tensor attention_output_7_perm_0 = const()[name = string("attention_output_7_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_242 = const()[name = string("op_242"), val = tensor([1, 256, 768])]; + tensor attention_output_7_cast_fp16 = transpose(perm = attention_output_7_perm_0, x = attention_output_5_cast_fp16)[name = string("transpose_149")]; + tensor input_21_cast_fp16 = reshape(shape = var_242, x = attention_output_7_cast_fp16)[name = string("input_21_cast_fp16")]; + tensor linear_10_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_21_cast_fp16)[name = string("linear_10_cast_fp16")]; + tensor input_23_cast_fp16 = add(x = hidden_states_3_cast_fp16, y = linear_10_cast_fp16)[name = string("input_23_cast_fp16")]; + tensor input_25_axes_0 = const()[name = string("input_25_axes_0"), val = tensor([-1])]; + tensor input_25_cast_fp16 = layer_norm(axes = input_25_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_23_cast_fp16)[name = string("input_25_cast_fp16")]; + tensor linear_11_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_25_cast_fp16)[name = string("linear_11_cast_fp16")]; + string input_29_mode_0 = const()[name = string("input_29_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_29_cast_fp16 = gelu(mode = input_29_mode_0, x = linear_11_cast_fp16)[name = string("input_29_cast_fp16")]; + tensor linear_12_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_29_cast_fp16)[name = string("linear_12_cast_fp16")]; + tensor input_31_cast_fp16 = add(x = linear_12_cast_fp16, y = input_25_cast_fp16)[name = string("input_31_cast_fp16")]; + tensor hidden_states_5_axes_0 = const()[name = string("hidden_states_5_axes_0"), val = tensor([-1])]; + tensor hidden_states_5_cast_fp16 = layer_norm(axes = hidden_states_5_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_31_cast_fp16)[name = string("hidden_states_5_cast_fp16")]; + tensor linear_13_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_5_cast_fp16)[name = string("linear_13_cast_fp16")]; + tensor var_293 = const()[name = string("op_293"), val = tensor([1, 256, 12, 64])]; + tensor x_27_cast_fp16 = reshape(shape = var_293, x = linear_13_cast_fp16)[name = string("x_27_cast_fp16")]; + tensor linear_14_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_5_cast_fp16)[name = string("linear_14_cast_fp16")]; + tensor var_302 = const()[name = string("op_302"), val = tensor([1, 256, 12, 64])]; + tensor x_31_cast_fp16 = reshape(shape = var_302, x = linear_14_cast_fp16)[name = string("x_31_cast_fp16")]; + tensor linear_15_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_5_cast_fp16)[name = string("linear_15_cast_fp16")]; + tensor var_311 = const()[name = string("op_311"), val = tensor([1, 256, 12, 64])]; + tensor x_35_cast_fp16 = reshape(shape = var_311, x = linear_15_cast_fp16)[name = string("x_35_cast_fp16")]; + tensor transpose_78_perm_0 = const()[name = string("transpose_78_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_79_perm_0 = const()[name = string("transpose_79_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_80_perm_0 = const()[name = string("transpose_80_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_80 = transpose(perm = transpose_80_perm_0, x = x_35_cast_fp16)[name = string("transpose_146")]; + tensor transpose_79 = transpose(perm = transpose_79_perm_0, x = x_31_cast_fp16)[name = string("transpose_147")]; + tensor transpose_78 = transpose(perm = transpose_78_perm_0, x = x_27_cast_fp16)[name = string("transpose_148")]; + tensor attention_output_9_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_79, query = transpose_78, value = transpose_80)[name = string("attention_output_9_cast_fp16")]; + tensor attention_output_11_perm_0 = const()[name = string("attention_output_11_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_317 = const()[name = string("op_317"), val = tensor([1, 256, 768])]; + tensor attention_output_11_cast_fp16 = transpose(perm = attention_output_11_perm_0, x = attention_output_9_cast_fp16)[name = string("transpose_145")]; + tensor input_33_cast_fp16 = reshape(shape = var_317, x = attention_output_11_cast_fp16)[name = string("input_33_cast_fp16")]; + tensor linear_16_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_33_cast_fp16)[name = string("linear_16_cast_fp16")]; + tensor input_35_cast_fp16 = add(x = hidden_states_5_cast_fp16, y = linear_16_cast_fp16)[name = string("input_35_cast_fp16")]; + tensor input_37_axes_0 = const()[name = string("input_37_axes_0"), val = tensor([-1])]; + tensor input_37_cast_fp16 = layer_norm(axes = input_37_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_35_cast_fp16)[name = string("input_37_cast_fp16")]; + tensor linear_17_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_37_cast_fp16)[name = string("linear_17_cast_fp16")]; + string input_41_mode_0 = const()[name = string("input_41_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_41_cast_fp16 = gelu(mode = input_41_mode_0, x = linear_17_cast_fp16)[name = string("input_41_cast_fp16")]; + tensor linear_18_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_41_cast_fp16)[name = string("linear_18_cast_fp16")]; + tensor input_43_cast_fp16 = add(x = linear_18_cast_fp16, y = input_37_cast_fp16)[name = string("input_43_cast_fp16")]; + tensor hidden_states_7_axes_0 = const()[name = string("hidden_states_7_axes_0"), val = tensor([-1])]; + tensor hidden_states_7_cast_fp16 = layer_norm(axes = hidden_states_7_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_43_cast_fp16)[name = string("hidden_states_7_cast_fp16")]; + tensor linear_19_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_7_cast_fp16)[name = string("linear_19_cast_fp16")]; + tensor var_368 = const()[name = string("op_368"), val = tensor([1, 256, 12, 64])]; + tensor x_39_cast_fp16 = reshape(shape = var_368, x = linear_19_cast_fp16)[name = string("x_39_cast_fp16")]; + tensor linear_20_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_7_cast_fp16)[name = string("linear_20_cast_fp16")]; + tensor var_377 = const()[name = string("op_377"), val = tensor([1, 256, 12, 64])]; + tensor x_43_cast_fp16 = reshape(shape = var_377, x = linear_20_cast_fp16)[name = string("x_43_cast_fp16")]; + tensor linear_21_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_7_cast_fp16)[name = string("linear_21_cast_fp16")]; + tensor var_386 = const()[name = string("op_386"), val = tensor([1, 256, 12, 64])]; + tensor x_47_cast_fp16 = reshape(shape = var_386, x = linear_21_cast_fp16)[name = string("x_47_cast_fp16")]; + tensor transpose_81_perm_0 = const()[name = string("transpose_81_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_82_perm_0 = const()[name = string("transpose_82_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_83_perm_0 = const()[name = string("transpose_83_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_83 = transpose(perm = transpose_83_perm_0, x = x_47_cast_fp16)[name = string("transpose_142")]; + tensor transpose_82 = transpose(perm = transpose_82_perm_0, x = x_43_cast_fp16)[name = string("transpose_143")]; + tensor transpose_81 = transpose(perm = transpose_81_perm_0, x = x_39_cast_fp16)[name = string("transpose_144")]; + tensor attention_output_13_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_82, query = transpose_81, value = transpose_83)[name = string("attention_output_13_cast_fp16")]; + tensor attention_output_15_perm_0 = const()[name = string("attention_output_15_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_392 = const()[name = string("op_392"), val = tensor([1, 256, 768])]; + tensor attention_output_15_cast_fp16 = transpose(perm = attention_output_15_perm_0, x = attention_output_13_cast_fp16)[name = string("transpose_141")]; + tensor input_45_cast_fp16 = reshape(shape = var_392, x = attention_output_15_cast_fp16)[name = string("input_45_cast_fp16")]; + tensor linear_22_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_45_cast_fp16)[name = string("linear_22_cast_fp16")]; + tensor input_47_cast_fp16 = add(x = hidden_states_7_cast_fp16, y = linear_22_cast_fp16)[name = string("input_47_cast_fp16")]; + tensor input_49_axes_0 = const()[name = string("input_49_axes_0"), val = tensor([-1])]; + tensor input_49_cast_fp16 = layer_norm(axes = input_49_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_47_cast_fp16)[name = string("input_49_cast_fp16")]; + tensor linear_23_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_49_cast_fp16)[name = string("linear_23_cast_fp16")]; + string input_53_mode_0 = const()[name = string("input_53_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_53_cast_fp16 = gelu(mode = input_53_mode_0, x = linear_23_cast_fp16)[name = string("input_53_cast_fp16")]; + tensor linear_24_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_53_cast_fp16)[name = string("linear_24_cast_fp16")]; + tensor input_55_cast_fp16 = add(x = linear_24_cast_fp16, y = input_49_cast_fp16)[name = string("input_55_cast_fp16")]; + tensor hidden_states_9_axes_0 = const()[name = string("hidden_states_9_axes_0"), val = tensor([-1])]; + tensor hidden_states_9_cast_fp16 = layer_norm(axes = hidden_states_9_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_55_cast_fp16)[name = string("hidden_states_9_cast_fp16")]; + tensor linear_25_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_9_cast_fp16)[name = string("linear_25_cast_fp16")]; + tensor var_443 = const()[name = string("op_443"), val = tensor([1, 256, 12, 64])]; + tensor x_51_cast_fp16 = reshape(shape = var_443, x = linear_25_cast_fp16)[name = string("x_51_cast_fp16")]; + tensor linear_26_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_9_cast_fp16)[name = string("linear_26_cast_fp16")]; + tensor var_452 = const()[name = string("op_452"), val = tensor([1, 256, 12, 64])]; + tensor x_55_cast_fp16 = reshape(shape = var_452, x = linear_26_cast_fp16)[name = string("x_55_cast_fp16")]; + tensor linear_27_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_9_cast_fp16)[name = string("linear_27_cast_fp16")]; + tensor var_461 = const()[name = string("op_461"), val = tensor([1, 256, 12, 64])]; + tensor x_59_cast_fp16 = reshape(shape = var_461, x = linear_27_cast_fp16)[name = string("x_59_cast_fp16")]; + tensor transpose_84_perm_0 = const()[name = string("transpose_84_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_85_perm_0 = const()[name = string("transpose_85_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_86_perm_0 = const()[name = string("transpose_86_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_86 = transpose(perm = transpose_86_perm_0, x = x_59_cast_fp16)[name = string("transpose_138")]; + tensor transpose_85 = transpose(perm = transpose_85_perm_0, x = x_55_cast_fp16)[name = string("transpose_139")]; + tensor transpose_84 = transpose(perm = transpose_84_perm_0, x = x_51_cast_fp16)[name = string("transpose_140")]; + tensor attention_output_17_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_85, query = transpose_84, value = transpose_86)[name = string("attention_output_17_cast_fp16")]; + tensor attention_output_19_perm_0 = const()[name = string("attention_output_19_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_467 = const()[name = string("op_467"), val = tensor([1, 256, 768])]; + tensor attention_output_19_cast_fp16 = transpose(perm = attention_output_19_perm_0, x = attention_output_17_cast_fp16)[name = string("transpose_137")]; + tensor input_57_cast_fp16 = reshape(shape = var_467, x = attention_output_19_cast_fp16)[name = string("input_57_cast_fp16")]; + tensor linear_28_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_57_cast_fp16)[name = string("linear_28_cast_fp16")]; + tensor input_59_cast_fp16 = add(x = hidden_states_9_cast_fp16, y = linear_28_cast_fp16)[name = string("input_59_cast_fp16")]; + tensor input_61_axes_0 = const()[name = string("input_61_axes_0"), val = tensor([-1])]; + tensor input_61_cast_fp16 = layer_norm(axes = input_61_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_59_cast_fp16)[name = string("input_61_cast_fp16")]; + tensor linear_29_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_61_cast_fp16)[name = string("linear_29_cast_fp16")]; + string input_65_mode_0 = const()[name = string("input_65_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_65_cast_fp16 = gelu(mode = input_65_mode_0, x = linear_29_cast_fp16)[name = string("input_65_cast_fp16")]; + tensor linear_30_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_65_cast_fp16)[name = string("linear_30_cast_fp16")]; + tensor input_67_cast_fp16 = add(x = linear_30_cast_fp16, y = input_61_cast_fp16)[name = string("input_67_cast_fp16")]; + tensor hidden_states_11_axes_0 = const()[name = string("hidden_states_11_axes_0"), val = tensor([-1])]; + tensor hidden_states_11_cast_fp16 = layer_norm(axes = hidden_states_11_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_67_cast_fp16)[name = string("hidden_states_11_cast_fp16")]; + tensor linear_31_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_11_cast_fp16)[name = string("linear_31_cast_fp16")]; + tensor var_518 = const()[name = string("op_518"), val = tensor([1, 256, 12, 64])]; + tensor x_63_cast_fp16 = reshape(shape = var_518, x = linear_31_cast_fp16)[name = string("x_63_cast_fp16")]; + tensor linear_32_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_11_cast_fp16)[name = string("linear_32_cast_fp16")]; + tensor var_527 = const()[name = string("op_527"), val = tensor([1, 256, 12, 64])]; + tensor x_67_cast_fp16 = reshape(shape = var_527, x = linear_32_cast_fp16)[name = string("x_67_cast_fp16")]; + tensor linear_33_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_11_cast_fp16)[name = string("linear_33_cast_fp16")]; + tensor var_536 = const()[name = string("op_536"), val = tensor([1, 256, 12, 64])]; + tensor x_71_cast_fp16 = reshape(shape = var_536, x = linear_33_cast_fp16)[name = string("x_71_cast_fp16")]; + tensor transpose_87_perm_0 = const()[name = string("transpose_87_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_88_perm_0 = const()[name = string("transpose_88_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_89_perm_0 = const()[name = string("transpose_89_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_89 = transpose(perm = transpose_89_perm_0, x = x_71_cast_fp16)[name = string("transpose_134")]; + tensor transpose_88 = transpose(perm = transpose_88_perm_0, x = x_67_cast_fp16)[name = string("transpose_135")]; + tensor transpose_87 = transpose(perm = transpose_87_perm_0, x = x_63_cast_fp16)[name = string("transpose_136")]; + tensor attention_output_21_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_88, query = transpose_87, value = transpose_89)[name = string("attention_output_21_cast_fp16")]; + tensor attention_output_23_perm_0 = const()[name = string("attention_output_23_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_542 = const()[name = string("op_542"), val = tensor([1, 256, 768])]; + tensor attention_output_23_cast_fp16 = transpose(perm = attention_output_23_perm_0, x = attention_output_21_cast_fp16)[name = string("transpose_133")]; + tensor input_69_cast_fp16 = reshape(shape = var_542, x = attention_output_23_cast_fp16)[name = string("input_69_cast_fp16")]; + tensor linear_34_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_69_cast_fp16)[name = string("linear_34_cast_fp16")]; + tensor input_71_cast_fp16 = add(x = hidden_states_11_cast_fp16, y = linear_34_cast_fp16)[name = string("input_71_cast_fp16")]; + tensor input_73_axes_0 = const()[name = string("input_73_axes_0"), val = tensor([-1])]; + tensor input_73_cast_fp16 = layer_norm(axes = input_73_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_71_cast_fp16)[name = string("input_73_cast_fp16")]; + tensor linear_35_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_73_cast_fp16)[name = string("linear_35_cast_fp16")]; + string input_77_mode_0 = const()[name = string("input_77_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_77_cast_fp16 = gelu(mode = input_77_mode_0, x = linear_35_cast_fp16)[name = string("input_77_cast_fp16")]; + tensor linear_36_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_77_cast_fp16)[name = string("linear_36_cast_fp16")]; + tensor input_79_cast_fp16 = add(x = linear_36_cast_fp16, y = input_73_cast_fp16)[name = string("input_79_cast_fp16")]; + tensor hidden_states_13_axes_0 = const()[name = string("hidden_states_13_axes_0"), val = tensor([-1])]; + tensor hidden_states_13_cast_fp16 = layer_norm(axes = hidden_states_13_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_79_cast_fp16)[name = string("hidden_states_13_cast_fp16")]; + tensor linear_37_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_13_cast_fp16)[name = string("linear_37_cast_fp16")]; + tensor var_593 = const()[name = string("op_593"), val = tensor([1, 256, 12, 64])]; + tensor x_75_cast_fp16 = reshape(shape = var_593, x = linear_37_cast_fp16)[name = string("x_75_cast_fp16")]; + tensor linear_38_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_13_cast_fp16)[name = string("linear_38_cast_fp16")]; + tensor var_602 = const()[name = string("op_602"), val = tensor([1, 256, 12, 64])]; + tensor x_79_cast_fp16 = reshape(shape = var_602, x = linear_38_cast_fp16)[name = string("x_79_cast_fp16")]; + tensor linear_39_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_13_cast_fp16)[name = string("linear_39_cast_fp16")]; + tensor var_611 = const()[name = string("op_611"), val = tensor([1, 256, 12, 64])]; + tensor x_83_cast_fp16 = reshape(shape = var_611, x = linear_39_cast_fp16)[name = string("x_83_cast_fp16")]; + tensor transpose_90_perm_0 = const()[name = string("transpose_90_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_91_perm_0 = const()[name = string("transpose_91_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_92_perm_0 = const()[name = string("transpose_92_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_92 = transpose(perm = transpose_92_perm_0, x = x_83_cast_fp16)[name = string("transpose_130")]; + tensor transpose_91 = transpose(perm = transpose_91_perm_0, x = x_79_cast_fp16)[name = string("transpose_131")]; + tensor transpose_90 = transpose(perm = transpose_90_perm_0, x = x_75_cast_fp16)[name = string("transpose_132")]; + tensor attention_output_25_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_91, query = transpose_90, value = transpose_92)[name = string("attention_output_25_cast_fp16")]; + tensor attention_output_27_perm_0 = const()[name = string("attention_output_27_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_617 = const()[name = string("op_617"), val = tensor([1, 256, 768])]; + tensor attention_output_27_cast_fp16 = transpose(perm = attention_output_27_perm_0, x = attention_output_25_cast_fp16)[name = string("transpose_129")]; + tensor input_81_cast_fp16 = reshape(shape = var_617, x = attention_output_27_cast_fp16)[name = string("input_81_cast_fp16")]; + tensor linear_40_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_81_cast_fp16)[name = string("linear_40_cast_fp16")]; + tensor input_83_cast_fp16 = add(x = hidden_states_13_cast_fp16, y = linear_40_cast_fp16)[name = string("input_83_cast_fp16")]; + tensor input_85_axes_0 = const()[name = string("input_85_axes_0"), val = tensor([-1])]; + tensor input_85_cast_fp16 = layer_norm(axes = input_85_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_83_cast_fp16)[name = string("input_85_cast_fp16")]; + tensor linear_41_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_85_cast_fp16)[name = string("linear_41_cast_fp16")]; + string input_89_mode_0 = const()[name = string("input_89_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_89_cast_fp16 = gelu(mode = input_89_mode_0, x = linear_41_cast_fp16)[name = string("input_89_cast_fp16")]; + tensor linear_42_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_89_cast_fp16)[name = string("linear_42_cast_fp16")]; + tensor input_91_cast_fp16 = add(x = linear_42_cast_fp16, y = input_85_cast_fp16)[name = string("input_91_cast_fp16")]; + tensor hidden_states_15_axes_0 = const()[name = string("hidden_states_15_axes_0"), val = tensor([-1])]; + tensor hidden_states_15_cast_fp16 = layer_norm(axes = hidden_states_15_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_91_cast_fp16)[name = string("hidden_states_15_cast_fp16")]; + tensor linear_43_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_15_cast_fp16)[name = string("linear_43_cast_fp16")]; + tensor var_668 = const()[name = string("op_668"), val = tensor([1, 256, 12, 64])]; + tensor x_87_cast_fp16 = reshape(shape = var_668, x = linear_43_cast_fp16)[name = string("x_87_cast_fp16")]; + tensor linear_44_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_15_cast_fp16)[name = string("linear_44_cast_fp16")]; + tensor var_677 = const()[name = string("op_677"), val = tensor([1, 256, 12, 64])]; + tensor x_91_cast_fp16 = reshape(shape = var_677, x = linear_44_cast_fp16)[name = string("x_91_cast_fp16")]; + tensor linear_45_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_15_cast_fp16)[name = string("linear_45_cast_fp16")]; + tensor var_686 = const()[name = string("op_686"), val = tensor([1, 256, 12, 64])]; + tensor x_95_cast_fp16 = reshape(shape = var_686, x = linear_45_cast_fp16)[name = string("x_95_cast_fp16")]; + tensor transpose_93_perm_0 = const()[name = string("transpose_93_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_94_perm_0 = const()[name = string("transpose_94_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_95_perm_0 = const()[name = string("transpose_95_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_95 = transpose(perm = transpose_95_perm_0, x = x_95_cast_fp16)[name = string("transpose_126")]; + tensor transpose_94 = transpose(perm = transpose_94_perm_0, x = x_91_cast_fp16)[name = string("transpose_127")]; + tensor transpose_93 = transpose(perm = transpose_93_perm_0, x = x_87_cast_fp16)[name = string("transpose_128")]; + tensor attention_output_29_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_94, query = transpose_93, value = transpose_95)[name = string("attention_output_29_cast_fp16")]; + tensor attention_output_31_perm_0 = const()[name = string("attention_output_31_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_692 = const()[name = string("op_692"), val = tensor([1, 256, 768])]; + tensor attention_output_31_cast_fp16 = transpose(perm = attention_output_31_perm_0, x = attention_output_29_cast_fp16)[name = string("transpose_125")]; + tensor input_93_cast_fp16 = reshape(shape = var_692, x = attention_output_31_cast_fp16)[name = string("input_93_cast_fp16")]; + tensor linear_46_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_93_cast_fp16)[name = string("linear_46_cast_fp16")]; + tensor input_95_cast_fp16 = add(x = hidden_states_15_cast_fp16, y = linear_46_cast_fp16)[name = string("input_95_cast_fp16")]; + tensor input_97_axes_0 = const()[name = string("input_97_axes_0"), val = tensor([-1])]; + tensor input_97_cast_fp16 = layer_norm(axes = input_97_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_95_cast_fp16)[name = string("input_97_cast_fp16")]; + tensor linear_47_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_97_cast_fp16)[name = string("linear_47_cast_fp16")]; + string input_101_mode_0 = const()[name = string("input_101_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_101_cast_fp16 = gelu(mode = input_101_mode_0, x = linear_47_cast_fp16)[name = string("input_101_cast_fp16")]; + tensor linear_48_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_101_cast_fp16)[name = string("linear_48_cast_fp16")]; + tensor input_103_cast_fp16 = add(x = linear_48_cast_fp16, y = input_97_cast_fp16)[name = string("input_103_cast_fp16")]; + tensor hidden_states_17_axes_0 = const()[name = string("hidden_states_17_axes_0"), val = tensor([-1])]; + tensor hidden_states_17_cast_fp16 = layer_norm(axes = hidden_states_17_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_103_cast_fp16)[name = string("hidden_states_17_cast_fp16")]; + tensor linear_49_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_17_cast_fp16)[name = string("linear_49_cast_fp16")]; + tensor var_743 = const()[name = string("op_743"), val = tensor([1, 256, 12, 64])]; + tensor x_99_cast_fp16 = reshape(shape = var_743, x = linear_49_cast_fp16)[name = string("x_99_cast_fp16")]; + tensor linear_50_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_17_cast_fp16)[name = string("linear_50_cast_fp16")]; + tensor var_752 = const()[name = string("op_752"), val = tensor([1, 256, 12, 64])]; + tensor x_103_cast_fp16 = reshape(shape = var_752, x = linear_50_cast_fp16)[name = string("x_103_cast_fp16")]; + tensor linear_51_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_17_cast_fp16)[name = string("linear_51_cast_fp16")]; + tensor var_761 = const()[name = string("op_761"), val = tensor([1, 256, 12, 64])]; + tensor x_107_cast_fp16 = reshape(shape = var_761, x = linear_51_cast_fp16)[name = string("x_107_cast_fp16")]; + tensor transpose_96_perm_0 = const()[name = string("transpose_96_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_97_perm_0 = const()[name = string("transpose_97_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_98_perm_0 = const()[name = string("transpose_98_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_98 = transpose(perm = transpose_98_perm_0, x = x_107_cast_fp16)[name = string("transpose_122")]; + tensor transpose_97 = transpose(perm = transpose_97_perm_0, x = x_103_cast_fp16)[name = string("transpose_123")]; + tensor transpose_96 = transpose(perm = transpose_96_perm_0, x = x_99_cast_fp16)[name = string("transpose_124")]; + tensor attention_output_33_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_97, query = transpose_96, value = transpose_98)[name = string("attention_output_33_cast_fp16")]; + tensor attention_output_35_perm_0 = const()[name = string("attention_output_35_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_767 = const()[name = string("op_767"), val = tensor([1, 256, 768])]; + tensor attention_output_35_cast_fp16 = transpose(perm = attention_output_35_perm_0, x = attention_output_33_cast_fp16)[name = string("transpose_121")]; + tensor input_105_cast_fp16 = reshape(shape = var_767, x = attention_output_35_cast_fp16)[name = string("input_105_cast_fp16")]; + tensor linear_52_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_105_cast_fp16)[name = string("linear_52_cast_fp16")]; + tensor input_107_cast_fp16 = add(x = hidden_states_17_cast_fp16, y = linear_52_cast_fp16)[name = string("input_107_cast_fp16")]; + tensor input_109_axes_0 = const()[name = string("input_109_axes_0"), val = tensor([-1])]; + tensor input_109_cast_fp16 = layer_norm(axes = input_109_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_107_cast_fp16)[name = string("input_109_cast_fp16")]; + tensor linear_53_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_109_cast_fp16)[name = string("linear_53_cast_fp16")]; + string input_113_mode_0 = const()[name = string("input_113_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_113_cast_fp16 = gelu(mode = input_113_mode_0, x = linear_53_cast_fp16)[name = string("input_113_cast_fp16")]; + tensor linear_54_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_113_cast_fp16)[name = string("linear_54_cast_fp16")]; + tensor input_115_cast_fp16 = add(x = linear_54_cast_fp16, y = input_109_cast_fp16)[name = string("input_115_cast_fp16")]; + tensor hidden_states_19_axes_0 = const()[name = string("hidden_states_19_axes_0"), val = tensor([-1])]; + tensor hidden_states_19_cast_fp16 = layer_norm(axes = hidden_states_19_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_115_cast_fp16)[name = string("hidden_states_19_cast_fp16")]; + tensor linear_55_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_19_cast_fp16)[name = string("linear_55_cast_fp16")]; + tensor var_818 = const()[name = string("op_818"), val = tensor([1, 256, 12, 64])]; + tensor x_111_cast_fp16 = reshape(shape = var_818, x = linear_55_cast_fp16)[name = string("x_111_cast_fp16")]; + tensor linear_56_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_19_cast_fp16)[name = string("linear_56_cast_fp16")]; + tensor var_827 = const()[name = string("op_827"), val = tensor([1, 256, 12, 64])]; + tensor x_115_cast_fp16 = reshape(shape = var_827, x = linear_56_cast_fp16)[name = string("x_115_cast_fp16")]; + tensor linear_57_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_19_cast_fp16)[name = string("linear_57_cast_fp16")]; + tensor var_836 = const()[name = string("op_836"), val = tensor([1, 256, 12, 64])]; + tensor x_119_cast_fp16 = reshape(shape = var_836, x = linear_57_cast_fp16)[name = string("x_119_cast_fp16")]; + tensor transpose_99_perm_0 = const()[name = string("transpose_99_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_100_perm_0 = const()[name = string("transpose_100_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_101_perm_0 = const()[name = string("transpose_101_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_101 = transpose(perm = transpose_101_perm_0, x = x_119_cast_fp16)[name = string("transpose_118")]; + tensor transpose_100 = transpose(perm = transpose_100_perm_0, x = x_115_cast_fp16)[name = string("transpose_119")]; + tensor transpose_99 = transpose(perm = transpose_99_perm_0, x = x_111_cast_fp16)[name = string("transpose_120")]; + tensor attention_output_37_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_100, query = transpose_99, value = transpose_101)[name = string("attention_output_37_cast_fp16")]; + tensor attention_output_39_perm_0 = const()[name = string("attention_output_39_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_842 = const()[name = string("op_842"), val = tensor([1, 256, 768])]; + tensor attention_output_39_cast_fp16 = transpose(perm = attention_output_39_perm_0, x = attention_output_37_cast_fp16)[name = string("transpose_117")]; + tensor input_117_cast_fp16 = reshape(shape = var_842, x = attention_output_39_cast_fp16)[name = string("input_117_cast_fp16")]; + tensor linear_58_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_117_cast_fp16)[name = string("linear_58_cast_fp16")]; + tensor input_119_cast_fp16 = add(x = hidden_states_19_cast_fp16, y = linear_58_cast_fp16)[name = string("input_119_cast_fp16")]; + tensor input_121_axes_0 = const()[name = string("input_121_axes_0"), val = tensor([-1])]; + tensor input_121_cast_fp16 = layer_norm(axes = input_121_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_119_cast_fp16)[name = string("input_121_cast_fp16")]; + tensor linear_59_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_121_cast_fp16)[name = string("linear_59_cast_fp16")]; + string input_125_mode_0 = const()[name = string("input_125_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_125_cast_fp16 = gelu(mode = input_125_mode_0, x = linear_59_cast_fp16)[name = string("input_125_cast_fp16")]; + tensor linear_60_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_125_cast_fp16)[name = string("linear_60_cast_fp16")]; + tensor input_127_cast_fp16 = add(x = linear_60_cast_fp16, y = input_121_cast_fp16)[name = string("input_127_cast_fp16")]; + tensor hidden_states_21_axes_0 = const()[name = string("hidden_states_21_axes_0"), val = tensor([-1])]; + tensor hidden_states_21_cast_fp16 = layer_norm(axes = hidden_states_21_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_127_cast_fp16)[name = string("hidden_states_21_cast_fp16")]; + tensor linear_61_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_21_cast_fp16)[name = string("linear_61_cast_fp16")]; + tensor var_893 = const()[name = string("op_893"), val = tensor([1, 256, 12, 64])]; + tensor x_123_cast_fp16 = reshape(shape = var_893, x = linear_61_cast_fp16)[name = string("x_123_cast_fp16")]; + tensor linear_62_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_21_cast_fp16)[name = string("linear_62_cast_fp16")]; + tensor var_902 = const()[name = string("op_902"), val = tensor([1, 256, 12, 64])]; + tensor x_127_cast_fp16 = reshape(shape = var_902, x = linear_62_cast_fp16)[name = string("x_127_cast_fp16")]; + tensor linear_63_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_21_cast_fp16)[name = string("linear_63_cast_fp16")]; + tensor var_911 = const()[name = string("op_911"), val = tensor([1, 256, 12, 64])]; + tensor x_131_cast_fp16 = reshape(shape = var_911, x = linear_63_cast_fp16)[name = string("x_131_cast_fp16")]; + tensor transpose_102_perm_0 = const()[name = string("transpose_102_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_103_perm_0 = const()[name = string("transpose_103_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_104_perm_0 = const()[name = string("transpose_104_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_104 = transpose(perm = transpose_104_perm_0, x = x_131_cast_fp16)[name = string("transpose_114")]; + tensor transpose_103 = transpose(perm = transpose_103_perm_0, x = x_127_cast_fp16)[name = string("transpose_115")]; + tensor transpose_102 = transpose(perm = transpose_102_perm_0, x = x_123_cast_fp16)[name = string("transpose_116")]; + tensor attention_output_41_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_103, query = transpose_102, value = transpose_104)[name = string("attention_output_41_cast_fp16")]; + tensor attention_output_43_perm_0 = const()[name = string("attention_output_43_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_917 = const()[name = string("op_917"), val = tensor([1, 256, 768])]; + tensor attention_output_43_cast_fp16 = transpose(perm = attention_output_43_perm_0, x = attention_output_41_cast_fp16)[name = string("transpose_113")]; + tensor input_129_cast_fp16 = reshape(shape = var_917, x = attention_output_43_cast_fp16)[name = string("input_129_cast_fp16")]; + tensor linear_64_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_129_cast_fp16)[name = string("linear_64_cast_fp16")]; + tensor input_131_cast_fp16 = add(x = hidden_states_21_cast_fp16, y = linear_64_cast_fp16)[name = string("input_131_cast_fp16")]; + tensor input_133_axes_0 = const()[name = string("input_133_axes_0"), val = tensor([-1])]; + tensor input_133_cast_fp16 = layer_norm(axes = input_133_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_131_cast_fp16)[name = string("input_133_cast_fp16")]; + tensor linear_65_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_133_cast_fp16)[name = string("linear_65_cast_fp16")]; + string input_137_mode_0 = const()[name = string("input_137_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_137_cast_fp16 = gelu(mode = input_137_mode_0, x = linear_65_cast_fp16)[name = string("input_137_cast_fp16")]; + tensor linear_66_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_137_cast_fp16)[name = string("linear_66_cast_fp16")]; + tensor input_139_cast_fp16 = add(x = linear_66_cast_fp16, y = input_133_cast_fp16)[name = string("input_139_cast_fp16")]; + tensor hidden_states_axes_0 = const()[name = string("hidden_states_axes_0"), val = tensor([-1])]; + tensor hidden_states_cast_fp16 = layer_norm(axes = hidden_states_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_139_cast_fp16)[name = string("hidden_states_cast_fp16")]; + tensor linear_67_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_cast_fp16)[name = string("linear_67_cast_fp16")]; + tensor var_968 = const()[name = string("op_968"), val = tensor([1, 256, 12, 64])]; + tensor x_135_cast_fp16 = reshape(shape = var_968, x = linear_67_cast_fp16)[name = string("x_135_cast_fp16")]; + tensor linear_68_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_cast_fp16)[name = string("linear_68_cast_fp16")]; + tensor var_977 = const()[name = string("op_977"), val = tensor([1, 256, 12, 64])]; + tensor x_139_cast_fp16 = reshape(shape = var_977, x = linear_68_cast_fp16)[name = string("x_139_cast_fp16")]; + tensor linear_69_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_cast_fp16)[name = string("linear_69_cast_fp16")]; + tensor var_986 = const()[name = string("op_986"), val = tensor([1, 256, 12, 64])]; + tensor x_cast_fp16 = reshape(shape = var_986, x = linear_69_cast_fp16)[name = string("x_cast_fp16")]; + tensor transpose_105_perm_0 = const()[name = string("transpose_105_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_106_perm_0 = const()[name = string("transpose_106_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_107_perm_0 = const()[name = string("transpose_107_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_107 = transpose(perm = transpose_107_perm_0, x = x_cast_fp16)[name = string("transpose_110")]; + tensor transpose_106 = transpose(perm = transpose_106_perm_0, x = x_139_cast_fp16)[name = string("transpose_111")]; + tensor transpose_105 = transpose(perm = transpose_105_perm_0, x = x_135_cast_fp16)[name = string("transpose_112")]; + tensor attention_output_45_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_106, query = transpose_105, value = transpose_107)[name = string("attention_output_45_cast_fp16")]; + tensor attention_output_perm_0 = const()[name = string("attention_output_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_992 = const()[name = string("op_992"), val = tensor([1, 256, 768])]; + tensor attention_output_cast_fp16 = transpose(perm = attention_output_perm_0, x = attention_output_45_cast_fp16)[name = string("transpose_109")]; + tensor input_141_cast_fp16 = reshape(shape = var_992, x = attention_output_cast_fp16)[name = string("input_141_cast_fp16")]; + tensor linear_70_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_141_cast_fp16)[name = string("linear_70_cast_fp16")]; + tensor input_143_cast_fp16 = add(x = hidden_states_cast_fp16, y = linear_70_cast_fp16)[name = string("input_143_cast_fp16")]; + tensor input_145_axes_0 = const()[name = string("input_145_axes_0"), val = tensor([-1])]; + tensor input_145_cast_fp16 = layer_norm(axes = input_145_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_143_cast_fp16)[name = string("input_145_cast_fp16")]; + tensor linear_71_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_145_cast_fp16)[name = string("linear_71_cast_fp16")]; + string input_149_mode_0 = const()[name = string("input_149_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_149_cast_fp16 = gelu(mode = input_149_mode_0, x = linear_71_cast_fp16)[name = string("input_149_cast_fp16")]; + tensor linear_72_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_149_cast_fp16)[name = string("linear_72_cast_fp16")]; + tensor input_151_cast_fp16 = add(x = linear_72_cast_fp16, y = input_145_cast_fp16)[name = string("input_151_cast_fp16")]; + tensor sequence_output_axes_0 = const()[name = string("sequence_output_axes_0"), val = tensor([-1])]; + tensor sequence_output = layer_norm(axes = sequence_output_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_151_cast_fp16)[name = string("sequence_output_cast_fp16")]; + tensor bert_encoder_weight_to_fp16 = const()[name = string("bert_encoder_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11404800)))]; + tensor bert_encoder_bias_to_fp16 = const()[name = string("bert_encoder_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(12191296)))]; + tensor linear_73_cast_fp16 = linear(bias = bert_encoder_bias_to_fp16, weight = bert_encoder_weight_to_fp16, x = sequence_output)[name = string("linear_73_cast_fp16")]; + tensor var_1030_perm_0 = const()[name = string("op_1030_perm_0"), val = tensor([0, -1, -2])]; + tensor var_1030 = transpose(perm = var_1030_perm_0, x = linear_73_cast_fp16)[name = string("transpose_108")]; + } -> (sequence_output, var_1030); +} \ No newline at end of file diff --git a/iteration_3/compiled/bert_fp16_t256.mlmodelc/weights/weight.bin b/iteration_3/compiled/bert_fp16_t256.mlmodelc/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..ee82062c83de4d5127acc3df81c91bfb2c075860 --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t256.mlmodelc/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7569b53a68b9664e246fda171851daa1dd5f01f64aa31533dfcaf40f4034fee3 +size 12192384 diff --git a/iteration_3/compiled/bert_fp16_t64.mlmodelc/analytics/coremldata.bin b/iteration_3/compiled/bert_fp16_t64.mlmodelc/analytics/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..50c193032d9f4fb764af46d3d9ff5152ce15d5c9 --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t64.mlmodelc/analytics/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fff70f6d02af452c0cfe9133a60c0ddaea7da51bca82b19885aaf90d49533955 +size 243 diff --git a/iteration_3/compiled/bert_fp16_t64.mlmodelc/coremldata.bin b/iteration_3/compiled/bert_fp16_t64.mlmodelc/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..af095b89e557f0defe6a490990fbc774f2332193 --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t64.mlmodelc/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1ab9e12f625df7bfa9daa01c7c3a319d095408cf032b482e2f6733748de5e850 +size 437 diff --git a/iteration_3/compiled/bert_fp16_t64.mlmodelc/metadata.json b/iteration_3/compiled/bert_fp16_t64.mlmodelc/metadata.json new file mode 100644 index 0000000000000000000000000000000000000000..fe6f1c33a9fbfc24994212c3a11c902ac6926fc8 --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t64.mlmodelc/metadata.json @@ -0,0 +1,94 @@ +[ + { + "metadataOutputVersion" : "3.0", + "storagePrecision" : "Float16", + "outputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float16", + "formattedType" : "MultiArray (Float16 1 × 64 × 768)", + "shortDescription" : "", + "shape" : "[1, 64, 768]", + "name" : "sequence_output", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float16", + "formattedType" : "MultiArray (Float16 1 × 512 × 64)", + "shortDescription" : "", + "shape" : "[1, 512, 64]", + "name" : "var_1030", + "type" : "MultiArray" + } + ], + "modelParameters" : [ + + ], + "specificationVersion" : 9, + "mlProgramOperationTypeHistogram" : { + "Ios18.linear" : 74, + "Ios18.scaledDotProductAttention" : 12, + "Ios18.sub" : 1, + "Select" : 2, + "Ios18.expandDims" : 2, + "Ios18.gelu" : 12, + "Ios18.gather" : 1, + "Ios18.add" : 27, + "Tile" : 1, + "Ios18.layerNorm" : 25, + "Ios18.transpose" : 49, + "Ios18.cast" : 5, + "Ios18.reshape" : 48, + "Ios18.greaterEqual" : 1 + }, + "computePrecision" : "Mixed (Float16, Int16, Int32)", + "isUpdatable" : "0", + "stateSchema" : [ + + ], + "availability" : { + "macOS" : "15.0", + "tvOS" : "18.0", + "visionOS" : "2.0", + "watchOS" : "11.0", + "iOS" : "18.0", + "macCatalyst" : "18.0" + }, + "modelType" : { + "name" : "MLModelType_mlProgram" + }, + "userDefinedMetadata" : { + "com.github.apple.coremltools.conversion_date" : "2026-05-08", + "com.github.apple.coremltools.source" : "torch==2.11.0", + "com.github.apple.coremltools.version" : "9.0", + "com.github.apple.coremltools.source_dialect" : "TorchScript" + }, + "inputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Int32", + "formattedType" : "MultiArray (Int32 1 × 64)", + "shortDescription" : "", + "shape" : "[1, 64]", + "name" : "tokens", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Int32", + "formattedType" : "MultiArray (Int32 1 × 64)", + "shortDescription" : "", + "shape" : "[1, 64]", + "name" : "attention_mask", + "type" : "MultiArray" + } + ], + "generatedClassName" : "bert_fp16_t64", + "method" : "predict" + } +] \ No newline at end of file diff --git a/iteration_3/compiled/bert_fp16_t64.mlmodelc/model.mil b/iteration_3/compiled/bert_fp16_t64.mlmodelc/model.mil new file mode 100644 index 0000000000000000000000000000000000000000..e2ae1b7a2204ff837123c2a5018e4e82ab5340c7 --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t64.mlmodelc/model.mil @@ -0,0 +1,442 @@ +program(1.3) +[buildInfo = dict({{"coremlc-component-MIL", "3520.4.1"}, {"coremlc-version", "3520.5.1"}, {"coremltools-component-torch", "2.11.0"}, {"coremltools-source-dialect", "TorchScript"}, {"coremltools-version", "9.0"}})] +{ + func main(tensor attention_mask, tensor tokens) { + int32 inputs_embeds_batch_dims_0 = const()[name = string("inputs_embeds_batch_dims_0"), val = int32(0)]; + bool inputs_embeds_validate_indices_0 = const()[name = string("inputs_embeds_validate_indices_0"), val = bool(false)]; + tensor bert_embeddings_word_embeddings_weight_to_fp16 = const()[name = string("bert_embeddings_word_embeddings_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(64)))]; + string tokens_to_int16_dtype_0 = const()[name = string("tokens_to_int16_dtype_0"), val = string("int16")]; + string cast_53_dtype_0 = const()[name = string("cast_53_dtype_0"), val = string("int32")]; + int32 greater_equal_0_y_0 = const()[name = string("greater_equal_0_y_0"), val = int32(0)]; + tensor tokens_to_int16 = cast(dtype = tokens_to_int16_dtype_0, x = tokens)[name = string("cast_58")]; + tensor cast_53 = cast(dtype = cast_53_dtype_0, x = tokens_to_int16)[name = string("cast_57")]; + tensor greater_equal_0 = greater_equal(x = cast_53, y = greater_equal_0_y_0)[name = string("greater_equal_0")]; + int32 slice_by_index_0 = const()[name = string("slice_by_index_0"), val = int32(178)]; + tensor add_0 = add(x = cast_53, y = slice_by_index_0)[name = string("add_0")]; + tensor select_0 = select(a = cast_53, b = add_0, cond = greater_equal_0)[name = string("select_0")]; + int32 inputs_embeds_cast_fp16_cast_uint16_axis_0 = const()[name = string("inputs_embeds_cast_fp16_cast_uint16_axis_0"), val = int32(0)]; + string select_0_to_int16_dtype_0 = const()[name = string("select_0_to_int16_dtype_0"), val = string("int16")]; + tensor select_0_to_int16 = cast(dtype = select_0_to_int16_dtype_0, x = select_0)[name = string("cast_56")]; + tensor inputs_embeds_cast_fp16_cast_uint16_cast_uint16 = gather(axis = inputs_embeds_cast_fp16_cast_uint16_axis_0, batch_dims = inputs_embeds_batch_dims_0, indices = select_0_to_int16, validate_indices = inputs_embeds_validate_indices_0, x = bert_embeddings_word_embeddings_weight_to_fp16)[name = string("inputs_embeds_cast_fp16_cast_uint16_cast_uint16")]; + tensor token_type_embeddings_1_to_fp16 = const()[name = string("token_type_embeddings_1_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(45696)))]; + tensor embeddings_1_cast_fp16 = add(x = inputs_embeds_cast_fp16_cast_uint16_cast_uint16, y = token_type_embeddings_1_to_fp16)[name = string("embeddings_1_cast_fp16")]; + tensor position_embeddings_1_to_fp16 = const()[name = string("position_embeddings_1_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(62144)))]; + tensor input_5_cast_fp16 = add(x = embeddings_1_cast_fp16, y = position_embeddings_1_to_fp16)[name = string("input_5_cast_fp16")]; + tensor input_7_axes_0 = const()[name = string("input_7_axes_0"), val = tensor([-1])]; + tensor bert_embeddings_LayerNorm_weight_to_fp16 = const()[name = string("bert_embeddings_LayerNorm_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(78592)))]; + tensor bert_embeddings_LayerNorm_bias_to_fp16 = const()[name = string("bert_embeddings_LayerNorm_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(78912)))]; + fp16 var_34_to_fp16 = const()[name = string("op_34_to_fp16"), val = fp16(0x1p-24)]; + tensor input_7_cast_fp16 = layer_norm(axes = input_7_axes_0, beta = bert_embeddings_LayerNorm_bias_to_fp16, epsilon = var_34_to_fp16, gamma = bert_embeddings_LayerNorm_weight_to_fp16, x = input_5_cast_fp16)[name = string("input_7_cast_fp16")]; + tensor var_79_axes_0 = const()[name = string("op_79_axes_0"), val = tensor([1])]; + tensor var_79 = expand_dims(axes = var_79_axes_0, x = attention_mask)[name = string("op_79")]; + tensor var_81_axes_0 = const()[name = string("op_81_axes_0"), val = tensor([2])]; + tensor var_81 = expand_dims(axes = var_81_axes_0, x = var_79)[name = string("op_81")]; + tensor var_90_reps_0 = const()[name = string("op_90_reps_0"), val = tensor([1, 1, 64, 1])]; + tensor var_90 = tile(reps = var_90_reps_0, x = var_81)[name = string("op_90")]; + fp16 var_96_to_fp16 = const()[name = string("op_96_to_fp16"), val = fp16(0x1p+0)]; + string var_95_to_fp16_dtype_0 = const()[name = string("op_95_to_fp16_dtype_0"), val = string("fp16")]; + tensor var_90_to_fp16 = cast(dtype = var_95_to_fp16_dtype_0, x = var_90)[name = string("cast_55")]; + tensor inverted_mask_cast_fp16 = sub(x = var_96_to_fp16, y = var_90_to_fp16)[name = string("inverted_mask_cast_fp16")]; + string var_103_dtype_0 = const()[name = string("op_103_dtype_0"), val = string("bool")]; + fp16 var_104_to_fp16 = const()[name = string("op_104_to_fp16"), val = fp16(-inf)]; + tensor inverted_mask_cast_fp16_to_bool = cast(dtype = var_103_dtype_0, x = inverted_mask_cast_fp16)[name = string("cast_54")]; + tensor attention_mask_cast_fp16 = select(a = var_104_to_fp16, b = inverted_mask_cast_fp16, cond = inverted_mask_cast_fp16_to_bool)[name = string("attention_mask_cast_fp16")]; + tensor bert_encoder_embedding_hidden_mapping_in_weight_to_fp16 = const()[name = string("bert_encoder_embedding_hidden_mapping_in_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(79232)))]; + tensor bert_encoder_embedding_hidden_mapping_in_bias_to_fp16 = const()[name = string("bert_encoder_embedding_hidden_mapping_in_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(275904)))]; + tensor linear_0_cast_fp16 = linear(bias = bert_encoder_embedding_hidden_mapping_in_bias_to_fp16, weight = bert_encoder_embedding_hidden_mapping_in_weight_to_fp16, x = input_7_cast_fp16)[name = string("linear_0_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(277504)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(1457216)))]; + tensor linear_1_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = linear_0_cast_fp16)[name = string("linear_1_cast_fp16")]; + tensor var_143 = const()[name = string("op_143"), val = tensor([1, 64, 12, 64])]; + tensor x_3_cast_fp16 = reshape(shape = var_143, x = linear_1_cast_fp16)[name = string("x_3_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(1458816)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2638528)))]; + tensor linear_2_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = linear_0_cast_fp16)[name = string("linear_2_cast_fp16")]; + tensor var_152 = const()[name = string("op_152"), val = tensor([1, 64, 12, 64])]; + tensor x_7_cast_fp16 = reshape(shape = var_152, x = linear_2_cast_fp16)[name = string("x_7_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2640128)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(3819840)))]; + tensor linear_3_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = linear_0_cast_fp16)[name = string("linear_3_cast_fp16")]; + tensor var_161 = const()[name = string("op_161"), val = tensor([1, 64, 12, 64])]; + tensor x_11_cast_fp16 = reshape(shape = var_161, x = linear_3_cast_fp16)[name = string("x_11_cast_fp16")]; + tensor transpose_72_perm_0 = const()[name = string("transpose_72_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_73_perm_0 = const()[name = string("transpose_73_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_74_perm_0 = const()[name = string("transpose_74_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_74 = transpose(perm = transpose_74_perm_0, x = x_11_cast_fp16)[name = string("transpose_154")]; + tensor transpose_73 = transpose(perm = transpose_73_perm_0, x = x_7_cast_fp16)[name = string("transpose_155")]; + tensor transpose_72 = transpose(perm = transpose_72_perm_0, x = x_3_cast_fp16)[name = string("transpose_156")]; + tensor attention_output_1_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_73, query = transpose_72, value = transpose_74)[name = string("attention_output_1_cast_fp16")]; + tensor attention_output_3_perm_0 = const()[name = string("attention_output_3_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_167 = const()[name = string("op_167"), val = tensor([1, 64, 768])]; + tensor attention_output_3_cast_fp16 = transpose(perm = attention_output_3_perm_0, x = attention_output_1_cast_fp16)[name = string("transpose_153")]; + tensor input_9_cast_fp16 = reshape(shape = var_167, x = attention_output_3_cast_fp16)[name = string("input_9_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(3821440)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5001152)))]; + tensor linear_4_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_9_cast_fp16)[name = string("linear_4_cast_fp16")]; + tensor input_11_cast_fp16 = add(x = linear_0_cast_fp16, y = linear_4_cast_fp16)[name = string("input_11_cast_fp16")]; + tensor input_13_axes_0 = const()[name = string("input_13_axes_0"), val = tensor([-1])]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5002752)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5004352)))]; + fp16 var_118_to_fp16 = const()[name = string("op_118_to_fp16"), val = fp16(0x1p-24)]; + tensor input_13_cast_fp16 = layer_norm(axes = input_13_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_11_cast_fp16)[name = string("input_13_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5005952)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(8151744)))]; + tensor linear_5_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_13_cast_fp16)[name = string("linear_5_cast_fp16")]; + string input_17_mode_0 = const()[name = string("input_17_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_17_cast_fp16 = gelu(mode = input_17_mode_0, x = linear_5_cast_fp16)[name = string("input_17_cast_fp16")]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(8155904)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11301696)))]; + tensor linear_6_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_17_cast_fp16)[name = string("linear_6_cast_fp16")]; + tensor input_19_cast_fp16 = add(x = linear_6_cast_fp16, y = input_13_cast_fp16)[name = string("input_19_cast_fp16")]; + tensor hidden_states_3_axes_0 = const()[name = string("hidden_states_3_axes_0"), val = tensor([-1])]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11303296)))]; + tensor bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16 = const()[name = string("bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11304896)))]; + tensor hidden_states_3_cast_fp16 = layer_norm(axes = hidden_states_3_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_19_cast_fp16)[name = string("hidden_states_3_cast_fp16")]; + tensor linear_7_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_3_cast_fp16)[name = string("linear_7_cast_fp16")]; + tensor var_218 = const()[name = string("op_218"), val = tensor([1, 64, 12, 64])]; + tensor x_15_cast_fp16 = reshape(shape = var_218, x = linear_7_cast_fp16)[name = string("x_15_cast_fp16")]; + tensor linear_8_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_3_cast_fp16)[name = string("linear_8_cast_fp16")]; + tensor var_227 = const()[name = string("op_227"), val = tensor([1, 64, 12, 64])]; + tensor x_19_cast_fp16 = reshape(shape = var_227, x = linear_8_cast_fp16)[name = string("x_19_cast_fp16")]; + tensor linear_9_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_3_cast_fp16)[name = string("linear_9_cast_fp16")]; + tensor var_236 = const()[name = string("op_236"), val = tensor([1, 64, 12, 64])]; + tensor x_23_cast_fp16 = reshape(shape = var_236, x = linear_9_cast_fp16)[name = string("x_23_cast_fp16")]; + tensor transpose_75_perm_0 = const()[name = string("transpose_75_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_76_perm_0 = const()[name = string("transpose_76_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_77_perm_0 = const()[name = string("transpose_77_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_77 = transpose(perm = transpose_77_perm_0, x = x_23_cast_fp16)[name = string("transpose_150")]; + tensor transpose_76 = transpose(perm = transpose_76_perm_0, x = x_19_cast_fp16)[name = string("transpose_151")]; + tensor transpose_75 = transpose(perm = transpose_75_perm_0, x = x_15_cast_fp16)[name = string("transpose_152")]; + tensor attention_output_5_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_76, query = transpose_75, value = transpose_77)[name = string("attention_output_5_cast_fp16")]; + tensor attention_output_7_perm_0 = const()[name = string("attention_output_7_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_242 = const()[name = string("op_242"), val = tensor([1, 64, 768])]; + tensor attention_output_7_cast_fp16 = transpose(perm = attention_output_7_perm_0, x = attention_output_5_cast_fp16)[name = string("transpose_149")]; + tensor input_21_cast_fp16 = reshape(shape = var_242, x = attention_output_7_cast_fp16)[name = string("input_21_cast_fp16")]; + tensor linear_10_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_21_cast_fp16)[name = string("linear_10_cast_fp16")]; + tensor input_23_cast_fp16 = add(x = hidden_states_3_cast_fp16, y = linear_10_cast_fp16)[name = string("input_23_cast_fp16")]; + tensor input_25_axes_0 = const()[name = string("input_25_axes_0"), val = tensor([-1])]; + tensor input_25_cast_fp16 = layer_norm(axes = input_25_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_23_cast_fp16)[name = string("input_25_cast_fp16")]; + tensor linear_11_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_25_cast_fp16)[name = string("linear_11_cast_fp16")]; + string input_29_mode_0 = const()[name = string("input_29_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_29_cast_fp16 = gelu(mode = input_29_mode_0, x = linear_11_cast_fp16)[name = string("input_29_cast_fp16")]; + tensor linear_12_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_29_cast_fp16)[name = string("linear_12_cast_fp16")]; + tensor input_31_cast_fp16 = add(x = linear_12_cast_fp16, y = input_25_cast_fp16)[name = string("input_31_cast_fp16")]; + tensor hidden_states_5_axes_0 = const()[name = string("hidden_states_5_axes_0"), val = tensor([-1])]; + tensor hidden_states_5_cast_fp16 = layer_norm(axes = hidden_states_5_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_31_cast_fp16)[name = string("hidden_states_5_cast_fp16")]; + tensor linear_13_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_5_cast_fp16)[name = string("linear_13_cast_fp16")]; + tensor var_293 = const()[name = string("op_293"), val = tensor([1, 64, 12, 64])]; + tensor x_27_cast_fp16 = reshape(shape = var_293, x = linear_13_cast_fp16)[name = string("x_27_cast_fp16")]; + tensor linear_14_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_5_cast_fp16)[name = string("linear_14_cast_fp16")]; + tensor var_302 = const()[name = string("op_302"), val = tensor([1, 64, 12, 64])]; + tensor x_31_cast_fp16 = reshape(shape = var_302, x = linear_14_cast_fp16)[name = string("x_31_cast_fp16")]; + tensor linear_15_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_5_cast_fp16)[name = string("linear_15_cast_fp16")]; + tensor var_311 = const()[name = string("op_311"), val = tensor([1, 64, 12, 64])]; + tensor x_35_cast_fp16 = reshape(shape = var_311, x = linear_15_cast_fp16)[name = string("x_35_cast_fp16")]; + tensor transpose_78_perm_0 = const()[name = string("transpose_78_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_79_perm_0 = const()[name = string("transpose_79_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_80_perm_0 = const()[name = string("transpose_80_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_80 = transpose(perm = transpose_80_perm_0, x = x_35_cast_fp16)[name = string("transpose_146")]; + tensor transpose_79 = transpose(perm = transpose_79_perm_0, x = x_31_cast_fp16)[name = string("transpose_147")]; + tensor transpose_78 = transpose(perm = transpose_78_perm_0, x = x_27_cast_fp16)[name = string("transpose_148")]; + tensor attention_output_9_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_79, query = transpose_78, value = transpose_80)[name = string("attention_output_9_cast_fp16")]; + tensor attention_output_11_perm_0 = const()[name = string("attention_output_11_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_317 = const()[name = string("op_317"), val = tensor([1, 64, 768])]; + tensor attention_output_11_cast_fp16 = transpose(perm = attention_output_11_perm_0, x = attention_output_9_cast_fp16)[name = string("transpose_145")]; + tensor input_33_cast_fp16 = reshape(shape = var_317, x = attention_output_11_cast_fp16)[name = string("input_33_cast_fp16")]; + tensor linear_16_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_33_cast_fp16)[name = string("linear_16_cast_fp16")]; + tensor input_35_cast_fp16 = add(x = hidden_states_5_cast_fp16, y = linear_16_cast_fp16)[name = string("input_35_cast_fp16")]; + tensor input_37_axes_0 = const()[name = string("input_37_axes_0"), val = tensor([-1])]; + tensor input_37_cast_fp16 = layer_norm(axes = input_37_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_35_cast_fp16)[name = string("input_37_cast_fp16")]; + tensor linear_17_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_37_cast_fp16)[name = string("linear_17_cast_fp16")]; + string input_41_mode_0 = const()[name = string("input_41_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_41_cast_fp16 = gelu(mode = input_41_mode_0, x = linear_17_cast_fp16)[name = string("input_41_cast_fp16")]; + tensor linear_18_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_41_cast_fp16)[name = string("linear_18_cast_fp16")]; + tensor input_43_cast_fp16 = add(x = linear_18_cast_fp16, y = input_37_cast_fp16)[name = string("input_43_cast_fp16")]; + tensor hidden_states_7_axes_0 = const()[name = string("hidden_states_7_axes_0"), val = tensor([-1])]; + tensor hidden_states_7_cast_fp16 = layer_norm(axes = hidden_states_7_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_43_cast_fp16)[name = string("hidden_states_7_cast_fp16")]; + tensor linear_19_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_7_cast_fp16)[name = string("linear_19_cast_fp16")]; + tensor var_368 = const()[name = string("op_368"), val = tensor([1, 64, 12, 64])]; + tensor x_39_cast_fp16 = reshape(shape = var_368, x = linear_19_cast_fp16)[name = string("x_39_cast_fp16")]; + tensor linear_20_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_7_cast_fp16)[name = string("linear_20_cast_fp16")]; + tensor var_377 = const()[name = string("op_377"), val = tensor([1, 64, 12, 64])]; + tensor x_43_cast_fp16 = reshape(shape = var_377, x = linear_20_cast_fp16)[name = string("x_43_cast_fp16")]; + tensor linear_21_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_7_cast_fp16)[name = string("linear_21_cast_fp16")]; + tensor var_386 = const()[name = string("op_386"), val = tensor([1, 64, 12, 64])]; + tensor x_47_cast_fp16 = reshape(shape = var_386, x = linear_21_cast_fp16)[name = string("x_47_cast_fp16")]; + tensor transpose_81_perm_0 = const()[name = string("transpose_81_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_82_perm_0 = const()[name = string("transpose_82_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_83_perm_0 = const()[name = string("transpose_83_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_83 = transpose(perm = transpose_83_perm_0, x = x_47_cast_fp16)[name = string("transpose_142")]; + tensor transpose_82 = transpose(perm = transpose_82_perm_0, x = x_43_cast_fp16)[name = string("transpose_143")]; + tensor transpose_81 = transpose(perm = transpose_81_perm_0, x = x_39_cast_fp16)[name = string("transpose_144")]; + tensor attention_output_13_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_82, query = transpose_81, value = transpose_83)[name = string("attention_output_13_cast_fp16")]; + tensor attention_output_15_perm_0 = const()[name = string("attention_output_15_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_392 = const()[name = string("op_392"), val = tensor([1, 64, 768])]; + tensor attention_output_15_cast_fp16 = transpose(perm = attention_output_15_perm_0, x = attention_output_13_cast_fp16)[name = string("transpose_141")]; + tensor input_45_cast_fp16 = reshape(shape = var_392, x = attention_output_15_cast_fp16)[name = string("input_45_cast_fp16")]; + tensor linear_22_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_45_cast_fp16)[name = string("linear_22_cast_fp16")]; + tensor input_47_cast_fp16 = add(x = hidden_states_7_cast_fp16, y = linear_22_cast_fp16)[name = string("input_47_cast_fp16")]; + tensor input_49_axes_0 = const()[name = string("input_49_axes_0"), val = tensor([-1])]; + tensor input_49_cast_fp16 = layer_norm(axes = input_49_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_47_cast_fp16)[name = string("input_49_cast_fp16")]; + tensor linear_23_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_49_cast_fp16)[name = string("linear_23_cast_fp16")]; + string input_53_mode_0 = const()[name = string("input_53_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_53_cast_fp16 = gelu(mode = input_53_mode_0, x = linear_23_cast_fp16)[name = string("input_53_cast_fp16")]; + tensor linear_24_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_53_cast_fp16)[name = string("linear_24_cast_fp16")]; + tensor input_55_cast_fp16 = add(x = linear_24_cast_fp16, y = input_49_cast_fp16)[name = string("input_55_cast_fp16")]; + tensor hidden_states_9_axes_0 = const()[name = string("hidden_states_9_axes_0"), val = tensor([-1])]; + tensor hidden_states_9_cast_fp16 = layer_norm(axes = hidden_states_9_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_55_cast_fp16)[name = string("hidden_states_9_cast_fp16")]; + tensor linear_25_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_9_cast_fp16)[name = string("linear_25_cast_fp16")]; + tensor var_443 = const()[name = string("op_443"), val = tensor([1, 64, 12, 64])]; + tensor x_51_cast_fp16 = reshape(shape = var_443, x = linear_25_cast_fp16)[name = string("x_51_cast_fp16")]; + tensor linear_26_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_9_cast_fp16)[name = string("linear_26_cast_fp16")]; + tensor var_452 = const()[name = string("op_452"), val = tensor([1, 64, 12, 64])]; + tensor x_55_cast_fp16 = reshape(shape = var_452, x = linear_26_cast_fp16)[name = string("x_55_cast_fp16")]; + tensor linear_27_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_9_cast_fp16)[name = string("linear_27_cast_fp16")]; + tensor var_461 = const()[name = string("op_461"), val = tensor([1, 64, 12, 64])]; + tensor x_59_cast_fp16 = reshape(shape = var_461, x = linear_27_cast_fp16)[name = string("x_59_cast_fp16")]; + tensor transpose_84_perm_0 = const()[name = string("transpose_84_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_85_perm_0 = const()[name = string("transpose_85_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_86_perm_0 = const()[name = string("transpose_86_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_86 = transpose(perm = transpose_86_perm_0, x = x_59_cast_fp16)[name = string("transpose_138")]; + tensor transpose_85 = transpose(perm = transpose_85_perm_0, x = x_55_cast_fp16)[name = string("transpose_139")]; + tensor transpose_84 = transpose(perm = transpose_84_perm_0, x = x_51_cast_fp16)[name = string("transpose_140")]; + tensor attention_output_17_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_85, query = transpose_84, value = transpose_86)[name = string("attention_output_17_cast_fp16")]; + tensor attention_output_19_perm_0 = const()[name = string("attention_output_19_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_467 = const()[name = string("op_467"), val = tensor([1, 64, 768])]; + tensor attention_output_19_cast_fp16 = transpose(perm = attention_output_19_perm_0, x = attention_output_17_cast_fp16)[name = string("transpose_137")]; + tensor input_57_cast_fp16 = reshape(shape = var_467, x = attention_output_19_cast_fp16)[name = string("input_57_cast_fp16")]; + tensor linear_28_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_57_cast_fp16)[name = string("linear_28_cast_fp16")]; + tensor input_59_cast_fp16 = add(x = hidden_states_9_cast_fp16, y = linear_28_cast_fp16)[name = string("input_59_cast_fp16")]; + tensor input_61_axes_0 = const()[name = string("input_61_axes_0"), val = tensor([-1])]; + tensor input_61_cast_fp16 = layer_norm(axes = input_61_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_59_cast_fp16)[name = string("input_61_cast_fp16")]; + tensor linear_29_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_61_cast_fp16)[name = string("linear_29_cast_fp16")]; + string input_65_mode_0 = const()[name = string("input_65_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_65_cast_fp16 = gelu(mode = input_65_mode_0, x = linear_29_cast_fp16)[name = string("input_65_cast_fp16")]; + tensor linear_30_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_65_cast_fp16)[name = string("linear_30_cast_fp16")]; + tensor input_67_cast_fp16 = add(x = linear_30_cast_fp16, y = input_61_cast_fp16)[name = string("input_67_cast_fp16")]; + tensor hidden_states_11_axes_0 = const()[name = string("hidden_states_11_axes_0"), val = tensor([-1])]; + tensor hidden_states_11_cast_fp16 = layer_norm(axes = hidden_states_11_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_67_cast_fp16)[name = string("hidden_states_11_cast_fp16")]; + tensor linear_31_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_11_cast_fp16)[name = string("linear_31_cast_fp16")]; + tensor var_518 = const()[name = string("op_518"), val = tensor([1, 64, 12, 64])]; + tensor x_63_cast_fp16 = reshape(shape = var_518, x = linear_31_cast_fp16)[name = string("x_63_cast_fp16")]; + tensor linear_32_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_11_cast_fp16)[name = string("linear_32_cast_fp16")]; + tensor var_527 = const()[name = string("op_527"), val = tensor([1, 64, 12, 64])]; + tensor x_67_cast_fp16 = reshape(shape = var_527, x = linear_32_cast_fp16)[name = string("x_67_cast_fp16")]; + tensor linear_33_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_11_cast_fp16)[name = string("linear_33_cast_fp16")]; + tensor var_536 = const()[name = string("op_536"), val = tensor([1, 64, 12, 64])]; + tensor x_71_cast_fp16 = reshape(shape = var_536, x = linear_33_cast_fp16)[name = string("x_71_cast_fp16")]; + tensor transpose_87_perm_0 = const()[name = string("transpose_87_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_88_perm_0 = const()[name = string("transpose_88_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_89_perm_0 = const()[name = string("transpose_89_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_89 = transpose(perm = transpose_89_perm_0, x = x_71_cast_fp16)[name = string("transpose_134")]; + tensor transpose_88 = transpose(perm = transpose_88_perm_0, x = x_67_cast_fp16)[name = string("transpose_135")]; + tensor transpose_87 = transpose(perm = transpose_87_perm_0, x = x_63_cast_fp16)[name = string("transpose_136")]; + tensor attention_output_21_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_88, query = transpose_87, value = transpose_89)[name = string("attention_output_21_cast_fp16")]; + tensor attention_output_23_perm_0 = const()[name = string("attention_output_23_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_542 = const()[name = string("op_542"), val = tensor([1, 64, 768])]; + tensor attention_output_23_cast_fp16 = transpose(perm = attention_output_23_perm_0, x = attention_output_21_cast_fp16)[name = string("transpose_133")]; + tensor input_69_cast_fp16 = reshape(shape = var_542, x = attention_output_23_cast_fp16)[name = string("input_69_cast_fp16")]; + tensor linear_34_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_69_cast_fp16)[name = string("linear_34_cast_fp16")]; + tensor input_71_cast_fp16 = add(x = hidden_states_11_cast_fp16, y = linear_34_cast_fp16)[name = string("input_71_cast_fp16")]; + tensor input_73_axes_0 = const()[name = string("input_73_axes_0"), val = tensor([-1])]; + tensor input_73_cast_fp16 = layer_norm(axes = input_73_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_71_cast_fp16)[name = string("input_73_cast_fp16")]; + tensor linear_35_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_73_cast_fp16)[name = string("linear_35_cast_fp16")]; + string input_77_mode_0 = const()[name = string("input_77_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_77_cast_fp16 = gelu(mode = input_77_mode_0, x = linear_35_cast_fp16)[name = string("input_77_cast_fp16")]; + tensor linear_36_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_77_cast_fp16)[name = string("linear_36_cast_fp16")]; + tensor input_79_cast_fp16 = add(x = linear_36_cast_fp16, y = input_73_cast_fp16)[name = string("input_79_cast_fp16")]; + tensor hidden_states_13_axes_0 = const()[name = string("hidden_states_13_axes_0"), val = tensor([-1])]; + tensor hidden_states_13_cast_fp16 = layer_norm(axes = hidden_states_13_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_79_cast_fp16)[name = string("hidden_states_13_cast_fp16")]; + tensor linear_37_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_13_cast_fp16)[name = string("linear_37_cast_fp16")]; + tensor var_593 = const()[name = string("op_593"), val = tensor([1, 64, 12, 64])]; + tensor x_75_cast_fp16 = reshape(shape = var_593, x = linear_37_cast_fp16)[name = string("x_75_cast_fp16")]; + tensor linear_38_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_13_cast_fp16)[name = string("linear_38_cast_fp16")]; + tensor var_602 = const()[name = string("op_602"), val = tensor([1, 64, 12, 64])]; + tensor x_79_cast_fp16 = reshape(shape = var_602, x = linear_38_cast_fp16)[name = string("x_79_cast_fp16")]; + tensor linear_39_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_13_cast_fp16)[name = string("linear_39_cast_fp16")]; + tensor var_611 = const()[name = string("op_611"), val = tensor([1, 64, 12, 64])]; + tensor x_83_cast_fp16 = reshape(shape = var_611, x = linear_39_cast_fp16)[name = string("x_83_cast_fp16")]; + tensor transpose_90_perm_0 = const()[name = string("transpose_90_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_91_perm_0 = const()[name = string("transpose_91_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_92_perm_0 = const()[name = string("transpose_92_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_92 = transpose(perm = transpose_92_perm_0, x = x_83_cast_fp16)[name = string("transpose_130")]; + tensor transpose_91 = transpose(perm = transpose_91_perm_0, x = x_79_cast_fp16)[name = string("transpose_131")]; + tensor transpose_90 = transpose(perm = transpose_90_perm_0, x = x_75_cast_fp16)[name = string("transpose_132")]; + tensor attention_output_25_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_91, query = transpose_90, value = transpose_92)[name = string("attention_output_25_cast_fp16")]; + tensor attention_output_27_perm_0 = const()[name = string("attention_output_27_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_617 = const()[name = string("op_617"), val = tensor([1, 64, 768])]; + tensor attention_output_27_cast_fp16 = transpose(perm = attention_output_27_perm_0, x = attention_output_25_cast_fp16)[name = string("transpose_129")]; + tensor input_81_cast_fp16 = reshape(shape = var_617, x = attention_output_27_cast_fp16)[name = string("input_81_cast_fp16")]; + tensor linear_40_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_81_cast_fp16)[name = string("linear_40_cast_fp16")]; + tensor input_83_cast_fp16 = add(x = hidden_states_13_cast_fp16, y = linear_40_cast_fp16)[name = string("input_83_cast_fp16")]; + tensor input_85_axes_0 = const()[name = string("input_85_axes_0"), val = tensor([-1])]; + tensor input_85_cast_fp16 = layer_norm(axes = input_85_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_83_cast_fp16)[name = string("input_85_cast_fp16")]; + tensor linear_41_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_85_cast_fp16)[name = string("linear_41_cast_fp16")]; + string input_89_mode_0 = const()[name = string("input_89_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_89_cast_fp16 = gelu(mode = input_89_mode_0, x = linear_41_cast_fp16)[name = string("input_89_cast_fp16")]; + tensor linear_42_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_89_cast_fp16)[name = string("linear_42_cast_fp16")]; + tensor input_91_cast_fp16 = add(x = linear_42_cast_fp16, y = input_85_cast_fp16)[name = string("input_91_cast_fp16")]; + tensor hidden_states_15_axes_0 = const()[name = string("hidden_states_15_axes_0"), val = tensor([-1])]; + tensor hidden_states_15_cast_fp16 = layer_norm(axes = hidden_states_15_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_91_cast_fp16)[name = string("hidden_states_15_cast_fp16")]; + tensor linear_43_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_15_cast_fp16)[name = string("linear_43_cast_fp16")]; + tensor var_668 = const()[name = string("op_668"), val = tensor([1, 64, 12, 64])]; + tensor x_87_cast_fp16 = reshape(shape = var_668, x = linear_43_cast_fp16)[name = string("x_87_cast_fp16")]; + tensor linear_44_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_15_cast_fp16)[name = string("linear_44_cast_fp16")]; + tensor var_677 = const()[name = string("op_677"), val = tensor([1, 64, 12, 64])]; + tensor x_91_cast_fp16 = reshape(shape = var_677, x = linear_44_cast_fp16)[name = string("x_91_cast_fp16")]; + tensor linear_45_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_15_cast_fp16)[name = string("linear_45_cast_fp16")]; + tensor var_686 = const()[name = string("op_686"), val = tensor([1, 64, 12, 64])]; + tensor x_95_cast_fp16 = reshape(shape = var_686, x = linear_45_cast_fp16)[name = string("x_95_cast_fp16")]; + tensor transpose_93_perm_0 = const()[name = string("transpose_93_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_94_perm_0 = const()[name = string("transpose_94_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_95_perm_0 = const()[name = string("transpose_95_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_95 = transpose(perm = transpose_95_perm_0, x = x_95_cast_fp16)[name = string("transpose_126")]; + tensor transpose_94 = transpose(perm = transpose_94_perm_0, x = x_91_cast_fp16)[name = string("transpose_127")]; + tensor transpose_93 = transpose(perm = transpose_93_perm_0, x = x_87_cast_fp16)[name = string("transpose_128")]; + tensor attention_output_29_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_94, query = transpose_93, value = transpose_95)[name = string("attention_output_29_cast_fp16")]; + tensor attention_output_31_perm_0 = const()[name = string("attention_output_31_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_692 = const()[name = string("op_692"), val = tensor([1, 64, 768])]; + tensor attention_output_31_cast_fp16 = transpose(perm = attention_output_31_perm_0, x = attention_output_29_cast_fp16)[name = string("transpose_125")]; + tensor input_93_cast_fp16 = reshape(shape = var_692, x = attention_output_31_cast_fp16)[name = string("input_93_cast_fp16")]; + tensor linear_46_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_93_cast_fp16)[name = string("linear_46_cast_fp16")]; + tensor input_95_cast_fp16 = add(x = hidden_states_15_cast_fp16, y = linear_46_cast_fp16)[name = string("input_95_cast_fp16")]; + tensor input_97_axes_0 = const()[name = string("input_97_axes_0"), val = tensor([-1])]; + tensor input_97_cast_fp16 = layer_norm(axes = input_97_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_95_cast_fp16)[name = string("input_97_cast_fp16")]; + tensor linear_47_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_97_cast_fp16)[name = string("linear_47_cast_fp16")]; + string input_101_mode_0 = const()[name = string("input_101_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_101_cast_fp16 = gelu(mode = input_101_mode_0, x = linear_47_cast_fp16)[name = string("input_101_cast_fp16")]; + tensor linear_48_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_101_cast_fp16)[name = string("linear_48_cast_fp16")]; + tensor input_103_cast_fp16 = add(x = linear_48_cast_fp16, y = input_97_cast_fp16)[name = string("input_103_cast_fp16")]; + tensor hidden_states_17_axes_0 = const()[name = string("hidden_states_17_axes_0"), val = tensor([-1])]; + tensor hidden_states_17_cast_fp16 = layer_norm(axes = hidden_states_17_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_103_cast_fp16)[name = string("hidden_states_17_cast_fp16")]; + tensor linear_49_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_17_cast_fp16)[name = string("linear_49_cast_fp16")]; + tensor var_743 = const()[name = string("op_743"), val = tensor([1, 64, 12, 64])]; + tensor x_99_cast_fp16 = reshape(shape = var_743, x = linear_49_cast_fp16)[name = string("x_99_cast_fp16")]; + tensor linear_50_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_17_cast_fp16)[name = string("linear_50_cast_fp16")]; + tensor var_752 = const()[name = string("op_752"), val = tensor([1, 64, 12, 64])]; + tensor x_103_cast_fp16 = reshape(shape = var_752, x = linear_50_cast_fp16)[name = string("x_103_cast_fp16")]; + tensor linear_51_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_17_cast_fp16)[name = string("linear_51_cast_fp16")]; + tensor var_761 = const()[name = string("op_761"), val = tensor([1, 64, 12, 64])]; + tensor x_107_cast_fp16 = reshape(shape = var_761, x = linear_51_cast_fp16)[name = string("x_107_cast_fp16")]; + tensor transpose_96_perm_0 = const()[name = string("transpose_96_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_97_perm_0 = const()[name = string("transpose_97_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_98_perm_0 = const()[name = string("transpose_98_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_98 = transpose(perm = transpose_98_perm_0, x = x_107_cast_fp16)[name = string("transpose_122")]; + tensor transpose_97 = transpose(perm = transpose_97_perm_0, x = x_103_cast_fp16)[name = string("transpose_123")]; + tensor transpose_96 = transpose(perm = transpose_96_perm_0, x = x_99_cast_fp16)[name = string("transpose_124")]; + tensor attention_output_33_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_97, query = transpose_96, value = transpose_98)[name = string("attention_output_33_cast_fp16")]; + tensor attention_output_35_perm_0 = const()[name = string("attention_output_35_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_767 = const()[name = string("op_767"), val = tensor([1, 64, 768])]; + tensor attention_output_35_cast_fp16 = transpose(perm = attention_output_35_perm_0, x = attention_output_33_cast_fp16)[name = string("transpose_121")]; + tensor input_105_cast_fp16 = reshape(shape = var_767, x = attention_output_35_cast_fp16)[name = string("input_105_cast_fp16")]; + tensor linear_52_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_105_cast_fp16)[name = string("linear_52_cast_fp16")]; + tensor input_107_cast_fp16 = add(x = hidden_states_17_cast_fp16, y = linear_52_cast_fp16)[name = string("input_107_cast_fp16")]; + tensor input_109_axes_0 = const()[name = string("input_109_axes_0"), val = tensor([-1])]; + tensor input_109_cast_fp16 = layer_norm(axes = input_109_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_107_cast_fp16)[name = string("input_109_cast_fp16")]; + tensor linear_53_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_109_cast_fp16)[name = string("linear_53_cast_fp16")]; + string input_113_mode_0 = const()[name = string("input_113_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_113_cast_fp16 = gelu(mode = input_113_mode_0, x = linear_53_cast_fp16)[name = string("input_113_cast_fp16")]; + tensor linear_54_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_113_cast_fp16)[name = string("linear_54_cast_fp16")]; + tensor input_115_cast_fp16 = add(x = linear_54_cast_fp16, y = input_109_cast_fp16)[name = string("input_115_cast_fp16")]; + tensor hidden_states_19_axes_0 = const()[name = string("hidden_states_19_axes_0"), val = tensor([-1])]; + tensor hidden_states_19_cast_fp16 = layer_norm(axes = hidden_states_19_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_115_cast_fp16)[name = string("hidden_states_19_cast_fp16")]; + tensor linear_55_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_19_cast_fp16)[name = string("linear_55_cast_fp16")]; + tensor var_818 = const()[name = string("op_818"), val = tensor([1, 64, 12, 64])]; + tensor x_111_cast_fp16 = reshape(shape = var_818, x = linear_55_cast_fp16)[name = string("x_111_cast_fp16")]; + tensor linear_56_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_19_cast_fp16)[name = string("linear_56_cast_fp16")]; + tensor var_827 = const()[name = string("op_827"), val = tensor([1, 64, 12, 64])]; + tensor x_115_cast_fp16 = reshape(shape = var_827, x = linear_56_cast_fp16)[name = string("x_115_cast_fp16")]; + tensor linear_57_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_19_cast_fp16)[name = string("linear_57_cast_fp16")]; + tensor var_836 = const()[name = string("op_836"), val = tensor([1, 64, 12, 64])]; + tensor x_119_cast_fp16 = reshape(shape = var_836, x = linear_57_cast_fp16)[name = string("x_119_cast_fp16")]; + tensor transpose_99_perm_0 = const()[name = string("transpose_99_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_100_perm_0 = const()[name = string("transpose_100_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_101_perm_0 = const()[name = string("transpose_101_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_101 = transpose(perm = transpose_101_perm_0, x = x_119_cast_fp16)[name = string("transpose_118")]; + tensor transpose_100 = transpose(perm = transpose_100_perm_0, x = x_115_cast_fp16)[name = string("transpose_119")]; + tensor transpose_99 = transpose(perm = transpose_99_perm_0, x = x_111_cast_fp16)[name = string("transpose_120")]; + tensor attention_output_37_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_100, query = transpose_99, value = transpose_101)[name = string("attention_output_37_cast_fp16")]; + tensor attention_output_39_perm_0 = const()[name = string("attention_output_39_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_842 = const()[name = string("op_842"), val = tensor([1, 64, 768])]; + tensor attention_output_39_cast_fp16 = transpose(perm = attention_output_39_perm_0, x = attention_output_37_cast_fp16)[name = string("transpose_117")]; + tensor input_117_cast_fp16 = reshape(shape = var_842, x = attention_output_39_cast_fp16)[name = string("input_117_cast_fp16")]; + tensor linear_58_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_117_cast_fp16)[name = string("linear_58_cast_fp16")]; + tensor input_119_cast_fp16 = add(x = hidden_states_19_cast_fp16, y = linear_58_cast_fp16)[name = string("input_119_cast_fp16")]; + tensor input_121_axes_0 = const()[name = string("input_121_axes_0"), val = tensor([-1])]; + tensor input_121_cast_fp16 = layer_norm(axes = input_121_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_119_cast_fp16)[name = string("input_121_cast_fp16")]; + tensor linear_59_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_121_cast_fp16)[name = string("linear_59_cast_fp16")]; + string input_125_mode_0 = const()[name = string("input_125_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_125_cast_fp16 = gelu(mode = input_125_mode_0, x = linear_59_cast_fp16)[name = string("input_125_cast_fp16")]; + tensor linear_60_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_125_cast_fp16)[name = string("linear_60_cast_fp16")]; + tensor input_127_cast_fp16 = add(x = linear_60_cast_fp16, y = input_121_cast_fp16)[name = string("input_127_cast_fp16")]; + tensor hidden_states_21_axes_0 = const()[name = string("hidden_states_21_axes_0"), val = tensor([-1])]; + tensor hidden_states_21_cast_fp16 = layer_norm(axes = hidden_states_21_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_127_cast_fp16)[name = string("hidden_states_21_cast_fp16")]; + tensor linear_61_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_21_cast_fp16)[name = string("linear_61_cast_fp16")]; + tensor var_893 = const()[name = string("op_893"), val = tensor([1, 64, 12, 64])]; + tensor x_123_cast_fp16 = reshape(shape = var_893, x = linear_61_cast_fp16)[name = string("x_123_cast_fp16")]; + tensor linear_62_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_21_cast_fp16)[name = string("linear_62_cast_fp16")]; + tensor var_902 = const()[name = string("op_902"), val = tensor([1, 64, 12, 64])]; + tensor x_127_cast_fp16 = reshape(shape = var_902, x = linear_62_cast_fp16)[name = string("x_127_cast_fp16")]; + tensor linear_63_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_21_cast_fp16)[name = string("linear_63_cast_fp16")]; + tensor var_911 = const()[name = string("op_911"), val = tensor([1, 64, 12, 64])]; + tensor x_131_cast_fp16 = reshape(shape = var_911, x = linear_63_cast_fp16)[name = string("x_131_cast_fp16")]; + tensor transpose_102_perm_0 = const()[name = string("transpose_102_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_103_perm_0 = const()[name = string("transpose_103_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_104_perm_0 = const()[name = string("transpose_104_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_104 = transpose(perm = transpose_104_perm_0, x = x_131_cast_fp16)[name = string("transpose_114")]; + tensor transpose_103 = transpose(perm = transpose_103_perm_0, x = x_127_cast_fp16)[name = string("transpose_115")]; + tensor transpose_102 = transpose(perm = transpose_102_perm_0, x = x_123_cast_fp16)[name = string("transpose_116")]; + tensor attention_output_41_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_103, query = transpose_102, value = transpose_104)[name = string("attention_output_41_cast_fp16")]; + tensor attention_output_43_perm_0 = const()[name = string("attention_output_43_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_917 = const()[name = string("op_917"), val = tensor([1, 64, 768])]; + tensor attention_output_43_cast_fp16 = transpose(perm = attention_output_43_perm_0, x = attention_output_41_cast_fp16)[name = string("transpose_113")]; + tensor input_129_cast_fp16 = reshape(shape = var_917, x = attention_output_43_cast_fp16)[name = string("input_129_cast_fp16")]; + tensor linear_64_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_129_cast_fp16)[name = string("linear_64_cast_fp16")]; + tensor input_131_cast_fp16 = add(x = hidden_states_21_cast_fp16, y = linear_64_cast_fp16)[name = string("input_131_cast_fp16")]; + tensor input_133_axes_0 = const()[name = string("input_133_axes_0"), val = tensor([-1])]; + tensor input_133_cast_fp16 = layer_norm(axes = input_133_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_131_cast_fp16)[name = string("input_133_cast_fp16")]; + tensor linear_65_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_133_cast_fp16)[name = string("linear_65_cast_fp16")]; + string input_137_mode_0 = const()[name = string("input_137_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_137_cast_fp16 = gelu(mode = input_137_mode_0, x = linear_65_cast_fp16)[name = string("input_137_cast_fp16")]; + tensor linear_66_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_137_cast_fp16)[name = string("linear_66_cast_fp16")]; + tensor input_139_cast_fp16 = add(x = linear_66_cast_fp16, y = input_133_cast_fp16)[name = string("input_139_cast_fp16")]; + tensor hidden_states_axes_0 = const()[name = string("hidden_states_axes_0"), val = tensor([-1])]; + tensor hidden_states_cast_fp16 = layer_norm(axes = hidden_states_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_139_cast_fp16)[name = string("hidden_states_cast_fp16")]; + tensor linear_67_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_query_weight_to_fp16, x = hidden_states_cast_fp16)[name = string("linear_67_cast_fp16")]; + tensor var_968 = const()[name = string("op_968"), val = tensor([1, 64, 12, 64])]; + tensor x_135_cast_fp16 = reshape(shape = var_968, x = linear_67_cast_fp16)[name = string("x_135_cast_fp16")]; + tensor linear_68_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_key_weight_to_fp16, x = hidden_states_cast_fp16)[name = string("linear_68_cast_fp16")]; + tensor var_977 = const()[name = string("op_977"), val = tensor([1, 64, 12, 64])]; + tensor x_139_cast_fp16 = reshape(shape = var_977, x = linear_68_cast_fp16)[name = string("x_139_cast_fp16")]; + tensor linear_69_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_value_weight_to_fp16, x = hidden_states_cast_fp16)[name = string("linear_69_cast_fp16")]; + tensor var_986 = const()[name = string("op_986"), val = tensor([1, 64, 12, 64])]; + tensor x_cast_fp16 = reshape(shape = var_986, x = linear_69_cast_fp16)[name = string("x_cast_fp16")]; + tensor transpose_105_perm_0 = const()[name = string("transpose_105_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_106_perm_0 = const()[name = string("transpose_106_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_107_perm_0 = const()[name = string("transpose_107_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor transpose_107 = transpose(perm = transpose_107_perm_0, x = x_cast_fp16)[name = string("transpose_110")]; + tensor transpose_106 = transpose(perm = transpose_106_perm_0, x = x_139_cast_fp16)[name = string("transpose_111")]; + tensor transpose_105 = transpose(perm = transpose_105_perm_0, x = x_135_cast_fp16)[name = string("transpose_112")]; + tensor attention_output_45_cast_fp16 = scaled_dot_product_attention(attn_mask = attention_mask_cast_fp16, key = transpose_106, query = transpose_105, value = transpose_107)[name = string("attention_output_45_cast_fp16")]; + tensor attention_output_perm_0 = const()[name = string("attention_output_perm_0"), val = tensor([0, 2, 1, 3])]; + tensor var_992 = const()[name = string("op_992"), val = tensor([1, 64, 768])]; + tensor attention_output_cast_fp16 = transpose(perm = attention_output_perm_0, x = attention_output_45_cast_fp16)[name = string("transpose_109")]; + tensor input_141_cast_fp16 = reshape(shape = var_992, x = attention_output_cast_fp16)[name = string("input_141_cast_fp16")]; + tensor linear_70_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_dense_weight_to_fp16, x = input_141_cast_fp16)[name = string("linear_70_cast_fp16")]; + tensor input_143_cast_fp16 = add(x = hidden_states_cast_fp16, y = linear_70_cast_fp16)[name = string("input_143_cast_fp16")]; + tensor input_145_axes_0 = const()[name = string("input_145_axes_0"), val = tensor([-1])]; + tensor input_145_cast_fp16 = layer_norm(axes = input_145_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_attention_LayerNorm_weight_to_fp16, x = input_143_cast_fp16)[name = string("input_145_cast_fp16")]; + tensor linear_71_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_weight_to_fp16, x = input_145_cast_fp16)[name = string("linear_71_cast_fp16")]; + string input_149_mode_0 = const()[name = string("input_149_mode_0"), val = string("TANH_APPROXIMATION")]; + tensor input_149_cast_fp16 = gelu(mode = input_149_mode_0, x = linear_71_cast_fp16)[name = string("input_149_cast_fp16")]; + tensor linear_72_cast_fp16 = linear(bias = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_bias_to_fp16, weight = bert_encoder_albert_layer_groups_0_albert_layers_0_ffn_output_weight_to_fp16, x = input_149_cast_fp16)[name = string("linear_72_cast_fp16")]; + tensor input_151_cast_fp16 = add(x = linear_72_cast_fp16, y = input_145_cast_fp16)[name = string("input_151_cast_fp16")]; + tensor sequence_output_axes_0 = const()[name = string("sequence_output_axes_0"), val = tensor([-1])]; + tensor sequence_output = layer_norm(axes = sequence_output_axes_0, beta = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_bias_to_fp16, epsilon = var_118_to_fp16, gamma = bert_encoder_albert_layer_groups_0_albert_layers_0_full_layer_layer_norm_weight_to_fp16, x = input_151_cast_fp16)[name = string("sequence_output_cast_fp16")]; + tensor bert_encoder_weight_to_fp16 = const()[name = string("bert_encoder_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11306496)))]; + tensor bert_encoder_bias_to_fp16 = const()[name = string("bert_encoder_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(12092992)))]; + tensor linear_73_cast_fp16 = linear(bias = bert_encoder_bias_to_fp16, weight = bert_encoder_weight_to_fp16, x = sequence_output)[name = string("linear_73_cast_fp16")]; + tensor var_1030_perm_0 = const()[name = string("op_1030_perm_0"), val = tensor([0, -1, -2])]; + tensor var_1030 = transpose(perm = var_1030_perm_0, x = linear_73_cast_fp16)[name = string("transpose_108")]; + } -> (sequence_output, var_1030); +} \ No newline at end of file diff --git a/iteration_3/compiled/bert_fp16_t64.mlmodelc/weights/weight.bin b/iteration_3/compiled/bert_fp16_t64.mlmodelc/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..af96b10f5f63bf47c6b0a8eeabea8ce8d0913aed --- /dev/null +++ b/iteration_3/compiled/bert_fp16_t64.mlmodelc/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:629b75e73fbceeb89e92b0b85548bde59918208b424b8d6467202d72d82629b2 +size 12094080 diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/analytics/coremldata.bin b/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/analytics/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..e736feb88ca538cdfbadc478aa6ec70b3b82a167 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/analytics/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc3f372a74ac4a7096282d6c8b15aca657091562cd9968823195b79b46634951 +size 243 diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/coremldata.bin b/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..ca93cf33b918bddd70e94856e692e01515f59042 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c248985ef0758e1fdb436873593f6c37e0a645295dbdcda5bdc8f1c0b6d9efe8 +size 462 diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/metadata.json b/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/metadata.json new file mode 100644 index 0000000000000000000000000000000000000000..f059289190223a6532152f28782fb8385cd3ba75 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/metadata.json @@ -0,0 +1,110 @@ +[ + { + "metadataOutputVersion" : "3.0", + "storagePrecision" : "Float16", + "outputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float16", + "formattedType" : "MultiArray (Float16 1 × 1 × 256)", + "shortDescription" : "", + "shape" : "[1, 1, 256]", + "name" : "var_6225", + "type" : "MultiArray" + } + ], + "modelParameters" : [ + + ], + "specificationVersion" : 9, + "mlProgramOperationTypeHistogram" : { + "Ios18.expandDims" : 16, + "Ios18.softmax" : 24, + "Ios18.mul" : 117, + "Ios18.matmul" : 48, + "Ios16.reduceMean" : 8, + "Split" : 72, + "Tile" : 16, + "Ios18.add" : 188, + "Ios16.reduceSum" : 8, + "Ios18.layerNorm" : 24, + "Ios18.reshape" : 102, + "Ios18.linear" : 143, + "Ios18.conv" : 8, + "Ios18.gelu" : 41, + "Ios18.sub" : 8, + "Ios18.concat" : 8, + "Stack" : 8, + "Ios18.transpose" : 216, + "Ios18.cast" : 4, + "Ios18.sliceByIndex" : 4 + }, + "computePrecision" : "Mixed (Float16, Int32)", + "isUpdatable" : "0", + "stateSchema" : [ + + ], + "availability" : { + "macOS" : "15.0", + "tvOS" : "18.0", + "visionOS" : "2.0", + "watchOS" : "11.0", + "iOS" : "18.0", + "macCatalyst" : "18.0" + }, + "modelType" : { + "name" : "MLModelType_mlProgram" + }, + "userDefinedMetadata" : { + "com.github.apple.coremltools.conversion_date" : "2026-05-08", + "com.github.apple.coremltools.source" : "torch==2.11.0", + "com.github.apple.coremltools.version" : "9.0", + "com.github.apple.coremltools.source_dialect" : "TorchScript" + }, + "inputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 1 × 1 × 256)", + "shortDescription" : "", + "shape" : "[1, 1, 256]", + "name" : "noise_init", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 4 × 1 × 1 × 256)", + "shortDescription" : "", + "shape" : "[4, 1, 1, 256]", + "name" : "noises_aux", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 1 × 128 × 768)", + "shortDescription" : "", + "shape" : "[1, 128, 768]", + "name" : "embedding", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 1 × 256)", + "shortDescription" : "", + "shape" : "[1, 256]", + "name" : "features", + "type" : "MultiArray" + } + ], + "generatedClassName" : "fused_diffusion_sampler_fp16_t128", + "method" : "predict" + } +] \ No newline at end of file diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/model.mil b/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/model.mil new file mode 100644 index 0000000000000000000000000000000000000000..0bd21659b7db183240573ce4212207c07b8b2401 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/model.mil @@ -0,0 +1,2019 @@ +program(1.3) +[buildInfo = dict({{"coremlc-component-MIL", "3520.4.1"}, {"coremlc-version", "3520.5.1"}, {"coremltools-component-torch", "2.11.0"}, {"coremltools-source-dialect", "TorchScript"}, {"coremltools-version", "9.0"}})] +{ + func main(tensor embedding, tensor features, tensor noise_init, tensor noises_aux) { + fp16 var_12_to_fp16 = const()[name = string("op_12_to_fp16"), val = fp16(0x1.8p+1)]; + string noise_init_to_fp16_dtype_0 = const()[name = string("noise_init_to_fp16_dtype_0"), val = string("fp16")]; + tensor noise_init_to_fp16 = cast(dtype = noise_init_to_fp16_dtype_0, x = noise_init)[name = string("cast_196")]; + tensor x_noisy_1_cast_fp16 = mul(x = var_12_to_fp16, y = noise_init_to_fp16)[name = string("x_noisy_1_cast_fp16")]; + int32 var_35 = const()[name = string("op_35"), val = int32(-1)]; + tensor c_in_1_to_fp16 = const()[name = string("c_in_1_to_fp16"), val = tensor([[[0x1.548p-2]]])]; + tensor x_11_cast_fp16 = mul(x = c_in_1_to_fp16, y = x_noisy_1_cast_fp16)[name = string("x_11_cast_fp16")]; + string features_to_fp16_dtype_0 = const()[name = string("features_to_fp16_dtype_0"), val = string("fp16")]; + tensor unet_step_kdiffusion_net_to_features_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_features_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(64)))]; + tensor unet_step_kdiffusion_net_to_features_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_features_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(524416)))]; + tensor features_to_fp16 = cast(dtype = features_to_fp16_dtype_0, x = features)[name = string("cast_195")]; + tensor linear_1_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_features_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_features_0_weight_to_fp16, x = features_to_fp16)[name = string("linear_1_cast_fp16")]; + string var_423_mode_0 = const()[name = string("op_423_mode_0"), val = string("EXACT")]; + tensor var_423_cast_fp16 = gelu(mode = var_423_mode_0, x = linear_1_cast_fp16)[name = string("op_423_cast_fp16")]; + int32 x_7_axis_0 = const()[name = string("x_7_axis_0"), val = int32(0)]; + tensor var_421_to_fp16 = const()[name = string("op_421_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(526528)))]; + tensor x_7_cast_fp16 = stack(axis = x_7_axis_0, values = (var_421_to_fp16, var_423_cast_fp16))[name = string("x_7_cast_fp16")]; + tensor var_426 = const()[name = string("op_426"), val = tensor([1, 2, 0])]; + tensor input_7_axes_0 = const()[name = string("input_7_axes_0"), val = tensor([2])]; + bool input_7_keep_dims_0 = const()[name = string("input_7_keep_dims_0"), val = bool(false)]; + tensor x_9_cast_fp16 = transpose(perm = var_426, x = x_7_cast_fp16)[name = string("transpose_335")]; + tensor input_7_cast_fp16 = reduce_sum(axes = input_7_axes_0, keep_dims = input_7_keep_dims_0, x = x_9_cast_fp16)[name = string("input_7_cast_fp16")]; + tensor unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(528640)))]; + tensor unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2625856)))]; + tensor linear_2_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_7_cast_fp16)[name = string("linear_2_cast_fp16")]; + string input_11_mode_0 = const()[name = string("input_11_mode_0"), val = string("EXACT")]; + tensor input_11_cast_fp16 = gelu(mode = input_11_mode_0, x = linear_2_cast_fp16)[name = string("input_11_cast_fp16")]; + tensor unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2627968)))]; + tensor unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(4725184)))]; + tensor linear_3_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_11_cast_fp16)[name = string("linear_3_cast_fp16")]; + string mapping_1_mode_0 = const()[name = string("mapping_1_mode_0"), val = string("EXACT")]; + tensor mapping_1_cast_fp16 = gelu(mode = mapping_1_mode_0, x = linear_3_cast_fp16)[name = string("mapping_1_cast_fp16")]; + tensor var_436_reps_0 = const()[name = string("op_436_reps_0"), val = tensor([1, 128, 1])]; + tensor var_436_cast_fp16 = tile(reps = var_436_reps_0, x = x_11_cast_fp16)[name = string("op_436_cast_fp16")]; + bool x_13_interleave_0 = const()[name = string("x_13_interleave_0"), val = bool(false)]; + string embedding_to_fp16_dtype_0 = const()[name = string("embedding_to_fp16_dtype_0"), val = string("fp16")]; + tensor embedding_to_fp16 = cast(dtype = embedding_to_fp16_dtype_0, x = embedding)[name = string("cast_194")]; + tensor x_13_cast_fp16 = concat(axis = var_35, interleave = x_13_interleave_0, values = (var_436_cast_fp16, embedding_to_fp16))[name = string("x_13_cast_fp16")]; + tensor var_439_axes_0 = const()[name = string("op_439_axes_0"), val = tensor([1])]; + tensor var_439_cast_fp16 = expand_dims(axes = var_439_axes_0, x = mapping_1_cast_fp16)[name = string("op_439_cast_fp16")]; + tensor mapping_3_reps_0 = const()[name = string("mapping_3_reps_0"), val = tensor([1, 128, 1])]; + tensor mapping_3_cast_fp16 = tile(reps = mapping_3_reps_0, x = var_439_cast_fp16)[name = string("mapping_3_cast_fp16")]; + tensor x_15_cast_fp16 = add(x = x_13_cast_fp16, y = mapping_3_cast_fp16)[name = string("x_15_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(4727296)))]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5775936)))]; + tensor linear_4_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_norm_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_norm_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_4_cast_fp16")]; + tensor var_449 = const()[name = string("op_449"), val = tensor([1, 2048, 1])]; + tensor h_3_cast_fp16 = reshape(shape = var_449, x = linear_4_cast_fp16)[name = string("h_3_cast_fp16")]; + tensor var_451_split_sizes_0 = const()[name = string("op_451_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_451_axis_0 = const()[name = string("op_451_axis_0"), val = int32(1)]; + tensor var_451_cast_fp16_0, tensor var_451_cast_fp16_1 = split(axis = var_451_axis_0, split_sizes = var_451_split_sizes_0, x = h_3_cast_fp16)[name = string("op_451_cast_fp16")]; + tensor gamma_3_perm_0 = const()[name = string("gamma_3_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_3_perm_0 = const()[name = string("beta_3_perm_0"), val = tensor([0, -1, 1])]; + tensor x_19_axes_0 = const()[name = string("x_19_axes_0"), val = tensor([-1])]; + fp16 var_31_to_fp16 = const()[name = string("op_31_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_19_cast_fp16 = layer_norm(axes = x_19_axes_0, epsilon = var_31_to_fp16, x = x_15_cast_fp16)[name = string("x_19_cast_fp16")]; + fp16 var_457_promoted_to_fp16 = const()[name = string("op_457_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_3_cast_fp16 = transpose(perm = gamma_3_perm_0, x = var_451_cast_fp16_0)[name = string("transpose_334")]; + tensor var_458_cast_fp16 = add(x = gamma_3_cast_fp16, y = var_457_promoted_to_fp16)[name = string("op_458_cast_fp16")]; + tensor var_459_cast_fp16 = mul(x = var_458_cast_fp16, y = x_19_cast_fp16)[name = string("op_459_cast_fp16")]; + tensor beta_3_cast_fp16 = transpose(perm = beta_3_perm_0, x = var_451_cast_fp16_1)[name = string("transpose_333")]; + tensor x_21_cast_fp16 = add(x = var_459_cast_fp16, y = beta_3_cast_fp16)[name = string("x_21_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5780096)))]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(6828736)))]; + tensor linear_5_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_5_cast_fp16")]; + tensor var_468 = const()[name = string("op_468"), val = tensor([1, 2048, 1])]; + tensor h_7_cast_fp16 = reshape(shape = var_468, x = linear_5_cast_fp16)[name = string("h_7_cast_fp16")]; + tensor var_470_split_sizes_0 = const()[name = string("op_470_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_470_axis_0 = const()[name = string("op_470_axis_0"), val = int32(1)]; + tensor var_470_cast_fp16_0, tensor var_470_cast_fp16_1 = split(axis = var_470_axis_0, split_sizes = var_470_split_sizes_0, x = h_7_cast_fp16)[name = string("op_470_cast_fp16")]; + tensor gamma_7_perm_0 = const()[name = string("gamma_7_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_7_perm_0 = const()[name = string("beta_7_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_476_promoted_to_fp16 = const()[name = string("op_476_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_7_cast_fp16 = transpose(perm = gamma_7_perm_0, x = var_470_cast_fp16_0)[name = string("transpose_332")]; + tensor var_477_cast_fp16 = add(x = gamma_7_cast_fp16, y = var_476_promoted_to_fp16)[name = string("op_477_cast_fp16")]; + tensor var_478_cast_fp16 = mul(x = var_477_cast_fp16, y = x_19_cast_fp16)[name = string("op_478_cast_fp16")]; + tensor beta_7_cast_fp16 = transpose(perm = beta_7_perm_0, x = var_470_cast_fp16_1)[name = string("transpose_331")]; + tensor x_27_cast_fp16 = add(x = var_478_cast_fp16, y = beta_7_cast_fp16)[name = string("x_27_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(6832896)))]; + tensor linear_6_bias_0_to_fp16 = const()[name = string("linear_6_bias_0_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(7881536)))]; + tensor linear_6_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_21_cast_fp16)[name = string("linear_6_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(7882624)))]; + tensor linear_7_bias_0_to_fp16 = const()[name = string("linear_7_bias_0_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(9979840)))]; + tensor linear_7_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_27_cast_fp16)[name = string("linear_7_cast_fp16")]; + tensor var_484_split_sizes_0 = const()[name = string("op_484_split_sizes_0"), val = tensor([512, 512])]; + int32 var_484_axis_0 = const()[name = string("op_484_axis_0"), val = int32(-1)]; + tensor var_484_cast_fp16_0, tensor var_484_cast_fp16_1 = split(axis = var_484_axis_0, split_sizes = var_484_split_sizes_0, x = linear_7_cast_fp16)[name = string("op_484_cast_fp16")]; + tensor var_492 = const()[name = string("op_492"), val = tensor([1, 128, 8, 64])]; + tensor x_31_cast_fp16 = reshape(shape = var_492, x = linear_6_cast_fp16)[name = string("x_31_cast_fp16")]; + tensor var_502 = const()[name = string("op_502"), val = tensor([1, 128, 8, 64])]; + tensor x_35_cast_fp16 = reshape(shape = var_502, x = var_484_cast_fp16_0)[name = string("x_35_cast_fp16")]; + tensor var_512 = const()[name = string("op_512"), val = tensor([1, 128, 8, 64])]; + tensor x_39_cast_fp16 = reshape(shape = var_512, x = var_484_cast_fp16_1)[name = string("x_39_cast_fp16")]; + tensor var_514 = const()[name = string("op_514"), val = tensor([0, 2, 1, 3])]; + bool sim_1_transpose_x_0 = const()[name = string("sim_1_transpose_x_0"), val = bool(false)]; + bool sim_1_transpose_y_0 = const()[name = string("sim_1_transpose_y_0"), val = bool(false)]; + tensor transpose_72_perm_0 = const()[name = string("transpose_72_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_73_perm_0 = const()[name = string("transpose_73_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_73 = transpose(perm = transpose_73_perm_0, x = x_35_cast_fp16)[name = string("transpose_328")]; + tensor transpose_72 = transpose(perm = transpose_72_perm_0, x = x_31_cast_fp16)[name = string("transpose_329")]; + tensor sim_1_cast_fp16 = matmul(transpose_x = sim_1_transpose_x_0, transpose_y = sim_1_transpose_y_0, x = transpose_72, y = transpose_73)[name = string("sim_1_cast_fp16")]; + fp16 var_518_to_fp16 = const()[name = string("op_518_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_3_cast_fp16 = mul(x = sim_1_cast_fp16, y = var_518_to_fp16)[name = string("sim_3_cast_fp16")]; + tensor attn_1_cast_fp16 = softmax(axis = var_35, x = sim_3_cast_fp16)[name = string("attn_1_cast_fp16")]; + bool x_41_transpose_x_0 = const()[name = string("x_41_transpose_x_0"), val = bool(false)]; + bool x_41_transpose_y_0 = const()[name = string("x_41_transpose_y_0"), val = bool(false)]; + tensor v_1_cast_fp16 = transpose(perm = var_514, x = x_39_cast_fp16)[name = string("transpose_330")]; + tensor x_41_cast_fp16 = matmul(transpose_x = x_41_transpose_x_0, transpose_y = x_41_transpose_y_0, x = attn_1_cast_fp16, y = v_1_cast_fp16)[name = string("x_41_cast_fp16")]; + tensor var_540 = const()[name = string("op_540"), val = tensor([0, 2, 1, 3])]; + tensor var_542 = const()[name = string("op_542"), val = tensor([1, 128, 512])]; + tensor x_43_cast_fp16 = transpose(perm = var_540, x = x_41_cast_fp16)[name = string("transpose_327")]; + tensor input_23_cast_fp16 = reshape(shape = var_542, x = x_43_cast_fp16)[name = string("input_23_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(9981952)))]; + tensor unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11030592)))]; + tensor linear_8_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_23_cast_fp16)[name = string("linear_8_cast_fp16")]; + tensor input_25_cast_fp16 = add(x = linear_8_cast_fp16, y = x_15_cast_fp16)[name = string("input_25_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11032704)))]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(15227072)))]; + tensor linear_9_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_25_cast_fp16)[name = string("linear_9_cast_fp16")]; + string input_29_mode_0 = const()[name = string("input_29_mode_0"), val = string("EXACT")]; + tensor input_29_cast_fp16 = gelu(mode = input_29_mode_0, x = linear_9_cast_fp16)[name = string("input_29_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(15231232)))]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(19425600)))]; + tensor linear_10_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_29_cast_fp16)[name = string("linear_10_cast_fp16")]; + tensor x_45_cast_fp16 = add(x = linear_10_cast_fp16, y = input_25_cast_fp16)[name = string("x_45_cast_fp16")]; + tensor x_47_cast_fp16 = add(x = x_45_cast_fp16, y = mapping_3_cast_fp16)[name = string("x_47_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(19427712)))]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(20476352)))]; + tensor linear_11_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_norm_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_norm_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_11_cast_fp16")]; + tensor var_556 = const()[name = string("op_556"), val = tensor([1, 2048, 1])]; + tensor h_11_cast_fp16 = reshape(shape = var_556, x = linear_11_cast_fp16)[name = string("h_11_cast_fp16")]; + tensor var_558_split_sizes_0 = const()[name = string("op_558_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_558_axis_0 = const()[name = string("op_558_axis_0"), val = int32(1)]; + tensor var_558_cast_fp16_0, tensor var_558_cast_fp16_1 = split(axis = var_558_axis_0, split_sizes = var_558_split_sizes_0, x = h_11_cast_fp16)[name = string("op_558_cast_fp16")]; + tensor gamma_11_perm_0 = const()[name = string("gamma_11_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_11_perm_0 = const()[name = string("beta_11_perm_0"), val = tensor([0, -1, 1])]; + tensor x_51_axes_0 = const()[name = string("x_51_axes_0"), val = tensor([-1])]; + tensor x_51_cast_fp16 = layer_norm(axes = x_51_axes_0, epsilon = var_31_to_fp16, x = x_47_cast_fp16)[name = string("x_51_cast_fp16")]; + fp16 var_564_promoted_to_fp16 = const()[name = string("op_564_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_11_cast_fp16 = transpose(perm = gamma_11_perm_0, x = var_558_cast_fp16_0)[name = string("transpose_326")]; + tensor var_565_cast_fp16 = add(x = gamma_11_cast_fp16, y = var_564_promoted_to_fp16)[name = string("op_565_cast_fp16")]; + tensor var_566_cast_fp16 = mul(x = var_565_cast_fp16, y = x_51_cast_fp16)[name = string("op_566_cast_fp16")]; + tensor beta_11_cast_fp16 = transpose(perm = beta_11_perm_0, x = var_558_cast_fp16_1)[name = string("transpose_325")]; + tensor x_53_cast_fp16 = add(x = var_566_cast_fp16, y = beta_11_cast_fp16)[name = string("x_53_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(20480512)))]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(21529152)))]; + tensor linear_12_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_12_cast_fp16")]; + tensor var_575 = const()[name = string("op_575"), val = tensor([1, 2048, 1])]; + tensor h_15_cast_fp16 = reshape(shape = var_575, x = linear_12_cast_fp16)[name = string("h_15_cast_fp16")]; + tensor var_577_split_sizes_0 = const()[name = string("op_577_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_577_axis_0 = const()[name = string("op_577_axis_0"), val = int32(1)]; + tensor var_577_cast_fp16_0, tensor var_577_cast_fp16_1 = split(axis = var_577_axis_0, split_sizes = var_577_split_sizes_0, x = h_15_cast_fp16)[name = string("op_577_cast_fp16")]; + tensor gamma_15_perm_0 = const()[name = string("gamma_15_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_15_perm_0 = const()[name = string("beta_15_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_583_promoted_to_fp16 = const()[name = string("op_583_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_15_cast_fp16 = transpose(perm = gamma_15_perm_0, x = var_577_cast_fp16_0)[name = string("transpose_324")]; + tensor var_584_cast_fp16 = add(x = gamma_15_cast_fp16, y = var_583_promoted_to_fp16)[name = string("op_584_cast_fp16")]; + tensor var_585_cast_fp16 = mul(x = var_584_cast_fp16, y = x_51_cast_fp16)[name = string("op_585_cast_fp16")]; + tensor beta_15_cast_fp16 = transpose(perm = beta_15_perm_0, x = var_577_cast_fp16_1)[name = string("transpose_323")]; + tensor x_59_cast_fp16 = add(x = var_585_cast_fp16, y = beta_15_cast_fp16)[name = string("x_59_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(21533312)))]; + tensor linear_13_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_53_cast_fp16)[name = string("linear_13_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(22581952)))]; + tensor linear_14_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_59_cast_fp16)[name = string("linear_14_cast_fp16")]; + tensor var_591_split_sizes_0 = const()[name = string("op_591_split_sizes_0"), val = tensor([512, 512])]; + int32 var_591_axis_0 = const()[name = string("op_591_axis_0"), val = int32(-1)]; + tensor var_591_cast_fp16_0, tensor var_591_cast_fp16_1 = split(axis = var_591_axis_0, split_sizes = var_591_split_sizes_0, x = linear_14_cast_fp16)[name = string("op_591_cast_fp16")]; + tensor var_599 = const()[name = string("op_599"), val = tensor([1, 128, 8, 64])]; + tensor x_63_cast_fp16 = reshape(shape = var_599, x = linear_13_cast_fp16)[name = string("x_63_cast_fp16")]; + tensor var_609 = const()[name = string("op_609"), val = tensor([1, 128, 8, 64])]; + tensor x_67_cast_fp16 = reshape(shape = var_609, x = var_591_cast_fp16_0)[name = string("x_67_cast_fp16")]; + tensor var_619 = const()[name = string("op_619"), val = tensor([1, 128, 8, 64])]; + tensor x_71_cast_fp16 = reshape(shape = var_619, x = var_591_cast_fp16_1)[name = string("x_71_cast_fp16")]; + tensor var_621 = const()[name = string("op_621"), val = tensor([0, 2, 1, 3])]; + bool sim_5_transpose_x_0 = const()[name = string("sim_5_transpose_x_0"), val = bool(false)]; + bool sim_5_transpose_y_0 = const()[name = string("sim_5_transpose_y_0"), val = bool(false)]; + tensor transpose_74_perm_0 = const()[name = string("transpose_74_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_75_perm_0 = const()[name = string("transpose_75_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_75 = transpose(perm = transpose_75_perm_0, x = x_67_cast_fp16)[name = string("transpose_320")]; + tensor transpose_74 = transpose(perm = transpose_74_perm_0, x = x_63_cast_fp16)[name = string("transpose_321")]; + tensor sim_5_cast_fp16 = matmul(transpose_x = sim_5_transpose_x_0, transpose_y = sim_5_transpose_y_0, x = transpose_74, y = transpose_75)[name = string("sim_5_cast_fp16")]; + fp16 var_625_to_fp16 = const()[name = string("op_625_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_7_cast_fp16 = mul(x = sim_5_cast_fp16, y = var_625_to_fp16)[name = string("sim_7_cast_fp16")]; + tensor attn_3_cast_fp16 = softmax(axis = var_35, x = sim_7_cast_fp16)[name = string("attn_3_cast_fp16")]; + bool x_73_transpose_x_0 = const()[name = string("x_73_transpose_x_0"), val = bool(false)]; + bool x_73_transpose_y_0 = const()[name = string("x_73_transpose_y_0"), val = bool(false)]; + tensor v_3_cast_fp16 = transpose(perm = var_621, x = x_71_cast_fp16)[name = string("transpose_322")]; + tensor x_73_cast_fp16 = matmul(transpose_x = x_73_transpose_x_0, transpose_y = x_73_transpose_y_0, x = attn_3_cast_fp16, y = v_3_cast_fp16)[name = string("x_73_cast_fp16")]; + tensor var_647 = const()[name = string("op_647"), val = tensor([0, 2, 1, 3])]; + tensor var_649 = const()[name = string("op_649"), val = tensor([1, 128, 512])]; + tensor x_75_cast_fp16 = transpose(perm = var_647, x = x_73_cast_fp16)[name = string("transpose_319")]; + tensor input_39_cast_fp16 = reshape(shape = var_649, x = x_75_cast_fp16)[name = string("input_39_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(24679168)))]; + tensor unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(25727808)))]; + tensor linear_15_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_39_cast_fp16)[name = string("linear_15_cast_fp16")]; + tensor input_41_cast_fp16 = add(x = linear_15_cast_fp16, y = x_47_cast_fp16)[name = string("input_41_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(25729920)))]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(29924288)))]; + tensor linear_16_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_41_cast_fp16)[name = string("linear_16_cast_fp16")]; + string input_45_mode_0 = const()[name = string("input_45_mode_0"), val = string("EXACT")]; + tensor input_45_cast_fp16 = gelu(mode = input_45_mode_0, x = linear_16_cast_fp16)[name = string("input_45_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(29928448)))]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(34122816)))]; + tensor linear_17_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_45_cast_fp16)[name = string("linear_17_cast_fp16")]; + tensor x_77_cast_fp16 = add(x = linear_17_cast_fp16, y = input_41_cast_fp16)[name = string("x_77_cast_fp16")]; + tensor x_79_cast_fp16 = add(x = x_77_cast_fp16, y = mapping_3_cast_fp16)[name = string("x_79_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(34124928)))]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(35173568)))]; + tensor linear_18_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_norm_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_norm_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_18_cast_fp16")]; + tensor var_663 = const()[name = string("op_663"), val = tensor([1, 2048, 1])]; + tensor h_19_cast_fp16 = reshape(shape = var_663, x = linear_18_cast_fp16)[name = string("h_19_cast_fp16")]; + tensor var_665_split_sizes_0 = const()[name = string("op_665_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_665_axis_0 = const()[name = string("op_665_axis_0"), val = int32(1)]; + tensor var_665_cast_fp16_0, tensor var_665_cast_fp16_1 = split(axis = var_665_axis_0, split_sizes = var_665_split_sizes_0, x = h_19_cast_fp16)[name = string("op_665_cast_fp16")]; + tensor gamma_19_perm_0 = const()[name = string("gamma_19_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_19_perm_0 = const()[name = string("beta_19_perm_0"), val = tensor([0, -1, 1])]; + tensor x_83_axes_0 = const()[name = string("x_83_axes_0"), val = tensor([-1])]; + tensor x_83_cast_fp16 = layer_norm(axes = x_83_axes_0, epsilon = var_31_to_fp16, x = x_79_cast_fp16)[name = string("x_83_cast_fp16")]; + fp16 var_671_promoted_to_fp16 = const()[name = string("op_671_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_19_cast_fp16 = transpose(perm = gamma_19_perm_0, x = var_665_cast_fp16_0)[name = string("transpose_318")]; + tensor var_672_cast_fp16 = add(x = gamma_19_cast_fp16, y = var_671_promoted_to_fp16)[name = string("op_672_cast_fp16")]; + tensor var_673_cast_fp16 = mul(x = var_672_cast_fp16, y = x_83_cast_fp16)[name = string("op_673_cast_fp16")]; + tensor beta_19_cast_fp16 = transpose(perm = beta_19_perm_0, x = var_665_cast_fp16_1)[name = string("transpose_317")]; + tensor x_85_cast_fp16 = add(x = var_673_cast_fp16, y = beta_19_cast_fp16)[name = string("x_85_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(35177728)))]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(36226368)))]; + tensor linear_19_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_19_cast_fp16")]; + tensor var_682 = const()[name = string("op_682"), val = tensor([1, 2048, 1])]; + tensor h_23_cast_fp16 = reshape(shape = var_682, x = linear_19_cast_fp16)[name = string("h_23_cast_fp16")]; + tensor var_684_split_sizes_0 = const()[name = string("op_684_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_684_axis_0 = const()[name = string("op_684_axis_0"), val = int32(1)]; + tensor var_684_cast_fp16_0, tensor var_684_cast_fp16_1 = split(axis = var_684_axis_0, split_sizes = var_684_split_sizes_0, x = h_23_cast_fp16)[name = string("op_684_cast_fp16")]; + tensor gamma_23_perm_0 = const()[name = string("gamma_23_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_23_perm_0 = const()[name = string("beta_23_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_690_promoted_to_fp16 = const()[name = string("op_690_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_23_cast_fp16 = transpose(perm = gamma_23_perm_0, x = var_684_cast_fp16_0)[name = string("transpose_316")]; + tensor var_691_cast_fp16 = add(x = gamma_23_cast_fp16, y = var_690_promoted_to_fp16)[name = string("op_691_cast_fp16")]; + tensor var_692_cast_fp16 = mul(x = var_691_cast_fp16, y = x_83_cast_fp16)[name = string("op_692_cast_fp16")]; + tensor beta_23_cast_fp16 = transpose(perm = beta_23_perm_0, x = var_684_cast_fp16_1)[name = string("transpose_315")]; + tensor x_91_cast_fp16 = add(x = var_692_cast_fp16, y = beta_23_cast_fp16)[name = string("x_91_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(36230528)))]; + tensor linear_20_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_85_cast_fp16)[name = string("linear_20_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(37279168)))]; + tensor linear_21_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_91_cast_fp16)[name = string("linear_21_cast_fp16")]; + tensor var_698_split_sizes_0 = const()[name = string("op_698_split_sizes_0"), val = tensor([512, 512])]; + int32 var_698_axis_0 = const()[name = string("op_698_axis_0"), val = int32(-1)]; + tensor var_698_cast_fp16_0, tensor var_698_cast_fp16_1 = split(axis = var_698_axis_0, split_sizes = var_698_split_sizes_0, x = linear_21_cast_fp16)[name = string("op_698_cast_fp16")]; + tensor var_706 = const()[name = string("op_706"), val = tensor([1, 128, 8, 64])]; + tensor x_95_cast_fp16 = reshape(shape = var_706, x = linear_20_cast_fp16)[name = string("x_95_cast_fp16")]; + tensor var_716 = const()[name = string("op_716"), val = tensor([1, 128, 8, 64])]; + tensor x_99_cast_fp16 = reshape(shape = var_716, x = var_698_cast_fp16_0)[name = string("x_99_cast_fp16")]; + tensor var_726 = const()[name = string("op_726"), val = tensor([1, 128, 8, 64])]; + tensor x_103_cast_fp16 = reshape(shape = var_726, x = var_698_cast_fp16_1)[name = string("x_103_cast_fp16")]; + tensor var_728 = const()[name = string("op_728"), val = tensor([0, 2, 1, 3])]; + bool sim_9_transpose_x_0 = const()[name = string("sim_9_transpose_x_0"), val = bool(false)]; + bool sim_9_transpose_y_0 = const()[name = string("sim_9_transpose_y_0"), val = bool(false)]; + tensor transpose_76_perm_0 = const()[name = string("transpose_76_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_77_perm_0 = const()[name = string("transpose_77_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_77 = transpose(perm = transpose_77_perm_0, x = x_99_cast_fp16)[name = string("transpose_312")]; + tensor transpose_76 = transpose(perm = transpose_76_perm_0, x = x_95_cast_fp16)[name = string("transpose_313")]; + tensor sim_9_cast_fp16 = matmul(transpose_x = sim_9_transpose_x_0, transpose_y = sim_9_transpose_y_0, x = transpose_76, y = transpose_77)[name = string("sim_9_cast_fp16")]; + fp16 var_732_to_fp16 = const()[name = string("op_732_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_11_cast_fp16 = mul(x = sim_9_cast_fp16, y = var_732_to_fp16)[name = string("sim_11_cast_fp16")]; + tensor attn_5_cast_fp16 = softmax(axis = var_35, x = sim_11_cast_fp16)[name = string("attn_5_cast_fp16")]; + bool x_105_transpose_x_0 = const()[name = string("x_105_transpose_x_0"), val = bool(false)]; + bool x_105_transpose_y_0 = const()[name = string("x_105_transpose_y_0"), val = bool(false)]; + tensor v_5_cast_fp16 = transpose(perm = var_728, x = x_103_cast_fp16)[name = string("transpose_314")]; + tensor x_105_cast_fp16 = matmul(transpose_x = x_105_transpose_x_0, transpose_y = x_105_transpose_y_0, x = attn_5_cast_fp16, y = v_5_cast_fp16)[name = string("x_105_cast_fp16")]; + tensor var_754 = const()[name = string("op_754"), val = tensor([0, 2, 1, 3])]; + tensor var_756 = const()[name = string("op_756"), val = tensor([1, 128, 512])]; + tensor x_107_cast_fp16 = transpose(perm = var_754, x = x_105_cast_fp16)[name = string("transpose_311")]; + tensor input_55_cast_fp16 = reshape(shape = var_756, x = x_107_cast_fp16)[name = string("input_55_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(39376384)))]; + tensor unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(40425024)))]; + tensor linear_22_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_55_cast_fp16)[name = string("linear_22_cast_fp16")]; + tensor input_57_cast_fp16 = add(x = linear_22_cast_fp16, y = x_79_cast_fp16)[name = string("input_57_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(40427136)))]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(44621504)))]; + tensor linear_23_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_57_cast_fp16)[name = string("linear_23_cast_fp16")]; + string input_61_mode_0 = const()[name = string("input_61_mode_0"), val = string("EXACT")]; + tensor input_61_cast_fp16 = gelu(mode = input_61_mode_0, x = linear_23_cast_fp16)[name = string("input_61_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(44625664)))]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(48820032)))]; + tensor linear_24_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_61_cast_fp16)[name = string("linear_24_cast_fp16")]; + tensor x_109_cast_fp16 = add(x = linear_24_cast_fp16, y = input_57_cast_fp16)[name = string("x_109_cast_fp16")]; + tensor var_765_axes_0 = const()[name = string("op_765_axes_0"), val = tensor([1])]; + bool var_765_keep_dims_0 = const()[name = string("op_765_keep_dims_0"), val = bool(false)]; + tensor var_765_cast_fp16 = reduce_mean(axes = var_765_axes_0, keep_dims = var_765_keep_dims_0, x = x_109_cast_fp16)[name = string("op_765_cast_fp16")]; + tensor x_111_axes_0 = const()[name = string("x_111_axes_0"), val = tensor([1])]; + tensor x_111_cast_fp16 = expand_dims(axes = x_111_axes_0, x = var_765_cast_fp16)[name = string("x_111_cast_fp16")]; + tensor var_767 = const()[name = string("op_767"), val = tensor([0, 2, 1])]; + string x_113_pad_type_0 = const()[name = string("x_113_pad_type_0"), val = string("valid")]; + tensor x_113_strides_0 = const()[name = string("x_113_strides_0"), val = tensor([1])]; + tensor x_113_pad_0 = const()[name = string("x_113_pad_0"), val = tensor([0, 0])]; + tensor x_113_dilations_0 = const()[name = string("x_113_dilations_0"), val = tensor([1])]; + int32 x_113_groups_0 = const()[name = string("x_113_groups_0"), val = int32(1)]; + tensor unet_step_kdiffusion_net_to_out_1_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_out_1_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(48822144)))]; + tensor unet_step_kdiffusion_net_to_out_1_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_out_1_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49346496)))]; + tensor input_63_cast_fp16 = transpose(perm = var_767, x = x_111_cast_fp16)[name = string("transpose_310")]; + tensor x_113_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_113_dilations_0, groups = x_113_groups_0, pad = x_113_pad_0, pad_type = x_113_pad_type_0, strides = x_113_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_63_cast_fp16)[name = string("x_113_cast_fp16")]; + tensor x_pred_1_perm_0 = const()[name = string("x_pred_1_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_1_to_fp16 = const()[name = string("c_skip_1_to_fp16"), val = tensor([[[0x1.1fcp-8]]])]; + tensor var_775_cast_fp16 = mul(x = c_skip_1_to_fp16, y = x_noisy_1_cast_fp16)[name = string("op_775_cast_fp16")]; + tensor c_out_1_to_fp16 = const()[name = string("c_out_1_to_fp16"), val = tensor([[[0x1.974p-3]]])]; + tensor x_pred_1_cast_fp16 = transpose(perm = x_pred_1_perm_0, x = x_113_cast_fp16)[name = string("transpose_309")]; + tensor var_776_cast_fp16 = mul(x = c_out_1_to_fp16, y = x_pred_1_cast_fp16)[name = string("op_776_cast_fp16")]; + tensor x_dn_1_cast_fp16 = add(x = var_775_cast_fp16, y = var_776_cast_fp16)[name = string("x_dn_1_cast_fp16")]; + tensor var_779_cast_fp16 = sub(x = x_noisy_1_cast_fp16, y = x_dn_1_cast_fp16)[name = string("op_779_cast_fp16")]; + tensor _inversed_d_1_y_0_to_fp16 = const()[name = string("_inversed_d_1_y_0_to_fp16"), val = tensor([0x1.554p-2])]; + tensor _inversed_d_1_cast_fp16 = mul(x = var_779_cast_fp16, y = _inversed_d_1_y_0_to_fp16)[name = string("_inversed_d_1_cast_fp16")]; + fp16 var_788_to_fp16 = const()[name = string("op_788_to_fp16"), val = fp16(-0x1.72cp+0)]; + tensor var_789_cast_fp16 = mul(x = _inversed_d_1_cast_fp16, y = var_788_to_fp16)[name = string("op_789_cast_fp16")]; + tensor x_noisy_3_cast_fp16 = add(x = x_noisy_1_cast_fp16, y = var_789_cast_fp16)[name = string("x_noisy_3_cast_fp16")]; + int32 var_801 = const()[name = string("op_801"), val = int32(-1)]; + tensor c_in_3_to_fp16 = const()[name = string("c_in_3_to_fp16"), val = tensor([[[0x1.474p-1]]])]; + tensor x_123_cast_fp16 = mul(x = c_in_3_to_fp16, y = x_noisy_3_cast_fp16)[name = string("x_123_cast_fp16")]; + int32 x_119_axis_0 = const()[name = string("x_119_axis_0"), val = int32(0)]; + tensor var_1187_to_fp16 = const()[name = string("op_1187_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49347072)))]; + tensor x_119_cast_fp16 = stack(axis = x_119_axis_0, values = (var_1187_to_fp16, var_423_cast_fp16))[name = string("x_119_cast_fp16")]; + tensor var_1192 = const()[name = string("op_1192"), val = tensor([1, 2, 0])]; + tensor input_71_axes_0 = const()[name = string("input_71_axes_0"), val = tensor([2])]; + bool input_71_keep_dims_0 = const()[name = string("input_71_keep_dims_0"), val = bool(false)]; + tensor x_121_cast_fp16 = transpose(perm = var_1192, x = x_119_cast_fp16)[name = string("transpose_308")]; + tensor input_71_cast_fp16 = reduce_sum(axes = input_71_axes_0, keep_dims = input_71_keep_dims_0, x = x_121_cast_fp16)[name = string("input_71_cast_fp16")]; + tensor linear_27_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_71_cast_fp16)[name = string("linear_27_cast_fp16")]; + string input_75_mode_0 = const()[name = string("input_75_mode_0"), val = string("EXACT")]; + tensor input_75_cast_fp16 = gelu(mode = input_75_mode_0, x = linear_27_cast_fp16)[name = string("input_75_cast_fp16")]; + tensor linear_28_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_75_cast_fp16)[name = string("linear_28_cast_fp16")]; + string mapping_5_mode_0 = const()[name = string("mapping_5_mode_0"), val = string("EXACT")]; + tensor mapping_5_cast_fp16 = gelu(mode = mapping_5_mode_0, x = linear_28_cast_fp16)[name = string("mapping_5_cast_fp16")]; + tensor var_1202_reps_0 = const()[name = string("op_1202_reps_0"), val = tensor([1, 128, 1])]; + tensor var_1202_cast_fp16 = tile(reps = var_1202_reps_0, x = x_123_cast_fp16)[name = string("op_1202_cast_fp16")]; + bool x_125_interleave_0 = const()[name = string("x_125_interleave_0"), val = bool(false)]; + tensor x_125_cast_fp16 = concat(axis = var_801, interleave = x_125_interleave_0, values = (var_1202_cast_fp16, embedding_to_fp16))[name = string("x_125_cast_fp16")]; + tensor var_1205_axes_0 = const()[name = string("op_1205_axes_0"), val = tensor([1])]; + tensor var_1205_cast_fp16 = expand_dims(axes = var_1205_axes_0, x = mapping_5_cast_fp16)[name = string("op_1205_cast_fp16")]; + tensor mapping_7_reps_0 = const()[name = string("mapping_7_reps_0"), val = tensor([1, 128, 1])]; + tensor mapping_7_cast_fp16 = tile(reps = mapping_7_reps_0, x = var_1205_cast_fp16)[name = string("mapping_7_cast_fp16")]; + tensor x_127_cast_fp16 = add(x = x_125_cast_fp16, y = mapping_7_cast_fp16)[name = string("x_127_cast_fp16")]; + tensor var_1217_split_sizes_0 = const()[name = string("op_1217_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1217_axis_0 = const()[name = string("op_1217_axis_0"), val = int32(1)]; + tensor var_1217_cast_fp16_0, tensor var_1217_cast_fp16_1 = split(axis = var_1217_axis_0, split_sizes = var_1217_split_sizes_0, x = h_3_cast_fp16)[name = string("op_1217_cast_fp16")]; + tensor gamma_27_perm_0 = const()[name = string("gamma_27_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_27_perm_0 = const()[name = string("beta_27_perm_0"), val = tensor([0, -1, 1])]; + tensor x_131_axes_0 = const()[name = string("x_131_axes_0"), val = tensor([-1])]; + fp16 var_797_to_fp16 = const()[name = string("op_797_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_131_cast_fp16 = layer_norm(axes = x_131_axes_0, epsilon = var_797_to_fp16, x = x_127_cast_fp16)[name = string("x_131_cast_fp16")]; + fp16 var_1223_promoted_to_fp16 = const()[name = string("op_1223_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_27_cast_fp16 = transpose(perm = gamma_27_perm_0, x = var_1217_cast_fp16_0)[name = string("transpose_307")]; + tensor var_1224_cast_fp16 = add(x = gamma_27_cast_fp16, y = var_1223_promoted_to_fp16)[name = string("op_1224_cast_fp16")]; + tensor var_1225_cast_fp16 = mul(x = var_1224_cast_fp16, y = x_131_cast_fp16)[name = string("op_1225_cast_fp16")]; + tensor beta_27_cast_fp16 = transpose(perm = beta_27_perm_0, x = var_1217_cast_fp16_1)[name = string("transpose_306")]; + tensor x_133_cast_fp16 = add(x = var_1225_cast_fp16, y = beta_27_cast_fp16)[name = string("x_133_cast_fp16")]; + tensor var_1236_split_sizes_0 = const()[name = string("op_1236_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1236_axis_0 = const()[name = string("op_1236_axis_0"), val = int32(1)]; + tensor var_1236_cast_fp16_0, tensor var_1236_cast_fp16_1 = split(axis = var_1236_axis_0, split_sizes = var_1236_split_sizes_0, x = h_7_cast_fp16)[name = string("op_1236_cast_fp16")]; + tensor gamma_31_perm_0 = const()[name = string("gamma_31_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_31_perm_0 = const()[name = string("beta_31_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_1242_promoted_to_fp16 = const()[name = string("op_1242_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_31_cast_fp16 = transpose(perm = gamma_31_perm_0, x = var_1236_cast_fp16_0)[name = string("transpose_305")]; + tensor var_1243_cast_fp16 = add(x = gamma_31_cast_fp16, y = var_1242_promoted_to_fp16)[name = string("op_1243_cast_fp16")]; + tensor var_1244_cast_fp16 = mul(x = var_1243_cast_fp16, y = x_131_cast_fp16)[name = string("op_1244_cast_fp16")]; + tensor beta_31_cast_fp16 = transpose(perm = beta_31_perm_0, x = var_1236_cast_fp16_1)[name = string("transpose_304")]; + tensor x_139_cast_fp16 = add(x = var_1244_cast_fp16, y = beta_31_cast_fp16)[name = string("x_139_cast_fp16")]; + tensor linear_31_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_133_cast_fp16)[name = string("linear_31_cast_fp16")]; + tensor linear_32_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_139_cast_fp16)[name = string("linear_32_cast_fp16")]; + tensor var_1250_split_sizes_0 = const()[name = string("op_1250_split_sizes_0"), val = tensor([512, 512])]; + int32 var_1250_axis_0 = const()[name = string("op_1250_axis_0"), val = int32(-1)]; + tensor var_1250_cast_fp16_0, tensor var_1250_cast_fp16_1 = split(axis = var_1250_axis_0, split_sizes = var_1250_split_sizes_0, x = linear_32_cast_fp16)[name = string("op_1250_cast_fp16")]; + tensor var_1258 = const()[name = string("op_1258"), val = tensor([1, 128, 8, 64])]; + tensor x_143_cast_fp16 = reshape(shape = var_1258, x = linear_31_cast_fp16)[name = string("x_143_cast_fp16")]; + tensor var_1268 = const()[name = string("op_1268"), val = tensor([1, 128, 8, 64])]; + tensor x_147_cast_fp16 = reshape(shape = var_1268, x = var_1250_cast_fp16_0)[name = string("x_147_cast_fp16")]; + tensor var_1278 = const()[name = string("op_1278"), val = tensor([1, 128, 8, 64])]; + tensor x_151_cast_fp16 = reshape(shape = var_1278, x = var_1250_cast_fp16_1)[name = string("x_151_cast_fp16")]; + tensor var_1280 = const()[name = string("op_1280"), val = tensor([0, 2, 1, 3])]; + bool sim_13_transpose_x_0 = const()[name = string("sim_13_transpose_x_0"), val = bool(false)]; + bool sim_13_transpose_y_0 = const()[name = string("sim_13_transpose_y_0"), val = bool(false)]; + tensor transpose_78_perm_0 = const()[name = string("transpose_78_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_79_perm_0 = const()[name = string("transpose_79_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_79 = transpose(perm = transpose_79_perm_0, x = x_147_cast_fp16)[name = string("transpose_301")]; + tensor transpose_78 = transpose(perm = transpose_78_perm_0, x = x_143_cast_fp16)[name = string("transpose_302")]; + tensor sim_13_cast_fp16 = matmul(transpose_x = sim_13_transpose_x_0, transpose_y = sim_13_transpose_y_0, x = transpose_78, y = transpose_79)[name = string("sim_13_cast_fp16")]; + fp16 var_1284_to_fp16 = const()[name = string("op_1284_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_15_cast_fp16 = mul(x = sim_13_cast_fp16, y = var_1284_to_fp16)[name = string("sim_15_cast_fp16")]; + tensor attn_7_cast_fp16 = softmax(axis = var_801, x = sim_15_cast_fp16)[name = string("attn_7_cast_fp16")]; + bool x_153_transpose_x_0 = const()[name = string("x_153_transpose_x_0"), val = bool(false)]; + bool x_153_transpose_y_0 = const()[name = string("x_153_transpose_y_0"), val = bool(false)]; + tensor v_7_cast_fp16 = transpose(perm = var_1280, x = x_151_cast_fp16)[name = string("transpose_303")]; + tensor x_153_cast_fp16 = matmul(transpose_x = x_153_transpose_x_0, transpose_y = x_153_transpose_y_0, x = attn_7_cast_fp16, y = v_7_cast_fp16)[name = string("x_153_cast_fp16")]; + tensor var_1306 = const()[name = string("op_1306"), val = tensor([0, 2, 1, 3])]; + tensor var_1308 = const()[name = string("op_1308"), val = tensor([1, 128, 512])]; + tensor x_155_cast_fp16 = transpose(perm = var_1306, x = x_153_cast_fp16)[name = string("transpose_300")]; + tensor input_87_cast_fp16 = reshape(shape = var_1308, x = x_155_cast_fp16)[name = string("input_87_cast_fp16")]; + tensor linear_33_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_87_cast_fp16)[name = string("linear_33_cast_fp16")]; + tensor input_89_cast_fp16 = add(x = linear_33_cast_fp16, y = x_127_cast_fp16)[name = string("input_89_cast_fp16")]; + tensor linear_34_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_89_cast_fp16)[name = string("linear_34_cast_fp16")]; + string input_93_mode_0 = const()[name = string("input_93_mode_0"), val = string("EXACT")]; + tensor input_93_cast_fp16 = gelu(mode = input_93_mode_0, x = linear_34_cast_fp16)[name = string("input_93_cast_fp16")]; + tensor linear_35_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_93_cast_fp16)[name = string("linear_35_cast_fp16")]; + tensor x_157_cast_fp16 = add(x = linear_35_cast_fp16, y = input_89_cast_fp16)[name = string("x_157_cast_fp16")]; + tensor x_159_cast_fp16 = add(x = x_157_cast_fp16, y = mapping_7_cast_fp16)[name = string("x_159_cast_fp16")]; + tensor var_1324_split_sizes_0 = const()[name = string("op_1324_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1324_axis_0 = const()[name = string("op_1324_axis_0"), val = int32(1)]; + tensor var_1324_cast_fp16_0, tensor var_1324_cast_fp16_1 = split(axis = var_1324_axis_0, split_sizes = var_1324_split_sizes_0, x = h_11_cast_fp16)[name = string("op_1324_cast_fp16")]; + tensor gamma_35_perm_0 = const()[name = string("gamma_35_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_35_perm_0 = const()[name = string("beta_35_perm_0"), val = tensor([0, -1, 1])]; + tensor x_163_axes_0 = const()[name = string("x_163_axes_0"), val = tensor([-1])]; + tensor x_163_cast_fp16 = layer_norm(axes = x_163_axes_0, epsilon = var_797_to_fp16, x = x_159_cast_fp16)[name = string("x_163_cast_fp16")]; + fp16 var_1330_promoted_to_fp16 = const()[name = string("op_1330_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_35_cast_fp16 = transpose(perm = gamma_35_perm_0, x = var_1324_cast_fp16_0)[name = string("transpose_299")]; + tensor var_1331_cast_fp16 = add(x = gamma_35_cast_fp16, y = var_1330_promoted_to_fp16)[name = string("op_1331_cast_fp16")]; + tensor var_1332_cast_fp16 = mul(x = var_1331_cast_fp16, y = x_163_cast_fp16)[name = string("op_1332_cast_fp16")]; + tensor beta_35_cast_fp16 = transpose(perm = beta_35_perm_0, x = var_1324_cast_fp16_1)[name = string("transpose_298")]; + tensor x_165_cast_fp16 = add(x = var_1332_cast_fp16, y = beta_35_cast_fp16)[name = string("x_165_cast_fp16")]; + tensor var_1343_split_sizes_0 = const()[name = string("op_1343_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1343_axis_0 = const()[name = string("op_1343_axis_0"), val = int32(1)]; + tensor var_1343_cast_fp16_0, tensor var_1343_cast_fp16_1 = split(axis = var_1343_axis_0, split_sizes = var_1343_split_sizes_0, x = h_15_cast_fp16)[name = string("op_1343_cast_fp16")]; + tensor gamma_39_perm_0 = const()[name = string("gamma_39_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_39_perm_0 = const()[name = string("beta_39_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_1349_promoted_to_fp16 = const()[name = string("op_1349_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_39_cast_fp16 = transpose(perm = gamma_39_perm_0, x = var_1343_cast_fp16_0)[name = string("transpose_297")]; + tensor var_1350_cast_fp16 = add(x = gamma_39_cast_fp16, y = var_1349_promoted_to_fp16)[name = string("op_1350_cast_fp16")]; + tensor var_1351_cast_fp16 = mul(x = var_1350_cast_fp16, y = x_163_cast_fp16)[name = string("op_1351_cast_fp16")]; + tensor beta_39_cast_fp16 = transpose(perm = beta_39_perm_0, x = var_1343_cast_fp16_1)[name = string("transpose_296")]; + tensor x_171_cast_fp16 = add(x = var_1351_cast_fp16, y = beta_39_cast_fp16)[name = string("x_171_cast_fp16")]; + tensor linear_38_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_165_cast_fp16)[name = string("linear_38_cast_fp16")]; + tensor linear_39_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_171_cast_fp16)[name = string("linear_39_cast_fp16")]; + tensor var_1357_split_sizes_0 = const()[name = string("op_1357_split_sizes_0"), val = tensor([512, 512])]; + int32 var_1357_axis_0 = const()[name = string("op_1357_axis_0"), val = int32(-1)]; + tensor var_1357_cast_fp16_0, tensor var_1357_cast_fp16_1 = split(axis = var_1357_axis_0, split_sizes = var_1357_split_sizes_0, x = linear_39_cast_fp16)[name = string("op_1357_cast_fp16")]; + tensor var_1365 = const()[name = string("op_1365"), val = tensor([1, 128, 8, 64])]; + tensor x_175_cast_fp16 = reshape(shape = var_1365, x = linear_38_cast_fp16)[name = string("x_175_cast_fp16")]; + tensor var_1375 = const()[name = string("op_1375"), val = tensor([1, 128, 8, 64])]; + tensor x_179_cast_fp16 = reshape(shape = var_1375, x = var_1357_cast_fp16_0)[name = string("x_179_cast_fp16")]; + tensor var_1385 = const()[name = string("op_1385"), val = tensor([1, 128, 8, 64])]; + tensor x_183_cast_fp16 = reshape(shape = var_1385, x = var_1357_cast_fp16_1)[name = string("x_183_cast_fp16")]; + tensor var_1387 = const()[name = string("op_1387"), val = tensor([0, 2, 1, 3])]; + bool sim_17_transpose_x_0 = const()[name = string("sim_17_transpose_x_0"), val = bool(false)]; + bool sim_17_transpose_y_0 = const()[name = string("sim_17_transpose_y_0"), val = bool(false)]; + tensor transpose_80_perm_0 = const()[name = string("transpose_80_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_81_perm_0 = const()[name = string("transpose_81_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_81 = transpose(perm = transpose_81_perm_0, x = x_179_cast_fp16)[name = string("transpose_293")]; + tensor transpose_80 = transpose(perm = transpose_80_perm_0, x = x_175_cast_fp16)[name = string("transpose_294")]; + tensor sim_17_cast_fp16 = matmul(transpose_x = sim_17_transpose_x_0, transpose_y = sim_17_transpose_y_0, x = transpose_80, y = transpose_81)[name = string("sim_17_cast_fp16")]; + fp16 var_1391_to_fp16 = const()[name = string("op_1391_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_19_cast_fp16 = mul(x = sim_17_cast_fp16, y = var_1391_to_fp16)[name = string("sim_19_cast_fp16")]; + tensor attn_9_cast_fp16 = softmax(axis = var_801, x = sim_19_cast_fp16)[name = string("attn_9_cast_fp16")]; + bool x_185_transpose_x_0 = const()[name = string("x_185_transpose_x_0"), val = bool(false)]; + bool x_185_transpose_y_0 = const()[name = string("x_185_transpose_y_0"), val = bool(false)]; + tensor v_9_cast_fp16 = transpose(perm = var_1387, x = x_183_cast_fp16)[name = string("transpose_295")]; + tensor x_185_cast_fp16 = matmul(transpose_x = x_185_transpose_x_0, transpose_y = x_185_transpose_y_0, x = attn_9_cast_fp16, y = v_9_cast_fp16)[name = string("x_185_cast_fp16")]; + tensor var_1413 = const()[name = string("op_1413"), val = tensor([0, 2, 1, 3])]; + tensor var_1415 = const()[name = string("op_1415"), val = tensor([1, 128, 512])]; + tensor x_187_cast_fp16 = transpose(perm = var_1413, x = x_185_cast_fp16)[name = string("transpose_292")]; + tensor input_103_cast_fp16 = reshape(shape = var_1415, x = x_187_cast_fp16)[name = string("input_103_cast_fp16")]; + tensor linear_40_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_103_cast_fp16)[name = string("linear_40_cast_fp16")]; + tensor input_105_cast_fp16 = add(x = linear_40_cast_fp16, y = x_159_cast_fp16)[name = string("input_105_cast_fp16")]; + tensor linear_41_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_105_cast_fp16)[name = string("linear_41_cast_fp16")]; + string input_109_mode_0 = const()[name = string("input_109_mode_0"), val = string("EXACT")]; + tensor input_109_cast_fp16 = gelu(mode = input_109_mode_0, x = linear_41_cast_fp16)[name = string("input_109_cast_fp16")]; + tensor linear_42_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_109_cast_fp16)[name = string("linear_42_cast_fp16")]; + tensor x_189_cast_fp16 = add(x = linear_42_cast_fp16, y = input_105_cast_fp16)[name = string("x_189_cast_fp16")]; + tensor x_191_cast_fp16 = add(x = x_189_cast_fp16, y = mapping_7_cast_fp16)[name = string("x_191_cast_fp16")]; + tensor var_1431_split_sizes_0 = const()[name = string("op_1431_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1431_axis_0 = const()[name = string("op_1431_axis_0"), val = int32(1)]; + tensor var_1431_cast_fp16_0, tensor var_1431_cast_fp16_1 = split(axis = var_1431_axis_0, split_sizes = var_1431_split_sizes_0, x = h_19_cast_fp16)[name = string("op_1431_cast_fp16")]; + tensor gamma_43_perm_0 = const()[name = string("gamma_43_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_43_perm_0 = const()[name = string("beta_43_perm_0"), val = tensor([0, -1, 1])]; + tensor x_195_axes_0 = const()[name = string("x_195_axes_0"), val = tensor([-1])]; + tensor x_195_cast_fp16 = layer_norm(axes = x_195_axes_0, epsilon = var_797_to_fp16, x = x_191_cast_fp16)[name = string("x_195_cast_fp16")]; + fp16 var_1437_promoted_to_fp16 = const()[name = string("op_1437_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_43_cast_fp16 = transpose(perm = gamma_43_perm_0, x = var_1431_cast_fp16_0)[name = string("transpose_291")]; + tensor var_1438_cast_fp16 = add(x = gamma_43_cast_fp16, y = var_1437_promoted_to_fp16)[name = string("op_1438_cast_fp16")]; + tensor var_1439_cast_fp16 = mul(x = var_1438_cast_fp16, y = x_195_cast_fp16)[name = string("op_1439_cast_fp16")]; + tensor beta_43_cast_fp16 = transpose(perm = beta_43_perm_0, x = var_1431_cast_fp16_1)[name = string("transpose_290")]; + tensor x_197_cast_fp16 = add(x = var_1439_cast_fp16, y = beta_43_cast_fp16)[name = string("x_197_cast_fp16")]; + tensor var_1450_split_sizes_0 = const()[name = string("op_1450_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1450_axis_0 = const()[name = string("op_1450_axis_0"), val = int32(1)]; + tensor var_1450_cast_fp16_0, tensor var_1450_cast_fp16_1 = split(axis = var_1450_axis_0, split_sizes = var_1450_split_sizes_0, x = h_23_cast_fp16)[name = string("op_1450_cast_fp16")]; + tensor gamma_47_perm_0 = const()[name = string("gamma_47_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_47_perm_0 = const()[name = string("beta_47_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_1456_promoted_to_fp16 = const()[name = string("op_1456_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_47_cast_fp16 = transpose(perm = gamma_47_perm_0, x = var_1450_cast_fp16_0)[name = string("transpose_289")]; + tensor var_1457_cast_fp16 = add(x = gamma_47_cast_fp16, y = var_1456_promoted_to_fp16)[name = string("op_1457_cast_fp16")]; + tensor var_1458_cast_fp16 = mul(x = var_1457_cast_fp16, y = x_195_cast_fp16)[name = string("op_1458_cast_fp16")]; + tensor beta_47_cast_fp16 = transpose(perm = beta_47_perm_0, x = var_1450_cast_fp16_1)[name = string("transpose_288")]; + tensor x_203_cast_fp16 = add(x = var_1458_cast_fp16, y = beta_47_cast_fp16)[name = string("x_203_cast_fp16")]; + tensor linear_45_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_197_cast_fp16)[name = string("linear_45_cast_fp16")]; + tensor linear_46_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_203_cast_fp16)[name = string("linear_46_cast_fp16")]; + tensor var_1464_split_sizes_0 = const()[name = string("op_1464_split_sizes_0"), val = tensor([512, 512])]; + int32 var_1464_axis_0 = const()[name = string("op_1464_axis_0"), val = int32(-1)]; + tensor var_1464_cast_fp16_0, tensor var_1464_cast_fp16_1 = split(axis = var_1464_axis_0, split_sizes = var_1464_split_sizes_0, x = linear_46_cast_fp16)[name = string("op_1464_cast_fp16")]; + tensor var_1472 = const()[name = string("op_1472"), val = tensor([1, 128, 8, 64])]; + tensor x_207_cast_fp16 = reshape(shape = var_1472, x = linear_45_cast_fp16)[name = string("x_207_cast_fp16")]; + tensor var_1482 = const()[name = string("op_1482"), val = tensor([1, 128, 8, 64])]; + tensor x_211_cast_fp16 = reshape(shape = var_1482, x = var_1464_cast_fp16_0)[name = string("x_211_cast_fp16")]; + tensor var_1492 = const()[name = string("op_1492"), val = tensor([1, 128, 8, 64])]; + tensor x_215_cast_fp16 = reshape(shape = var_1492, x = var_1464_cast_fp16_1)[name = string("x_215_cast_fp16")]; + tensor var_1494 = const()[name = string("op_1494"), val = tensor([0, 2, 1, 3])]; + bool sim_21_transpose_x_0 = const()[name = string("sim_21_transpose_x_0"), val = bool(false)]; + bool sim_21_transpose_y_0 = const()[name = string("sim_21_transpose_y_0"), val = bool(false)]; + tensor transpose_82_perm_0 = const()[name = string("transpose_82_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_83_perm_0 = const()[name = string("transpose_83_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_83 = transpose(perm = transpose_83_perm_0, x = x_211_cast_fp16)[name = string("transpose_285")]; + tensor transpose_82 = transpose(perm = transpose_82_perm_0, x = x_207_cast_fp16)[name = string("transpose_286")]; + tensor sim_21_cast_fp16 = matmul(transpose_x = sim_21_transpose_x_0, transpose_y = sim_21_transpose_y_0, x = transpose_82, y = transpose_83)[name = string("sim_21_cast_fp16")]; + fp16 var_1498_to_fp16 = const()[name = string("op_1498_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_23_cast_fp16 = mul(x = sim_21_cast_fp16, y = var_1498_to_fp16)[name = string("sim_23_cast_fp16")]; + tensor attn_11_cast_fp16 = softmax(axis = var_801, x = sim_23_cast_fp16)[name = string("attn_11_cast_fp16")]; + bool x_217_transpose_x_0 = const()[name = string("x_217_transpose_x_0"), val = bool(false)]; + bool x_217_transpose_y_0 = const()[name = string("x_217_transpose_y_0"), val = bool(false)]; + tensor v_11_cast_fp16 = transpose(perm = var_1494, x = x_215_cast_fp16)[name = string("transpose_287")]; + tensor x_217_cast_fp16 = matmul(transpose_x = x_217_transpose_x_0, transpose_y = x_217_transpose_y_0, x = attn_11_cast_fp16, y = v_11_cast_fp16)[name = string("x_217_cast_fp16")]; + tensor var_1520 = const()[name = string("op_1520"), val = tensor([0, 2, 1, 3])]; + tensor var_1522 = const()[name = string("op_1522"), val = tensor([1, 128, 512])]; + tensor x_219_cast_fp16 = transpose(perm = var_1520, x = x_217_cast_fp16)[name = string("transpose_284")]; + tensor input_119_cast_fp16 = reshape(shape = var_1522, x = x_219_cast_fp16)[name = string("input_119_cast_fp16")]; + tensor linear_47_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_119_cast_fp16)[name = string("linear_47_cast_fp16")]; + tensor input_121_cast_fp16 = add(x = linear_47_cast_fp16, y = x_191_cast_fp16)[name = string("input_121_cast_fp16")]; + tensor linear_48_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_121_cast_fp16)[name = string("linear_48_cast_fp16")]; + string input_125_mode_0 = const()[name = string("input_125_mode_0"), val = string("EXACT")]; + tensor input_125_cast_fp16 = gelu(mode = input_125_mode_0, x = linear_48_cast_fp16)[name = string("input_125_cast_fp16")]; + tensor linear_49_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_125_cast_fp16)[name = string("linear_49_cast_fp16")]; + tensor x_221_cast_fp16 = add(x = linear_49_cast_fp16, y = input_121_cast_fp16)[name = string("x_221_cast_fp16")]; + tensor var_1531_axes_0 = const()[name = string("op_1531_axes_0"), val = tensor([1])]; + bool var_1531_keep_dims_0 = const()[name = string("op_1531_keep_dims_0"), val = bool(false)]; + tensor var_1531_cast_fp16 = reduce_mean(axes = var_1531_axes_0, keep_dims = var_1531_keep_dims_0, x = x_221_cast_fp16)[name = string("op_1531_cast_fp16")]; + tensor x_223_axes_0 = const()[name = string("x_223_axes_0"), val = tensor([1])]; + tensor x_223_cast_fp16 = expand_dims(axes = x_223_axes_0, x = var_1531_cast_fp16)[name = string("x_223_cast_fp16")]; + tensor var_1533 = const()[name = string("op_1533"), val = tensor([0, 2, 1])]; + string x_225_pad_type_0 = const()[name = string("x_225_pad_type_0"), val = string("valid")]; + tensor x_225_strides_0 = const()[name = string("x_225_strides_0"), val = tensor([1])]; + tensor x_225_pad_0 = const()[name = string("x_225_pad_0"), val = tensor([0, 0])]; + tensor x_225_dilations_0 = const()[name = string("x_225_dilations_0"), val = tensor([1])]; + int32 x_225_groups_0 = const()[name = string("x_225_groups_0"), val = int32(1)]; + tensor input_127_cast_fp16 = transpose(perm = var_1533, x = x_223_cast_fp16)[name = string("transpose_283")]; + tensor x_225_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_225_dilations_0, groups = x_225_groups_0, pad = x_225_pad_0, pad_type = x_225_pad_type_0, strides = x_225_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_127_cast_fp16)[name = string("x_225_cast_fp16")]; + tensor x_pred_3_perm_0 = const()[name = string("x_pred_3_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_3_to_fp16 = const()[name = string("c_skip_3_to_fp16"), val = tensor([[[0x1.09cp-6]]])]; + tensor var_1541_cast_fp16 = mul(x = c_skip_3_to_fp16, y = x_noisy_3_cast_fp16)[name = string("op_1541_cast_fp16")]; + tensor c_out_3_to_fp16 = const()[name = string("c_out_3_to_fp16"), val = tensor([[[0x1.94cp-3]]])]; + tensor x_pred_3_cast_fp16 = transpose(perm = x_pred_3_perm_0, x = x_225_cast_fp16)[name = string("transpose_282")]; + tensor var_1542_cast_fp16 = mul(x = c_out_3_to_fp16, y = x_pred_3_cast_fp16)[name = string("op_1542_cast_fp16")]; + tensor x_mid_dn_1_cast_fp16 = add(x = var_1541_cast_fp16, y = var_1542_cast_fp16)[name = string("x_mid_dn_1_cast_fp16")]; + tensor var_1545_cast_fp16 = sub(x = x_noisy_3_cast_fp16, y = x_mid_dn_1_cast_fp16)[name = string("op_1545_cast_fp16")]; + tensor _inversed_d_mid_1_y_0_to_fp16 = const()[name = string("_inversed_d_mid_1_y_0_to_fp16"), val = tensor([0x1.4ap-1])]; + tensor _inversed_d_mid_1_cast_fp16 = mul(x = var_1545_cast_fp16, y = _inversed_d_mid_1_y_0_to_fp16)[name = string("_inversed_d_mid_1_cast_fp16")]; + fp16 var_1554_to_fp16 = const()[name = string("op_1554_to_fp16"), val = fp16(-0x1.72cp+1)]; + tensor var_1555_cast_fp16 = mul(x = _inversed_d_mid_1_cast_fp16, y = var_1554_to_fp16)[name = string("op_1555_cast_fp16")]; + tensor x_227_cast_fp16 = add(x = x_noisy_1_cast_fp16, y = var_1555_cast_fp16)[name = string("x_227_cast_fp16")]; + tensor var_1560_begin_0 = const()[name = string("op_1560_begin_0"), val = tensor([0, 0, 0, 0])]; + tensor var_1560_end_0 = const()[name = string("op_1560_end_0"), val = tensor([1, 1, 1, 256])]; + tensor var_1560_end_mask_0 = const()[name = string("op_1560_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_1560_squeeze_mask_0 = const()[name = string("op_1560_squeeze_mask_0"), val = tensor([true, false, false, false])]; + string noises_aux_to_fp16_dtype_0 = const()[name = string("noises_aux_to_fp16_dtype_0"), val = string("fp16")]; + tensor noises_aux_to_fp16 = cast(dtype = noises_aux_to_fp16_dtype_0, x = noises_aux)[name = string("cast_193")]; + tensor var_1560_cast_fp16 = slice_by_index(begin = var_1560_begin_0, end = var_1560_end_0, end_mask = var_1560_end_mask_0, squeeze_mask = var_1560_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_1560_cast_fp16")]; + fp16 var_1563_to_fp16 = const()[name = string("op_1563_to_fp16"), val = fp16(0x1.18cp-1)]; + tensor var_1564_cast_fp16 = mul(x = var_1560_cast_fp16, y = var_1563_to_fp16)[name = string("op_1564_cast_fp16")]; + tensor x_noisy_5_cast_fp16 = add(x = x_227_cast_fp16, y = var_1564_cast_fp16)[name = string("x_noisy_5_cast_fp16")]; + int32 var_1588 = const()[name = string("op_1588"), val = int32(-1)]; + tensor c_in_5_to_fp16 = const()[name = string("c_in_5_to_fp16"), val = tensor([[[0x1.bp+0]]])]; + tensor x_237_cast_fp16 = mul(x = c_in_5_to_fp16, y = x_noisy_5_cast_fp16)[name = string("x_237_cast_fp16")]; + int32 x_233_axis_0 = const()[name = string("x_233_axis_0"), val = int32(0)]; + tensor var_1974_to_fp16 = const()[name = string("op_1974_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49349184)))]; + tensor x_233_cast_fp16 = stack(axis = x_233_axis_0, values = (var_1974_to_fp16, var_423_cast_fp16))[name = string("x_233_cast_fp16")]; + tensor var_1979 = const()[name = string("op_1979"), val = tensor([1, 2, 0])]; + tensor input_135_axes_0 = const()[name = string("input_135_axes_0"), val = tensor([2])]; + bool input_135_keep_dims_0 = const()[name = string("input_135_keep_dims_0"), val = bool(false)]; + tensor x_235_cast_fp16 = transpose(perm = var_1979, x = x_233_cast_fp16)[name = string("transpose_281")]; + tensor input_135_cast_fp16 = reduce_sum(axes = input_135_axes_0, keep_dims = input_135_keep_dims_0, x = x_235_cast_fp16)[name = string("input_135_cast_fp16")]; + tensor linear_52_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_135_cast_fp16)[name = string("linear_52_cast_fp16")]; + string input_139_mode_0 = const()[name = string("input_139_mode_0"), val = string("EXACT")]; + tensor input_139_cast_fp16 = gelu(mode = input_139_mode_0, x = linear_52_cast_fp16)[name = string("input_139_cast_fp16")]; + tensor linear_53_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_139_cast_fp16)[name = string("linear_53_cast_fp16")]; + string mapping_9_mode_0 = const()[name = string("mapping_9_mode_0"), val = string("EXACT")]; + tensor mapping_9_cast_fp16 = gelu(mode = mapping_9_mode_0, x = linear_53_cast_fp16)[name = string("mapping_9_cast_fp16")]; + tensor var_1989_reps_0 = const()[name = string("op_1989_reps_0"), val = tensor([1, 128, 1])]; + tensor var_1989_cast_fp16 = tile(reps = var_1989_reps_0, x = x_237_cast_fp16)[name = string("op_1989_cast_fp16")]; + bool x_239_interleave_0 = const()[name = string("x_239_interleave_0"), val = bool(false)]; + tensor x_239_cast_fp16 = concat(axis = var_1588, interleave = x_239_interleave_0, values = (var_1989_cast_fp16, embedding_to_fp16))[name = string("x_239_cast_fp16")]; + tensor var_1992_axes_0 = const()[name = string("op_1992_axes_0"), val = tensor([1])]; + tensor var_1992_cast_fp16 = expand_dims(axes = var_1992_axes_0, x = mapping_9_cast_fp16)[name = string("op_1992_cast_fp16")]; + tensor mapping_11_reps_0 = const()[name = string("mapping_11_reps_0"), val = tensor([1, 128, 1])]; + tensor mapping_11_cast_fp16 = tile(reps = mapping_11_reps_0, x = var_1992_cast_fp16)[name = string("mapping_11_cast_fp16")]; + tensor x_241_cast_fp16 = add(x = x_239_cast_fp16, y = mapping_11_cast_fp16)[name = string("x_241_cast_fp16")]; + tensor var_2004_split_sizes_0 = const()[name = string("op_2004_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2004_axis_0 = const()[name = string("op_2004_axis_0"), val = int32(1)]; + tensor var_2004_cast_fp16_0, tensor var_2004_cast_fp16_1 = split(axis = var_2004_axis_0, split_sizes = var_2004_split_sizes_0, x = h_3_cast_fp16)[name = string("op_2004_cast_fp16")]; + tensor gamma_51_perm_0 = const()[name = string("gamma_51_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_51_perm_0 = const()[name = string("beta_51_perm_0"), val = tensor([0, -1, 1])]; + tensor x_245_axes_0 = const()[name = string("x_245_axes_0"), val = tensor([-1])]; + fp16 var_1584_to_fp16 = const()[name = string("op_1584_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_245_cast_fp16 = layer_norm(axes = x_245_axes_0, epsilon = var_1584_to_fp16, x = x_241_cast_fp16)[name = string("x_245_cast_fp16")]; + fp16 var_2010_promoted_to_fp16 = const()[name = string("op_2010_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_51_cast_fp16 = transpose(perm = gamma_51_perm_0, x = var_2004_cast_fp16_0)[name = string("transpose_280")]; + tensor var_2011_cast_fp16 = add(x = gamma_51_cast_fp16, y = var_2010_promoted_to_fp16)[name = string("op_2011_cast_fp16")]; + tensor var_2012_cast_fp16 = mul(x = var_2011_cast_fp16, y = x_245_cast_fp16)[name = string("op_2012_cast_fp16")]; + tensor beta_51_cast_fp16 = transpose(perm = beta_51_perm_0, x = var_2004_cast_fp16_1)[name = string("transpose_279")]; + tensor x_247_cast_fp16 = add(x = var_2012_cast_fp16, y = beta_51_cast_fp16)[name = string("x_247_cast_fp16")]; + tensor var_2023_split_sizes_0 = const()[name = string("op_2023_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2023_axis_0 = const()[name = string("op_2023_axis_0"), val = int32(1)]; + tensor var_2023_cast_fp16_0, tensor var_2023_cast_fp16_1 = split(axis = var_2023_axis_0, split_sizes = var_2023_split_sizes_0, x = h_7_cast_fp16)[name = string("op_2023_cast_fp16")]; + tensor gamma_55_perm_0 = const()[name = string("gamma_55_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_55_perm_0 = const()[name = string("beta_55_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2029_promoted_to_fp16 = const()[name = string("op_2029_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_55_cast_fp16 = transpose(perm = gamma_55_perm_0, x = var_2023_cast_fp16_0)[name = string("transpose_278")]; + tensor var_2030_cast_fp16 = add(x = gamma_55_cast_fp16, y = var_2029_promoted_to_fp16)[name = string("op_2030_cast_fp16")]; + tensor var_2031_cast_fp16 = mul(x = var_2030_cast_fp16, y = x_245_cast_fp16)[name = string("op_2031_cast_fp16")]; + tensor beta_55_cast_fp16 = transpose(perm = beta_55_perm_0, x = var_2023_cast_fp16_1)[name = string("transpose_277")]; + tensor x_253_cast_fp16 = add(x = var_2031_cast_fp16, y = beta_55_cast_fp16)[name = string("x_253_cast_fp16")]; + tensor linear_56_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_247_cast_fp16)[name = string("linear_56_cast_fp16")]; + tensor linear_57_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_253_cast_fp16)[name = string("linear_57_cast_fp16")]; + tensor var_2037_split_sizes_0 = const()[name = string("op_2037_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2037_axis_0 = const()[name = string("op_2037_axis_0"), val = int32(-1)]; + tensor var_2037_cast_fp16_0, tensor var_2037_cast_fp16_1 = split(axis = var_2037_axis_0, split_sizes = var_2037_split_sizes_0, x = linear_57_cast_fp16)[name = string("op_2037_cast_fp16")]; + tensor var_2045 = const()[name = string("op_2045"), val = tensor([1, 128, 8, 64])]; + tensor x_257_cast_fp16 = reshape(shape = var_2045, x = linear_56_cast_fp16)[name = string("x_257_cast_fp16")]; + tensor var_2055 = const()[name = string("op_2055"), val = tensor([1, 128, 8, 64])]; + tensor x_261_cast_fp16 = reshape(shape = var_2055, x = var_2037_cast_fp16_0)[name = string("x_261_cast_fp16")]; + tensor var_2065 = const()[name = string("op_2065"), val = tensor([1, 128, 8, 64])]; + tensor x_265_cast_fp16 = reshape(shape = var_2065, x = var_2037_cast_fp16_1)[name = string("x_265_cast_fp16")]; + tensor var_2067 = const()[name = string("op_2067"), val = tensor([0, 2, 1, 3])]; + bool sim_25_transpose_x_0 = const()[name = string("sim_25_transpose_x_0"), val = bool(false)]; + bool sim_25_transpose_y_0 = const()[name = string("sim_25_transpose_y_0"), val = bool(false)]; + tensor transpose_84_perm_0 = const()[name = string("transpose_84_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_85_perm_0 = const()[name = string("transpose_85_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_85 = transpose(perm = transpose_85_perm_0, x = x_261_cast_fp16)[name = string("transpose_274")]; + tensor transpose_84 = transpose(perm = transpose_84_perm_0, x = x_257_cast_fp16)[name = string("transpose_275")]; + tensor sim_25_cast_fp16 = matmul(transpose_x = sim_25_transpose_x_0, transpose_y = sim_25_transpose_y_0, x = transpose_84, y = transpose_85)[name = string("sim_25_cast_fp16")]; + fp16 var_2071_to_fp16 = const()[name = string("op_2071_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_27_cast_fp16 = mul(x = sim_25_cast_fp16, y = var_2071_to_fp16)[name = string("sim_27_cast_fp16")]; + tensor attn_13_cast_fp16 = softmax(axis = var_1588, x = sim_27_cast_fp16)[name = string("attn_13_cast_fp16")]; + bool x_267_transpose_x_0 = const()[name = string("x_267_transpose_x_0"), val = bool(false)]; + bool x_267_transpose_y_0 = const()[name = string("x_267_transpose_y_0"), val = bool(false)]; + tensor v_13_cast_fp16 = transpose(perm = var_2067, x = x_265_cast_fp16)[name = string("transpose_276")]; + tensor x_267_cast_fp16 = matmul(transpose_x = x_267_transpose_x_0, transpose_y = x_267_transpose_y_0, x = attn_13_cast_fp16, y = v_13_cast_fp16)[name = string("x_267_cast_fp16")]; + tensor var_2093 = const()[name = string("op_2093"), val = tensor([0, 2, 1, 3])]; + tensor var_2095 = const()[name = string("op_2095"), val = tensor([1, 128, 512])]; + tensor x_269_cast_fp16 = transpose(perm = var_2093, x = x_267_cast_fp16)[name = string("transpose_273")]; + tensor input_151_cast_fp16 = reshape(shape = var_2095, x = x_269_cast_fp16)[name = string("input_151_cast_fp16")]; + tensor linear_58_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_151_cast_fp16)[name = string("linear_58_cast_fp16")]; + tensor input_153_cast_fp16 = add(x = linear_58_cast_fp16, y = x_241_cast_fp16)[name = string("input_153_cast_fp16")]; + tensor linear_59_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_153_cast_fp16)[name = string("linear_59_cast_fp16")]; + string input_157_mode_0 = const()[name = string("input_157_mode_0"), val = string("EXACT")]; + tensor input_157_cast_fp16 = gelu(mode = input_157_mode_0, x = linear_59_cast_fp16)[name = string("input_157_cast_fp16")]; + tensor linear_60_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_157_cast_fp16)[name = string("linear_60_cast_fp16")]; + tensor x_271_cast_fp16 = add(x = linear_60_cast_fp16, y = input_153_cast_fp16)[name = string("x_271_cast_fp16")]; + tensor x_273_cast_fp16 = add(x = x_271_cast_fp16, y = mapping_11_cast_fp16)[name = string("x_273_cast_fp16")]; + tensor var_2111_split_sizes_0 = const()[name = string("op_2111_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2111_axis_0 = const()[name = string("op_2111_axis_0"), val = int32(1)]; + tensor var_2111_cast_fp16_0, tensor var_2111_cast_fp16_1 = split(axis = var_2111_axis_0, split_sizes = var_2111_split_sizes_0, x = h_11_cast_fp16)[name = string("op_2111_cast_fp16")]; + tensor gamma_59_perm_0 = const()[name = string("gamma_59_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_59_perm_0 = const()[name = string("beta_59_perm_0"), val = tensor([0, -1, 1])]; + tensor x_277_axes_0 = const()[name = string("x_277_axes_0"), val = tensor([-1])]; + tensor x_277_cast_fp16 = layer_norm(axes = x_277_axes_0, epsilon = var_1584_to_fp16, x = x_273_cast_fp16)[name = string("x_277_cast_fp16")]; + fp16 var_2117_promoted_to_fp16 = const()[name = string("op_2117_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_59_cast_fp16 = transpose(perm = gamma_59_perm_0, x = var_2111_cast_fp16_0)[name = string("transpose_272")]; + tensor var_2118_cast_fp16 = add(x = gamma_59_cast_fp16, y = var_2117_promoted_to_fp16)[name = string("op_2118_cast_fp16")]; + tensor var_2119_cast_fp16 = mul(x = var_2118_cast_fp16, y = x_277_cast_fp16)[name = string("op_2119_cast_fp16")]; + tensor beta_59_cast_fp16 = transpose(perm = beta_59_perm_0, x = var_2111_cast_fp16_1)[name = string("transpose_271")]; + tensor x_279_cast_fp16 = add(x = var_2119_cast_fp16, y = beta_59_cast_fp16)[name = string("x_279_cast_fp16")]; + tensor var_2130_split_sizes_0 = const()[name = string("op_2130_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2130_axis_0 = const()[name = string("op_2130_axis_0"), val = int32(1)]; + tensor var_2130_cast_fp16_0, tensor var_2130_cast_fp16_1 = split(axis = var_2130_axis_0, split_sizes = var_2130_split_sizes_0, x = h_15_cast_fp16)[name = string("op_2130_cast_fp16")]; + tensor gamma_63_perm_0 = const()[name = string("gamma_63_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_63_perm_0 = const()[name = string("beta_63_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2136_promoted_to_fp16 = const()[name = string("op_2136_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_63_cast_fp16 = transpose(perm = gamma_63_perm_0, x = var_2130_cast_fp16_0)[name = string("transpose_270")]; + tensor var_2137_cast_fp16 = add(x = gamma_63_cast_fp16, y = var_2136_promoted_to_fp16)[name = string("op_2137_cast_fp16")]; + tensor var_2138_cast_fp16 = mul(x = var_2137_cast_fp16, y = x_277_cast_fp16)[name = string("op_2138_cast_fp16")]; + tensor beta_63_cast_fp16 = transpose(perm = beta_63_perm_0, x = var_2130_cast_fp16_1)[name = string("transpose_269")]; + tensor x_285_cast_fp16 = add(x = var_2138_cast_fp16, y = beta_63_cast_fp16)[name = string("x_285_cast_fp16")]; + tensor linear_63_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_279_cast_fp16)[name = string("linear_63_cast_fp16")]; + tensor linear_64_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_285_cast_fp16)[name = string("linear_64_cast_fp16")]; + tensor var_2144_split_sizes_0 = const()[name = string("op_2144_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2144_axis_0 = const()[name = string("op_2144_axis_0"), val = int32(-1)]; + tensor var_2144_cast_fp16_0, tensor var_2144_cast_fp16_1 = split(axis = var_2144_axis_0, split_sizes = var_2144_split_sizes_0, x = linear_64_cast_fp16)[name = string("op_2144_cast_fp16")]; + tensor var_2152 = const()[name = string("op_2152"), val = tensor([1, 128, 8, 64])]; + tensor x_289_cast_fp16 = reshape(shape = var_2152, x = linear_63_cast_fp16)[name = string("x_289_cast_fp16")]; + tensor var_2162 = const()[name = string("op_2162"), val = tensor([1, 128, 8, 64])]; + tensor x_293_cast_fp16 = reshape(shape = var_2162, x = var_2144_cast_fp16_0)[name = string("x_293_cast_fp16")]; + tensor var_2172 = const()[name = string("op_2172"), val = tensor([1, 128, 8, 64])]; + tensor x_297_cast_fp16 = reshape(shape = var_2172, x = var_2144_cast_fp16_1)[name = string("x_297_cast_fp16")]; + tensor var_2174 = const()[name = string("op_2174"), val = tensor([0, 2, 1, 3])]; + bool sim_29_transpose_x_0 = const()[name = string("sim_29_transpose_x_0"), val = bool(false)]; + bool sim_29_transpose_y_0 = const()[name = string("sim_29_transpose_y_0"), val = bool(false)]; + tensor transpose_86_perm_0 = const()[name = string("transpose_86_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_87_perm_0 = const()[name = string("transpose_87_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_87 = transpose(perm = transpose_87_perm_0, x = x_293_cast_fp16)[name = string("transpose_266")]; + tensor transpose_86 = transpose(perm = transpose_86_perm_0, x = x_289_cast_fp16)[name = string("transpose_267")]; + tensor sim_29_cast_fp16 = matmul(transpose_x = sim_29_transpose_x_0, transpose_y = sim_29_transpose_y_0, x = transpose_86, y = transpose_87)[name = string("sim_29_cast_fp16")]; + fp16 var_2178_to_fp16 = const()[name = string("op_2178_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_31_cast_fp16 = mul(x = sim_29_cast_fp16, y = var_2178_to_fp16)[name = string("sim_31_cast_fp16")]; + tensor attn_15_cast_fp16 = softmax(axis = var_1588, x = sim_31_cast_fp16)[name = string("attn_15_cast_fp16")]; + bool x_299_transpose_x_0 = const()[name = string("x_299_transpose_x_0"), val = bool(false)]; + bool x_299_transpose_y_0 = const()[name = string("x_299_transpose_y_0"), val = bool(false)]; + tensor v_15_cast_fp16 = transpose(perm = var_2174, x = x_297_cast_fp16)[name = string("transpose_268")]; + tensor x_299_cast_fp16 = matmul(transpose_x = x_299_transpose_x_0, transpose_y = x_299_transpose_y_0, x = attn_15_cast_fp16, y = v_15_cast_fp16)[name = string("x_299_cast_fp16")]; + tensor var_2200 = const()[name = string("op_2200"), val = tensor([0, 2, 1, 3])]; + tensor var_2202 = const()[name = string("op_2202"), val = tensor([1, 128, 512])]; + tensor x_301_cast_fp16 = transpose(perm = var_2200, x = x_299_cast_fp16)[name = string("transpose_265")]; + tensor input_167_cast_fp16 = reshape(shape = var_2202, x = x_301_cast_fp16)[name = string("input_167_cast_fp16")]; + tensor linear_65_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_167_cast_fp16)[name = string("linear_65_cast_fp16")]; + tensor input_169_cast_fp16 = add(x = linear_65_cast_fp16, y = x_273_cast_fp16)[name = string("input_169_cast_fp16")]; + tensor linear_66_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_169_cast_fp16)[name = string("linear_66_cast_fp16")]; + string input_173_mode_0 = const()[name = string("input_173_mode_0"), val = string("EXACT")]; + tensor input_173_cast_fp16 = gelu(mode = input_173_mode_0, x = linear_66_cast_fp16)[name = string("input_173_cast_fp16")]; + tensor linear_67_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_173_cast_fp16)[name = string("linear_67_cast_fp16")]; + tensor x_303_cast_fp16 = add(x = linear_67_cast_fp16, y = input_169_cast_fp16)[name = string("x_303_cast_fp16")]; + tensor x_305_cast_fp16 = add(x = x_303_cast_fp16, y = mapping_11_cast_fp16)[name = string("x_305_cast_fp16")]; + tensor var_2218_split_sizes_0 = const()[name = string("op_2218_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2218_axis_0 = const()[name = string("op_2218_axis_0"), val = int32(1)]; + tensor var_2218_cast_fp16_0, tensor var_2218_cast_fp16_1 = split(axis = var_2218_axis_0, split_sizes = var_2218_split_sizes_0, x = h_19_cast_fp16)[name = string("op_2218_cast_fp16")]; + tensor gamma_67_perm_0 = const()[name = string("gamma_67_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_67_perm_0 = const()[name = string("beta_67_perm_0"), val = tensor([0, -1, 1])]; + tensor x_309_axes_0 = const()[name = string("x_309_axes_0"), val = tensor([-1])]; + tensor x_309_cast_fp16 = layer_norm(axes = x_309_axes_0, epsilon = var_1584_to_fp16, x = x_305_cast_fp16)[name = string("x_309_cast_fp16")]; + fp16 var_2224_promoted_to_fp16 = const()[name = string("op_2224_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_67_cast_fp16 = transpose(perm = gamma_67_perm_0, x = var_2218_cast_fp16_0)[name = string("transpose_264")]; + tensor var_2225_cast_fp16 = add(x = gamma_67_cast_fp16, y = var_2224_promoted_to_fp16)[name = string("op_2225_cast_fp16")]; + tensor var_2226_cast_fp16 = mul(x = var_2225_cast_fp16, y = x_309_cast_fp16)[name = string("op_2226_cast_fp16")]; + tensor beta_67_cast_fp16 = transpose(perm = beta_67_perm_0, x = var_2218_cast_fp16_1)[name = string("transpose_263")]; + tensor x_311_cast_fp16 = add(x = var_2226_cast_fp16, y = beta_67_cast_fp16)[name = string("x_311_cast_fp16")]; + tensor var_2237_split_sizes_0 = const()[name = string("op_2237_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2237_axis_0 = const()[name = string("op_2237_axis_0"), val = int32(1)]; + tensor var_2237_cast_fp16_0, tensor var_2237_cast_fp16_1 = split(axis = var_2237_axis_0, split_sizes = var_2237_split_sizes_0, x = h_23_cast_fp16)[name = string("op_2237_cast_fp16")]; + tensor gamma_71_perm_0 = const()[name = string("gamma_71_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_71_perm_0 = const()[name = string("beta_71_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2243_promoted_to_fp16 = const()[name = string("op_2243_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_71_cast_fp16 = transpose(perm = gamma_71_perm_0, x = var_2237_cast_fp16_0)[name = string("transpose_262")]; + tensor var_2244_cast_fp16 = add(x = gamma_71_cast_fp16, y = var_2243_promoted_to_fp16)[name = string("op_2244_cast_fp16")]; + tensor var_2245_cast_fp16 = mul(x = var_2244_cast_fp16, y = x_309_cast_fp16)[name = string("op_2245_cast_fp16")]; + tensor beta_71_cast_fp16 = transpose(perm = beta_71_perm_0, x = var_2237_cast_fp16_1)[name = string("transpose_261")]; + tensor x_317_cast_fp16 = add(x = var_2245_cast_fp16, y = beta_71_cast_fp16)[name = string("x_317_cast_fp16")]; + tensor linear_70_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_311_cast_fp16)[name = string("linear_70_cast_fp16")]; + tensor linear_71_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_317_cast_fp16)[name = string("linear_71_cast_fp16")]; + tensor var_2251_split_sizes_0 = const()[name = string("op_2251_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2251_axis_0 = const()[name = string("op_2251_axis_0"), val = int32(-1)]; + tensor var_2251_cast_fp16_0, tensor var_2251_cast_fp16_1 = split(axis = var_2251_axis_0, split_sizes = var_2251_split_sizes_0, x = linear_71_cast_fp16)[name = string("op_2251_cast_fp16")]; + tensor var_2259 = const()[name = string("op_2259"), val = tensor([1, 128, 8, 64])]; + tensor x_321_cast_fp16 = reshape(shape = var_2259, x = linear_70_cast_fp16)[name = string("x_321_cast_fp16")]; + tensor var_2269 = const()[name = string("op_2269"), val = tensor([1, 128, 8, 64])]; + tensor x_325_cast_fp16 = reshape(shape = var_2269, x = var_2251_cast_fp16_0)[name = string("x_325_cast_fp16")]; + tensor var_2279 = const()[name = string("op_2279"), val = tensor([1, 128, 8, 64])]; + tensor x_329_cast_fp16 = reshape(shape = var_2279, x = var_2251_cast_fp16_1)[name = string("x_329_cast_fp16")]; + tensor var_2281 = const()[name = string("op_2281"), val = tensor([0, 2, 1, 3])]; + bool sim_33_transpose_x_0 = const()[name = string("sim_33_transpose_x_0"), val = bool(false)]; + bool sim_33_transpose_y_0 = const()[name = string("sim_33_transpose_y_0"), val = bool(false)]; + tensor transpose_88_perm_0 = const()[name = string("transpose_88_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_89_perm_0 = const()[name = string("transpose_89_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_89 = transpose(perm = transpose_89_perm_0, x = x_325_cast_fp16)[name = string("transpose_258")]; + tensor transpose_88 = transpose(perm = transpose_88_perm_0, x = x_321_cast_fp16)[name = string("transpose_259")]; + tensor sim_33_cast_fp16 = matmul(transpose_x = sim_33_transpose_x_0, transpose_y = sim_33_transpose_y_0, x = transpose_88, y = transpose_89)[name = string("sim_33_cast_fp16")]; + fp16 var_2285_to_fp16 = const()[name = string("op_2285_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_35_cast_fp16 = mul(x = sim_33_cast_fp16, y = var_2285_to_fp16)[name = string("sim_35_cast_fp16")]; + tensor attn_17_cast_fp16 = softmax(axis = var_1588, x = sim_35_cast_fp16)[name = string("attn_17_cast_fp16")]; + bool x_331_transpose_x_0 = const()[name = string("x_331_transpose_x_0"), val = bool(false)]; + bool x_331_transpose_y_0 = const()[name = string("x_331_transpose_y_0"), val = bool(false)]; + tensor v_17_cast_fp16 = transpose(perm = var_2281, x = x_329_cast_fp16)[name = string("transpose_260")]; + tensor x_331_cast_fp16 = matmul(transpose_x = x_331_transpose_x_0, transpose_y = x_331_transpose_y_0, x = attn_17_cast_fp16, y = v_17_cast_fp16)[name = string("x_331_cast_fp16")]; + tensor var_2307 = const()[name = string("op_2307"), val = tensor([0, 2, 1, 3])]; + tensor var_2309 = const()[name = string("op_2309"), val = tensor([1, 128, 512])]; + tensor x_333_cast_fp16 = transpose(perm = var_2307, x = x_331_cast_fp16)[name = string("transpose_257")]; + tensor input_183_cast_fp16 = reshape(shape = var_2309, x = x_333_cast_fp16)[name = string("input_183_cast_fp16")]; + tensor linear_72_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_183_cast_fp16)[name = string("linear_72_cast_fp16")]; + tensor input_185_cast_fp16 = add(x = linear_72_cast_fp16, y = x_305_cast_fp16)[name = string("input_185_cast_fp16")]; + tensor linear_73_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_185_cast_fp16)[name = string("linear_73_cast_fp16")]; + string input_189_mode_0 = const()[name = string("input_189_mode_0"), val = string("EXACT")]; + tensor input_189_cast_fp16 = gelu(mode = input_189_mode_0, x = linear_73_cast_fp16)[name = string("input_189_cast_fp16")]; + tensor linear_74_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_189_cast_fp16)[name = string("linear_74_cast_fp16")]; + tensor x_335_cast_fp16 = add(x = linear_74_cast_fp16, y = input_185_cast_fp16)[name = string("x_335_cast_fp16")]; + tensor var_2318_axes_0 = const()[name = string("op_2318_axes_0"), val = tensor([1])]; + bool var_2318_keep_dims_0 = const()[name = string("op_2318_keep_dims_0"), val = bool(false)]; + tensor var_2318_cast_fp16 = reduce_mean(axes = var_2318_axes_0, keep_dims = var_2318_keep_dims_0, x = x_335_cast_fp16)[name = string("op_2318_cast_fp16")]; + tensor x_337_axes_0 = const()[name = string("x_337_axes_0"), val = tensor([1])]; + tensor x_337_cast_fp16 = expand_dims(axes = x_337_axes_0, x = var_2318_cast_fp16)[name = string("x_337_cast_fp16")]; + tensor var_2320 = const()[name = string("op_2320"), val = tensor([0, 2, 1])]; + string x_339_pad_type_0 = const()[name = string("x_339_pad_type_0"), val = string("valid")]; + tensor x_339_strides_0 = const()[name = string("x_339_strides_0"), val = tensor([1])]; + tensor x_339_pad_0 = const()[name = string("x_339_pad_0"), val = tensor([0, 0])]; + tensor x_339_dilations_0 = const()[name = string("x_339_dilations_0"), val = tensor([1])]; + int32 x_339_groups_0 = const()[name = string("x_339_groups_0"), val = int32(1)]; + tensor input_191_cast_fp16 = transpose(perm = var_2320, x = x_337_cast_fp16)[name = string("transpose_256")]; + tensor x_339_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_339_dilations_0, groups = x_339_groups_0, pad = x_339_pad_0, pad_type = x_339_pad_type_0, strides = x_339_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_191_cast_fp16)[name = string("x_339_cast_fp16")]; + tensor x_pred_5_perm_0 = const()[name = string("x_pred_5_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_5_to_fp16 = const()[name = string("c_skip_5_to_fp16"), val = tensor([[[0x1.cf4p-4]]])]; + tensor var_2328_cast_fp16 = mul(x = c_skip_5_to_fp16, y = x_noisy_5_cast_fp16)[name = string("op_2328_cast_fp16")]; + tensor c_out_5_to_fp16 = const()[name = string("c_out_5_to_fp16"), val = tensor([[[0x1.804p-3]]])]; + tensor x_pred_5_cast_fp16 = transpose(perm = x_pred_5_perm_0, x = x_339_cast_fp16)[name = string("transpose_255")]; + tensor var_2329_cast_fp16 = mul(x = c_out_5_to_fp16, y = x_pred_5_cast_fp16)[name = string("op_2329_cast_fp16")]; + tensor x_dn_3_cast_fp16 = add(x = var_2328_cast_fp16, y = var_2329_cast_fp16)[name = string("x_dn_3_cast_fp16")]; + tensor var_2332_cast_fp16 = sub(x = x_noisy_5_cast_fp16, y = x_dn_3_cast_fp16)[name = string("op_2332_cast_fp16")]; + tensor _inversed_d_3_y_0_to_fp16 = const()[name = string("_inversed_d_3_y_0_to_fp16"), val = tensor([0x1.cacp+0])]; + tensor _inversed_d_3_cast_fp16 = mul(x = var_2332_cast_fp16, y = _inversed_d_3_y_0_to_fp16)[name = string("_inversed_d_3_cast_fp16")]; + fp16 var_2341_to_fp16 = const()[name = string("op_2341_to_fp16"), val = fp16(-0x1.19p-2)]; + tensor var_2342_cast_fp16 = mul(x = _inversed_d_3_cast_fp16, y = var_2341_to_fp16)[name = string("op_2342_cast_fp16")]; + tensor x_noisy_7_cast_fp16 = add(x = x_noisy_5_cast_fp16, y = var_2342_cast_fp16)[name = string("x_noisy_7_cast_fp16")]; + int32 var_2354 = const()[name = string("op_2354"), val = int32(-1)]; + tensor c_in_7_to_fp16 = const()[name = string("c_in_7_to_fp16"), val = tensor([[[0x1.718p+1]]])]; + tensor x_349_cast_fp16 = mul(x = c_in_7_to_fp16, y = x_noisy_7_cast_fp16)[name = string("x_349_cast_fp16")]; + int32 x_345_axis_0 = const()[name = string("x_345_axis_0"), val = int32(0)]; + tensor var_2740_to_fp16 = const()[name = string("op_2740_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49351296)))]; + tensor x_345_cast_fp16 = stack(axis = x_345_axis_0, values = (var_2740_to_fp16, var_423_cast_fp16))[name = string("x_345_cast_fp16")]; + tensor var_2745 = const()[name = string("op_2745"), val = tensor([1, 2, 0])]; + tensor input_199_axes_0 = const()[name = string("input_199_axes_0"), val = tensor([2])]; + bool input_199_keep_dims_0 = const()[name = string("input_199_keep_dims_0"), val = bool(false)]; + tensor x_347_cast_fp16 = transpose(perm = var_2745, x = x_345_cast_fp16)[name = string("transpose_254")]; + tensor input_199_cast_fp16 = reduce_sum(axes = input_199_axes_0, keep_dims = input_199_keep_dims_0, x = x_347_cast_fp16)[name = string("input_199_cast_fp16")]; + tensor linear_77_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_199_cast_fp16)[name = string("linear_77_cast_fp16")]; + string input_203_mode_0 = const()[name = string("input_203_mode_0"), val = string("EXACT")]; + tensor input_203_cast_fp16 = gelu(mode = input_203_mode_0, x = linear_77_cast_fp16)[name = string("input_203_cast_fp16")]; + tensor linear_78_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_203_cast_fp16)[name = string("linear_78_cast_fp16")]; + string mapping_13_mode_0 = const()[name = string("mapping_13_mode_0"), val = string("EXACT")]; + tensor mapping_13_cast_fp16 = gelu(mode = mapping_13_mode_0, x = linear_78_cast_fp16)[name = string("mapping_13_cast_fp16")]; + tensor var_2755_reps_0 = const()[name = string("op_2755_reps_0"), val = tensor([1, 128, 1])]; + tensor var_2755_cast_fp16 = tile(reps = var_2755_reps_0, x = x_349_cast_fp16)[name = string("op_2755_cast_fp16")]; + bool x_351_interleave_0 = const()[name = string("x_351_interleave_0"), val = bool(false)]; + tensor x_351_cast_fp16 = concat(axis = var_2354, interleave = x_351_interleave_0, values = (var_2755_cast_fp16, embedding_to_fp16))[name = string("x_351_cast_fp16")]; + tensor var_2758_axes_0 = const()[name = string("op_2758_axes_0"), val = tensor([1])]; + tensor var_2758_cast_fp16 = expand_dims(axes = var_2758_axes_0, x = mapping_13_cast_fp16)[name = string("op_2758_cast_fp16")]; + tensor mapping_15_reps_0 = const()[name = string("mapping_15_reps_0"), val = tensor([1, 128, 1])]; + tensor mapping_15_cast_fp16 = tile(reps = mapping_15_reps_0, x = var_2758_cast_fp16)[name = string("mapping_15_cast_fp16")]; + tensor x_353_cast_fp16 = add(x = x_351_cast_fp16, y = mapping_15_cast_fp16)[name = string("x_353_cast_fp16")]; + tensor var_2770_split_sizes_0 = const()[name = string("op_2770_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2770_axis_0 = const()[name = string("op_2770_axis_0"), val = int32(1)]; + tensor var_2770_cast_fp16_0, tensor var_2770_cast_fp16_1 = split(axis = var_2770_axis_0, split_sizes = var_2770_split_sizes_0, x = h_3_cast_fp16)[name = string("op_2770_cast_fp16")]; + tensor gamma_75_perm_0 = const()[name = string("gamma_75_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_75_perm_0 = const()[name = string("beta_75_perm_0"), val = tensor([0, -1, 1])]; + tensor x_357_axes_0 = const()[name = string("x_357_axes_0"), val = tensor([-1])]; + fp16 var_2350_to_fp16 = const()[name = string("op_2350_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_357_cast_fp16 = layer_norm(axes = x_357_axes_0, epsilon = var_2350_to_fp16, x = x_353_cast_fp16)[name = string("x_357_cast_fp16")]; + fp16 var_2776_promoted_to_fp16 = const()[name = string("op_2776_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_75_cast_fp16 = transpose(perm = gamma_75_perm_0, x = var_2770_cast_fp16_0)[name = string("transpose_253")]; + tensor var_2777_cast_fp16 = add(x = gamma_75_cast_fp16, y = var_2776_promoted_to_fp16)[name = string("op_2777_cast_fp16")]; + tensor var_2778_cast_fp16 = mul(x = var_2777_cast_fp16, y = x_357_cast_fp16)[name = string("op_2778_cast_fp16")]; + tensor beta_75_cast_fp16 = transpose(perm = beta_75_perm_0, x = var_2770_cast_fp16_1)[name = string("transpose_252")]; + tensor x_359_cast_fp16 = add(x = var_2778_cast_fp16, y = beta_75_cast_fp16)[name = string("x_359_cast_fp16")]; + tensor var_2789_split_sizes_0 = const()[name = string("op_2789_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2789_axis_0 = const()[name = string("op_2789_axis_0"), val = int32(1)]; + tensor var_2789_cast_fp16_0, tensor var_2789_cast_fp16_1 = split(axis = var_2789_axis_0, split_sizes = var_2789_split_sizes_0, x = h_7_cast_fp16)[name = string("op_2789_cast_fp16")]; + tensor gamma_79_perm_0 = const()[name = string("gamma_79_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_79_perm_0 = const()[name = string("beta_79_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2795_promoted_to_fp16 = const()[name = string("op_2795_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_79_cast_fp16 = transpose(perm = gamma_79_perm_0, x = var_2789_cast_fp16_0)[name = string("transpose_251")]; + tensor var_2796_cast_fp16 = add(x = gamma_79_cast_fp16, y = var_2795_promoted_to_fp16)[name = string("op_2796_cast_fp16")]; + tensor var_2797_cast_fp16 = mul(x = var_2796_cast_fp16, y = x_357_cast_fp16)[name = string("op_2797_cast_fp16")]; + tensor beta_79_cast_fp16 = transpose(perm = beta_79_perm_0, x = var_2789_cast_fp16_1)[name = string("transpose_250")]; + tensor x_365_cast_fp16 = add(x = var_2797_cast_fp16, y = beta_79_cast_fp16)[name = string("x_365_cast_fp16")]; + tensor linear_81_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_359_cast_fp16)[name = string("linear_81_cast_fp16")]; + tensor linear_82_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_365_cast_fp16)[name = string("linear_82_cast_fp16")]; + tensor var_2803_split_sizes_0 = const()[name = string("op_2803_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2803_axis_0 = const()[name = string("op_2803_axis_0"), val = int32(-1)]; + tensor var_2803_cast_fp16_0, tensor var_2803_cast_fp16_1 = split(axis = var_2803_axis_0, split_sizes = var_2803_split_sizes_0, x = linear_82_cast_fp16)[name = string("op_2803_cast_fp16")]; + tensor var_2811 = const()[name = string("op_2811"), val = tensor([1, 128, 8, 64])]; + tensor x_369_cast_fp16 = reshape(shape = var_2811, x = linear_81_cast_fp16)[name = string("x_369_cast_fp16")]; + tensor var_2821 = const()[name = string("op_2821"), val = tensor([1, 128, 8, 64])]; + tensor x_373_cast_fp16 = reshape(shape = var_2821, x = var_2803_cast_fp16_0)[name = string("x_373_cast_fp16")]; + tensor var_2831 = const()[name = string("op_2831"), val = tensor([1, 128, 8, 64])]; + tensor x_377_cast_fp16 = reshape(shape = var_2831, x = var_2803_cast_fp16_1)[name = string("x_377_cast_fp16")]; + tensor var_2833 = const()[name = string("op_2833"), val = tensor([0, 2, 1, 3])]; + bool sim_37_transpose_x_0 = const()[name = string("sim_37_transpose_x_0"), val = bool(false)]; + bool sim_37_transpose_y_0 = const()[name = string("sim_37_transpose_y_0"), val = bool(false)]; + tensor transpose_90_perm_0 = const()[name = string("transpose_90_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_91_perm_0 = const()[name = string("transpose_91_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_91 = transpose(perm = transpose_91_perm_0, x = x_373_cast_fp16)[name = string("transpose_247")]; + tensor transpose_90 = transpose(perm = transpose_90_perm_0, x = x_369_cast_fp16)[name = string("transpose_248")]; + tensor sim_37_cast_fp16 = matmul(transpose_x = sim_37_transpose_x_0, transpose_y = sim_37_transpose_y_0, x = transpose_90, y = transpose_91)[name = string("sim_37_cast_fp16")]; + fp16 var_2837_to_fp16 = const()[name = string("op_2837_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_39_cast_fp16 = mul(x = sim_37_cast_fp16, y = var_2837_to_fp16)[name = string("sim_39_cast_fp16")]; + tensor attn_19_cast_fp16 = softmax(axis = var_2354, x = sim_39_cast_fp16)[name = string("attn_19_cast_fp16")]; + bool x_379_transpose_x_0 = const()[name = string("x_379_transpose_x_0"), val = bool(false)]; + bool x_379_transpose_y_0 = const()[name = string("x_379_transpose_y_0"), val = bool(false)]; + tensor v_19_cast_fp16 = transpose(perm = var_2833, x = x_377_cast_fp16)[name = string("transpose_249")]; + tensor x_379_cast_fp16 = matmul(transpose_x = x_379_transpose_x_0, transpose_y = x_379_transpose_y_0, x = attn_19_cast_fp16, y = v_19_cast_fp16)[name = string("x_379_cast_fp16")]; + tensor var_2859 = const()[name = string("op_2859"), val = tensor([0, 2, 1, 3])]; + tensor var_2861 = const()[name = string("op_2861"), val = tensor([1, 128, 512])]; + tensor x_381_cast_fp16 = transpose(perm = var_2859, x = x_379_cast_fp16)[name = string("transpose_246")]; + tensor input_215_cast_fp16 = reshape(shape = var_2861, x = x_381_cast_fp16)[name = string("input_215_cast_fp16")]; + tensor linear_83_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_215_cast_fp16)[name = string("linear_83_cast_fp16")]; + tensor input_217_cast_fp16 = add(x = linear_83_cast_fp16, y = x_353_cast_fp16)[name = string("input_217_cast_fp16")]; + tensor linear_84_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_217_cast_fp16)[name = string("linear_84_cast_fp16")]; + string input_221_mode_0 = const()[name = string("input_221_mode_0"), val = string("EXACT")]; + tensor input_221_cast_fp16 = gelu(mode = input_221_mode_0, x = linear_84_cast_fp16)[name = string("input_221_cast_fp16")]; + tensor linear_85_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_221_cast_fp16)[name = string("linear_85_cast_fp16")]; + tensor x_383_cast_fp16 = add(x = linear_85_cast_fp16, y = input_217_cast_fp16)[name = string("x_383_cast_fp16")]; + tensor x_385_cast_fp16 = add(x = x_383_cast_fp16, y = mapping_15_cast_fp16)[name = string("x_385_cast_fp16")]; + tensor var_2877_split_sizes_0 = const()[name = string("op_2877_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2877_axis_0 = const()[name = string("op_2877_axis_0"), val = int32(1)]; + tensor var_2877_cast_fp16_0, tensor var_2877_cast_fp16_1 = split(axis = var_2877_axis_0, split_sizes = var_2877_split_sizes_0, x = h_11_cast_fp16)[name = string("op_2877_cast_fp16")]; + tensor gamma_83_perm_0 = const()[name = string("gamma_83_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_83_perm_0 = const()[name = string("beta_83_perm_0"), val = tensor([0, -1, 1])]; + tensor x_389_axes_0 = const()[name = string("x_389_axes_0"), val = tensor([-1])]; + tensor x_389_cast_fp16 = layer_norm(axes = x_389_axes_0, epsilon = var_2350_to_fp16, x = x_385_cast_fp16)[name = string("x_389_cast_fp16")]; + fp16 var_2883_promoted_to_fp16 = const()[name = string("op_2883_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_83_cast_fp16 = transpose(perm = gamma_83_perm_0, x = var_2877_cast_fp16_0)[name = string("transpose_245")]; + tensor var_2884_cast_fp16 = add(x = gamma_83_cast_fp16, y = var_2883_promoted_to_fp16)[name = string("op_2884_cast_fp16")]; + tensor var_2885_cast_fp16 = mul(x = var_2884_cast_fp16, y = x_389_cast_fp16)[name = string("op_2885_cast_fp16")]; + tensor beta_83_cast_fp16 = transpose(perm = beta_83_perm_0, x = var_2877_cast_fp16_1)[name = string("transpose_244")]; + tensor x_391_cast_fp16 = add(x = var_2885_cast_fp16, y = beta_83_cast_fp16)[name = string("x_391_cast_fp16")]; + tensor var_2896_split_sizes_0 = const()[name = string("op_2896_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2896_axis_0 = const()[name = string("op_2896_axis_0"), val = int32(1)]; + tensor var_2896_cast_fp16_0, tensor var_2896_cast_fp16_1 = split(axis = var_2896_axis_0, split_sizes = var_2896_split_sizes_0, x = h_15_cast_fp16)[name = string("op_2896_cast_fp16")]; + tensor gamma_87_perm_0 = const()[name = string("gamma_87_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_87_perm_0 = const()[name = string("beta_87_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2902_promoted_to_fp16 = const()[name = string("op_2902_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_87_cast_fp16 = transpose(perm = gamma_87_perm_0, x = var_2896_cast_fp16_0)[name = string("transpose_243")]; + tensor var_2903_cast_fp16 = add(x = gamma_87_cast_fp16, y = var_2902_promoted_to_fp16)[name = string("op_2903_cast_fp16")]; + tensor var_2904_cast_fp16 = mul(x = var_2903_cast_fp16, y = x_389_cast_fp16)[name = string("op_2904_cast_fp16")]; + tensor beta_87_cast_fp16 = transpose(perm = beta_87_perm_0, x = var_2896_cast_fp16_1)[name = string("transpose_242")]; + tensor x_397_cast_fp16 = add(x = var_2904_cast_fp16, y = beta_87_cast_fp16)[name = string("x_397_cast_fp16")]; + tensor linear_88_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_391_cast_fp16)[name = string("linear_88_cast_fp16")]; + tensor linear_89_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_397_cast_fp16)[name = string("linear_89_cast_fp16")]; + tensor var_2910_split_sizes_0 = const()[name = string("op_2910_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2910_axis_0 = const()[name = string("op_2910_axis_0"), val = int32(-1)]; + tensor var_2910_cast_fp16_0, tensor var_2910_cast_fp16_1 = split(axis = var_2910_axis_0, split_sizes = var_2910_split_sizes_0, x = linear_89_cast_fp16)[name = string("op_2910_cast_fp16")]; + tensor var_2918 = const()[name = string("op_2918"), val = tensor([1, 128, 8, 64])]; + tensor x_401_cast_fp16 = reshape(shape = var_2918, x = linear_88_cast_fp16)[name = string("x_401_cast_fp16")]; + tensor var_2928 = const()[name = string("op_2928"), val = tensor([1, 128, 8, 64])]; + tensor x_405_cast_fp16 = reshape(shape = var_2928, x = var_2910_cast_fp16_0)[name = string("x_405_cast_fp16")]; + tensor var_2938 = const()[name = string("op_2938"), val = tensor([1, 128, 8, 64])]; + tensor x_409_cast_fp16 = reshape(shape = var_2938, x = var_2910_cast_fp16_1)[name = string("x_409_cast_fp16")]; + tensor var_2940 = const()[name = string("op_2940"), val = tensor([0, 2, 1, 3])]; + bool sim_41_transpose_x_0 = const()[name = string("sim_41_transpose_x_0"), val = bool(false)]; + bool sim_41_transpose_y_0 = const()[name = string("sim_41_transpose_y_0"), val = bool(false)]; + tensor transpose_92_perm_0 = const()[name = string("transpose_92_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_93_perm_0 = const()[name = string("transpose_93_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_93 = transpose(perm = transpose_93_perm_0, x = x_405_cast_fp16)[name = string("transpose_239")]; + tensor transpose_92 = transpose(perm = transpose_92_perm_0, x = x_401_cast_fp16)[name = string("transpose_240")]; + tensor sim_41_cast_fp16 = matmul(transpose_x = sim_41_transpose_x_0, transpose_y = sim_41_transpose_y_0, x = transpose_92, y = transpose_93)[name = string("sim_41_cast_fp16")]; + fp16 var_2944_to_fp16 = const()[name = string("op_2944_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_43_cast_fp16 = mul(x = sim_41_cast_fp16, y = var_2944_to_fp16)[name = string("sim_43_cast_fp16")]; + tensor attn_21_cast_fp16 = softmax(axis = var_2354, x = sim_43_cast_fp16)[name = string("attn_21_cast_fp16")]; + bool x_411_transpose_x_0 = const()[name = string("x_411_transpose_x_0"), val = bool(false)]; + bool x_411_transpose_y_0 = const()[name = string("x_411_transpose_y_0"), val = bool(false)]; + tensor v_21_cast_fp16 = transpose(perm = var_2940, x = x_409_cast_fp16)[name = string("transpose_241")]; + tensor x_411_cast_fp16 = matmul(transpose_x = x_411_transpose_x_0, transpose_y = x_411_transpose_y_0, x = attn_21_cast_fp16, y = v_21_cast_fp16)[name = string("x_411_cast_fp16")]; + tensor var_2966 = const()[name = string("op_2966"), val = tensor([0, 2, 1, 3])]; + tensor var_2968 = const()[name = string("op_2968"), val = tensor([1, 128, 512])]; + tensor x_413_cast_fp16 = transpose(perm = var_2966, x = x_411_cast_fp16)[name = string("transpose_238")]; + tensor input_231_cast_fp16 = reshape(shape = var_2968, x = x_413_cast_fp16)[name = string("input_231_cast_fp16")]; + tensor linear_90_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_231_cast_fp16)[name = string("linear_90_cast_fp16")]; + tensor input_233_cast_fp16 = add(x = linear_90_cast_fp16, y = x_385_cast_fp16)[name = string("input_233_cast_fp16")]; + tensor linear_91_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_233_cast_fp16)[name = string("linear_91_cast_fp16")]; + string input_237_mode_0 = const()[name = string("input_237_mode_0"), val = string("EXACT")]; + tensor input_237_cast_fp16 = gelu(mode = input_237_mode_0, x = linear_91_cast_fp16)[name = string("input_237_cast_fp16")]; + tensor linear_92_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_237_cast_fp16)[name = string("linear_92_cast_fp16")]; + tensor x_415_cast_fp16 = add(x = linear_92_cast_fp16, y = input_233_cast_fp16)[name = string("x_415_cast_fp16")]; + tensor x_417_cast_fp16 = add(x = x_415_cast_fp16, y = mapping_15_cast_fp16)[name = string("x_417_cast_fp16")]; + tensor var_2984_split_sizes_0 = const()[name = string("op_2984_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2984_axis_0 = const()[name = string("op_2984_axis_0"), val = int32(1)]; + tensor var_2984_cast_fp16_0, tensor var_2984_cast_fp16_1 = split(axis = var_2984_axis_0, split_sizes = var_2984_split_sizes_0, x = h_19_cast_fp16)[name = string("op_2984_cast_fp16")]; + tensor gamma_91_perm_0 = const()[name = string("gamma_91_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_91_perm_0 = const()[name = string("beta_91_perm_0"), val = tensor([0, -1, 1])]; + tensor x_421_axes_0 = const()[name = string("x_421_axes_0"), val = tensor([-1])]; + tensor x_421_cast_fp16 = layer_norm(axes = x_421_axes_0, epsilon = var_2350_to_fp16, x = x_417_cast_fp16)[name = string("x_421_cast_fp16")]; + fp16 var_2990_promoted_to_fp16 = const()[name = string("op_2990_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_91_cast_fp16 = transpose(perm = gamma_91_perm_0, x = var_2984_cast_fp16_0)[name = string("transpose_237")]; + tensor var_2991_cast_fp16 = add(x = gamma_91_cast_fp16, y = var_2990_promoted_to_fp16)[name = string("op_2991_cast_fp16")]; + tensor var_2992_cast_fp16 = mul(x = var_2991_cast_fp16, y = x_421_cast_fp16)[name = string("op_2992_cast_fp16")]; + tensor beta_91_cast_fp16 = transpose(perm = beta_91_perm_0, x = var_2984_cast_fp16_1)[name = string("transpose_236")]; + tensor x_423_cast_fp16 = add(x = var_2992_cast_fp16, y = beta_91_cast_fp16)[name = string("x_423_cast_fp16")]; + tensor var_3003_split_sizes_0 = const()[name = string("op_3003_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3003_axis_0 = const()[name = string("op_3003_axis_0"), val = int32(1)]; + tensor var_3003_cast_fp16_0, tensor var_3003_cast_fp16_1 = split(axis = var_3003_axis_0, split_sizes = var_3003_split_sizes_0, x = h_23_cast_fp16)[name = string("op_3003_cast_fp16")]; + tensor gamma_95_perm_0 = const()[name = string("gamma_95_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_95_perm_0 = const()[name = string("beta_95_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3009_promoted_to_fp16 = const()[name = string("op_3009_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_95_cast_fp16 = transpose(perm = gamma_95_perm_0, x = var_3003_cast_fp16_0)[name = string("transpose_235")]; + tensor var_3010_cast_fp16 = add(x = gamma_95_cast_fp16, y = var_3009_promoted_to_fp16)[name = string("op_3010_cast_fp16")]; + tensor var_3011_cast_fp16 = mul(x = var_3010_cast_fp16, y = x_421_cast_fp16)[name = string("op_3011_cast_fp16")]; + tensor beta_95_cast_fp16 = transpose(perm = beta_95_perm_0, x = var_3003_cast_fp16_1)[name = string("transpose_234")]; + tensor x_429_cast_fp16 = add(x = var_3011_cast_fp16, y = beta_95_cast_fp16)[name = string("x_429_cast_fp16")]; + tensor linear_95_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_423_cast_fp16)[name = string("linear_95_cast_fp16")]; + tensor linear_96_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_429_cast_fp16)[name = string("linear_96_cast_fp16")]; + tensor var_3017_split_sizes_0 = const()[name = string("op_3017_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3017_axis_0 = const()[name = string("op_3017_axis_0"), val = int32(-1)]; + tensor var_3017_cast_fp16_0, tensor var_3017_cast_fp16_1 = split(axis = var_3017_axis_0, split_sizes = var_3017_split_sizes_0, x = linear_96_cast_fp16)[name = string("op_3017_cast_fp16")]; + tensor var_3025 = const()[name = string("op_3025"), val = tensor([1, 128, 8, 64])]; + tensor x_433_cast_fp16 = reshape(shape = var_3025, x = linear_95_cast_fp16)[name = string("x_433_cast_fp16")]; + tensor var_3035 = const()[name = string("op_3035"), val = tensor([1, 128, 8, 64])]; + tensor x_437_cast_fp16 = reshape(shape = var_3035, x = var_3017_cast_fp16_0)[name = string("x_437_cast_fp16")]; + tensor var_3045 = const()[name = string("op_3045"), val = tensor([1, 128, 8, 64])]; + tensor x_441_cast_fp16 = reshape(shape = var_3045, x = var_3017_cast_fp16_1)[name = string("x_441_cast_fp16")]; + tensor var_3047 = const()[name = string("op_3047"), val = tensor([0, 2, 1, 3])]; + bool sim_45_transpose_x_0 = const()[name = string("sim_45_transpose_x_0"), val = bool(false)]; + bool sim_45_transpose_y_0 = const()[name = string("sim_45_transpose_y_0"), val = bool(false)]; + tensor transpose_94_perm_0 = const()[name = string("transpose_94_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_95_perm_0 = const()[name = string("transpose_95_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_95 = transpose(perm = transpose_95_perm_0, x = x_437_cast_fp16)[name = string("transpose_231")]; + tensor transpose_94 = transpose(perm = transpose_94_perm_0, x = x_433_cast_fp16)[name = string("transpose_232")]; + tensor sim_45_cast_fp16 = matmul(transpose_x = sim_45_transpose_x_0, transpose_y = sim_45_transpose_y_0, x = transpose_94, y = transpose_95)[name = string("sim_45_cast_fp16")]; + fp16 var_3051_to_fp16 = const()[name = string("op_3051_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_47_cast_fp16 = mul(x = sim_45_cast_fp16, y = var_3051_to_fp16)[name = string("sim_47_cast_fp16")]; + tensor attn_23_cast_fp16 = softmax(axis = var_2354, x = sim_47_cast_fp16)[name = string("attn_23_cast_fp16")]; + bool x_443_transpose_x_0 = const()[name = string("x_443_transpose_x_0"), val = bool(false)]; + bool x_443_transpose_y_0 = const()[name = string("x_443_transpose_y_0"), val = bool(false)]; + tensor v_23_cast_fp16 = transpose(perm = var_3047, x = x_441_cast_fp16)[name = string("transpose_233")]; + tensor x_443_cast_fp16 = matmul(transpose_x = x_443_transpose_x_0, transpose_y = x_443_transpose_y_0, x = attn_23_cast_fp16, y = v_23_cast_fp16)[name = string("x_443_cast_fp16")]; + tensor var_3073 = const()[name = string("op_3073"), val = tensor([0, 2, 1, 3])]; + tensor var_3075 = const()[name = string("op_3075"), val = tensor([1, 128, 512])]; + tensor x_445_cast_fp16 = transpose(perm = var_3073, x = x_443_cast_fp16)[name = string("transpose_230")]; + tensor input_247_cast_fp16 = reshape(shape = var_3075, x = x_445_cast_fp16)[name = string("input_247_cast_fp16")]; + tensor linear_97_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_247_cast_fp16)[name = string("linear_97_cast_fp16")]; + tensor input_249_cast_fp16 = add(x = linear_97_cast_fp16, y = x_417_cast_fp16)[name = string("input_249_cast_fp16")]; + tensor linear_98_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_249_cast_fp16)[name = string("linear_98_cast_fp16")]; + string input_253_mode_0 = const()[name = string("input_253_mode_0"), val = string("EXACT")]; + tensor input_253_cast_fp16 = gelu(mode = input_253_mode_0, x = linear_98_cast_fp16)[name = string("input_253_cast_fp16")]; + tensor linear_99_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_253_cast_fp16)[name = string("linear_99_cast_fp16")]; + tensor x_447_cast_fp16 = add(x = linear_99_cast_fp16, y = input_249_cast_fp16)[name = string("x_447_cast_fp16")]; + tensor var_3084_axes_0 = const()[name = string("op_3084_axes_0"), val = tensor([1])]; + bool var_3084_keep_dims_0 = const()[name = string("op_3084_keep_dims_0"), val = bool(false)]; + tensor var_3084_cast_fp16 = reduce_mean(axes = var_3084_axes_0, keep_dims = var_3084_keep_dims_0, x = x_447_cast_fp16)[name = string("op_3084_cast_fp16")]; + tensor x_449_axes_0 = const()[name = string("x_449_axes_0"), val = tensor([1])]; + tensor x_449_cast_fp16 = expand_dims(axes = x_449_axes_0, x = var_3084_cast_fp16)[name = string("x_449_cast_fp16")]; + tensor var_3086 = const()[name = string("op_3086"), val = tensor([0, 2, 1])]; + string x_451_pad_type_0 = const()[name = string("x_451_pad_type_0"), val = string("valid")]; + tensor x_451_strides_0 = const()[name = string("x_451_strides_0"), val = tensor([1])]; + tensor x_451_pad_0 = const()[name = string("x_451_pad_0"), val = tensor([0, 0])]; + tensor x_451_dilations_0 = const()[name = string("x_451_dilations_0"), val = tensor([1])]; + int32 x_451_groups_0 = const()[name = string("x_451_groups_0"), val = int32(1)]; + tensor input_255_cast_fp16 = transpose(perm = var_3086, x = x_449_cast_fp16)[name = string("transpose_229")]; + tensor x_451_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_451_dilations_0, groups = x_451_groups_0, pad = x_451_pad_0, pad_type = x_451_pad_type_0, strides = x_451_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_255_cast_fp16)[name = string("x_451_cast_fp16")]; + tensor x_pred_7_perm_0 = const()[name = string("x_pred_7_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_7_to_fp16 = const()[name = string("c_skip_7_to_fp16"), val = tensor([[[0x1.528p-2]]])]; + tensor var_3094_cast_fp16 = mul(x = c_skip_7_to_fp16, y = x_noisy_7_cast_fp16)[name = string("op_3094_cast_fp16")]; + tensor c_out_7_to_fp16 = const()[name = string("c_out_7_to_fp16"), val = tensor([[[0x1.4dcp-3]]])]; + tensor x_pred_7_cast_fp16 = transpose(perm = x_pred_7_perm_0, x = x_451_cast_fp16)[name = string("transpose_228")]; + tensor var_3095_cast_fp16 = mul(x = c_out_7_to_fp16, y = x_pred_7_cast_fp16)[name = string("op_3095_cast_fp16")]; + tensor x_mid_dn_3_cast_fp16 = add(x = var_3094_cast_fp16, y = var_3095_cast_fp16)[name = string("x_mid_dn_3_cast_fp16")]; + tensor var_3098_cast_fp16 = sub(x = x_noisy_7_cast_fp16, y = x_mid_dn_3_cast_fp16)[name = string("op_3098_cast_fp16")]; + tensor _inversed_d_mid_3_y_0_to_fp16 = const()[name = string("_inversed_d_mid_3_y_0_to_fp16"), val = tensor([0x1.c3cp+1])]; + tensor _inversed_d_mid_3_cast_fp16 = mul(x = var_3098_cast_fp16, y = _inversed_d_mid_3_y_0_to_fp16)[name = string("_inversed_d_mid_3_cast_fp16")]; + fp16 var_3107_to_fp16 = const()[name = string("op_3107_to_fp16"), val = fp16(-0x1.19p-1)]; + tensor var_3108_cast_fp16 = mul(x = _inversed_d_mid_3_cast_fp16, y = var_3107_to_fp16)[name = string("op_3108_cast_fp16")]; + tensor x_453_cast_fp16 = add(x = x_noisy_5_cast_fp16, y = var_3108_cast_fp16)[name = string("x_453_cast_fp16")]; + tensor var_3113_begin_0 = const()[name = string("op_3113_begin_0"), val = tensor([1, 0, 0, 0])]; + tensor var_3113_end_0 = const()[name = string("op_3113_end_0"), val = tensor([2, 1, 1, 256])]; + tensor var_3113_end_mask_0 = const()[name = string("op_3113_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_3113_squeeze_mask_0 = const()[name = string("op_3113_squeeze_mask_0"), val = tensor([true, false, false, false])]; + tensor var_3113_cast_fp16 = slice_by_index(begin = var_3113_begin_0, end = var_3113_end_0, end_mask = var_3113_end_mask_0, squeeze_mask = var_3113_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_3113_cast_fp16")]; + fp16 var_3116_to_fp16 = const()[name = string("op_3116_to_fp16"), val = fp16(0x1.1ep-4)]; + tensor var_3117_cast_fp16 = mul(x = var_3113_cast_fp16, y = var_3116_to_fp16)[name = string("op_3117_cast_fp16")]; + tensor x_noisy_9_cast_fp16 = add(x = x_453_cast_fp16, y = var_3117_cast_fp16)[name = string("x_noisy_9_cast_fp16")]; + int32 var_3141 = const()[name = string("op_3141"), val = int32(-1)]; + tensor c_in_9_to_fp16 = const()[name = string("c_in_9_to_fp16"), val = tensor([[[0x1.2ecp+2]]])]; + tensor x_463_cast_fp16 = mul(x = c_in_9_to_fp16, y = x_noisy_9_cast_fp16)[name = string("x_463_cast_fp16")]; + int32 x_459_axis_0 = const()[name = string("x_459_axis_0"), val = int32(0)]; + tensor var_3527_to_fp16 = const()[name = string("op_3527_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49353408)))]; + tensor x_459_cast_fp16 = stack(axis = x_459_axis_0, values = (var_3527_to_fp16, var_423_cast_fp16))[name = string("x_459_cast_fp16")]; + tensor var_3532 = const()[name = string("op_3532"), val = tensor([1, 2, 0])]; + tensor input_263_axes_0 = const()[name = string("input_263_axes_0"), val = tensor([2])]; + bool input_263_keep_dims_0 = const()[name = string("input_263_keep_dims_0"), val = bool(false)]; + tensor x_461_cast_fp16 = transpose(perm = var_3532, x = x_459_cast_fp16)[name = string("transpose_227")]; + tensor input_263_cast_fp16 = reduce_sum(axes = input_263_axes_0, keep_dims = input_263_keep_dims_0, x = x_461_cast_fp16)[name = string("input_263_cast_fp16")]; + tensor linear_102_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_263_cast_fp16)[name = string("linear_102_cast_fp16")]; + string input_267_mode_0 = const()[name = string("input_267_mode_0"), val = string("EXACT")]; + tensor input_267_cast_fp16 = gelu(mode = input_267_mode_0, x = linear_102_cast_fp16)[name = string("input_267_cast_fp16")]; + tensor linear_103_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_267_cast_fp16)[name = string("linear_103_cast_fp16")]; + string mapping_17_mode_0 = const()[name = string("mapping_17_mode_0"), val = string("EXACT")]; + tensor mapping_17_cast_fp16 = gelu(mode = mapping_17_mode_0, x = linear_103_cast_fp16)[name = string("mapping_17_cast_fp16")]; + tensor var_3542_reps_0 = const()[name = string("op_3542_reps_0"), val = tensor([1, 128, 1])]; + tensor var_3542_cast_fp16 = tile(reps = var_3542_reps_0, x = x_463_cast_fp16)[name = string("op_3542_cast_fp16")]; + bool x_465_interleave_0 = const()[name = string("x_465_interleave_0"), val = bool(false)]; + tensor x_465_cast_fp16 = concat(axis = var_3141, interleave = x_465_interleave_0, values = (var_3542_cast_fp16, embedding_to_fp16))[name = string("x_465_cast_fp16")]; + tensor var_3545_axes_0 = const()[name = string("op_3545_axes_0"), val = tensor([1])]; + tensor var_3545_cast_fp16 = expand_dims(axes = var_3545_axes_0, x = mapping_17_cast_fp16)[name = string("op_3545_cast_fp16")]; + tensor mapping_19_reps_0 = const()[name = string("mapping_19_reps_0"), val = tensor([1, 128, 1])]; + tensor mapping_19_cast_fp16 = tile(reps = mapping_19_reps_0, x = var_3545_cast_fp16)[name = string("mapping_19_cast_fp16")]; + tensor x_467_cast_fp16 = add(x = x_465_cast_fp16, y = mapping_19_cast_fp16)[name = string("x_467_cast_fp16")]; + tensor var_3557_split_sizes_0 = const()[name = string("op_3557_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3557_axis_0 = const()[name = string("op_3557_axis_0"), val = int32(1)]; + tensor var_3557_cast_fp16_0, tensor var_3557_cast_fp16_1 = split(axis = var_3557_axis_0, split_sizes = var_3557_split_sizes_0, x = h_3_cast_fp16)[name = string("op_3557_cast_fp16")]; + tensor gamma_99_perm_0 = const()[name = string("gamma_99_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_99_perm_0 = const()[name = string("beta_99_perm_0"), val = tensor([0, -1, 1])]; + tensor x_471_axes_0 = const()[name = string("x_471_axes_0"), val = tensor([-1])]; + fp16 var_3137_to_fp16 = const()[name = string("op_3137_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_471_cast_fp16 = layer_norm(axes = x_471_axes_0, epsilon = var_3137_to_fp16, x = x_467_cast_fp16)[name = string("x_471_cast_fp16")]; + fp16 var_3563_promoted_to_fp16 = const()[name = string("op_3563_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_99_cast_fp16 = transpose(perm = gamma_99_perm_0, x = var_3557_cast_fp16_0)[name = string("transpose_226")]; + tensor var_3564_cast_fp16 = add(x = gamma_99_cast_fp16, y = var_3563_promoted_to_fp16)[name = string("op_3564_cast_fp16")]; + tensor var_3565_cast_fp16 = mul(x = var_3564_cast_fp16, y = x_471_cast_fp16)[name = string("op_3565_cast_fp16")]; + tensor beta_99_cast_fp16 = transpose(perm = beta_99_perm_0, x = var_3557_cast_fp16_1)[name = string("transpose_225")]; + tensor x_473_cast_fp16 = add(x = var_3565_cast_fp16, y = beta_99_cast_fp16)[name = string("x_473_cast_fp16")]; + tensor var_3576_split_sizes_0 = const()[name = string("op_3576_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3576_axis_0 = const()[name = string("op_3576_axis_0"), val = int32(1)]; + tensor var_3576_cast_fp16_0, tensor var_3576_cast_fp16_1 = split(axis = var_3576_axis_0, split_sizes = var_3576_split_sizes_0, x = h_7_cast_fp16)[name = string("op_3576_cast_fp16")]; + tensor gamma_103_perm_0 = const()[name = string("gamma_103_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_103_perm_0 = const()[name = string("beta_103_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3582_promoted_to_fp16 = const()[name = string("op_3582_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_103_cast_fp16 = transpose(perm = gamma_103_perm_0, x = var_3576_cast_fp16_0)[name = string("transpose_224")]; + tensor var_3583_cast_fp16 = add(x = gamma_103_cast_fp16, y = var_3582_promoted_to_fp16)[name = string("op_3583_cast_fp16")]; + tensor var_3584_cast_fp16 = mul(x = var_3583_cast_fp16, y = x_471_cast_fp16)[name = string("op_3584_cast_fp16")]; + tensor beta_103_cast_fp16 = transpose(perm = beta_103_perm_0, x = var_3576_cast_fp16_1)[name = string("transpose_223")]; + tensor x_479_cast_fp16 = add(x = var_3584_cast_fp16, y = beta_103_cast_fp16)[name = string("x_479_cast_fp16")]; + tensor linear_106_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_473_cast_fp16)[name = string("linear_106_cast_fp16")]; + tensor linear_107_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_479_cast_fp16)[name = string("linear_107_cast_fp16")]; + tensor var_3590_split_sizes_0 = const()[name = string("op_3590_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3590_axis_0 = const()[name = string("op_3590_axis_0"), val = int32(-1)]; + tensor var_3590_cast_fp16_0, tensor var_3590_cast_fp16_1 = split(axis = var_3590_axis_0, split_sizes = var_3590_split_sizes_0, x = linear_107_cast_fp16)[name = string("op_3590_cast_fp16")]; + tensor var_3598 = const()[name = string("op_3598"), val = tensor([1, 128, 8, 64])]; + tensor x_483_cast_fp16 = reshape(shape = var_3598, x = linear_106_cast_fp16)[name = string("x_483_cast_fp16")]; + tensor var_3608 = const()[name = string("op_3608"), val = tensor([1, 128, 8, 64])]; + tensor x_487_cast_fp16 = reshape(shape = var_3608, x = var_3590_cast_fp16_0)[name = string("x_487_cast_fp16")]; + tensor var_3618 = const()[name = string("op_3618"), val = tensor([1, 128, 8, 64])]; + tensor x_491_cast_fp16 = reshape(shape = var_3618, x = var_3590_cast_fp16_1)[name = string("x_491_cast_fp16")]; + tensor var_3620 = const()[name = string("op_3620"), val = tensor([0, 2, 1, 3])]; + bool sim_49_transpose_x_0 = const()[name = string("sim_49_transpose_x_0"), val = bool(false)]; + bool sim_49_transpose_y_0 = const()[name = string("sim_49_transpose_y_0"), val = bool(false)]; + tensor transpose_96_perm_0 = const()[name = string("transpose_96_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_97_perm_0 = const()[name = string("transpose_97_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_97 = transpose(perm = transpose_97_perm_0, x = x_487_cast_fp16)[name = string("transpose_220")]; + tensor transpose_96 = transpose(perm = transpose_96_perm_0, x = x_483_cast_fp16)[name = string("transpose_221")]; + tensor sim_49_cast_fp16 = matmul(transpose_x = sim_49_transpose_x_0, transpose_y = sim_49_transpose_y_0, x = transpose_96, y = transpose_97)[name = string("sim_49_cast_fp16")]; + fp16 var_3624_to_fp16 = const()[name = string("op_3624_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_51_cast_fp16 = mul(x = sim_49_cast_fp16, y = var_3624_to_fp16)[name = string("sim_51_cast_fp16")]; + tensor attn_25_cast_fp16 = softmax(axis = var_3141, x = sim_51_cast_fp16)[name = string("attn_25_cast_fp16")]; + bool x_493_transpose_x_0 = const()[name = string("x_493_transpose_x_0"), val = bool(false)]; + bool x_493_transpose_y_0 = const()[name = string("x_493_transpose_y_0"), val = bool(false)]; + tensor v_25_cast_fp16 = transpose(perm = var_3620, x = x_491_cast_fp16)[name = string("transpose_222")]; + tensor x_493_cast_fp16 = matmul(transpose_x = x_493_transpose_x_0, transpose_y = x_493_transpose_y_0, x = attn_25_cast_fp16, y = v_25_cast_fp16)[name = string("x_493_cast_fp16")]; + tensor var_3646 = const()[name = string("op_3646"), val = tensor([0, 2, 1, 3])]; + tensor var_3648 = const()[name = string("op_3648"), val = tensor([1, 128, 512])]; + tensor x_495_cast_fp16 = transpose(perm = var_3646, x = x_493_cast_fp16)[name = string("transpose_219")]; + tensor input_279_cast_fp16 = reshape(shape = var_3648, x = x_495_cast_fp16)[name = string("input_279_cast_fp16")]; + tensor linear_108_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_279_cast_fp16)[name = string("linear_108_cast_fp16")]; + tensor input_281_cast_fp16 = add(x = linear_108_cast_fp16, y = x_467_cast_fp16)[name = string("input_281_cast_fp16")]; + tensor linear_109_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_281_cast_fp16)[name = string("linear_109_cast_fp16")]; + string input_285_mode_0 = const()[name = string("input_285_mode_0"), val = string("EXACT")]; + tensor input_285_cast_fp16 = gelu(mode = input_285_mode_0, x = linear_109_cast_fp16)[name = string("input_285_cast_fp16")]; + tensor linear_110_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_285_cast_fp16)[name = string("linear_110_cast_fp16")]; + tensor x_497_cast_fp16 = add(x = linear_110_cast_fp16, y = input_281_cast_fp16)[name = string("x_497_cast_fp16")]; + tensor x_499_cast_fp16 = add(x = x_497_cast_fp16, y = mapping_19_cast_fp16)[name = string("x_499_cast_fp16")]; + tensor var_3664_split_sizes_0 = const()[name = string("op_3664_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3664_axis_0 = const()[name = string("op_3664_axis_0"), val = int32(1)]; + tensor var_3664_cast_fp16_0, tensor var_3664_cast_fp16_1 = split(axis = var_3664_axis_0, split_sizes = var_3664_split_sizes_0, x = h_11_cast_fp16)[name = string("op_3664_cast_fp16")]; + tensor gamma_107_perm_0 = const()[name = string("gamma_107_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_107_perm_0 = const()[name = string("beta_107_perm_0"), val = tensor([0, -1, 1])]; + tensor x_503_axes_0 = const()[name = string("x_503_axes_0"), val = tensor([-1])]; + tensor x_503_cast_fp16 = layer_norm(axes = x_503_axes_0, epsilon = var_3137_to_fp16, x = x_499_cast_fp16)[name = string("x_503_cast_fp16")]; + fp16 var_3670_promoted_to_fp16 = const()[name = string("op_3670_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_107_cast_fp16 = transpose(perm = gamma_107_perm_0, x = var_3664_cast_fp16_0)[name = string("transpose_218")]; + tensor var_3671_cast_fp16 = add(x = gamma_107_cast_fp16, y = var_3670_promoted_to_fp16)[name = string("op_3671_cast_fp16")]; + tensor var_3672_cast_fp16 = mul(x = var_3671_cast_fp16, y = x_503_cast_fp16)[name = string("op_3672_cast_fp16")]; + tensor beta_107_cast_fp16 = transpose(perm = beta_107_perm_0, x = var_3664_cast_fp16_1)[name = string("transpose_217")]; + tensor x_505_cast_fp16 = add(x = var_3672_cast_fp16, y = beta_107_cast_fp16)[name = string("x_505_cast_fp16")]; + tensor var_3683_split_sizes_0 = const()[name = string("op_3683_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3683_axis_0 = const()[name = string("op_3683_axis_0"), val = int32(1)]; + tensor var_3683_cast_fp16_0, tensor var_3683_cast_fp16_1 = split(axis = var_3683_axis_0, split_sizes = var_3683_split_sizes_0, x = h_15_cast_fp16)[name = string("op_3683_cast_fp16")]; + tensor gamma_111_perm_0 = const()[name = string("gamma_111_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_111_perm_0 = const()[name = string("beta_111_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3689_promoted_to_fp16 = const()[name = string("op_3689_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_111_cast_fp16 = transpose(perm = gamma_111_perm_0, x = var_3683_cast_fp16_0)[name = string("transpose_216")]; + tensor var_3690_cast_fp16 = add(x = gamma_111_cast_fp16, y = var_3689_promoted_to_fp16)[name = string("op_3690_cast_fp16")]; + tensor var_3691_cast_fp16 = mul(x = var_3690_cast_fp16, y = x_503_cast_fp16)[name = string("op_3691_cast_fp16")]; + tensor beta_111_cast_fp16 = transpose(perm = beta_111_perm_0, x = var_3683_cast_fp16_1)[name = string("transpose_215")]; + tensor x_511_cast_fp16 = add(x = var_3691_cast_fp16, y = beta_111_cast_fp16)[name = string("x_511_cast_fp16")]; + tensor linear_113_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_505_cast_fp16)[name = string("linear_113_cast_fp16")]; + tensor linear_114_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_511_cast_fp16)[name = string("linear_114_cast_fp16")]; + tensor var_3697_split_sizes_0 = const()[name = string("op_3697_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3697_axis_0 = const()[name = string("op_3697_axis_0"), val = int32(-1)]; + tensor var_3697_cast_fp16_0, tensor var_3697_cast_fp16_1 = split(axis = var_3697_axis_0, split_sizes = var_3697_split_sizes_0, x = linear_114_cast_fp16)[name = string("op_3697_cast_fp16")]; + tensor var_3705 = const()[name = string("op_3705"), val = tensor([1, 128, 8, 64])]; + tensor x_515_cast_fp16 = reshape(shape = var_3705, x = linear_113_cast_fp16)[name = string("x_515_cast_fp16")]; + tensor var_3715 = const()[name = string("op_3715"), val = tensor([1, 128, 8, 64])]; + tensor x_519_cast_fp16 = reshape(shape = var_3715, x = var_3697_cast_fp16_0)[name = string("x_519_cast_fp16")]; + tensor var_3725 = const()[name = string("op_3725"), val = tensor([1, 128, 8, 64])]; + tensor x_523_cast_fp16 = reshape(shape = var_3725, x = var_3697_cast_fp16_1)[name = string("x_523_cast_fp16")]; + tensor var_3727 = const()[name = string("op_3727"), val = tensor([0, 2, 1, 3])]; + bool sim_53_transpose_x_0 = const()[name = string("sim_53_transpose_x_0"), val = bool(false)]; + bool sim_53_transpose_y_0 = const()[name = string("sim_53_transpose_y_0"), val = bool(false)]; + tensor transpose_98_perm_0 = const()[name = string("transpose_98_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_99_perm_0 = const()[name = string("transpose_99_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_99 = transpose(perm = transpose_99_perm_0, x = x_519_cast_fp16)[name = string("transpose_212")]; + tensor transpose_98 = transpose(perm = transpose_98_perm_0, x = x_515_cast_fp16)[name = string("transpose_213")]; + tensor sim_53_cast_fp16 = matmul(transpose_x = sim_53_transpose_x_0, transpose_y = sim_53_transpose_y_0, x = transpose_98, y = transpose_99)[name = string("sim_53_cast_fp16")]; + fp16 var_3731_to_fp16 = const()[name = string("op_3731_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_55_cast_fp16 = mul(x = sim_53_cast_fp16, y = var_3731_to_fp16)[name = string("sim_55_cast_fp16")]; + tensor attn_27_cast_fp16 = softmax(axis = var_3141, x = sim_55_cast_fp16)[name = string("attn_27_cast_fp16")]; + bool x_525_transpose_x_0 = const()[name = string("x_525_transpose_x_0"), val = bool(false)]; + bool x_525_transpose_y_0 = const()[name = string("x_525_transpose_y_0"), val = bool(false)]; + tensor v_27_cast_fp16 = transpose(perm = var_3727, x = x_523_cast_fp16)[name = string("transpose_214")]; + tensor x_525_cast_fp16 = matmul(transpose_x = x_525_transpose_x_0, transpose_y = x_525_transpose_y_0, x = attn_27_cast_fp16, y = v_27_cast_fp16)[name = string("x_525_cast_fp16")]; + tensor var_3753 = const()[name = string("op_3753"), val = tensor([0, 2, 1, 3])]; + tensor var_3755 = const()[name = string("op_3755"), val = tensor([1, 128, 512])]; + tensor x_527_cast_fp16 = transpose(perm = var_3753, x = x_525_cast_fp16)[name = string("transpose_211")]; + tensor input_295_cast_fp16 = reshape(shape = var_3755, x = x_527_cast_fp16)[name = string("input_295_cast_fp16")]; + tensor linear_115_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_295_cast_fp16)[name = string("linear_115_cast_fp16")]; + tensor input_297_cast_fp16 = add(x = linear_115_cast_fp16, y = x_499_cast_fp16)[name = string("input_297_cast_fp16")]; + tensor linear_116_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_297_cast_fp16)[name = string("linear_116_cast_fp16")]; + string input_301_mode_0 = const()[name = string("input_301_mode_0"), val = string("EXACT")]; + tensor input_301_cast_fp16 = gelu(mode = input_301_mode_0, x = linear_116_cast_fp16)[name = string("input_301_cast_fp16")]; + tensor linear_117_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_301_cast_fp16)[name = string("linear_117_cast_fp16")]; + tensor x_529_cast_fp16 = add(x = linear_117_cast_fp16, y = input_297_cast_fp16)[name = string("x_529_cast_fp16")]; + tensor x_531_cast_fp16 = add(x = x_529_cast_fp16, y = mapping_19_cast_fp16)[name = string("x_531_cast_fp16")]; + tensor var_3771_split_sizes_0 = const()[name = string("op_3771_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3771_axis_0 = const()[name = string("op_3771_axis_0"), val = int32(1)]; + tensor var_3771_cast_fp16_0, tensor var_3771_cast_fp16_1 = split(axis = var_3771_axis_0, split_sizes = var_3771_split_sizes_0, x = h_19_cast_fp16)[name = string("op_3771_cast_fp16")]; + tensor gamma_115_perm_0 = const()[name = string("gamma_115_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_115_perm_0 = const()[name = string("beta_115_perm_0"), val = tensor([0, -1, 1])]; + tensor x_535_axes_0 = const()[name = string("x_535_axes_0"), val = tensor([-1])]; + tensor x_535_cast_fp16 = layer_norm(axes = x_535_axes_0, epsilon = var_3137_to_fp16, x = x_531_cast_fp16)[name = string("x_535_cast_fp16")]; + fp16 var_3777_promoted_to_fp16 = const()[name = string("op_3777_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_115_cast_fp16 = transpose(perm = gamma_115_perm_0, x = var_3771_cast_fp16_0)[name = string("transpose_210")]; + tensor var_3778_cast_fp16 = add(x = gamma_115_cast_fp16, y = var_3777_promoted_to_fp16)[name = string("op_3778_cast_fp16")]; + tensor var_3779_cast_fp16 = mul(x = var_3778_cast_fp16, y = x_535_cast_fp16)[name = string("op_3779_cast_fp16")]; + tensor beta_115_cast_fp16 = transpose(perm = beta_115_perm_0, x = var_3771_cast_fp16_1)[name = string("transpose_209")]; + tensor x_537_cast_fp16 = add(x = var_3779_cast_fp16, y = beta_115_cast_fp16)[name = string("x_537_cast_fp16")]; + tensor var_3790_split_sizes_0 = const()[name = string("op_3790_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3790_axis_0 = const()[name = string("op_3790_axis_0"), val = int32(1)]; + tensor var_3790_cast_fp16_0, tensor var_3790_cast_fp16_1 = split(axis = var_3790_axis_0, split_sizes = var_3790_split_sizes_0, x = h_23_cast_fp16)[name = string("op_3790_cast_fp16")]; + tensor gamma_119_perm_0 = const()[name = string("gamma_119_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_119_perm_0 = const()[name = string("beta_119_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3796_promoted_to_fp16 = const()[name = string("op_3796_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_119_cast_fp16 = transpose(perm = gamma_119_perm_0, x = var_3790_cast_fp16_0)[name = string("transpose_208")]; + tensor var_3797_cast_fp16 = add(x = gamma_119_cast_fp16, y = var_3796_promoted_to_fp16)[name = string("op_3797_cast_fp16")]; + tensor var_3798_cast_fp16 = mul(x = var_3797_cast_fp16, y = x_535_cast_fp16)[name = string("op_3798_cast_fp16")]; + tensor beta_119_cast_fp16 = transpose(perm = beta_119_perm_0, x = var_3790_cast_fp16_1)[name = string("transpose_207")]; + tensor x_543_cast_fp16 = add(x = var_3798_cast_fp16, y = beta_119_cast_fp16)[name = string("x_543_cast_fp16")]; + tensor linear_120_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_537_cast_fp16)[name = string("linear_120_cast_fp16")]; + tensor linear_121_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_543_cast_fp16)[name = string("linear_121_cast_fp16")]; + tensor var_3804_split_sizes_0 = const()[name = string("op_3804_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3804_axis_0 = const()[name = string("op_3804_axis_0"), val = int32(-1)]; + tensor var_3804_cast_fp16_0, tensor var_3804_cast_fp16_1 = split(axis = var_3804_axis_0, split_sizes = var_3804_split_sizes_0, x = linear_121_cast_fp16)[name = string("op_3804_cast_fp16")]; + tensor var_3812 = const()[name = string("op_3812"), val = tensor([1, 128, 8, 64])]; + tensor x_547_cast_fp16 = reshape(shape = var_3812, x = linear_120_cast_fp16)[name = string("x_547_cast_fp16")]; + tensor var_3822 = const()[name = string("op_3822"), val = tensor([1, 128, 8, 64])]; + tensor x_551_cast_fp16 = reshape(shape = var_3822, x = var_3804_cast_fp16_0)[name = string("x_551_cast_fp16")]; + tensor var_3832 = const()[name = string("op_3832"), val = tensor([1, 128, 8, 64])]; + tensor x_555_cast_fp16 = reshape(shape = var_3832, x = var_3804_cast_fp16_1)[name = string("x_555_cast_fp16")]; + tensor var_3834 = const()[name = string("op_3834"), val = tensor([0, 2, 1, 3])]; + bool sim_57_transpose_x_0 = const()[name = string("sim_57_transpose_x_0"), val = bool(false)]; + bool sim_57_transpose_y_0 = const()[name = string("sim_57_transpose_y_0"), val = bool(false)]; + tensor transpose_100_perm_0 = const()[name = string("transpose_100_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_101_perm_0 = const()[name = string("transpose_101_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_101 = transpose(perm = transpose_101_perm_0, x = x_551_cast_fp16)[name = string("transpose_204")]; + tensor transpose_100 = transpose(perm = transpose_100_perm_0, x = x_547_cast_fp16)[name = string("transpose_205")]; + tensor sim_57_cast_fp16 = matmul(transpose_x = sim_57_transpose_x_0, transpose_y = sim_57_transpose_y_0, x = transpose_100, y = transpose_101)[name = string("sim_57_cast_fp16")]; + fp16 var_3838_to_fp16 = const()[name = string("op_3838_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_59_cast_fp16 = mul(x = sim_57_cast_fp16, y = var_3838_to_fp16)[name = string("sim_59_cast_fp16")]; + tensor attn_29_cast_fp16 = softmax(axis = var_3141, x = sim_59_cast_fp16)[name = string("attn_29_cast_fp16")]; + bool x_557_transpose_x_0 = const()[name = string("x_557_transpose_x_0"), val = bool(false)]; + bool x_557_transpose_y_0 = const()[name = string("x_557_transpose_y_0"), val = bool(false)]; + tensor v_29_cast_fp16 = transpose(perm = var_3834, x = x_555_cast_fp16)[name = string("transpose_206")]; + tensor x_557_cast_fp16 = matmul(transpose_x = x_557_transpose_x_0, transpose_y = x_557_transpose_y_0, x = attn_29_cast_fp16, y = v_29_cast_fp16)[name = string("x_557_cast_fp16")]; + tensor var_3860 = const()[name = string("op_3860"), val = tensor([0, 2, 1, 3])]; + tensor var_3862 = const()[name = string("op_3862"), val = tensor([1, 128, 512])]; + tensor x_559_cast_fp16 = transpose(perm = var_3860, x = x_557_cast_fp16)[name = string("transpose_203")]; + tensor input_311_cast_fp16 = reshape(shape = var_3862, x = x_559_cast_fp16)[name = string("input_311_cast_fp16")]; + tensor linear_122_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_311_cast_fp16)[name = string("linear_122_cast_fp16")]; + tensor input_313_cast_fp16 = add(x = linear_122_cast_fp16, y = x_531_cast_fp16)[name = string("input_313_cast_fp16")]; + tensor linear_123_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_313_cast_fp16)[name = string("linear_123_cast_fp16")]; + string input_317_mode_0 = const()[name = string("input_317_mode_0"), val = string("EXACT")]; + tensor input_317_cast_fp16 = gelu(mode = input_317_mode_0, x = linear_123_cast_fp16)[name = string("input_317_cast_fp16")]; + tensor linear_124_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_317_cast_fp16)[name = string("linear_124_cast_fp16")]; + tensor x_561_cast_fp16 = add(x = linear_124_cast_fp16, y = input_313_cast_fp16)[name = string("x_561_cast_fp16")]; + tensor var_3871_axes_0 = const()[name = string("op_3871_axes_0"), val = tensor([1])]; + bool var_3871_keep_dims_0 = const()[name = string("op_3871_keep_dims_0"), val = bool(false)]; + tensor var_3871_cast_fp16 = reduce_mean(axes = var_3871_axes_0, keep_dims = var_3871_keep_dims_0, x = x_561_cast_fp16)[name = string("op_3871_cast_fp16")]; + tensor x_563_axes_0 = const()[name = string("x_563_axes_0"), val = tensor([1])]; + tensor x_563_cast_fp16 = expand_dims(axes = x_563_axes_0, x = var_3871_cast_fp16)[name = string("x_563_cast_fp16")]; + tensor var_3873 = const()[name = string("op_3873"), val = tensor([0, 2, 1])]; + string x_565_pad_type_0 = const()[name = string("x_565_pad_type_0"), val = string("valid")]; + tensor x_565_strides_0 = const()[name = string("x_565_strides_0"), val = tensor([1])]; + tensor x_565_pad_0 = const()[name = string("x_565_pad_0"), val = tensor([0, 0])]; + tensor x_565_dilations_0 = const()[name = string("x_565_dilations_0"), val = tensor([1])]; + int32 x_565_groups_0 = const()[name = string("x_565_groups_0"), val = int32(1)]; + tensor input_319_cast_fp16 = transpose(perm = var_3873, x = x_563_cast_fp16)[name = string("transpose_202")]; + tensor x_565_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_565_dilations_0, groups = x_565_groups_0, pad = x_565_pad_0, pad_type = x_565_pad_type_0, strides = x_565_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_319_cast_fp16)[name = string("x_565_cast_fp16")]; + tensor x_pred_9_perm_0 = const()[name = string("x_pred_9_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_9_to_fp16 = const()[name = string("c_skip_9_to_fp16"), val = tensor([[[0x1.c64p-1]]])]; + tensor var_3881_cast_fp16 = mul(x = c_skip_9_to_fp16, y = x_noisy_9_cast_fp16)[name = string("op_3881_cast_fp16")]; + tensor c_out_9_to_fp16 = const()[name = string("c_out_9_to_fp16"), val = tensor([[[0x1.0fcp-4]]])]; + tensor x_pred_9_cast_fp16 = transpose(perm = x_pred_9_perm_0, x = x_565_cast_fp16)[name = string("transpose_201")]; + tensor var_3882_cast_fp16 = mul(x = c_out_9_to_fp16, y = x_pred_9_cast_fp16)[name = string("op_3882_cast_fp16")]; + tensor x_dn_5_cast_fp16 = add(x = var_3881_cast_fp16, y = var_3882_cast_fp16)[name = string("x_dn_5_cast_fp16")]; + tensor var_3885_cast_fp16 = sub(x = x_noisy_9_cast_fp16, y = x_dn_5_cast_fp16)[name = string("op_3885_cast_fp16")]; + tensor _inversed_d_5_y_0_to_fp16 = const()[name = string("_inversed_d_5_y_0_to_fp16"), val = tensor([0x1.c6cp+3])]; + tensor _inversed_d_5_cast_fp16 = mul(x = var_3885_cast_fp16, y = _inversed_d_5_y_0_to_fp16)[name = string("_inversed_d_5_cast_fp16")]; + fp16 var_3894_to_fp16 = const()[name = string("op_3894_to_fp16"), val = fp16(-0x1.1fp-5)]; + tensor var_3895_cast_fp16 = mul(x = _inversed_d_5_cast_fp16, y = var_3894_to_fp16)[name = string("op_3895_cast_fp16")]; + tensor x_noisy_11_cast_fp16 = add(x = x_noisy_9_cast_fp16, y = var_3895_cast_fp16)[name = string("x_noisy_11_cast_fp16")]; + int32 var_3907 = const()[name = string("op_3907"), val = int32(-1)]; + tensor c_in_11_to_fp16 = const()[name = string("c_in_11_to_fp16"), val = tensor([[[0x1.3c4p+2]]])]; + tensor x_575_cast_fp16 = mul(x = c_in_11_to_fp16, y = x_noisy_11_cast_fp16)[name = string("x_575_cast_fp16")]; + int32 x_571_axis_0 = const()[name = string("x_571_axis_0"), val = int32(0)]; + tensor var_4293_to_fp16 = const()[name = string("op_4293_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49355520)))]; + tensor x_571_cast_fp16 = stack(axis = x_571_axis_0, values = (var_4293_to_fp16, var_423_cast_fp16))[name = string("x_571_cast_fp16")]; + tensor var_4298 = const()[name = string("op_4298"), val = tensor([1, 2, 0])]; + tensor input_327_axes_0 = const()[name = string("input_327_axes_0"), val = tensor([2])]; + bool input_327_keep_dims_0 = const()[name = string("input_327_keep_dims_0"), val = bool(false)]; + tensor x_573_cast_fp16 = transpose(perm = var_4298, x = x_571_cast_fp16)[name = string("transpose_200")]; + tensor input_327_cast_fp16 = reduce_sum(axes = input_327_axes_0, keep_dims = input_327_keep_dims_0, x = x_573_cast_fp16)[name = string("input_327_cast_fp16")]; + tensor linear_127_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_327_cast_fp16)[name = string("linear_127_cast_fp16")]; + string input_331_mode_0 = const()[name = string("input_331_mode_0"), val = string("EXACT")]; + tensor input_331_cast_fp16 = gelu(mode = input_331_mode_0, x = linear_127_cast_fp16)[name = string("input_331_cast_fp16")]; + tensor linear_128_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_331_cast_fp16)[name = string("linear_128_cast_fp16")]; + string mapping_21_mode_0 = const()[name = string("mapping_21_mode_0"), val = string("EXACT")]; + tensor mapping_21_cast_fp16 = gelu(mode = mapping_21_mode_0, x = linear_128_cast_fp16)[name = string("mapping_21_cast_fp16")]; + tensor var_4308_reps_0 = const()[name = string("op_4308_reps_0"), val = tensor([1, 128, 1])]; + tensor var_4308_cast_fp16 = tile(reps = var_4308_reps_0, x = x_575_cast_fp16)[name = string("op_4308_cast_fp16")]; + bool x_577_interleave_0 = const()[name = string("x_577_interleave_0"), val = bool(false)]; + tensor x_577_cast_fp16 = concat(axis = var_3907, interleave = x_577_interleave_0, values = (var_4308_cast_fp16, embedding_to_fp16))[name = string("x_577_cast_fp16")]; + tensor var_4311_axes_0 = const()[name = string("op_4311_axes_0"), val = tensor([1])]; + tensor var_4311_cast_fp16 = expand_dims(axes = var_4311_axes_0, x = mapping_21_cast_fp16)[name = string("op_4311_cast_fp16")]; + tensor mapping_23_reps_0 = const()[name = string("mapping_23_reps_0"), val = tensor([1, 128, 1])]; + tensor mapping_23_cast_fp16 = tile(reps = mapping_23_reps_0, x = var_4311_cast_fp16)[name = string("mapping_23_cast_fp16")]; + tensor x_579_cast_fp16 = add(x = x_577_cast_fp16, y = mapping_23_cast_fp16)[name = string("x_579_cast_fp16")]; + tensor var_4323_split_sizes_0 = const()[name = string("op_4323_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4323_axis_0 = const()[name = string("op_4323_axis_0"), val = int32(1)]; + tensor var_4323_cast_fp16_0, tensor var_4323_cast_fp16_1 = split(axis = var_4323_axis_0, split_sizes = var_4323_split_sizes_0, x = h_3_cast_fp16)[name = string("op_4323_cast_fp16")]; + tensor gamma_123_perm_0 = const()[name = string("gamma_123_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_123_perm_0 = const()[name = string("beta_123_perm_0"), val = tensor([0, -1, 1])]; + tensor x_583_axes_0 = const()[name = string("x_583_axes_0"), val = tensor([-1])]; + fp16 var_3903_to_fp16 = const()[name = string("op_3903_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_583_cast_fp16 = layer_norm(axes = x_583_axes_0, epsilon = var_3903_to_fp16, x = x_579_cast_fp16)[name = string("x_583_cast_fp16")]; + fp16 var_4329_promoted_to_fp16 = const()[name = string("op_4329_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_123_cast_fp16 = transpose(perm = gamma_123_perm_0, x = var_4323_cast_fp16_0)[name = string("transpose_199")]; + tensor var_4330_cast_fp16 = add(x = gamma_123_cast_fp16, y = var_4329_promoted_to_fp16)[name = string("op_4330_cast_fp16")]; + tensor var_4331_cast_fp16 = mul(x = var_4330_cast_fp16, y = x_583_cast_fp16)[name = string("op_4331_cast_fp16")]; + tensor beta_123_cast_fp16 = transpose(perm = beta_123_perm_0, x = var_4323_cast_fp16_1)[name = string("transpose_198")]; + tensor x_585_cast_fp16 = add(x = var_4331_cast_fp16, y = beta_123_cast_fp16)[name = string("x_585_cast_fp16")]; + tensor var_4342_split_sizes_0 = const()[name = string("op_4342_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4342_axis_0 = const()[name = string("op_4342_axis_0"), val = int32(1)]; + tensor var_4342_cast_fp16_0, tensor var_4342_cast_fp16_1 = split(axis = var_4342_axis_0, split_sizes = var_4342_split_sizes_0, x = h_7_cast_fp16)[name = string("op_4342_cast_fp16")]; + tensor gamma_127_perm_0 = const()[name = string("gamma_127_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_127_perm_0 = const()[name = string("beta_127_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_4348_promoted_to_fp16 = const()[name = string("op_4348_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_127_cast_fp16 = transpose(perm = gamma_127_perm_0, x = var_4342_cast_fp16_0)[name = string("transpose_197")]; + tensor var_4349_cast_fp16 = add(x = gamma_127_cast_fp16, y = var_4348_promoted_to_fp16)[name = string("op_4349_cast_fp16")]; + tensor var_4350_cast_fp16 = mul(x = var_4349_cast_fp16, y = x_583_cast_fp16)[name = string("op_4350_cast_fp16")]; + tensor beta_127_cast_fp16 = transpose(perm = beta_127_perm_0, x = var_4342_cast_fp16_1)[name = string("transpose_196")]; + tensor x_591_cast_fp16 = add(x = var_4350_cast_fp16, y = beta_127_cast_fp16)[name = string("x_591_cast_fp16")]; + tensor linear_131_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_585_cast_fp16)[name = string("linear_131_cast_fp16")]; + tensor linear_132_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_591_cast_fp16)[name = string("linear_132_cast_fp16")]; + tensor var_4356_split_sizes_0 = const()[name = string("op_4356_split_sizes_0"), val = tensor([512, 512])]; + int32 var_4356_axis_0 = const()[name = string("op_4356_axis_0"), val = int32(-1)]; + tensor var_4356_cast_fp16_0, tensor var_4356_cast_fp16_1 = split(axis = var_4356_axis_0, split_sizes = var_4356_split_sizes_0, x = linear_132_cast_fp16)[name = string("op_4356_cast_fp16")]; + tensor var_4364 = const()[name = string("op_4364"), val = tensor([1, 128, 8, 64])]; + tensor x_595_cast_fp16 = reshape(shape = var_4364, x = linear_131_cast_fp16)[name = string("x_595_cast_fp16")]; + tensor var_4374 = const()[name = string("op_4374"), val = tensor([1, 128, 8, 64])]; + tensor x_599_cast_fp16 = reshape(shape = var_4374, x = var_4356_cast_fp16_0)[name = string("x_599_cast_fp16")]; + tensor var_4384 = const()[name = string("op_4384"), val = tensor([1, 128, 8, 64])]; + tensor x_603_cast_fp16 = reshape(shape = var_4384, x = var_4356_cast_fp16_1)[name = string("x_603_cast_fp16")]; + tensor var_4386 = const()[name = string("op_4386"), val = tensor([0, 2, 1, 3])]; + bool sim_61_transpose_x_0 = const()[name = string("sim_61_transpose_x_0"), val = bool(false)]; + bool sim_61_transpose_y_0 = const()[name = string("sim_61_transpose_y_0"), val = bool(false)]; + tensor transpose_102_perm_0 = const()[name = string("transpose_102_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_103_perm_0 = const()[name = string("transpose_103_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_103 = transpose(perm = transpose_103_perm_0, x = x_599_cast_fp16)[name = string("transpose_193")]; + tensor transpose_102 = transpose(perm = transpose_102_perm_0, x = x_595_cast_fp16)[name = string("transpose_194")]; + tensor sim_61_cast_fp16 = matmul(transpose_x = sim_61_transpose_x_0, transpose_y = sim_61_transpose_y_0, x = transpose_102, y = transpose_103)[name = string("sim_61_cast_fp16")]; + fp16 var_4390_to_fp16 = const()[name = string("op_4390_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_63_cast_fp16 = mul(x = sim_61_cast_fp16, y = var_4390_to_fp16)[name = string("sim_63_cast_fp16")]; + tensor attn_31_cast_fp16 = softmax(axis = var_3907, x = sim_63_cast_fp16)[name = string("attn_31_cast_fp16")]; + bool x_605_transpose_x_0 = const()[name = string("x_605_transpose_x_0"), val = bool(false)]; + bool x_605_transpose_y_0 = const()[name = string("x_605_transpose_y_0"), val = bool(false)]; + tensor v_31_cast_fp16 = transpose(perm = var_4386, x = x_603_cast_fp16)[name = string("transpose_195")]; + tensor x_605_cast_fp16 = matmul(transpose_x = x_605_transpose_x_0, transpose_y = x_605_transpose_y_0, x = attn_31_cast_fp16, y = v_31_cast_fp16)[name = string("x_605_cast_fp16")]; + tensor var_4412 = const()[name = string("op_4412"), val = tensor([0, 2, 1, 3])]; + tensor var_4414 = const()[name = string("op_4414"), val = tensor([1, 128, 512])]; + tensor x_607_cast_fp16 = transpose(perm = var_4412, x = x_605_cast_fp16)[name = string("transpose_192")]; + tensor input_343_cast_fp16 = reshape(shape = var_4414, x = x_607_cast_fp16)[name = string("input_343_cast_fp16")]; + tensor linear_133_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_343_cast_fp16)[name = string("linear_133_cast_fp16")]; + tensor input_345_cast_fp16 = add(x = linear_133_cast_fp16, y = x_579_cast_fp16)[name = string("input_345_cast_fp16")]; + tensor linear_134_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_345_cast_fp16)[name = string("linear_134_cast_fp16")]; + string input_349_mode_0 = const()[name = string("input_349_mode_0"), val = string("EXACT")]; + tensor input_349_cast_fp16 = gelu(mode = input_349_mode_0, x = linear_134_cast_fp16)[name = string("input_349_cast_fp16")]; + tensor linear_135_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_349_cast_fp16)[name = string("linear_135_cast_fp16")]; + tensor x_609_cast_fp16 = add(x = linear_135_cast_fp16, y = input_345_cast_fp16)[name = string("x_609_cast_fp16")]; + tensor x_611_cast_fp16 = add(x = x_609_cast_fp16, y = mapping_23_cast_fp16)[name = string("x_611_cast_fp16")]; + tensor var_4430_split_sizes_0 = const()[name = string("op_4430_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4430_axis_0 = const()[name = string("op_4430_axis_0"), val = int32(1)]; + tensor var_4430_cast_fp16_0, tensor var_4430_cast_fp16_1 = split(axis = var_4430_axis_0, split_sizes = var_4430_split_sizes_0, x = h_11_cast_fp16)[name = string("op_4430_cast_fp16")]; + tensor gamma_131_perm_0 = const()[name = string("gamma_131_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_131_perm_0 = const()[name = string("beta_131_perm_0"), val = tensor([0, -1, 1])]; + tensor x_615_axes_0 = const()[name = string("x_615_axes_0"), val = tensor([-1])]; + tensor x_615_cast_fp16 = layer_norm(axes = x_615_axes_0, epsilon = var_3903_to_fp16, x = x_611_cast_fp16)[name = string("x_615_cast_fp16")]; + fp16 var_4436_promoted_to_fp16 = const()[name = string("op_4436_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_131_cast_fp16 = transpose(perm = gamma_131_perm_0, x = var_4430_cast_fp16_0)[name = string("transpose_191")]; + tensor var_4437_cast_fp16 = add(x = gamma_131_cast_fp16, y = var_4436_promoted_to_fp16)[name = string("op_4437_cast_fp16")]; + tensor var_4438_cast_fp16 = mul(x = var_4437_cast_fp16, y = x_615_cast_fp16)[name = string("op_4438_cast_fp16")]; + tensor beta_131_cast_fp16 = transpose(perm = beta_131_perm_0, x = var_4430_cast_fp16_1)[name = string("transpose_190")]; + tensor x_617_cast_fp16 = add(x = var_4438_cast_fp16, y = beta_131_cast_fp16)[name = string("x_617_cast_fp16")]; + tensor var_4449_split_sizes_0 = const()[name = string("op_4449_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4449_axis_0 = const()[name = string("op_4449_axis_0"), val = int32(1)]; + tensor var_4449_cast_fp16_0, tensor var_4449_cast_fp16_1 = split(axis = var_4449_axis_0, split_sizes = var_4449_split_sizes_0, x = h_15_cast_fp16)[name = string("op_4449_cast_fp16")]; + tensor gamma_135_perm_0 = const()[name = string("gamma_135_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_135_perm_0 = const()[name = string("beta_135_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_4455_promoted_to_fp16 = const()[name = string("op_4455_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_135_cast_fp16 = transpose(perm = gamma_135_perm_0, x = var_4449_cast_fp16_0)[name = string("transpose_189")]; + tensor var_4456_cast_fp16 = add(x = gamma_135_cast_fp16, y = var_4455_promoted_to_fp16)[name = string("op_4456_cast_fp16")]; + tensor var_4457_cast_fp16 = mul(x = var_4456_cast_fp16, y = x_615_cast_fp16)[name = string("op_4457_cast_fp16")]; + tensor beta_135_cast_fp16 = transpose(perm = beta_135_perm_0, x = var_4449_cast_fp16_1)[name = string("transpose_188")]; + tensor x_623_cast_fp16 = add(x = var_4457_cast_fp16, y = beta_135_cast_fp16)[name = string("x_623_cast_fp16")]; + tensor linear_138_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_617_cast_fp16)[name = string("linear_138_cast_fp16")]; + tensor linear_139_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_623_cast_fp16)[name = string("linear_139_cast_fp16")]; + tensor var_4463_split_sizes_0 = const()[name = string("op_4463_split_sizes_0"), val = tensor([512, 512])]; + int32 var_4463_axis_0 = const()[name = string("op_4463_axis_0"), val = int32(-1)]; + tensor var_4463_cast_fp16_0, tensor var_4463_cast_fp16_1 = split(axis = var_4463_axis_0, split_sizes = var_4463_split_sizes_0, x = linear_139_cast_fp16)[name = string("op_4463_cast_fp16")]; + tensor var_4471 = const()[name = string("op_4471"), val = tensor([1, 128, 8, 64])]; + tensor x_627_cast_fp16 = reshape(shape = var_4471, x = linear_138_cast_fp16)[name = string("x_627_cast_fp16")]; + tensor var_4481 = const()[name = string("op_4481"), val = tensor([1, 128, 8, 64])]; + tensor x_631_cast_fp16 = reshape(shape = var_4481, x = var_4463_cast_fp16_0)[name = string("x_631_cast_fp16")]; + tensor var_4491 = const()[name = string("op_4491"), val = tensor([1, 128, 8, 64])]; + tensor x_635_cast_fp16 = reshape(shape = var_4491, x = var_4463_cast_fp16_1)[name = string("x_635_cast_fp16")]; + tensor var_4493 = const()[name = string("op_4493"), val = tensor([0, 2, 1, 3])]; + bool sim_65_transpose_x_0 = const()[name = string("sim_65_transpose_x_0"), val = bool(false)]; + bool sim_65_transpose_y_0 = const()[name = string("sim_65_transpose_y_0"), val = bool(false)]; + tensor transpose_104_perm_0 = const()[name = string("transpose_104_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_105_perm_0 = const()[name = string("transpose_105_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_105 = transpose(perm = transpose_105_perm_0, x = x_631_cast_fp16)[name = string("transpose_185")]; + tensor transpose_104 = transpose(perm = transpose_104_perm_0, x = x_627_cast_fp16)[name = string("transpose_186")]; + tensor sim_65_cast_fp16 = matmul(transpose_x = sim_65_transpose_x_0, transpose_y = sim_65_transpose_y_0, x = transpose_104, y = transpose_105)[name = string("sim_65_cast_fp16")]; + fp16 var_4497_to_fp16 = const()[name = string("op_4497_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_67_cast_fp16 = mul(x = sim_65_cast_fp16, y = var_4497_to_fp16)[name = string("sim_67_cast_fp16")]; + tensor attn_33_cast_fp16 = softmax(axis = var_3907, x = sim_67_cast_fp16)[name = string("attn_33_cast_fp16")]; + bool x_637_transpose_x_0 = const()[name = string("x_637_transpose_x_0"), val = bool(false)]; + bool x_637_transpose_y_0 = const()[name = string("x_637_transpose_y_0"), val = bool(false)]; + tensor v_33_cast_fp16 = transpose(perm = var_4493, x = x_635_cast_fp16)[name = string("transpose_187")]; + tensor x_637_cast_fp16 = matmul(transpose_x = x_637_transpose_x_0, transpose_y = x_637_transpose_y_0, x = attn_33_cast_fp16, y = v_33_cast_fp16)[name = string("x_637_cast_fp16")]; + tensor var_4519 = const()[name = string("op_4519"), val = tensor([0, 2, 1, 3])]; + tensor var_4521 = const()[name = string("op_4521"), val = tensor([1, 128, 512])]; + tensor x_639_cast_fp16 = transpose(perm = var_4519, x = x_637_cast_fp16)[name = string("transpose_184")]; + tensor input_359_cast_fp16 = reshape(shape = var_4521, x = x_639_cast_fp16)[name = string("input_359_cast_fp16")]; + tensor linear_140_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_359_cast_fp16)[name = string("linear_140_cast_fp16")]; + tensor input_361_cast_fp16 = add(x = linear_140_cast_fp16, y = x_611_cast_fp16)[name = string("input_361_cast_fp16")]; + tensor linear_141_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_361_cast_fp16)[name = string("linear_141_cast_fp16")]; + string input_365_mode_0 = const()[name = string("input_365_mode_0"), val = string("EXACT")]; + tensor input_365_cast_fp16 = gelu(mode = input_365_mode_0, x = linear_141_cast_fp16)[name = string("input_365_cast_fp16")]; + tensor linear_142_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_365_cast_fp16)[name = string("linear_142_cast_fp16")]; + tensor x_641_cast_fp16 = add(x = linear_142_cast_fp16, y = input_361_cast_fp16)[name = string("x_641_cast_fp16")]; + tensor x_643_cast_fp16 = add(x = x_641_cast_fp16, y = mapping_23_cast_fp16)[name = string("x_643_cast_fp16")]; + tensor var_4537_split_sizes_0 = const()[name = string("op_4537_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4537_axis_0 = const()[name = string("op_4537_axis_0"), val = int32(1)]; + tensor var_4537_cast_fp16_0, tensor var_4537_cast_fp16_1 = split(axis = var_4537_axis_0, split_sizes = var_4537_split_sizes_0, x = h_19_cast_fp16)[name = string("op_4537_cast_fp16")]; + tensor gamma_139_perm_0 = const()[name = string("gamma_139_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_139_perm_0 = const()[name = string("beta_139_perm_0"), val = tensor([0, -1, 1])]; + tensor x_647_axes_0 = const()[name = string("x_647_axes_0"), val = tensor([-1])]; + tensor x_647_cast_fp16 = layer_norm(axes = x_647_axes_0, epsilon = var_3903_to_fp16, x = x_643_cast_fp16)[name = string("x_647_cast_fp16")]; + fp16 var_4543_promoted_to_fp16 = const()[name = string("op_4543_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_139_cast_fp16 = transpose(perm = gamma_139_perm_0, x = var_4537_cast_fp16_0)[name = string("transpose_183")]; + tensor var_4544_cast_fp16 = add(x = gamma_139_cast_fp16, y = var_4543_promoted_to_fp16)[name = string("op_4544_cast_fp16")]; + tensor var_4545_cast_fp16 = mul(x = var_4544_cast_fp16, y = x_647_cast_fp16)[name = string("op_4545_cast_fp16")]; + tensor beta_139_cast_fp16 = transpose(perm = beta_139_perm_0, x = var_4537_cast_fp16_1)[name = string("transpose_182")]; + tensor x_649_cast_fp16 = add(x = var_4545_cast_fp16, y = beta_139_cast_fp16)[name = string("x_649_cast_fp16")]; + tensor var_4556_split_sizes_0 = const()[name = string("op_4556_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4556_axis_0 = const()[name = string("op_4556_axis_0"), val = int32(1)]; + tensor var_4556_cast_fp16_0, tensor var_4556_cast_fp16_1 = split(axis = var_4556_axis_0, split_sizes = var_4556_split_sizes_0, x = h_23_cast_fp16)[name = string("op_4556_cast_fp16")]; + tensor gamma_143_perm_0 = const()[name = string("gamma_143_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_143_perm_0 = const()[name = string("beta_143_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_4562_promoted_to_fp16 = const()[name = string("op_4562_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_143_cast_fp16 = transpose(perm = gamma_143_perm_0, x = var_4556_cast_fp16_0)[name = string("transpose_181")]; + tensor var_4563_cast_fp16 = add(x = gamma_143_cast_fp16, y = var_4562_promoted_to_fp16)[name = string("op_4563_cast_fp16")]; + tensor var_4564_cast_fp16 = mul(x = var_4563_cast_fp16, y = x_647_cast_fp16)[name = string("op_4564_cast_fp16")]; + tensor beta_143_cast_fp16 = transpose(perm = beta_143_perm_0, x = var_4556_cast_fp16_1)[name = string("transpose_180")]; + tensor x_655_cast_fp16 = add(x = var_4564_cast_fp16, y = beta_143_cast_fp16)[name = string("x_655_cast_fp16")]; + tensor linear_145_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_649_cast_fp16)[name = string("linear_145_cast_fp16")]; + tensor linear_146_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_655_cast_fp16)[name = string("linear_146_cast_fp16")]; + tensor var_4570_split_sizes_0 = const()[name = string("op_4570_split_sizes_0"), val = tensor([512, 512])]; + int32 var_4570_axis_0 = const()[name = string("op_4570_axis_0"), val = int32(-1)]; + tensor var_4570_cast_fp16_0, tensor var_4570_cast_fp16_1 = split(axis = var_4570_axis_0, split_sizes = var_4570_split_sizes_0, x = linear_146_cast_fp16)[name = string("op_4570_cast_fp16")]; + tensor var_4578 = const()[name = string("op_4578"), val = tensor([1, 128, 8, 64])]; + tensor x_659_cast_fp16 = reshape(shape = var_4578, x = linear_145_cast_fp16)[name = string("x_659_cast_fp16")]; + tensor var_4588 = const()[name = string("op_4588"), val = tensor([1, 128, 8, 64])]; + tensor x_663_cast_fp16 = reshape(shape = var_4588, x = var_4570_cast_fp16_0)[name = string("x_663_cast_fp16")]; + tensor var_4598 = const()[name = string("op_4598"), val = tensor([1, 128, 8, 64])]; + tensor x_667_cast_fp16 = reshape(shape = var_4598, x = var_4570_cast_fp16_1)[name = string("x_667_cast_fp16")]; + tensor var_4600 = const()[name = string("op_4600"), val = tensor([0, 2, 1, 3])]; + bool sim_69_transpose_x_0 = const()[name = string("sim_69_transpose_x_0"), val = bool(false)]; + bool sim_69_transpose_y_0 = const()[name = string("sim_69_transpose_y_0"), val = bool(false)]; + tensor transpose_106_perm_0 = const()[name = string("transpose_106_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_107_perm_0 = const()[name = string("transpose_107_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_107 = transpose(perm = transpose_107_perm_0, x = x_663_cast_fp16)[name = string("transpose_177")]; + tensor transpose_106 = transpose(perm = transpose_106_perm_0, x = x_659_cast_fp16)[name = string("transpose_178")]; + tensor sim_69_cast_fp16 = matmul(transpose_x = sim_69_transpose_x_0, transpose_y = sim_69_transpose_y_0, x = transpose_106, y = transpose_107)[name = string("sim_69_cast_fp16")]; + fp16 var_4604_to_fp16 = const()[name = string("op_4604_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_71_cast_fp16 = mul(x = sim_69_cast_fp16, y = var_4604_to_fp16)[name = string("sim_71_cast_fp16")]; + tensor attn_35_cast_fp16 = softmax(axis = var_3907, x = sim_71_cast_fp16)[name = string("attn_35_cast_fp16")]; + bool x_669_transpose_x_0 = const()[name = string("x_669_transpose_x_0"), val = bool(false)]; + bool x_669_transpose_y_0 = const()[name = string("x_669_transpose_y_0"), val = bool(false)]; + tensor v_35_cast_fp16 = transpose(perm = var_4600, x = x_667_cast_fp16)[name = string("transpose_179")]; + tensor x_669_cast_fp16 = matmul(transpose_x = x_669_transpose_x_0, transpose_y = x_669_transpose_y_0, x = attn_35_cast_fp16, y = v_35_cast_fp16)[name = string("x_669_cast_fp16")]; + tensor var_4626 = const()[name = string("op_4626"), val = tensor([0, 2, 1, 3])]; + tensor var_4628 = const()[name = string("op_4628"), val = tensor([1, 128, 512])]; + tensor x_671_cast_fp16 = transpose(perm = var_4626, x = x_669_cast_fp16)[name = string("transpose_176")]; + tensor input_375_cast_fp16 = reshape(shape = var_4628, x = x_671_cast_fp16)[name = string("input_375_cast_fp16")]; + tensor linear_147_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_375_cast_fp16)[name = string("linear_147_cast_fp16")]; + tensor input_377_cast_fp16 = add(x = linear_147_cast_fp16, y = x_643_cast_fp16)[name = string("input_377_cast_fp16")]; + tensor linear_148_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_377_cast_fp16)[name = string("linear_148_cast_fp16")]; + string input_381_mode_0 = const()[name = string("input_381_mode_0"), val = string("EXACT")]; + tensor input_381_cast_fp16 = gelu(mode = input_381_mode_0, x = linear_148_cast_fp16)[name = string("input_381_cast_fp16")]; + tensor linear_149_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_381_cast_fp16)[name = string("linear_149_cast_fp16")]; + tensor x_673_cast_fp16 = add(x = linear_149_cast_fp16, y = input_377_cast_fp16)[name = string("x_673_cast_fp16")]; + tensor var_4637_axes_0 = const()[name = string("op_4637_axes_0"), val = tensor([1])]; + bool var_4637_keep_dims_0 = const()[name = string("op_4637_keep_dims_0"), val = bool(false)]; + tensor var_4637_cast_fp16 = reduce_mean(axes = var_4637_axes_0, keep_dims = var_4637_keep_dims_0, x = x_673_cast_fp16)[name = string("op_4637_cast_fp16")]; + tensor x_675_axes_0 = const()[name = string("x_675_axes_0"), val = tensor([1])]; + tensor x_675_cast_fp16 = expand_dims(axes = x_675_axes_0, x = var_4637_cast_fp16)[name = string("x_675_cast_fp16")]; + tensor var_4639 = const()[name = string("op_4639"), val = tensor([0, 2, 1])]; + string x_677_pad_type_0 = const()[name = string("x_677_pad_type_0"), val = string("valid")]; + tensor x_677_strides_0 = const()[name = string("x_677_strides_0"), val = tensor([1])]; + tensor x_677_pad_0 = const()[name = string("x_677_pad_0"), val = tensor([0, 0])]; + tensor x_677_dilations_0 = const()[name = string("x_677_dilations_0"), val = tensor([1])]; + int32 x_677_groups_0 = const()[name = string("x_677_groups_0"), val = int32(1)]; + tensor input_383_cast_fp16 = transpose(perm = var_4639, x = x_675_cast_fp16)[name = string("transpose_175")]; + tensor x_677_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_677_dilations_0, groups = x_677_groups_0, pad = x_677_pad_0, pad_type = x_677_pad_type_0, strides = x_677_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_383_cast_fp16)[name = string("x_677_cast_fp16")]; + tensor x_pred_11_perm_0 = const()[name = string("x_pred_11_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_11_to_fp16 = const()[name = string("c_skip_11_to_fp16"), val = tensor([[[0x1.ef4p-1]]])]; + tensor var_4647_cast_fp16 = mul(x = c_skip_11_to_fp16, y = x_noisy_11_cast_fp16)[name = string("op_4647_cast_fp16")]; + tensor c_out_11_to_fp16 = const()[name = string("c_out_11_to_fp16"), val = tensor([[[0x1.1dp-5]]])]; + tensor x_pred_11_cast_fp16 = transpose(perm = x_pred_11_perm_0, x = x_677_cast_fp16)[name = string("transpose_174")]; + tensor var_4648_cast_fp16 = mul(x = c_out_11_to_fp16, y = x_pred_11_cast_fp16)[name = string("op_4648_cast_fp16")]; + tensor x_mid_dn_5_cast_fp16 = add(x = var_4647_cast_fp16, y = var_4648_cast_fp16)[name = string("x_mid_dn_5_cast_fp16")]; + tensor var_4651_cast_fp16 = sub(x = x_noisy_11_cast_fp16, y = x_mid_dn_5_cast_fp16)[name = string("op_4651_cast_fp16")]; + tensor _inversed_d_mid_5_y_0_to_fp16 = const()[name = string("_inversed_d_mid_5_y_0_to_fp16"), val = tensor([0x1.c4cp+4])]; + tensor _inversed_d_mid_5_cast_fp16 = mul(x = var_4651_cast_fp16, y = _inversed_d_mid_5_y_0_to_fp16)[name = string("_inversed_d_mid_5_cast_fp16")]; + fp16 var_4660_to_fp16 = const()[name = string("op_4660_to_fp16"), val = fp16(-0x1.1fp-4)]; + tensor var_4661_cast_fp16 = mul(x = _inversed_d_mid_5_cast_fp16, y = var_4660_to_fp16)[name = string("op_4661_cast_fp16")]; + tensor x_679_cast_fp16 = add(x = x_noisy_9_cast_fp16, y = var_4661_cast_fp16)[name = string("x_679_cast_fp16")]; + tensor var_4666_begin_0 = const()[name = string("op_4666_begin_0"), val = tensor([2, 0, 0, 0])]; + tensor var_4666_end_0 = const()[name = string("op_4666_end_0"), val = tensor([3, 1, 1, 256])]; + tensor var_4666_end_mask_0 = const()[name = string("op_4666_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_4666_squeeze_mask_0 = const()[name = string("op_4666_squeeze_mask_0"), val = tensor([true, false, false, false])]; + tensor var_4666_cast_fp16 = slice_by_index(begin = var_4666_begin_0, end = var_4666_end_0, end_mask = var_4666_end_mask_0, squeeze_mask = var_4666_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_4666_cast_fp16")]; + fp16 var_4669_to_fp16 = const()[name = string("op_4669_to_fp16"), val = fp16(0x1.37p-8)]; + tensor var_4670_cast_fp16 = mul(x = var_4666_cast_fp16, y = var_4669_to_fp16)[name = string("op_4670_cast_fp16")]; + tensor x_noisy_13_cast_fp16 = add(x = x_679_cast_fp16, y = var_4670_cast_fp16)[name = string("x_noisy_13_cast_fp16")]; + int32 var_4694 = const()[name = string("op_4694"), val = int32(-1)]; + tensor c_in_13_to_fp16 = const()[name = string("c_in_13_to_fp16"), val = tensor([[[0x1.41p+2]]])]; + tensor x_689_cast_fp16 = mul(x = c_in_13_to_fp16, y = x_noisy_13_cast_fp16)[name = string("x_689_cast_fp16")]; + int32 x_685_axis_0 = const()[name = string("x_685_axis_0"), val = int32(0)]; + tensor var_5080_to_fp16 = const()[name = string("op_5080_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49357632)))]; + tensor x_685_cast_fp16 = stack(axis = x_685_axis_0, values = (var_5080_to_fp16, var_423_cast_fp16))[name = string("x_685_cast_fp16")]; + tensor var_5085 = const()[name = string("op_5085"), val = tensor([1, 2, 0])]; + tensor input_391_axes_0 = const()[name = string("input_391_axes_0"), val = tensor([2])]; + bool input_391_keep_dims_0 = const()[name = string("input_391_keep_dims_0"), val = bool(false)]; + tensor x_687_cast_fp16 = transpose(perm = var_5085, x = x_685_cast_fp16)[name = string("transpose_173")]; + tensor input_391_cast_fp16 = reduce_sum(axes = input_391_axes_0, keep_dims = input_391_keep_dims_0, x = x_687_cast_fp16)[name = string("input_391_cast_fp16")]; + tensor linear_152_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_391_cast_fp16)[name = string("linear_152_cast_fp16")]; + string input_395_mode_0 = const()[name = string("input_395_mode_0"), val = string("EXACT")]; + tensor input_395_cast_fp16 = gelu(mode = input_395_mode_0, x = linear_152_cast_fp16)[name = string("input_395_cast_fp16")]; + tensor linear_153_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_395_cast_fp16)[name = string("linear_153_cast_fp16")]; + string mapping_25_mode_0 = const()[name = string("mapping_25_mode_0"), val = string("EXACT")]; + tensor mapping_25_cast_fp16 = gelu(mode = mapping_25_mode_0, x = linear_153_cast_fp16)[name = string("mapping_25_cast_fp16")]; + tensor var_5095_reps_0 = const()[name = string("op_5095_reps_0"), val = tensor([1, 128, 1])]; + tensor var_5095_cast_fp16 = tile(reps = var_5095_reps_0, x = x_689_cast_fp16)[name = string("op_5095_cast_fp16")]; + bool x_691_interleave_0 = const()[name = string("x_691_interleave_0"), val = bool(false)]; + tensor x_691_cast_fp16 = concat(axis = var_4694, interleave = x_691_interleave_0, values = (var_5095_cast_fp16, embedding_to_fp16))[name = string("x_691_cast_fp16")]; + tensor var_5098_axes_0 = const()[name = string("op_5098_axes_0"), val = tensor([1])]; + tensor var_5098_cast_fp16 = expand_dims(axes = var_5098_axes_0, x = mapping_25_cast_fp16)[name = string("op_5098_cast_fp16")]; + tensor mapping_27_reps_0 = const()[name = string("mapping_27_reps_0"), val = tensor([1, 128, 1])]; + tensor mapping_27_cast_fp16 = tile(reps = mapping_27_reps_0, x = var_5098_cast_fp16)[name = string("mapping_27_cast_fp16")]; + tensor x_693_cast_fp16 = add(x = x_691_cast_fp16, y = mapping_27_cast_fp16)[name = string("x_693_cast_fp16")]; + tensor var_5110_split_sizes_0 = const()[name = string("op_5110_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5110_axis_0 = const()[name = string("op_5110_axis_0"), val = int32(1)]; + tensor var_5110_cast_fp16_0, tensor var_5110_cast_fp16_1 = split(axis = var_5110_axis_0, split_sizes = var_5110_split_sizes_0, x = h_3_cast_fp16)[name = string("op_5110_cast_fp16")]; + tensor gamma_147_perm_0 = const()[name = string("gamma_147_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_147_perm_0 = const()[name = string("beta_147_perm_0"), val = tensor([0, -1, 1])]; + tensor x_697_axes_0 = const()[name = string("x_697_axes_0"), val = tensor([-1])]; + fp16 var_4690_to_fp16 = const()[name = string("op_4690_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_697_cast_fp16 = layer_norm(axes = x_697_axes_0, epsilon = var_4690_to_fp16, x = x_693_cast_fp16)[name = string("x_697_cast_fp16")]; + fp16 var_5116_promoted_to_fp16 = const()[name = string("op_5116_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_147_cast_fp16 = transpose(perm = gamma_147_perm_0, x = var_5110_cast_fp16_0)[name = string("transpose_172")]; + tensor var_5117_cast_fp16 = add(x = gamma_147_cast_fp16, y = var_5116_promoted_to_fp16)[name = string("op_5117_cast_fp16")]; + tensor var_5118_cast_fp16 = mul(x = var_5117_cast_fp16, y = x_697_cast_fp16)[name = string("op_5118_cast_fp16")]; + tensor beta_147_cast_fp16 = transpose(perm = beta_147_perm_0, x = var_5110_cast_fp16_1)[name = string("transpose_171")]; + tensor x_699_cast_fp16 = add(x = var_5118_cast_fp16, y = beta_147_cast_fp16)[name = string("x_699_cast_fp16")]; + tensor var_5129_split_sizes_0 = const()[name = string("op_5129_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5129_axis_0 = const()[name = string("op_5129_axis_0"), val = int32(1)]; + tensor var_5129_cast_fp16_0, tensor var_5129_cast_fp16_1 = split(axis = var_5129_axis_0, split_sizes = var_5129_split_sizes_0, x = h_7_cast_fp16)[name = string("op_5129_cast_fp16")]; + tensor gamma_151_perm_0 = const()[name = string("gamma_151_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_151_perm_0 = const()[name = string("beta_151_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5135_promoted_to_fp16 = const()[name = string("op_5135_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_151_cast_fp16 = transpose(perm = gamma_151_perm_0, x = var_5129_cast_fp16_0)[name = string("transpose_170")]; + tensor var_5136_cast_fp16 = add(x = gamma_151_cast_fp16, y = var_5135_promoted_to_fp16)[name = string("op_5136_cast_fp16")]; + tensor var_5137_cast_fp16 = mul(x = var_5136_cast_fp16, y = x_697_cast_fp16)[name = string("op_5137_cast_fp16")]; + tensor beta_151_cast_fp16 = transpose(perm = beta_151_perm_0, x = var_5129_cast_fp16_1)[name = string("transpose_169")]; + tensor x_705_cast_fp16 = add(x = var_5137_cast_fp16, y = beta_151_cast_fp16)[name = string("x_705_cast_fp16")]; + tensor linear_156_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_699_cast_fp16)[name = string("linear_156_cast_fp16")]; + tensor linear_157_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_705_cast_fp16)[name = string("linear_157_cast_fp16")]; + tensor var_5143_split_sizes_0 = const()[name = string("op_5143_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5143_axis_0 = const()[name = string("op_5143_axis_0"), val = int32(-1)]; + tensor var_5143_cast_fp16_0, tensor var_5143_cast_fp16_1 = split(axis = var_5143_axis_0, split_sizes = var_5143_split_sizes_0, x = linear_157_cast_fp16)[name = string("op_5143_cast_fp16")]; + tensor var_5151 = const()[name = string("op_5151"), val = tensor([1, 128, 8, 64])]; + tensor x_709_cast_fp16 = reshape(shape = var_5151, x = linear_156_cast_fp16)[name = string("x_709_cast_fp16")]; + tensor var_5161 = const()[name = string("op_5161"), val = tensor([1, 128, 8, 64])]; + tensor x_713_cast_fp16 = reshape(shape = var_5161, x = var_5143_cast_fp16_0)[name = string("x_713_cast_fp16")]; + tensor var_5171 = const()[name = string("op_5171"), val = tensor([1, 128, 8, 64])]; + tensor x_717_cast_fp16 = reshape(shape = var_5171, x = var_5143_cast_fp16_1)[name = string("x_717_cast_fp16")]; + tensor var_5173 = const()[name = string("op_5173"), val = tensor([0, 2, 1, 3])]; + bool sim_73_transpose_x_0 = const()[name = string("sim_73_transpose_x_0"), val = bool(false)]; + bool sim_73_transpose_y_0 = const()[name = string("sim_73_transpose_y_0"), val = bool(false)]; + tensor transpose_108_perm_0 = const()[name = string("transpose_108_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_109_perm_0 = const()[name = string("transpose_109_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_109 = transpose(perm = transpose_109_perm_0, x = x_713_cast_fp16)[name = string("transpose_166")]; + tensor transpose_108 = transpose(perm = transpose_108_perm_0, x = x_709_cast_fp16)[name = string("transpose_167")]; + tensor sim_73_cast_fp16 = matmul(transpose_x = sim_73_transpose_x_0, transpose_y = sim_73_transpose_y_0, x = transpose_108, y = transpose_109)[name = string("sim_73_cast_fp16")]; + fp16 var_5177_to_fp16 = const()[name = string("op_5177_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_75_cast_fp16 = mul(x = sim_73_cast_fp16, y = var_5177_to_fp16)[name = string("sim_75_cast_fp16")]; + tensor attn_37_cast_fp16 = softmax(axis = var_4694, x = sim_75_cast_fp16)[name = string("attn_37_cast_fp16")]; + bool x_719_transpose_x_0 = const()[name = string("x_719_transpose_x_0"), val = bool(false)]; + bool x_719_transpose_y_0 = const()[name = string("x_719_transpose_y_0"), val = bool(false)]; + tensor v_37_cast_fp16 = transpose(perm = var_5173, x = x_717_cast_fp16)[name = string("transpose_168")]; + tensor x_719_cast_fp16 = matmul(transpose_x = x_719_transpose_x_0, transpose_y = x_719_transpose_y_0, x = attn_37_cast_fp16, y = v_37_cast_fp16)[name = string("x_719_cast_fp16")]; + tensor var_5199 = const()[name = string("op_5199"), val = tensor([0, 2, 1, 3])]; + tensor var_5201 = const()[name = string("op_5201"), val = tensor([1, 128, 512])]; + tensor x_721_cast_fp16 = transpose(perm = var_5199, x = x_719_cast_fp16)[name = string("transpose_165")]; + tensor input_407_cast_fp16 = reshape(shape = var_5201, x = x_721_cast_fp16)[name = string("input_407_cast_fp16")]; + tensor linear_158_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_407_cast_fp16)[name = string("linear_158_cast_fp16")]; + tensor input_409_cast_fp16 = add(x = linear_158_cast_fp16, y = x_693_cast_fp16)[name = string("input_409_cast_fp16")]; + tensor linear_159_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_409_cast_fp16)[name = string("linear_159_cast_fp16")]; + string input_413_mode_0 = const()[name = string("input_413_mode_0"), val = string("EXACT")]; + tensor input_413_cast_fp16 = gelu(mode = input_413_mode_0, x = linear_159_cast_fp16)[name = string("input_413_cast_fp16")]; + tensor linear_160_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_413_cast_fp16)[name = string("linear_160_cast_fp16")]; + tensor x_723_cast_fp16 = add(x = linear_160_cast_fp16, y = input_409_cast_fp16)[name = string("x_723_cast_fp16")]; + tensor x_725_cast_fp16 = add(x = x_723_cast_fp16, y = mapping_27_cast_fp16)[name = string("x_725_cast_fp16")]; + tensor var_5217_split_sizes_0 = const()[name = string("op_5217_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5217_axis_0 = const()[name = string("op_5217_axis_0"), val = int32(1)]; + tensor var_5217_cast_fp16_0, tensor var_5217_cast_fp16_1 = split(axis = var_5217_axis_0, split_sizes = var_5217_split_sizes_0, x = h_11_cast_fp16)[name = string("op_5217_cast_fp16")]; + tensor gamma_155_perm_0 = const()[name = string("gamma_155_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_155_perm_0 = const()[name = string("beta_155_perm_0"), val = tensor([0, -1, 1])]; + tensor x_729_axes_0 = const()[name = string("x_729_axes_0"), val = tensor([-1])]; + tensor x_729_cast_fp16 = layer_norm(axes = x_729_axes_0, epsilon = var_4690_to_fp16, x = x_725_cast_fp16)[name = string("x_729_cast_fp16")]; + fp16 var_5223_promoted_to_fp16 = const()[name = string("op_5223_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_155_cast_fp16 = transpose(perm = gamma_155_perm_0, x = var_5217_cast_fp16_0)[name = string("transpose_164")]; + tensor var_5224_cast_fp16 = add(x = gamma_155_cast_fp16, y = var_5223_promoted_to_fp16)[name = string("op_5224_cast_fp16")]; + tensor var_5225_cast_fp16 = mul(x = var_5224_cast_fp16, y = x_729_cast_fp16)[name = string("op_5225_cast_fp16")]; + tensor beta_155_cast_fp16 = transpose(perm = beta_155_perm_0, x = var_5217_cast_fp16_1)[name = string("transpose_163")]; + tensor x_731_cast_fp16 = add(x = var_5225_cast_fp16, y = beta_155_cast_fp16)[name = string("x_731_cast_fp16")]; + tensor var_5236_split_sizes_0 = const()[name = string("op_5236_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5236_axis_0 = const()[name = string("op_5236_axis_0"), val = int32(1)]; + tensor var_5236_cast_fp16_0, tensor var_5236_cast_fp16_1 = split(axis = var_5236_axis_0, split_sizes = var_5236_split_sizes_0, x = h_15_cast_fp16)[name = string("op_5236_cast_fp16")]; + tensor gamma_159_perm_0 = const()[name = string("gamma_159_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_159_perm_0 = const()[name = string("beta_159_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5242_promoted_to_fp16 = const()[name = string("op_5242_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_159_cast_fp16 = transpose(perm = gamma_159_perm_0, x = var_5236_cast_fp16_0)[name = string("transpose_162")]; + tensor var_5243_cast_fp16 = add(x = gamma_159_cast_fp16, y = var_5242_promoted_to_fp16)[name = string("op_5243_cast_fp16")]; + tensor var_5244_cast_fp16 = mul(x = var_5243_cast_fp16, y = x_729_cast_fp16)[name = string("op_5244_cast_fp16")]; + tensor beta_159_cast_fp16 = transpose(perm = beta_159_perm_0, x = var_5236_cast_fp16_1)[name = string("transpose_161")]; + tensor x_737_cast_fp16 = add(x = var_5244_cast_fp16, y = beta_159_cast_fp16)[name = string("x_737_cast_fp16")]; + tensor linear_163_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_731_cast_fp16)[name = string("linear_163_cast_fp16")]; + tensor linear_164_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_737_cast_fp16)[name = string("linear_164_cast_fp16")]; + tensor var_5250_split_sizes_0 = const()[name = string("op_5250_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5250_axis_0 = const()[name = string("op_5250_axis_0"), val = int32(-1)]; + tensor var_5250_cast_fp16_0, tensor var_5250_cast_fp16_1 = split(axis = var_5250_axis_0, split_sizes = var_5250_split_sizes_0, x = linear_164_cast_fp16)[name = string("op_5250_cast_fp16")]; + tensor var_5258 = const()[name = string("op_5258"), val = tensor([1, 128, 8, 64])]; + tensor x_741_cast_fp16 = reshape(shape = var_5258, x = linear_163_cast_fp16)[name = string("x_741_cast_fp16")]; + tensor var_5268 = const()[name = string("op_5268"), val = tensor([1, 128, 8, 64])]; + tensor x_745_cast_fp16 = reshape(shape = var_5268, x = var_5250_cast_fp16_0)[name = string("x_745_cast_fp16")]; + tensor var_5278 = const()[name = string("op_5278"), val = tensor([1, 128, 8, 64])]; + tensor x_749_cast_fp16 = reshape(shape = var_5278, x = var_5250_cast_fp16_1)[name = string("x_749_cast_fp16")]; + tensor var_5280 = const()[name = string("op_5280"), val = tensor([0, 2, 1, 3])]; + bool sim_77_transpose_x_0 = const()[name = string("sim_77_transpose_x_0"), val = bool(false)]; + bool sim_77_transpose_y_0 = const()[name = string("sim_77_transpose_y_0"), val = bool(false)]; + tensor transpose_110_perm_0 = const()[name = string("transpose_110_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_111_perm_0 = const()[name = string("transpose_111_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_111 = transpose(perm = transpose_111_perm_0, x = x_745_cast_fp16)[name = string("transpose_158")]; + tensor transpose_110 = transpose(perm = transpose_110_perm_0, x = x_741_cast_fp16)[name = string("transpose_159")]; + tensor sim_77_cast_fp16 = matmul(transpose_x = sim_77_transpose_x_0, transpose_y = sim_77_transpose_y_0, x = transpose_110, y = transpose_111)[name = string("sim_77_cast_fp16")]; + fp16 var_5284_to_fp16 = const()[name = string("op_5284_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_79_cast_fp16 = mul(x = sim_77_cast_fp16, y = var_5284_to_fp16)[name = string("sim_79_cast_fp16")]; + tensor attn_39_cast_fp16 = softmax(axis = var_4694, x = sim_79_cast_fp16)[name = string("attn_39_cast_fp16")]; + bool x_751_transpose_x_0 = const()[name = string("x_751_transpose_x_0"), val = bool(false)]; + bool x_751_transpose_y_0 = const()[name = string("x_751_transpose_y_0"), val = bool(false)]; + tensor v_39_cast_fp16 = transpose(perm = var_5280, x = x_749_cast_fp16)[name = string("transpose_160")]; + tensor x_751_cast_fp16 = matmul(transpose_x = x_751_transpose_x_0, transpose_y = x_751_transpose_y_0, x = attn_39_cast_fp16, y = v_39_cast_fp16)[name = string("x_751_cast_fp16")]; + tensor var_5306 = const()[name = string("op_5306"), val = tensor([0, 2, 1, 3])]; + tensor var_5308 = const()[name = string("op_5308"), val = tensor([1, 128, 512])]; + tensor x_753_cast_fp16 = transpose(perm = var_5306, x = x_751_cast_fp16)[name = string("transpose_157")]; + tensor input_423_cast_fp16 = reshape(shape = var_5308, x = x_753_cast_fp16)[name = string("input_423_cast_fp16")]; + tensor linear_165_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_423_cast_fp16)[name = string("linear_165_cast_fp16")]; + tensor input_425_cast_fp16 = add(x = linear_165_cast_fp16, y = x_725_cast_fp16)[name = string("input_425_cast_fp16")]; + tensor linear_166_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_425_cast_fp16)[name = string("linear_166_cast_fp16")]; + string input_429_mode_0 = const()[name = string("input_429_mode_0"), val = string("EXACT")]; + tensor input_429_cast_fp16 = gelu(mode = input_429_mode_0, x = linear_166_cast_fp16)[name = string("input_429_cast_fp16")]; + tensor linear_167_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_429_cast_fp16)[name = string("linear_167_cast_fp16")]; + tensor x_755_cast_fp16 = add(x = linear_167_cast_fp16, y = input_425_cast_fp16)[name = string("x_755_cast_fp16")]; + tensor x_757_cast_fp16 = add(x = x_755_cast_fp16, y = mapping_27_cast_fp16)[name = string("x_757_cast_fp16")]; + tensor var_5324_split_sizes_0 = const()[name = string("op_5324_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5324_axis_0 = const()[name = string("op_5324_axis_0"), val = int32(1)]; + tensor var_5324_cast_fp16_0, tensor var_5324_cast_fp16_1 = split(axis = var_5324_axis_0, split_sizes = var_5324_split_sizes_0, x = h_19_cast_fp16)[name = string("op_5324_cast_fp16")]; + tensor gamma_163_perm_0 = const()[name = string("gamma_163_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_163_perm_0 = const()[name = string("beta_163_perm_0"), val = tensor([0, -1, 1])]; + tensor x_761_axes_0 = const()[name = string("x_761_axes_0"), val = tensor([-1])]; + tensor x_761_cast_fp16 = layer_norm(axes = x_761_axes_0, epsilon = var_4690_to_fp16, x = x_757_cast_fp16)[name = string("x_761_cast_fp16")]; + fp16 var_5330_promoted_to_fp16 = const()[name = string("op_5330_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_163_cast_fp16 = transpose(perm = gamma_163_perm_0, x = var_5324_cast_fp16_0)[name = string("transpose_156")]; + tensor var_5331_cast_fp16 = add(x = gamma_163_cast_fp16, y = var_5330_promoted_to_fp16)[name = string("op_5331_cast_fp16")]; + tensor var_5332_cast_fp16 = mul(x = var_5331_cast_fp16, y = x_761_cast_fp16)[name = string("op_5332_cast_fp16")]; + tensor beta_163_cast_fp16 = transpose(perm = beta_163_perm_0, x = var_5324_cast_fp16_1)[name = string("transpose_155")]; + tensor x_763_cast_fp16 = add(x = var_5332_cast_fp16, y = beta_163_cast_fp16)[name = string("x_763_cast_fp16")]; + tensor var_5343_split_sizes_0 = const()[name = string("op_5343_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5343_axis_0 = const()[name = string("op_5343_axis_0"), val = int32(1)]; + tensor var_5343_cast_fp16_0, tensor var_5343_cast_fp16_1 = split(axis = var_5343_axis_0, split_sizes = var_5343_split_sizes_0, x = h_23_cast_fp16)[name = string("op_5343_cast_fp16")]; + tensor gamma_167_perm_0 = const()[name = string("gamma_167_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_167_perm_0 = const()[name = string("beta_167_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5349_promoted_to_fp16 = const()[name = string("op_5349_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_167_cast_fp16 = transpose(perm = gamma_167_perm_0, x = var_5343_cast_fp16_0)[name = string("transpose_154")]; + tensor var_5350_cast_fp16 = add(x = gamma_167_cast_fp16, y = var_5349_promoted_to_fp16)[name = string("op_5350_cast_fp16")]; + tensor var_5351_cast_fp16 = mul(x = var_5350_cast_fp16, y = x_761_cast_fp16)[name = string("op_5351_cast_fp16")]; + tensor beta_167_cast_fp16 = transpose(perm = beta_167_perm_0, x = var_5343_cast_fp16_1)[name = string("transpose_153")]; + tensor x_769_cast_fp16 = add(x = var_5351_cast_fp16, y = beta_167_cast_fp16)[name = string("x_769_cast_fp16")]; + tensor linear_170_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_763_cast_fp16)[name = string("linear_170_cast_fp16")]; + tensor linear_171_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_769_cast_fp16)[name = string("linear_171_cast_fp16")]; + tensor var_5357_split_sizes_0 = const()[name = string("op_5357_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5357_axis_0 = const()[name = string("op_5357_axis_0"), val = int32(-1)]; + tensor var_5357_cast_fp16_0, tensor var_5357_cast_fp16_1 = split(axis = var_5357_axis_0, split_sizes = var_5357_split_sizes_0, x = linear_171_cast_fp16)[name = string("op_5357_cast_fp16")]; + tensor var_5365 = const()[name = string("op_5365"), val = tensor([1, 128, 8, 64])]; + tensor x_773_cast_fp16 = reshape(shape = var_5365, x = linear_170_cast_fp16)[name = string("x_773_cast_fp16")]; + tensor var_5375 = const()[name = string("op_5375"), val = tensor([1, 128, 8, 64])]; + tensor x_777_cast_fp16 = reshape(shape = var_5375, x = var_5357_cast_fp16_0)[name = string("x_777_cast_fp16")]; + tensor var_5385 = const()[name = string("op_5385"), val = tensor([1, 128, 8, 64])]; + tensor x_781_cast_fp16 = reshape(shape = var_5385, x = var_5357_cast_fp16_1)[name = string("x_781_cast_fp16")]; + tensor var_5387 = const()[name = string("op_5387"), val = tensor([0, 2, 1, 3])]; + bool sim_81_transpose_x_0 = const()[name = string("sim_81_transpose_x_0"), val = bool(false)]; + bool sim_81_transpose_y_0 = const()[name = string("sim_81_transpose_y_0"), val = bool(false)]; + tensor transpose_112_perm_0 = const()[name = string("transpose_112_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_113_perm_0 = const()[name = string("transpose_113_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_113 = transpose(perm = transpose_113_perm_0, x = x_777_cast_fp16)[name = string("transpose_150")]; + tensor transpose_112 = transpose(perm = transpose_112_perm_0, x = x_773_cast_fp16)[name = string("transpose_151")]; + tensor sim_81_cast_fp16 = matmul(transpose_x = sim_81_transpose_x_0, transpose_y = sim_81_transpose_y_0, x = transpose_112, y = transpose_113)[name = string("sim_81_cast_fp16")]; + fp16 var_5391_to_fp16 = const()[name = string("op_5391_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_83_cast_fp16 = mul(x = sim_81_cast_fp16, y = var_5391_to_fp16)[name = string("sim_83_cast_fp16")]; + tensor attn_41_cast_fp16 = softmax(axis = var_4694, x = sim_83_cast_fp16)[name = string("attn_41_cast_fp16")]; + bool x_783_transpose_x_0 = const()[name = string("x_783_transpose_x_0"), val = bool(false)]; + bool x_783_transpose_y_0 = const()[name = string("x_783_transpose_y_0"), val = bool(false)]; + tensor v_41_cast_fp16 = transpose(perm = var_5387, x = x_781_cast_fp16)[name = string("transpose_152")]; + tensor x_783_cast_fp16 = matmul(transpose_x = x_783_transpose_x_0, transpose_y = x_783_transpose_y_0, x = attn_41_cast_fp16, y = v_41_cast_fp16)[name = string("x_783_cast_fp16")]; + tensor var_5413 = const()[name = string("op_5413"), val = tensor([0, 2, 1, 3])]; + tensor var_5415 = const()[name = string("op_5415"), val = tensor([1, 128, 512])]; + tensor x_785_cast_fp16 = transpose(perm = var_5413, x = x_783_cast_fp16)[name = string("transpose_149")]; + tensor input_439_cast_fp16 = reshape(shape = var_5415, x = x_785_cast_fp16)[name = string("input_439_cast_fp16")]; + tensor linear_172_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_439_cast_fp16)[name = string("linear_172_cast_fp16")]; + tensor input_441_cast_fp16 = add(x = linear_172_cast_fp16, y = x_757_cast_fp16)[name = string("input_441_cast_fp16")]; + tensor linear_173_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_441_cast_fp16)[name = string("linear_173_cast_fp16")]; + string input_445_mode_0 = const()[name = string("input_445_mode_0"), val = string("EXACT")]; + tensor input_445_cast_fp16 = gelu(mode = input_445_mode_0, x = linear_173_cast_fp16)[name = string("input_445_cast_fp16")]; + tensor linear_174_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_445_cast_fp16)[name = string("linear_174_cast_fp16")]; + tensor x_787_cast_fp16 = add(x = linear_174_cast_fp16, y = input_441_cast_fp16)[name = string("x_787_cast_fp16")]; + tensor var_5424_axes_0 = const()[name = string("op_5424_axes_0"), val = tensor([1])]; + bool var_5424_keep_dims_0 = const()[name = string("op_5424_keep_dims_0"), val = bool(false)]; + tensor var_5424_cast_fp16 = reduce_mean(axes = var_5424_axes_0, keep_dims = var_5424_keep_dims_0, x = x_787_cast_fp16)[name = string("op_5424_cast_fp16")]; + tensor x_789_axes_0 = const()[name = string("x_789_axes_0"), val = tensor([1])]; + tensor x_789_cast_fp16 = expand_dims(axes = x_789_axes_0, x = var_5424_cast_fp16)[name = string("x_789_cast_fp16")]; + tensor var_5426 = const()[name = string("op_5426"), val = tensor([0, 2, 1])]; + string x_791_pad_type_0 = const()[name = string("x_791_pad_type_0"), val = string("valid")]; + tensor x_791_strides_0 = const()[name = string("x_791_strides_0"), val = tensor([1])]; + tensor x_791_pad_0 = const()[name = string("x_791_pad_0"), val = tensor([0, 0])]; + tensor x_791_dilations_0 = const()[name = string("x_791_dilations_0"), val = tensor([1])]; + int32 x_791_groups_0 = const()[name = string("x_791_groups_0"), val = int32(1)]; + tensor input_447_cast_fp16 = transpose(perm = var_5426, x = x_789_cast_fp16)[name = string("transpose_148")]; + tensor x_791_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_791_dilations_0, groups = x_791_groups_0, pad = x_791_pad_0, pad_type = x_791_pad_type_0, strides = x_791_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_447_cast_fp16)[name = string("x_791_cast_fp16")]; + tensor x_pred_13_perm_0 = const()[name = string("x_pred_13_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_13_to_fp16 = const()[name = string("c_skip_13_to_fp16"), val = tensor([[[0x1.fe8p-1]]])]; + tensor var_5434_cast_fp16 = mul(x = c_skip_13_to_fp16, y = x_noisy_13_cast_fp16)[name = string("op_5434_cast_fp16")]; + tensor c_out_13_to_fp16 = const()[name = string("c_out_13_to_fp16"), val = tensor([[[0x1.37cp-8]]])]; + tensor x_pred_13_cast_fp16 = transpose(perm = x_pred_13_perm_0, x = x_791_cast_fp16)[name = string("transpose_147")]; + tensor var_5435_cast_fp16 = mul(x = c_out_13_to_fp16, y = x_pred_13_cast_fp16)[name = string("op_5435_cast_fp16")]; + tensor x_dn_cast_fp16 = add(x = var_5434_cast_fp16, y = var_5435_cast_fp16)[name = string("x_dn_cast_fp16")]; + tensor var_5438_cast_fp16 = sub(x = x_noisy_13_cast_fp16, y = x_dn_cast_fp16)[name = string("op_5438_cast_fp16")]; + tensor _inversed_d_y_0_to_fp16 = const()[name = string("_inversed_d_y_0_to_fp16"), val = tensor([0x1.a44p+7])]; + tensor _inversed_d_cast_fp16 = mul(x = var_5438_cast_fp16, y = _inversed_d_y_0_to_fp16)[name = string("_inversed_d_cast_fp16")]; + fp16 var_5447_to_fp16 = const()[name = string("op_5447_to_fp16"), val = fp16(-0x1.37cp-9)]; + tensor var_5448_cast_fp16 = mul(x = _inversed_d_cast_fp16, y = var_5447_to_fp16)[name = string("op_5448_cast_fp16")]; + tensor x_noisy_cast_fp16 = add(x = x_noisy_13_cast_fp16, y = var_5448_cast_fp16)[name = string("x_noisy_cast_fp16")]; + int32 var_5460 = const()[name = string("op_5460"), val = int32(-1)]; + tensor c_in_to_fp16 = const()[name = string("c_in_to_fp16"), val = tensor([[[0x1.414p+2]]])]; + tensor x_801_cast_fp16 = mul(x = c_in_to_fp16, y = x_noisy_cast_fp16)[name = string("x_801_cast_fp16")]; + int32 x_797_axis_0 = const()[name = string("x_797_axis_0"), val = int32(0)]; + tensor var_5846_to_fp16 = const()[name = string("op_5846_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49359744)))]; + tensor x_797_cast_fp16 = stack(axis = x_797_axis_0, values = (var_5846_to_fp16, var_423_cast_fp16))[name = string("x_797_cast_fp16")]; + tensor var_5851 = const()[name = string("op_5851"), val = tensor([1, 2, 0])]; + tensor input_455_axes_0 = const()[name = string("input_455_axes_0"), val = tensor([2])]; + bool input_455_keep_dims_0 = const()[name = string("input_455_keep_dims_0"), val = bool(false)]; + tensor x_799_cast_fp16 = transpose(perm = var_5851, x = x_797_cast_fp16)[name = string("transpose_146")]; + tensor input_455_cast_fp16 = reduce_sum(axes = input_455_axes_0, keep_dims = input_455_keep_dims_0, x = x_799_cast_fp16)[name = string("input_455_cast_fp16")]; + tensor linear_177_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_455_cast_fp16)[name = string("linear_177_cast_fp16")]; + string input_459_mode_0 = const()[name = string("input_459_mode_0"), val = string("EXACT")]; + tensor input_459_cast_fp16 = gelu(mode = input_459_mode_0, x = linear_177_cast_fp16)[name = string("input_459_cast_fp16")]; + tensor linear_178_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_459_cast_fp16)[name = string("linear_178_cast_fp16")]; + string mapping_29_mode_0 = const()[name = string("mapping_29_mode_0"), val = string("EXACT")]; + tensor mapping_29_cast_fp16 = gelu(mode = mapping_29_mode_0, x = linear_178_cast_fp16)[name = string("mapping_29_cast_fp16")]; + tensor var_5861_reps_0 = const()[name = string("op_5861_reps_0"), val = tensor([1, 128, 1])]; + tensor var_5861_cast_fp16 = tile(reps = var_5861_reps_0, x = x_801_cast_fp16)[name = string("op_5861_cast_fp16")]; + bool x_803_interleave_0 = const()[name = string("x_803_interleave_0"), val = bool(false)]; + tensor x_803_cast_fp16 = concat(axis = var_5460, interleave = x_803_interleave_0, values = (var_5861_cast_fp16, embedding_to_fp16))[name = string("x_803_cast_fp16")]; + tensor var_5864_axes_0 = const()[name = string("op_5864_axes_0"), val = tensor([1])]; + tensor var_5864_cast_fp16 = expand_dims(axes = var_5864_axes_0, x = mapping_29_cast_fp16)[name = string("op_5864_cast_fp16")]; + tensor mapping_reps_0 = const()[name = string("mapping_reps_0"), val = tensor([1, 128, 1])]; + tensor mapping_cast_fp16 = tile(reps = mapping_reps_0, x = var_5864_cast_fp16)[name = string("mapping_cast_fp16")]; + tensor x_805_cast_fp16 = add(x = x_803_cast_fp16, y = mapping_cast_fp16)[name = string("x_805_cast_fp16")]; + tensor var_5876_split_sizes_0 = const()[name = string("op_5876_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5876_axis_0 = const()[name = string("op_5876_axis_0"), val = int32(1)]; + tensor var_5876_cast_fp16_0, tensor var_5876_cast_fp16_1 = split(axis = var_5876_axis_0, split_sizes = var_5876_split_sizes_0, x = h_3_cast_fp16)[name = string("op_5876_cast_fp16")]; + tensor gamma_171_perm_0 = const()[name = string("gamma_171_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_171_perm_0 = const()[name = string("beta_171_perm_0"), val = tensor([0, -1, 1])]; + tensor x_809_axes_0 = const()[name = string("x_809_axes_0"), val = tensor([-1])]; + fp16 var_5456_to_fp16 = const()[name = string("op_5456_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_809_cast_fp16 = layer_norm(axes = x_809_axes_0, epsilon = var_5456_to_fp16, x = x_805_cast_fp16)[name = string("x_809_cast_fp16")]; + fp16 var_5882_promoted_to_fp16 = const()[name = string("op_5882_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_171_cast_fp16 = transpose(perm = gamma_171_perm_0, x = var_5876_cast_fp16_0)[name = string("transpose_145")]; + tensor var_5883_cast_fp16 = add(x = gamma_171_cast_fp16, y = var_5882_promoted_to_fp16)[name = string("op_5883_cast_fp16")]; + tensor var_5884_cast_fp16 = mul(x = var_5883_cast_fp16, y = x_809_cast_fp16)[name = string("op_5884_cast_fp16")]; + tensor beta_171_cast_fp16 = transpose(perm = beta_171_perm_0, x = var_5876_cast_fp16_1)[name = string("transpose_144")]; + tensor x_811_cast_fp16 = add(x = var_5884_cast_fp16, y = beta_171_cast_fp16)[name = string("x_811_cast_fp16")]; + tensor var_5895_split_sizes_0 = const()[name = string("op_5895_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5895_axis_0 = const()[name = string("op_5895_axis_0"), val = int32(1)]; + tensor var_5895_cast_fp16_0, tensor var_5895_cast_fp16_1 = split(axis = var_5895_axis_0, split_sizes = var_5895_split_sizes_0, x = h_7_cast_fp16)[name = string("op_5895_cast_fp16")]; + tensor gamma_175_perm_0 = const()[name = string("gamma_175_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_175_perm_0 = const()[name = string("beta_175_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5901_promoted_to_fp16 = const()[name = string("op_5901_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_175_cast_fp16 = transpose(perm = gamma_175_perm_0, x = var_5895_cast_fp16_0)[name = string("transpose_143")]; + tensor var_5902_cast_fp16 = add(x = gamma_175_cast_fp16, y = var_5901_promoted_to_fp16)[name = string("op_5902_cast_fp16")]; + tensor var_5903_cast_fp16 = mul(x = var_5902_cast_fp16, y = x_809_cast_fp16)[name = string("op_5903_cast_fp16")]; + tensor beta_175_cast_fp16 = transpose(perm = beta_175_perm_0, x = var_5895_cast_fp16_1)[name = string("transpose_142")]; + tensor x_817_cast_fp16 = add(x = var_5903_cast_fp16, y = beta_175_cast_fp16)[name = string("x_817_cast_fp16")]; + tensor linear_181_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_811_cast_fp16)[name = string("linear_181_cast_fp16")]; + tensor linear_182_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_817_cast_fp16)[name = string("linear_182_cast_fp16")]; + tensor var_5909_split_sizes_0 = const()[name = string("op_5909_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5909_axis_0 = const()[name = string("op_5909_axis_0"), val = int32(-1)]; + tensor var_5909_cast_fp16_0, tensor var_5909_cast_fp16_1 = split(axis = var_5909_axis_0, split_sizes = var_5909_split_sizes_0, x = linear_182_cast_fp16)[name = string("op_5909_cast_fp16")]; + tensor var_5917 = const()[name = string("op_5917"), val = tensor([1, 128, 8, 64])]; + tensor x_821_cast_fp16 = reshape(shape = var_5917, x = linear_181_cast_fp16)[name = string("x_821_cast_fp16")]; + tensor var_5927 = const()[name = string("op_5927"), val = tensor([1, 128, 8, 64])]; + tensor x_825_cast_fp16 = reshape(shape = var_5927, x = var_5909_cast_fp16_0)[name = string("x_825_cast_fp16")]; + tensor var_5937 = const()[name = string("op_5937"), val = tensor([1, 128, 8, 64])]; + tensor x_829_cast_fp16 = reshape(shape = var_5937, x = var_5909_cast_fp16_1)[name = string("x_829_cast_fp16")]; + tensor var_5939 = const()[name = string("op_5939"), val = tensor([0, 2, 1, 3])]; + bool sim_85_transpose_x_0 = const()[name = string("sim_85_transpose_x_0"), val = bool(false)]; + bool sim_85_transpose_y_0 = const()[name = string("sim_85_transpose_y_0"), val = bool(false)]; + tensor transpose_114_perm_0 = const()[name = string("transpose_114_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_115_perm_0 = const()[name = string("transpose_115_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_115 = transpose(perm = transpose_115_perm_0, x = x_825_cast_fp16)[name = string("transpose_139")]; + tensor transpose_114 = transpose(perm = transpose_114_perm_0, x = x_821_cast_fp16)[name = string("transpose_140")]; + tensor sim_85_cast_fp16 = matmul(transpose_x = sim_85_transpose_x_0, transpose_y = sim_85_transpose_y_0, x = transpose_114, y = transpose_115)[name = string("sim_85_cast_fp16")]; + fp16 var_5943_to_fp16 = const()[name = string("op_5943_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_87_cast_fp16 = mul(x = sim_85_cast_fp16, y = var_5943_to_fp16)[name = string("sim_87_cast_fp16")]; + tensor attn_43_cast_fp16 = softmax(axis = var_5460, x = sim_87_cast_fp16)[name = string("attn_43_cast_fp16")]; + bool x_831_transpose_x_0 = const()[name = string("x_831_transpose_x_0"), val = bool(false)]; + bool x_831_transpose_y_0 = const()[name = string("x_831_transpose_y_0"), val = bool(false)]; + tensor v_43_cast_fp16 = transpose(perm = var_5939, x = x_829_cast_fp16)[name = string("transpose_141")]; + tensor x_831_cast_fp16 = matmul(transpose_x = x_831_transpose_x_0, transpose_y = x_831_transpose_y_0, x = attn_43_cast_fp16, y = v_43_cast_fp16)[name = string("x_831_cast_fp16")]; + tensor var_5965 = const()[name = string("op_5965"), val = tensor([0, 2, 1, 3])]; + tensor var_5967 = const()[name = string("op_5967"), val = tensor([1, 128, 512])]; + tensor x_833_cast_fp16 = transpose(perm = var_5965, x = x_831_cast_fp16)[name = string("transpose_138")]; + tensor input_471_cast_fp16 = reshape(shape = var_5967, x = x_833_cast_fp16)[name = string("input_471_cast_fp16")]; + tensor linear_183_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_471_cast_fp16)[name = string("linear_183_cast_fp16")]; + tensor input_473_cast_fp16 = add(x = linear_183_cast_fp16, y = x_805_cast_fp16)[name = string("input_473_cast_fp16")]; + tensor linear_184_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_473_cast_fp16)[name = string("linear_184_cast_fp16")]; + string input_477_mode_0 = const()[name = string("input_477_mode_0"), val = string("EXACT")]; + tensor input_477_cast_fp16 = gelu(mode = input_477_mode_0, x = linear_184_cast_fp16)[name = string("input_477_cast_fp16")]; + tensor linear_185_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_477_cast_fp16)[name = string("linear_185_cast_fp16")]; + tensor x_835_cast_fp16 = add(x = linear_185_cast_fp16, y = input_473_cast_fp16)[name = string("x_835_cast_fp16")]; + tensor x_837_cast_fp16 = add(x = x_835_cast_fp16, y = mapping_cast_fp16)[name = string("x_837_cast_fp16")]; + tensor var_5983_split_sizes_0 = const()[name = string("op_5983_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5983_axis_0 = const()[name = string("op_5983_axis_0"), val = int32(1)]; + tensor var_5983_cast_fp16_0, tensor var_5983_cast_fp16_1 = split(axis = var_5983_axis_0, split_sizes = var_5983_split_sizes_0, x = h_11_cast_fp16)[name = string("op_5983_cast_fp16")]; + tensor gamma_179_perm_0 = const()[name = string("gamma_179_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_179_perm_0 = const()[name = string("beta_179_perm_0"), val = tensor([0, -1, 1])]; + tensor x_841_axes_0 = const()[name = string("x_841_axes_0"), val = tensor([-1])]; + tensor x_841_cast_fp16 = layer_norm(axes = x_841_axes_0, epsilon = var_5456_to_fp16, x = x_837_cast_fp16)[name = string("x_841_cast_fp16")]; + fp16 var_5989_promoted_to_fp16 = const()[name = string("op_5989_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_179_cast_fp16 = transpose(perm = gamma_179_perm_0, x = var_5983_cast_fp16_0)[name = string("transpose_137")]; + tensor var_5990_cast_fp16 = add(x = gamma_179_cast_fp16, y = var_5989_promoted_to_fp16)[name = string("op_5990_cast_fp16")]; + tensor var_5991_cast_fp16 = mul(x = var_5990_cast_fp16, y = x_841_cast_fp16)[name = string("op_5991_cast_fp16")]; + tensor beta_179_cast_fp16 = transpose(perm = beta_179_perm_0, x = var_5983_cast_fp16_1)[name = string("transpose_136")]; + tensor x_843_cast_fp16 = add(x = var_5991_cast_fp16, y = beta_179_cast_fp16)[name = string("x_843_cast_fp16")]; + tensor var_6002_split_sizes_0 = const()[name = string("op_6002_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_6002_axis_0 = const()[name = string("op_6002_axis_0"), val = int32(1)]; + tensor var_6002_cast_fp16_0, tensor var_6002_cast_fp16_1 = split(axis = var_6002_axis_0, split_sizes = var_6002_split_sizes_0, x = h_15_cast_fp16)[name = string("op_6002_cast_fp16")]; + tensor gamma_183_perm_0 = const()[name = string("gamma_183_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_183_perm_0 = const()[name = string("beta_183_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_6008_promoted_to_fp16 = const()[name = string("op_6008_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_183_cast_fp16 = transpose(perm = gamma_183_perm_0, x = var_6002_cast_fp16_0)[name = string("transpose_135")]; + tensor var_6009_cast_fp16 = add(x = gamma_183_cast_fp16, y = var_6008_promoted_to_fp16)[name = string("op_6009_cast_fp16")]; + tensor var_6010_cast_fp16 = mul(x = var_6009_cast_fp16, y = x_841_cast_fp16)[name = string("op_6010_cast_fp16")]; + tensor beta_183_cast_fp16 = transpose(perm = beta_183_perm_0, x = var_6002_cast_fp16_1)[name = string("transpose_134")]; + tensor x_849_cast_fp16 = add(x = var_6010_cast_fp16, y = beta_183_cast_fp16)[name = string("x_849_cast_fp16")]; + tensor linear_188_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_843_cast_fp16)[name = string("linear_188_cast_fp16")]; + tensor linear_189_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_849_cast_fp16)[name = string("linear_189_cast_fp16")]; + tensor var_6016_split_sizes_0 = const()[name = string("op_6016_split_sizes_0"), val = tensor([512, 512])]; + int32 var_6016_axis_0 = const()[name = string("op_6016_axis_0"), val = int32(-1)]; + tensor var_6016_cast_fp16_0, tensor var_6016_cast_fp16_1 = split(axis = var_6016_axis_0, split_sizes = var_6016_split_sizes_0, x = linear_189_cast_fp16)[name = string("op_6016_cast_fp16")]; + tensor var_6024 = const()[name = string("op_6024"), val = tensor([1, 128, 8, 64])]; + tensor x_853_cast_fp16 = reshape(shape = var_6024, x = linear_188_cast_fp16)[name = string("x_853_cast_fp16")]; + tensor var_6034 = const()[name = string("op_6034"), val = tensor([1, 128, 8, 64])]; + tensor x_857_cast_fp16 = reshape(shape = var_6034, x = var_6016_cast_fp16_0)[name = string("x_857_cast_fp16")]; + tensor var_6044 = const()[name = string("op_6044"), val = tensor([1, 128, 8, 64])]; + tensor x_861_cast_fp16 = reshape(shape = var_6044, x = var_6016_cast_fp16_1)[name = string("x_861_cast_fp16")]; + tensor var_6046 = const()[name = string("op_6046"), val = tensor([0, 2, 1, 3])]; + bool sim_89_transpose_x_0 = const()[name = string("sim_89_transpose_x_0"), val = bool(false)]; + bool sim_89_transpose_y_0 = const()[name = string("sim_89_transpose_y_0"), val = bool(false)]; + tensor transpose_116_perm_0 = const()[name = string("transpose_116_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_117_perm_0 = const()[name = string("transpose_117_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_117 = transpose(perm = transpose_117_perm_0, x = x_857_cast_fp16)[name = string("transpose_131")]; + tensor transpose_116 = transpose(perm = transpose_116_perm_0, x = x_853_cast_fp16)[name = string("transpose_132")]; + tensor sim_89_cast_fp16 = matmul(transpose_x = sim_89_transpose_x_0, transpose_y = sim_89_transpose_y_0, x = transpose_116, y = transpose_117)[name = string("sim_89_cast_fp16")]; + fp16 var_6050_to_fp16 = const()[name = string("op_6050_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_91_cast_fp16 = mul(x = sim_89_cast_fp16, y = var_6050_to_fp16)[name = string("sim_91_cast_fp16")]; + tensor attn_45_cast_fp16 = softmax(axis = var_5460, x = sim_91_cast_fp16)[name = string("attn_45_cast_fp16")]; + bool x_863_transpose_x_0 = const()[name = string("x_863_transpose_x_0"), val = bool(false)]; + bool x_863_transpose_y_0 = const()[name = string("x_863_transpose_y_0"), val = bool(false)]; + tensor v_45_cast_fp16 = transpose(perm = var_6046, x = x_861_cast_fp16)[name = string("transpose_133")]; + tensor x_863_cast_fp16 = matmul(transpose_x = x_863_transpose_x_0, transpose_y = x_863_transpose_y_0, x = attn_45_cast_fp16, y = v_45_cast_fp16)[name = string("x_863_cast_fp16")]; + tensor var_6072 = const()[name = string("op_6072"), val = tensor([0, 2, 1, 3])]; + tensor var_6074 = const()[name = string("op_6074"), val = tensor([1, 128, 512])]; + tensor x_865_cast_fp16 = transpose(perm = var_6072, x = x_863_cast_fp16)[name = string("transpose_130")]; + tensor input_487_cast_fp16 = reshape(shape = var_6074, x = x_865_cast_fp16)[name = string("input_487_cast_fp16")]; + tensor linear_190_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_487_cast_fp16)[name = string("linear_190_cast_fp16")]; + tensor input_489_cast_fp16 = add(x = linear_190_cast_fp16, y = x_837_cast_fp16)[name = string("input_489_cast_fp16")]; + tensor linear_191_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_489_cast_fp16)[name = string("linear_191_cast_fp16")]; + string input_493_mode_0 = const()[name = string("input_493_mode_0"), val = string("EXACT")]; + tensor input_493_cast_fp16 = gelu(mode = input_493_mode_0, x = linear_191_cast_fp16)[name = string("input_493_cast_fp16")]; + tensor linear_192_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_493_cast_fp16)[name = string("linear_192_cast_fp16")]; + tensor x_867_cast_fp16 = add(x = linear_192_cast_fp16, y = input_489_cast_fp16)[name = string("x_867_cast_fp16")]; + tensor x_869_cast_fp16 = add(x = x_867_cast_fp16, y = mapping_cast_fp16)[name = string("x_869_cast_fp16")]; + tensor var_6090_split_sizes_0 = const()[name = string("op_6090_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_6090_axis_0 = const()[name = string("op_6090_axis_0"), val = int32(1)]; + tensor var_6090_cast_fp16_0, tensor var_6090_cast_fp16_1 = split(axis = var_6090_axis_0, split_sizes = var_6090_split_sizes_0, x = h_19_cast_fp16)[name = string("op_6090_cast_fp16")]; + tensor gamma_187_perm_0 = const()[name = string("gamma_187_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_187_perm_0 = const()[name = string("beta_187_perm_0"), val = tensor([0, -1, 1])]; + tensor x_873_axes_0 = const()[name = string("x_873_axes_0"), val = tensor([-1])]; + tensor x_873_cast_fp16 = layer_norm(axes = x_873_axes_0, epsilon = var_5456_to_fp16, x = x_869_cast_fp16)[name = string("x_873_cast_fp16")]; + fp16 var_6096_promoted_to_fp16 = const()[name = string("op_6096_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_187_cast_fp16 = transpose(perm = gamma_187_perm_0, x = var_6090_cast_fp16_0)[name = string("transpose_129")]; + tensor var_6097_cast_fp16 = add(x = gamma_187_cast_fp16, y = var_6096_promoted_to_fp16)[name = string("op_6097_cast_fp16")]; + tensor var_6098_cast_fp16 = mul(x = var_6097_cast_fp16, y = x_873_cast_fp16)[name = string("op_6098_cast_fp16")]; + tensor beta_187_cast_fp16 = transpose(perm = beta_187_perm_0, x = var_6090_cast_fp16_1)[name = string("transpose_128")]; + tensor x_875_cast_fp16 = add(x = var_6098_cast_fp16, y = beta_187_cast_fp16)[name = string("x_875_cast_fp16")]; + tensor var_6109_split_sizes_0 = const()[name = string("op_6109_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_6109_axis_0 = const()[name = string("op_6109_axis_0"), val = int32(1)]; + tensor var_6109_cast_fp16_0, tensor var_6109_cast_fp16_1 = split(axis = var_6109_axis_0, split_sizes = var_6109_split_sizes_0, x = h_23_cast_fp16)[name = string("op_6109_cast_fp16")]; + tensor gamma_perm_0 = const()[name = string("gamma_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_perm_0 = const()[name = string("beta_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_6115_promoted_to_fp16 = const()[name = string("op_6115_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_cast_fp16 = transpose(perm = gamma_perm_0, x = var_6109_cast_fp16_0)[name = string("transpose_127")]; + tensor var_6116_cast_fp16 = add(x = gamma_cast_fp16, y = var_6115_promoted_to_fp16)[name = string("op_6116_cast_fp16")]; + tensor var_6117_cast_fp16 = mul(x = var_6116_cast_fp16, y = x_873_cast_fp16)[name = string("op_6117_cast_fp16")]; + tensor beta_cast_fp16 = transpose(perm = beta_perm_0, x = var_6109_cast_fp16_1)[name = string("transpose_126")]; + tensor x_881_cast_fp16 = add(x = var_6117_cast_fp16, y = beta_cast_fp16)[name = string("x_881_cast_fp16")]; + tensor linear_195_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_875_cast_fp16)[name = string("linear_195_cast_fp16")]; + tensor linear_196_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_881_cast_fp16)[name = string("linear_196_cast_fp16")]; + tensor var_6123_split_sizes_0 = const()[name = string("op_6123_split_sizes_0"), val = tensor([512, 512])]; + int32 var_6123_axis_0 = const()[name = string("op_6123_axis_0"), val = int32(-1)]; + tensor var_6123_cast_fp16_0, tensor var_6123_cast_fp16_1 = split(axis = var_6123_axis_0, split_sizes = var_6123_split_sizes_0, x = linear_196_cast_fp16)[name = string("op_6123_cast_fp16")]; + tensor var_6131 = const()[name = string("op_6131"), val = tensor([1, 128, 8, 64])]; + tensor x_885_cast_fp16 = reshape(shape = var_6131, x = linear_195_cast_fp16)[name = string("x_885_cast_fp16")]; + tensor var_6141 = const()[name = string("op_6141"), val = tensor([1, 128, 8, 64])]; + tensor x_889_cast_fp16 = reshape(shape = var_6141, x = var_6123_cast_fp16_0)[name = string("x_889_cast_fp16")]; + tensor var_6151 = const()[name = string("op_6151"), val = tensor([1, 128, 8, 64])]; + tensor x_893_cast_fp16 = reshape(shape = var_6151, x = var_6123_cast_fp16_1)[name = string("x_893_cast_fp16")]; + tensor var_6153 = const()[name = string("op_6153"), val = tensor([0, 2, 1, 3])]; + bool sim_93_transpose_x_0 = const()[name = string("sim_93_transpose_x_0"), val = bool(false)]; + bool sim_93_transpose_y_0 = const()[name = string("sim_93_transpose_y_0"), val = bool(false)]; + tensor transpose_118_perm_0 = const()[name = string("transpose_118_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_119_perm_0 = const()[name = string("transpose_119_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_119 = transpose(perm = transpose_119_perm_0, x = x_889_cast_fp16)[name = string("transpose_123")]; + tensor transpose_118 = transpose(perm = transpose_118_perm_0, x = x_885_cast_fp16)[name = string("transpose_124")]; + tensor sim_93_cast_fp16 = matmul(transpose_x = sim_93_transpose_x_0, transpose_y = sim_93_transpose_y_0, x = transpose_118, y = transpose_119)[name = string("sim_93_cast_fp16")]; + fp16 var_6157_to_fp16 = const()[name = string("op_6157_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_cast_fp16 = mul(x = sim_93_cast_fp16, y = var_6157_to_fp16)[name = string("sim_cast_fp16")]; + tensor attn_cast_fp16 = softmax(axis = var_5460, x = sim_cast_fp16)[name = string("attn_cast_fp16")]; + bool x_895_transpose_x_0 = const()[name = string("x_895_transpose_x_0"), val = bool(false)]; + bool x_895_transpose_y_0 = const()[name = string("x_895_transpose_y_0"), val = bool(false)]; + tensor v_cast_fp16 = transpose(perm = var_6153, x = x_893_cast_fp16)[name = string("transpose_125")]; + tensor x_895_cast_fp16 = matmul(transpose_x = x_895_transpose_x_0, transpose_y = x_895_transpose_y_0, x = attn_cast_fp16, y = v_cast_fp16)[name = string("x_895_cast_fp16")]; + tensor var_6179 = const()[name = string("op_6179"), val = tensor([0, 2, 1, 3])]; + tensor var_6181 = const()[name = string("op_6181"), val = tensor([1, 128, 512])]; + tensor x_897_cast_fp16 = transpose(perm = var_6179, x = x_895_cast_fp16)[name = string("transpose_122")]; + tensor input_503_cast_fp16 = reshape(shape = var_6181, x = x_897_cast_fp16)[name = string("input_503_cast_fp16")]; + tensor linear_197_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_503_cast_fp16)[name = string("linear_197_cast_fp16")]; + tensor input_505_cast_fp16 = add(x = linear_197_cast_fp16, y = x_869_cast_fp16)[name = string("input_505_cast_fp16")]; + tensor linear_198_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_505_cast_fp16)[name = string("linear_198_cast_fp16")]; + string input_509_mode_0 = const()[name = string("input_509_mode_0"), val = string("EXACT")]; + tensor input_509_cast_fp16 = gelu(mode = input_509_mode_0, x = linear_198_cast_fp16)[name = string("input_509_cast_fp16")]; + tensor linear_199_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_509_cast_fp16)[name = string("linear_199_cast_fp16")]; + tensor x_899_cast_fp16 = add(x = linear_199_cast_fp16, y = input_505_cast_fp16)[name = string("x_899_cast_fp16")]; + tensor var_6190_axes_0 = const()[name = string("op_6190_axes_0"), val = tensor([1])]; + bool var_6190_keep_dims_0 = const()[name = string("op_6190_keep_dims_0"), val = bool(false)]; + tensor var_6190_cast_fp16 = reduce_mean(axes = var_6190_axes_0, keep_dims = var_6190_keep_dims_0, x = x_899_cast_fp16)[name = string("op_6190_cast_fp16")]; + tensor x_901_axes_0 = const()[name = string("x_901_axes_0"), val = tensor([1])]; + tensor x_901_cast_fp16 = expand_dims(axes = x_901_axes_0, x = var_6190_cast_fp16)[name = string("x_901_cast_fp16")]; + tensor var_6192 = const()[name = string("op_6192"), val = tensor([0, 2, 1])]; + string x_903_pad_type_0 = const()[name = string("x_903_pad_type_0"), val = string("valid")]; + tensor x_903_strides_0 = const()[name = string("x_903_strides_0"), val = tensor([1])]; + tensor x_903_pad_0 = const()[name = string("x_903_pad_0"), val = tensor([0, 0])]; + tensor x_903_dilations_0 = const()[name = string("x_903_dilations_0"), val = tensor([1])]; + int32 x_903_groups_0 = const()[name = string("x_903_groups_0"), val = int32(1)]; + tensor input_cast_fp16 = transpose(perm = var_6192, x = x_901_cast_fp16)[name = string("transpose_121")]; + tensor x_903_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_903_dilations_0, groups = x_903_groups_0, pad = x_903_pad_0, pad_type = x_903_pad_type_0, strides = x_903_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_cast_fp16)[name = string("x_903_cast_fp16")]; + tensor x_pred_perm_0 = const()[name = string("x_pred_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_to_fp16 = const()[name = string("c_skip_to_fp16"), val = tensor([[[0x1.fecp-1]]])]; + tensor var_6200_cast_fp16 = mul(x = c_skip_to_fp16, y = x_noisy_cast_fp16)[name = string("op_6200_cast_fp16")]; + tensor c_out_to_fp16 = const()[name = string("c_out_to_fp16"), val = tensor([[[0x1.38p-9]]])]; + tensor x_pred_cast_fp16 = transpose(perm = x_pred_perm_0, x = x_903_cast_fp16)[name = string("transpose_120")]; + tensor var_6201_cast_fp16 = mul(x = c_out_to_fp16, y = x_pred_cast_fp16)[name = string("op_6201_cast_fp16")]; + tensor x_mid_dn_cast_fp16 = add(x = var_6200_cast_fp16, y = var_6201_cast_fp16)[name = string("x_mid_dn_cast_fp16")]; + tensor var_6204_cast_fp16 = sub(x = x_noisy_cast_fp16, y = x_mid_dn_cast_fp16)[name = string("op_6204_cast_fp16")]; + tensor _inversed_d_mid_y_0_to_fp16 = const()[name = string("_inversed_d_mid_y_0_to_fp16"), val = tensor([0x1.a44p+8])]; + tensor _inversed_d_mid_cast_fp16 = mul(x = var_6204_cast_fp16, y = _inversed_d_mid_y_0_to_fp16)[name = string("_inversed_d_mid_cast_fp16")]; + fp16 var_6213_to_fp16 = const()[name = string("op_6213_to_fp16"), val = fp16(-0x1.37cp-8)]; + tensor var_6214_cast_fp16 = mul(x = _inversed_d_mid_cast_fp16, y = var_6213_to_fp16)[name = string("op_6214_cast_fp16")]; + tensor x_cast_fp16 = add(x = x_noisy_13_cast_fp16, y = var_6214_cast_fp16)[name = string("x_cast_fp16")]; + tensor var_6219_begin_0 = const()[name = string("op_6219_begin_0"), val = tensor([3, 0, 0, 0])]; + tensor var_6219_end_0 = const()[name = string("op_6219_end_0"), val = tensor([4, 1, 1, 256])]; + tensor var_6219_end_mask_0 = const()[name = string("op_6219_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_6219_squeeze_mask_0 = const()[name = string("op_6219_squeeze_mask_0"), val = tensor([true, false, false, false])]; + tensor var_6219_cast_fp16 = slice_by_index(begin = var_6219_begin_0, end = var_6219_end_0, end_mask = var_6219_end_mask_0, squeeze_mask = var_6219_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_6219_cast_fp16")]; + fp16 var_6222_to_fp16 = const()[name = string("op_6222_to_fp16"), val = fp16(0x1.a34p-14)]; + tensor var_6223_cast_fp16 = mul(x = var_6219_cast_fp16, y = var_6222_to_fp16)[name = string("op_6223_cast_fp16")]; + tensor var_6225 = add(x = x_cast_fp16, y = var_6223_cast_fp16)[name = string("op_6225_cast_fp16")]; + } -> (var_6225); +} \ No newline at end of file diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/weights/weight.bin b/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..5d524e318268dea3c586fd0de5ce641710361300 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t128.mlmodelc/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f823b5c638d2eb2fd91bf8e4efe4a90b2e1d3d9e2f5ab40e7e93cb03cd212aca +size 49361856 diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/analytics/coremldata.bin b/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/analytics/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..f525332daa247c23122963d0e8a9b868988e72f7 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/analytics/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2455735a48a29c69ac89e1af3a8255d27c8900fc1a8a9818fa2b7482ba74ed20 +size 243 diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/coremldata.bin b/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..3352a3b452332766ccea7c1ca6703bb0e8d71727 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6a02ae96aa0d591c0718c7e108d017012d034e6511a5a9132b62d9120b1a4db +size 462 diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/metadata.json b/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/metadata.json new file mode 100644 index 0000000000000000000000000000000000000000..782e2e2920fb0fdf39a12dbec0c0b6bab404c6f8 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/metadata.json @@ -0,0 +1,110 @@ +[ + { + "metadataOutputVersion" : "3.0", + "storagePrecision" : "Float16", + "outputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float16", + "formattedType" : "MultiArray (Float16 1 × 1 × 256)", + "shortDescription" : "", + "shape" : "[1, 1, 256]", + "name" : "var_6225", + "type" : "MultiArray" + } + ], + "modelParameters" : [ + + ], + "specificationVersion" : 9, + "mlProgramOperationTypeHistogram" : { + "Ios18.expandDims" : 16, + "Ios18.softmax" : 24, + "Ios18.mul" : 117, + "Ios18.matmul" : 48, + "Ios16.reduceMean" : 8, + "Split" : 72, + "Tile" : 16, + "Ios18.add" : 188, + "Ios16.reduceSum" : 8, + "Ios18.layerNorm" : 24, + "Ios18.reshape" : 102, + "Ios18.linear" : 143, + "Ios18.conv" : 8, + "Ios18.gelu" : 41, + "Ios18.sub" : 8, + "Ios18.concat" : 8, + "Stack" : 8, + "Ios18.transpose" : 216, + "Ios18.cast" : 4, + "Ios18.sliceByIndex" : 4 + }, + "computePrecision" : "Mixed (Float16, Int32)", + "isUpdatable" : "0", + "stateSchema" : [ + + ], + "availability" : { + "macOS" : "15.0", + "tvOS" : "18.0", + "visionOS" : "2.0", + "watchOS" : "11.0", + "iOS" : "18.0", + "macCatalyst" : "18.0" + }, + "modelType" : { + "name" : "MLModelType_mlProgram" + }, + "userDefinedMetadata" : { + "com.github.apple.coremltools.conversion_date" : "2026-05-08", + "com.github.apple.coremltools.source" : "torch==2.11.0", + "com.github.apple.coremltools.version" : "9.0", + "com.github.apple.coremltools.source_dialect" : "TorchScript" + }, + "inputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 1 × 1 × 256)", + "shortDescription" : "", + "shape" : "[1, 1, 256]", + "name" : "noise_init", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 4 × 1 × 1 × 256)", + "shortDescription" : "", + "shape" : "[4, 1, 1, 256]", + "name" : "noises_aux", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 1 × 256 × 768)", + "shortDescription" : "", + "shape" : "[1, 256, 768]", + "name" : "embedding", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 1 × 256)", + "shortDescription" : "", + "shape" : "[1, 256]", + "name" : "features", + "type" : "MultiArray" + } + ], + "generatedClassName" : "fused_diffusion_sampler_fp16_t256", + "method" : "predict" + } +] \ No newline at end of file diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/model.mil b/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/model.mil new file mode 100644 index 0000000000000000000000000000000000000000..50545741b94577cfbf41b201107a8babf02f021f --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/model.mil @@ -0,0 +1,2019 @@ +program(1.3) +[buildInfo = dict({{"coremlc-component-MIL", "3520.4.1"}, {"coremlc-version", "3520.5.1"}, {"coremltools-component-torch", "2.11.0"}, {"coremltools-source-dialect", "TorchScript"}, {"coremltools-version", "9.0"}})] +{ + func main(tensor embedding, tensor features, tensor noise_init, tensor noises_aux) { + fp16 var_12_to_fp16 = const()[name = string("op_12_to_fp16"), val = fp16(0x1.8p+1)]; + string noise_init_to_fp16_dtype_0 = const()[name = string("noise_init_to_fp16_dtype_0"), val = string("fp16")]; + tensor noise_init_to_fp16 = cast(dtype = noise_init_to_fp16_dtype_0, x = noise_init)[name = string("cast_196")]; + tensor x_noisy_1_cast_fp16 = mul(x = var_12_to_fp16, y = noise_init_to_fp16)[name = string("x_noisy_1_cast_fp16")]; + int32 var_35 = const()[name = string("op_35"), val = int32(-1)]; + tensor c_in_1_to_fp16 = const()[name = string("c_in_1_to_fp16"), val = tensor([[[0x1.548p-2]]])]; + tensor x_11_cast_fp16 = mul(x = c_in_1_to_fp16, y = x_noisy_1_cast_fp16)[name = string("x_11_cast_fp16")]; + string features_to_fp16_dtype_0 = const()[name = string("features_to_fp16_dtype_0"), val = string("fp16")]; + tensor unet_step_kdiffusion_net_to_features_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_features_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(64)))]; + tensor unet_step_kdiffusion_net_to_features_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_features_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(524416)))]; + tensor features_to_fp16 = cast(dtype = features_to_fp16_dtype_0, x = features)[name = string("cast_195")]; + tensor linear_1_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_features_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_features_0_weight_to_fp16, x = features_to_fp16)[name = string("linear_1_cast_fp16")]; + string var_423_mode_0 = const()[name = string("op_423_mode_0"), val = string("EXACT")]; + tensor var_423_cast_fp16 = gelu(mode = var_423_mode_0, x = linear_1_cast_fp16)[name = string("op_423_cast_fp16")]; + int32 x_7_axis_0 = const()[name = string("x_7_axis_0"), val = int32(0)]; + tensor var_421_to_fp16 = const()[name = string("op_421_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(526528)))]; + tensor x_7_cast_fp16 = stack(axis = x_7_axis_0, values = (var_421_to_fp16, var_423_cast_fp16))[name = string("x_7_cast_fp16")]; + tensor var_426 = const()[name = string("op_426"), val = tensor([1, 2, 0])]; + tensor input_7_axes_0 = const()[name = string("input_7_axes_0"), val = tensor([2])]; + bool input_7_keep_dims_0 = const()[name = string("input_7_keep_dims_0"), val = bool(false)]; + tensor x_9_cast_fp16 = transpose(perm = var_426, x = x_7_cast_fp16)[name = string("transpose_335")]; + tensor input_7_cast_fp16 = reduce_sum(axes = input_7_axes_0, keep_dims = input_7_keep_dims_0, x = x_9_cast_fp16)[name = string("input_7_cast_fp16")]; + tensor unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(528640)))]; + tensor unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2625856)))]; + tensor linear_2_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_7_cast_fp16)[name = string("linear_2_cast_fp16")]; + string input_11_mode_0 = const()[name = string("input_11_mode_0"), val = string("EXACT")]; + tensor input_11_cast_fp16 = gelu(mode = input_11_mode_0, x = linear_2_cast_fp16)[name = string("input_11_cast_fp16")]; + tensor unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2627968)))]; + tensor unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(4725184)))]; + tensor linear_3_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_11_cast_fp16)[name = string("linear_3_cast_fp16")]; + string mapping_1_mode_0 = const()[name = string("mapping_1_mode_0"), val = string("EXACT")]; + tensor mapping_1_cast_fp16 = gelu(mode = mapping_1_mode_0, x = linear_3_cast_fp16)[name = string("mapping_1_cast_fp16")]; + tensor var_436_reps_0 = const()[name = string("op_436_reps_0"), val = tensor([1, 256, 1])]; + tensor var_436_cast_fp16 = tile(reps = var_436_reps_0, x = x_11_cast_fp16)[name = string("op_436_cast_fp16")]; + bool x_13_interleave_0 = const()[name = string("x_13_interleave_0"), val = bool(false)]; + string embedding_to_fp16_dtype_0 = const()[name = string("embedding_to_fp16_dtype_0"), val = string("fp16")]; + tensor embedding_to_fp16 = cast(dtype = embedding_to_fp16_dtype_0, x = embedding)[name = string("cast_194")]; + tensor x_13_cast_fp16 = concat(axis = var_35, interleave = x_13_interleave_0, values = (var_436_cast_fp16, embedding_to_fp16))[name = string("x_13_cast_fp16")]; + tensor var_439_axes_0 = const()[name = string("op_439_axes_0"), val = tensor([1])]; + tensor var_439_cast_fp16 = expand_dims(axes = var_439_axes_0, x = mapping_1_cast_fp16)[name = string("op_439_cast_fp16")]; + tensor mapping_3_reps_0 = const()[name = string("mapping_3_reps_0"), val = tensor([1, 256, 1])]; + tensor mapping_3_cast_fp16 = tile(reps = mapping_3_reps_0, x = var_439_cast_fp16)[name = string("mapping_3_cast_fp16")]; + tensor x_15_cast_fp16 = add(x = x_13_cast_fp16, y = mapping_3_cast_fp16)[name = string("x_15_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(4727296)))]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5775936)))]; + tensor linear_4_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_norm_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_norm_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_4_cast_fp16")]; + tensor var_449 = const()[name = string("op_449"), val = tensor([1, 2048, 1])]; + tensor h_3_cast_fp16 = reshape(shape = var_449, x = linear_4_cast_fp16)[name = string("h_3_cast_fp16")]; + tensor var_451_split_sizes_0 = const()[name = string("op_451_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_451_axis_0 = const()[name = string("op_451_axis_0"), val = int32(1)]; + tensor var_451_cast_fp16_0, tensor var_451_cast_fp16_1 = split(axis = var_451_axis_0, split_sizes = var_451_split_sizes_0, x = h_3_cast_fp16)[name = string("op_451_cast_fp16")]; + tensor gamma_3_perm_0 = const()[name = string("gamma_3_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_3_perm_0 = const()[name = string("beta_3_perm_0"), val = tensor([0, -1, 1])]; + tensor x_19_axes_0 = const()[name = string("x_19_axes_0"), val = tensor([-1])]; + fp16 var_31_to_fp16 = const()[name = string("op_31_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_19_cast_fp16 = layer_norm(axes = x_19_axes_0, epsilon = var_31_to_fp16, x = x_15_cast_fp16)[name = string("x_19_cast_fp16")]; + fp16 var_457_promoted_to_fp16 = const()[name = string("op_457_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_3_cast_fp16 = transpose(perm = gamma_3_perm_0, x = var_451_cast_fp16_0)[name = string("transpose_334")]; + tensor var_458_cast_fp16 = add(x = gamma_3_cast_fp16, y = var_457_promoted_to_fp16)[name = string("op_458_cast_fp16")]; + tensor var_459_cast_fp16 = mul(x = var_458_cast_fp16, y = x_19_cast_fp16)[name = string("op_459_cast_fp16")]; + tensor beta_3_cast_fp16 = transpose(perm = beta_3_perm_0, x = var_451_cast_fp16_1)[name = string("transpose_333")]; + tensor x_21_cast_fp16 = add(x = var_459_cast_fp16, y = beta_3_cast_fp16)[name = string("x_21_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5780096)))]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(6828736)))]; + tensor linear_5_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_5_cast_fp16")]; + tensor var_468 = const()[name = string("op_468"), val = tensor([1, 2048, 1])]; + tensor h_7_cast_fp16 = reshape(shape = var_468, x = linear_5_cast_fp16)[name = string("h_7_cast_fp16")]; + tensor var_470_split_sizes_0 = const()[name = string("op_470_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_470_axis_0 = const()[name = string("op_470_axis_0"), val = int32(1)]; + tensor var_470_cast_fp16_0, tensor var_470_cast_fp16_1 = split(axis = var_470_axis_0, split_sizes = var_470_split_sizes_0, x = h_7_cast_fp16)[name = string("op_470_cast_fp16")]; + tensor gamma_7_perm_0 = const()[name = string("gamma_7_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_7_perm_0 = const()[name = string("beta_7_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_476_promoted_to_fp16 = const()[name = string("op_476_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_7_cast_fp16 = transpose(perm = gamma_7_perm_0, x = var_470_cast_fp16_0)[name = string("transpose_332")]; + tensor var_477_cast_fp16 = add(x = gamma_7_cast_fp16, y = var_476_promoted_to_fp16)[name = string("op_477_cast_fp16")]; + tensor var_478_cast_fp16 = mul(x = var_477_cast_fp16, y = x_19_cast_fp16)[name = string("op_478_cast_fp16")]; + tensor beta_7_cast_fp16 = transpose(perm = beta_7_perm_0, x = var_470_cast_fp16_1)[name = string("transpose_331")]; + tensor x_27_cast_fp16 = add(x = var_478_cast_fp16, y = beta_7_cast_fp16)[name = string("x_27_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(6832896)))]; + tensor linear_6_bias_0_to_fp16 = const()[name = string("linear_6_bias_0_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(7881536)))]; + tensor linear_6_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_21_cast_fp16)[name = string("linear_6_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(7882624)))]; + tensor linear_7_bias_0_to_fp16 = const()[name = string("linear_7_bias_0_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(9979840)))]; + tensor linear_7_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_27_cast_fp16)[name = string("linear_7_cast_fp16")]; + tensor var_484_split_sizes_0 = const()[name = string("op_484_split_sizes_0"), val = tensor([512, 512])]; + int32 var_484_axis_0 = const()[name = string("op_484_axis_0"), val = int32(-1)]; + tensor var_484_cast_fp16_0, tensor var_484_cast_fp16_1 = split(axis = var_484_axis_0, split_sizes = var_484_split_sizes_0, x = linear_7_cast_fp16)[name = string("op_484_cast_fp16")]; + tensor var_492 = const()[name = string("op_492"), val = tensor([1, 256, 8, 64])]; + tensor x_31_cast_fp16 = reshape(shape = var_492, x = linear_6_cast_fp16)[name = string("x_31_cast_fp16")]; + tensor var_502 = const()[name = string("op_502"), val = tensor([1, 256, 8, 64])]; + tensor x_35_cast_fp16 = reshape(shape = var_502, x = var_484_cast_fp16_0)[name = string("x_35_cast_fp16")]; + tensor var_512 = const()[name = string("op_512"), val = tensor([1, 256, 8, 64])]; + tensor x_39_cast_fp16 = reshape(shape = var_512, x = var_484_cast_fp16_1)[name = string("x_39_cast_fp16")]; + tensor var_514 = const()[name = string("op_514"), val = tensor([0, 2, 1, 3])]; + bool sim_1_transpose_x_0 = const()[name = string("sim_1_transpose_x_0"), val = bool(false)]; + bool sim_1_transpose_y_0 = const()[name = string("sim_1_transpose_y_0"), val = bool(false)]; + tensor transpose_72_perm_0 = const()[name = string("transpose_72_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_73_perm_0 = const()[name = string("transpose_73_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_73 = transpose(perm = transpose_73_perm_0, x = x_35_cast_fp16)[name = string("transpose_328")]; + tensor transpose_72 = transpose(perm = transpose_72_perm_0, x = x_31_cast_fp16)[name = string("transpose_329")]; + tensor sim_1_cast_fp16 = matmul(transpose_x = sim_1_transpose_x_0, transpose_y = sim_1_transpose_y_0, x = transpose_72, y = transpose_73)[name = string("sim_1_cast_fp16")]; + fp16 var_518_to_fp16 = const()[name = string("op_518_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_3_cast_fp16 = mul(x = sim_1_cast_fp16, y = var_518_to_fp16)[name = string("sim_3_cast_fp16")]; + tensor attn_1_cast_fp16 = softmax(axis = var_35, x = sim_3_cast_fp16)[name = string("attn_1_cast_fp16")]; + bool x_41_transpose_x_0 = const()[name = string("x_41_transpose_x_0"), val = bool(false)]; + bool x_41_transpose_y_0 = const()[name = string("x_41_transpose_y_0"), val = bool(false)]; + tensor v_1_cast_fp16 = transpose(perm = var_514, x = x_39_cast_fp16)[name = string("transpose_330")]; + tensor x_41_cast_fp16 = matmul(transpose_x = x_41_transpose_x_0, transpose_y = x_41_transpose_y_0, x = attn_1_cast_fp16, y = v_1_cast_fp16)[name = string("x_41_cast_fp16")]; + tensor var_540 = const()[name = string("op_540"), val = tensor([0, 2, 1, 3])]; + tensor var_542 = const()[name = string("op_542"), val = tensor([1, 256, 512])]; + tensor x_43_cast_fp16 = transpose(perm = var_540, x = x_41_cast_fp16)[name = string("transpose_327")]; + tensor input_23_cast_fp16 = reshape(shape = var_542, x = x_43_cast_fp16)[name = string("input_23_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(9981952)))]; + tensor unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11030592)))]; + tensor linear_8_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_23_cast_fp16)[name = string("linear_8_cast_fp16")]; + tensor input_25_cast_fp16 = add(x = linear_8_cast_fp16, y = x_15_cast_fp16)[name = string("input_25_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11032704)))]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(15227072)))]; + tensor linear_9_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_25_cast_fp16)[name = string("linear_9_cast_fp16")]; + string input_29_mode_0 = const()[name = string("input_29_mode_0"), val = string("EXACT")]; + tensor input_29_cast_fp16 = gelu(mode = input_29_mode_0, x = linear_9_cast_fp16)[name = string("input_29_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(15231232)))]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(19425600)))]; + tensor linear_10_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_29_cast_fp16)[name = string("linear_10_cast_fp16")]; + tensor x_45_cast_fp16 = add(x = linear_10_cast_fp16, y = input_25_cast_fp16)[name = string("x_45_cast_fp16")]; + tensor x_47_cast_fp16 = add(x = x_45_cast_fp16, y = mapping_3_cast_fp16)[name = string("x_47_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(19427712)))]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(20476352)))]; + tensor linear_11_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_norm_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_norm_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_11_cast_fp16")]; + tensor var_556 = const()[name = string("op_556"), val = tensor([1, 2048, 1])]; + tensor h_11_cast_fp16 = reshape(shape = var_556, x = linear_11_cast_fp16)[name = string("h_11_cast_fp16")]; + tensor var_558_split_sizes_0 = const()[name = string("op_558_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_558_axis_0 = const()[name = string("op_558_axis_0"), val = int32(1)]; + tensor var_558_cast_fp16_0, tensor var_558_cast_fp16_1 = split(axis = var_558_axis_0, split_sizes = var_558_split_sizes_0, x = h_11_cast_fp16)[name = string("op_558_cast_fp16")]; + tensor gamma_11_perm_0 = const()[name = string("gamma_11_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_11_perm_0 = const()[name = string("beta_11_perm_0"), val = tensor([0, -1, 1])]; + tensor x_51_axes_0 = const()[name = string("x_51_axes_0"), val = tensor([-1])]; + tensor x_51_cast_fp16 = layer_norm(axes = x_51_axes_0, epsilon = var_31_to_fp16, x = x_47_cast_fp16)[name = string("x_51_cast_fp16")]; + fp16 var_564_promoted_to_fp16 = const()[name = string("op_564_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_11_cast_fp16 = transpose(perm = gamma_11_perm_0, x = var_558_cast_fp16_0)[name = string("transpose_326")]; + tensor var_565_cast_fp16 = add(x = gamma_11_cast_fp16, y = var_564_promoted_to_fp16)[name = string("op_565_cast_fp16")]; + tensor var_566_cast_fp16 = mul(x = var_565_cast_fp16, y = x_51_cast_fp16)[name = string("op_566_cast_fp16")]; + tensor beta_11_cast_fp16 = transpose(perm = beta_11_perm_0, x = var_558_cast_fp16_1)[name = string("transpose_325")]; + tensor x_53_cast_fp16 = add(x = var_566_cast_fp16, y = beta_11_cast_fp16)[name = string("x_53_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(20480512)))]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(21529152)))]; + tensor linear_12_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_12_cast_fp16")]; + tensor var_575 = const()[name = string("op_575"), val = tensor([1, 2048, 1])]; + tensor h_15_cast_fp16 = reshape(shape = var_575, x = linear_12_cast_fp16)[name = string("h_15_cast_fp16")]; + tensor var_577_split_sizes_0 = const()[name = string("op_577_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_577_axis_0 = const()[name = string("op_577_axis_0"), val = int32(1)]; + tensor var_577_cast_fp16_0, tensor var_577_cast_fp16_1 = split(axis = var_577_axis_0, split_sizes = var_577_split_sizes_0, x = h_15_cast_fp16)[name = string("op_577_cast_fp16")]; + tensor gamma_15_perm_0 = const()[name = string("gamma_15_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_15_perm_0 = const()[name = string("beta_15_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_583_promoted_to_fp16 = const()[name = string("op_583_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_15_cast_fp16 = transpose(perm = gamma_15_perm_0, x = var_577_cast_fp16_0)[name = string("transpose_324")]; + tensor var_584_cast_fp16 = add(x = gamma_15_cast_fp16, y = var_583_promoted_to_fp16)[name = string("op_584_cast_fp16")]; + tensor var_585_cast_fp16 = mul(x = var_584_cast_fp16, y = x_51_cast_fp16)[name = string("op_585_cast_fp16")]; + tensor beta_15_cast_fp16 = transpose(perm = beta_15_perm_0, x = var_577_cast_fp16_1)[name = string("transpose_323")]; + tensor x_59_cast_fp16 = add(x = var_585_cast_fp16, y = beta_15_cast_fp16)[name = string("x_59_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(21533312)))]; + tensor linear_13_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_53_cast_fp16)[name = string("linear_13_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(22581952)))]; + tensor linear_14_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_59_cast_fp16)[name = string("linear_14_cast_fp16")]; + tensor var_591_split_sizes_0 = const()[name = string("op_591_split_sizes_0"), val = tensor([512, 512])]; + int32 var_591_axis_0 = const()[name = string("op_591_axis_0"), val = int32(-1)]; + tensor var_591_cast_fp16_0, tensor var_591_cast_fp16_1 = split(axis = var_591_axis_0, split_sizes = var_591_split_sizes_0, x = linear_14_cast_fp16)[name = string("op_591_cast_fp16")]; + tensor var_599 = const()[name = string("op_599"), val = tensor([1, 256, 8, 64])]; + tensor x_63_cast_fp16 = reshape(shape = var_599, x = linear_13_cast_fp16)[name = string("x_63_cast_fp16")]; + tensor var_609 = const()[name = string("op_609"), val = tensor([1, 256, 8, 64])]; + tensor x_67_cast_fp16 = reshape(shape = var_609, x = var_591_cast_fp16_0)[name = string("x_67_cast_fp16")]; + tensor var_619 = const()[name = string("op_619"), val = tensor([1, 256, 8, 64])]; + tensor x_71_cast_fp16 = reshape(shape = var_619, x = var_591_cast_fp16_1)[name = string("x_71_cast_fp16")]; + tensor var_621 = const()[name = string("op_621"), val = tensor([0, 2, 1, 3])]; + bool sim_5_transpose_x_0 = const()[name = string("sim_5_transpose_x_0"), val = bool(false)]; + bool sim_5_transpose_y_0 = const()[name = string("sim_5_transpose_y_0"), val = bool(false)]; + tensor transpose_74_perm_0 = const()[name = string("transpose_74_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_75_perm_0 = const()[name = string("transpose_75_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_75 = transpose(perm = transpose_75_perm_0, x = x_67_cast_fp16)[name = string("transpose_320")]; + tensor transpose_74 = transpose(perm = transpose_74_perm_0, x = x_63_cast_fp16)[name = string("transpose_321")]; + tensor sim_5_cast_fp16 = matmul(transpose_x = sim_5_transpose_x_0, transpose_y = sim_5_transpose_y_0, x = transpose_74, y = transpose_75)[name = string("sim_5_cast_fp16")]; + fp16 var_625_to_fp16 = const()[name = string("op_625_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_7_cast_fp16 = mul(x = sim_5_cast_fp16, y = var_625_to_fp16)[name = string("sim_7_cast_fp16")]; + tensor attn_3_cast_fp16 = softmax(axis = var_35, x = sim_7_cast_fp16)[name = string("attn_3_cast_fp16")]; + bool x_73_transpose_x_0 = const()[name = string("x_73_transpose_x_0"), val = bool(false)]; + bool x_73_transpose_y_0 = const()[name = string("x_73_transpose_y_0"), val = bool(false)]; + tensor v_3_cast_fp16 = transpose(perm = var_621, x = x_71_cast_fp16)[name = string("transpose_322")]; + tensor x_73_cast_fp16 = matmul(transpose_x = x_73_transpose_x_0, transpose_y = x_73_transpose_y_0, x = attn_3_cast_fp16, y = v_3_cast_fp16)[name = string("x_73_cast_fp16")]; + tensor var_647 = const()[name = string("op_647"), val = tensor([0, 2, 1, 3])]; + tensor var_649 = const()[name = string("op_649"), val = tensor([1, 256, 512])]; + tensor x_75_cast_fp16 = transpose(perm = var_647, x = x_73_cast_fp16)[name = string("transpose_319")]; + tensor input_39_cast_fp16 = reshape(shape = var_649, x = x_75_cast_fp16)[name = string("input_39_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(24679168)))]; + tensor unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(25727808)))]; + tensor linear_15_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_39_cast_fp16)[name = string("linear_15_cast_fp16")]; + tensor input_41_cast_fp16 = add(x = linear_15_cast_fp16, y = x_47_cast_fp16)[name = string("input_41_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(25729920)))]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(29924288)))]; + tensor linear_16_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_41_cast_fp16)[name = string("linear_16_cast_fp16")]; + string input_45_mode_0 = const()[name = string("input_45_mode_0"), val = string("EXACT")]; + tensor input_45_cast_fp16 = gelu(mode = input_45_mode_0, x = linear_16_cast_fp16)[name = string("input_45_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(29928448)))]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(34122816)))]; + tensor linear_17_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_45_cast_fp16)[name = string("linear_17_cast_fp16")]; + tensor x_77_cast_fp16 = add(x = linear_17_cast_fp16, y = input_41_cast_fp16)[name = string("x_77_cast_fp16")]; + tensor x_79_cast_fp16 = add(x = x_77_cast_fp16, y = mapping_3_cast_fp16)[name = string("x_79_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(34124928)))]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(35173568)))]; + tensor linear_18_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_norm_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_norm_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_18_cast_fp16")]; + tensor var_663 = const()[name = string("op_663"), val = tensor([1, 2048, 1])]; + tensor h_19_cast_fp16 = reshape(shape = var_663, x = linear_18_cast_fp16)[name = string("h_19_cast_fp16")]; + tensor var_665_split_sizes_0 = const()[name = string("op_665_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_665_axis_0 = const()[name = string("op_665_axis_0"), val = int32(1)]; + tensor var_665_cast_fp16_0, tensor var_665_cast_fp16_1 = split(axis = var_665_axis_0, split_sizes = var_665_split_sizes_0, x = h_19_cast_fp16)[name = string("op_665_cast_fp16")]; + tensor gamma_19_perm_0 = const()[name = string("gamma_19_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_19_perm_0 = const()[name = string("beta_19_perm_0"), val = tensor([0, -1, 1])]; + tensor x_83_axes_0 = const()[name = string("x_83_axes_0"), val = tensor([-1])]; + tensor x_83_cast_fp16 = layer_norm(axes = x_83_axes_0, epsilon = var_31_to_fp16, x = x_79_cast_fp16)[name = string("x_83_cast_fp16")]; + fp16 var_671_promoted_to_fp16 = const()[name = string("op_671_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_19_cast_fp16 = transpose(perm = gamma_19_perm_0, x = var_665_cast_fp16_0)[name = string("transpose_318")]; + tensor var_672_cast_fp16 = add(x = gamma_19_cast_fp16, y = var_671_promoted_to_fp16)[name = string("op_672_cast_fp16")]; + tensor var_673_cast_fp16 = mul(x = var_672_cast_fp16, y = x_83_cast_fp16)[name = string("op_673_cast_fp16")]; + tensor beta_19_cast_fp16 = transpose(perm = beta_19_perm_0, x = var_665_cast_fp16_1)[name = string("transpose_317")]; + tensor x_85_cast_fp16 = add(x = var_673_cast_fp16, y = beta_19_cast_fp16)[name = string("x_85_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(35177728)))]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(36226368)))]; + tensor linear_19_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_19_cast_fp16")]; + tensor var_682 = const()[name = string("op_682"), val = tensor([1, 2048, 1])]; + tensor h_23_cast_fp16 = reshape(shape = var_682, x = linear_19_cast_fp16)[name = string("h_23_cast_fp16")]; + tensor var_684_split_sizes_0 = const()[name = string("op_684_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_684_axis_0 = const()[name = string("op_684_axis_0"), val = int32(1)]; + tensor var_684_cast_fp16_0, tensor var_684_cast_fp16_1 = split(axis = var_684_axis_0, split_sizes = var_684_split_sizes_0, x = h_23_cast_fp16)[name = string("op_684_cast_fp16")]; + tensor gamma_23_perm_0 = const()[name = string("gamma_23_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_23_perm_0 = const()[name = string("beta_23_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_690_promoted_to_fp16 = const()[name = string("op_690_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_23_cast_fp16 = transpose(perm = gamma_23_perm_0, x = var_684_cast_fp16_0)[name = string("transpose_316")]; + tensor var_691_cast_fp16 = add(x = gamma_23_cast_fp16, y = var_690_promoted_to_fp16)[name = string("op_691_cast_fp16")]; + tensor var_692_cast_fp16 = mul(x = var_691_cast_fp16, y = x_83_cast_fp16)[name = string("op_692_cast_fp16")]; + tensor beta_23_cast_fp16 = transpose(perm = beta_23_perm_0, x = var_684_cast_fp16_1)[name = string("transpose_315")]; + tensor x_91_cast_fp16 = add(x = var_692_cast_fp16, y = beta_23_cast_fp16)[name = string("x_91_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(36230528)))]; + tensor linear_20_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_85_cast_fp16)[name = string("linear_20_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(37279168)))]; + tensor linear_21_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_91_cast_fp16)[name = string("linear_21_cast_fp16")]; + tensor var_698_split_sizes_0 = const()[name = string("op_698_split_sizes_0"), val = tensor([512, 512])]; + int32 var_698_axis_0 = const()[name = string("op_698_axis_0"), val = int32(-1)]; + tensor var_698_cast_fp16_0, tensor var_698_cast_fp16_1 = split(axis = var_698_axis_0, split_sizes = var_698_split_sizes_0, x = linear_21_cast_fp16)[name = string("op_698_cast_fp16")]; + tensor var_706 = const()[name = string("op_706"), val = tensor([1, 256, 8, 64])]; + tensor x_95_cast_fp16 = reshape(shape = var_706, x = linear_20_cast_fp16)[name = string("x_95_cast_fp16")]; + tensor var_716 = const()[name = string("op_716"), val = tensor([1, 256, 8, 64])]; + tensor x_99_cast_fp16 = reshape(shape = var_716, x = var_698_cast_fp16_0)[name = string("x_99_cast_fp16")]; + tensor var_726 = const()[name = string("op_726"), val = tensor([1, 256, 8, 64])]; + tensor x_103_cast_fp16 = reshape(shape = var_726, x = var_698_cast_fp16_1)[name = string("x_103_cast_fp16")]; + tensor var_728 = const()[name = string("op_728"), val = tensor([0, 2, 1, 3])]; + bool sim_9_transpose_x_0 = const()[name = string("sim_9_transpose_x_0"), val = bool(false)]; + bool sim_9_transpose_y_0 = const()[name = string("sim_9_transpose_y_0"), val = bool(false)]; + tensor transpose_76_perm_0 = const()[name = string("transpose_76_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_77_perm_0 = const()[name = string("transpose_77_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_77 = transpose(perm = transpose_77_perm_0, x = x_99_cast_fp16)[name = string("transpose_312")]; + tensor transpose_76 = transpose(perm = transpose_76_perm_0, x = x_95_cast_fp16)[name = string("transpose_313")]; + tensor sim_9_cast_fp16 = matmul(transpose_x = sim_9_transpose_x_0, transpose_y = sim_9_transpose_y_0, x = transpose_76, y = transpose_77)[name = string("sim_9_cast_fp16")]; + fp16 var_732_to_fp16 = const()[name = string("op_732_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_11_cast_fp16 = mul(x = sim_9_cast_fp16, y = var_732_to_fp16)[name = string("sim_11_cast_fp16")]; + tensor attn_5_cast_fp16 = softmax(axis = var_35, x = sim_11_cast_fp16)[name = string("attn_5_cast_fp16")]; + bool x_105_transpose_x_0 = const()[name = string("x_105_transpose_x_0"), val = bool(false)]; + bool x_105_transpose_y_0 = const()[name = string("x_105_transpose_y_0"), val = bool(false)]; + tensor v_5_cast_fp16 = transpose(perm = var_728, x = x_103_cast_fp16)[name = string("transpose_314")]; + tensor x_105_cast_fp16 = matmul(transpose_x = x_105_transpose_x_0, transpose_y = x_105_transpose_y_0, x = attn_5_cast_fp16, y = v_5_cast_fp16)[name = string("x_105_cast_fp16")]; + tensor var_754 = const()[name = string("op_754"), val = tensor([0, 2, 1, 3])]; + tensor var_756 = const()[name = string("op_756"), val = tensor([1, 256, 512])]; + tensor x_107_cast_fp16 = transpose(perm = var_754, x = x_105_cast_fp16)[name = string("transpose_311")]; + tensor input_55_cast_fp16 = reshape(shape = var_756, x = x_107_cast_fp16)[name = string("input_55_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(39376384)))]; + tensor unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(40425024)))]; + tensor linear_22_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_55_cast_fp16)[name = string("linear_22_cast_fp16")]; + tensor input_57_cast_fp16 = add(x = linear_22_cast_fp16, y = x_79_cast_fp16)[name = string("input_57_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(40427136)))]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(44621504)))]; + tensor linear_23_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_57_cast_fp16)[name = string("linear_23_cast_fp16")]; + string input_61_mode_0 = const()[name = string("input_61_mode_0"), val = string("EXACT")]; + tensor input_61_cast_fp16 = gelu(mode = input_61_mode_0, x = linear_23_cast_fp16)[name = string("input_61_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(44625664)))]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(48820032)))]; + tensor linear_24_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_61_cast_fp16)[name = string("linear_24_cast_fp16")]; + tensor x_109_cast_fp16 = add(x = linear_24_cast_fp16, y = input_57_cast_fp16)[name = string("x_109_cast_fp16")]; + tensor var_765_axes_0 = const()[name = string("op_765_axes_0"), val = tensor([1])]; + bool var_765_keep_dims_0 = const()[name = string("op_765_keep_dims_0"), val = bool(false)]; + tensor var_765_cast_fp16 = reduce_mean(axes = var_765_axes_0, keep_dims = var_765_keep_dims_0, x = x_109_cast_fp16)[name = string("op_765_cast_fp16")]; + tensor x_111_axes_0 = const()[name = string("x_111_axes_0"), val = tensor([1])]; + tensor x_111_cast_fp16 = expand_dims(axes = x_111_axes_0, x = var_765_cast_fp16)[name = string("x_111_cast_fp16")]; + tensor var_767 = const()[name = string("op_767"), val = tensor([0, 2, 1])]; + string x_113_pad_type_0 = const()[name = string("x_113_pad_type_0"), val = string("valid")]; + tensor x_113_strides_0 = const()[name = string("x_113_strides_0"), val = tensor([1])]; + tensor x_113_pad_0 = const()[name = string("x_113_pad_0"), val = tensor([0, 0])]; + tensor x_113_dilations_0 = const()[name = string("x_113_dilations_0"), val = tensor([1])]; + int32 x_113_groups_0 = const()[name = string("x_113_groups_0"), val = int32(1)]; + tensor unet_step_kdiffusion_net_to_out_1_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_out_1_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(48822144)))]; + tensor unet_step_kdiffusion_net_to_out_1_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_out_1_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49346496)))]; + tensor input_63_cast_fp16 = transpose(perm = var_767, x = x_111_cast_fp16)[name = string("transpose_310")]; + tensor x_113_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_113_dilations_0, groups = x_113_groups_0, pad = x_113_pad_0, pad_type = x_113_pad_type_0, strides = x_113_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_63_cast_fp16)[name = string("x_113_cast_fp16")]; + tensor x_pred_1_perm_0 = const()[name = string("x_pred_1_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_1_to_fp16 = const()[name = string("c_skip_1_to_fp16"), val = tensor([[[0x1.1fcp-8]]])]; + tensor var_775_cast_fp16 = mul(x = c_skip_1_to_fp16, y = x_noisy_1_cast_fp16)[name = string("op_775_cast_fp16")]; + tensor c_out_1_to_fp16 = const()[name = string("c_out_1_to_fp16"), val = tensor([[[0x1.974p-3]]])]; + tensor x_pred_1_cast_fp16 = transpose(perm = x_pred_1_perm_0, x = x_113_cast_fp16)[name = string("transpose_309")]; + tensor var_776_cast_fp16 = mul(x = c_out_1_to_fp16, y = x_pred_1_cast_fp16)[name = string("op_776_cast_fp16")]; + tensor x_dn_1_cast_fp16 = add(x = var_775_cast_fp16, y = var_776_cast_fp16)[name = string("x_dn_1_cast_fp16")]; + tensor var_779_cast_fp16 = sub(x = x_noisy_1_cast_fp16, y = x_dn_1_cast_fp16)[name = string("op_779_cast_fp16")]; + tensor _inversed_d_1_y_0_to_fp16 = const()[name = string("_inversed_d_1_y_0_to_fp16"), val = tensor([0x1.554p-2])]; + tensor _inversed_d_1_cast_fp16 = mul(x = var_779_cast_fp16, y = _inversed_d_1_y_0_to_fp16)[name = string("_inversed_d_1_cast_fp16")]; + fp16 var_788_to_fp16 = const()[name = string("op_788_to_fp16"), val = fp16(-0x1.72cp+0)]; + tensor var_789_cast_fp16 = mul(x = _inversed_d_1_cast_fp16, y = var_788_to_fp16)[name = string("op_789_cast_fp16")]; + tensor x_noisy_3_cast_fp16 = add(x = x_noisy_1_cast_fp16, y = var_789_cast_fp16)[name = string("x_noisy_3_cast_fp16")]; + int32 var_801 = const()[name = string("op_801"), val = int32(-1)]; + tensor c_in_3_to_fp16 = const()[name = string("c_in_3_to_fp16"), val = tensor([[[0x1.474p-1]]])]; + tensor x_123_cast_fp16 = mul(x = c_in_3_to_fp16, y = x_noisy_3_cast_fp16)[name = string("x_123_cast_fp16")]; + int32 x_119_axis_0 = const()[name = string("x_119_axis_0"), val = int32(0)]; + tensor var_1187_to_fp16 = const()[name = string("op_1187_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49347072)))]; + tensor x_119_cast_fp16 = stack(axis = x_119_axis_0, values = (var_1187_to_fp16, var_423_cast_fp16))[name = string("x_119_cast_fp16")]; + tensor var_1192 = const()[name = string("op_1192"), val = tensor([1, 2, 0])]; + tensor input_71_axes_0 = const()[name = string("input_71_axes_0"), val = tensor([2])]; + bool input_71_keep_dims_0 = const()[name = string("input_71_keep_dims_0"), val = bool(false)]; + tensor x_121_cast_fp16 = transpose(perm = var_1192, x = x_119_cast_fp16)[name = string("transpose_308")]; + tensor input_71_cast_fp16 = reduce_sum(axes = input_71_axes_0, keep_dims = input_71_keep_dims_0, x = x_121_cast_fp16)[name = string("input_71_cast_fp16")]; + tensor linear_27_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_71_cast_fp16)[name = string("linear_27_cast_fp16")]; + string input_75_mode_0 = const()[name = string("input_75_mode_0"), val = string("EXACT")]; + tensor input_75_cast_fp16 = gelu(mode = input_75_mode_0, x = linear_27_cast_fp16)[name = string("input_75_cast_fp16")]; + tensor linear_28_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_75_cast_fp16)[name = string("linear_28_cast_fp16")]; + string mapping_5_mode_0 = const()[name = string("mapping_5_mode_0"), val = string("EXACT")]; + tensor mapping_5_cast_fp16 = gelu(mode = mapping_5_mode_0, x = linear_28_cast_fp16)[name = string("mapping_5_cast_fp16")]; + tensor var_1202_reps_0 = const()[name = string("op_1202_reps_0"), val = tensor([1, 256, 1])]; + tensor var_1202_cast_fp16 = tile(reps = var_1202_reps_0, x = x_123_cast_fp16)[name = string("op_1202_cast_fp16")]; + bool x_125_interleave_0 = const()[name = string("x_125_interleave_0"), val = bool(false)]; + tensor x_125_cast_fp16 = concat(axis = var_801, interleave = x_125_interleave_0, values = (var_1202_cast_fp16, embedding_to_fp16))[name = string("x_125_cast_fp16")]; + tensor var_1205_axes_0 = const()[name = string("op_1205_axes_0"), val = tensor([1])]; + tensor var_1205_cast_fp16 = expand_dims(axes = var_1205_axes_0, x = mapping_5_cast_fp16)[name = string("op_1205_cast_fp16")]; + tensor mapping_7_reps_0 = const()[name = string("mapping_7_reps_0"), val = tensor([1, 256, 1])]; + tensor mapping_7_cast_fp16 = tile(reps = mapping_7_reps_0, x = var_1205_cast_fp16)[name = string("mapping_7_cast_fp16")]; + tensor x_127_cast_fp16 = add(x = x_125_cast_fp16, y = mapping_7_cast_fp16)[name = string("x_127_cast_fp16")]; + tensor var_1217_split_sizes_0 = const()[name = string("op_1217_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1217_axis_0 = const()[name = string("op_1217_axis_0"), val = int32(1)]; + tensor var_1217_cast_fp16_0, tensor var_1217_cast_fp16_1 = split(axis = var_1217_axis_0, split_sizes = var_1217_split_sizes_0, x = h_3_cast_fp16)[name = string("op_1217_cast_fp16")]; + tensor gamma_27_perm_0 = const()[name = string("gamma_27_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_27_perm_0 = const()[name = string("beta_27_perm_0"), val = tensor([0, -1, 1])]; + tensor x_131_axes_0 = const()[name = string("x_131_axes_0"), val = tensor([-1])]; + fp16 var_797_to_fp16 = const()[name = string("op_797_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_131_cast_fp16 = layer_norm(axes = x_131_axes_0, epsilon = var_797_to_fp16, x = x_127_cast_fp16)[name = string("x_131_cast_fp16")]; + fp16 var_1223_promoted_to_fp16 = const()[name = string("op_1223_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_27_cast_fp16 = transpose(perm = gamma_27_perm_0, x = var_1217_cast_fp16_0)[name = string("transpose_307")]; + tensor var_1224_cast_fp16 = add(x = gamma_27_cast_fp16, y = var_1223_promoted_to_fp16)[name = string("op_1224_cast_fp16")]; + tensor var_1225_cast_fp16 = mul(x = var_1224_cast_fp16, y = x_131_cast_fp16)[name = string("op_1225_cast_fp16")]; + tensor beta_27_cast_fp16 = transpose(perm = beta_27_perm_0, x = var_1217_cast_fp16_1)[name = string("transpose_306")]; + tensor x_133_cast_fp16 = add(x = var_1225_cast_fp16, y = beta_27_cast_fp16)[name = string("x_133_cast_fp16")]; + tensor var_1236_split_sizes_0 = const()[name = string("op_1236_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1236_axis_0 = const()[name = string("op_1236_axis_0"), val = int32(1)]; + tensor var_1236_cast_fp16_0, tensor var_1236_cast_fp16_1 = split(axis = var_1236_axis_0, split_sizes = var_1236_split_sizes_0, x = h_7_cast_fp16)[name = string("op_1236_cast_fp16")]; + tensor gamma_31_perm_0 = const()[name = string("gamma_31_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_31_perm_0 = const()[name = string("beta_31_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_1242_promoted_to_fp16 = const()[name = string("op_1242_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_31_cast_fp16 = transpose(perm = gamma_31_perm_0, x = var_1236_cast_fp16_0)[name = string("transpose_305")]; + tensor var_1243_cast_fp16 = add(x = gamma_31_cast_fp16, y = var_1242_promoted_to_fp16)[name = string("op_1243_cast_fp16")]; + tensor var_1244_cast_fp16 = mul(x = var_1243_cast_fp16, y = x_131_cast_fp16)[name = string("op_1244_cast_fp16")]; + tensor beta_31_cast_fp16 = transpose(perm = beta_31_perm_0, x = var_1236_cast_fp16_1)[name = string("transpose_304")]; + tensor x_139_cast_fp16 = add(x = var_1244_cast_fp16, y = beta_31_cast_fp16)[name = string("x_139_cast_fp16")]; + tensor linear_31_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_133_cast_fp16)[name = string("linear_31_cast_fp16")]; + tensor linear_32_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_139_cast_fp16)[name = string("linear_32_cast_fp16")]; + tensor var_1250_split_sizes_0 = const()[name = string("op_1250_split_sizes_0"), val = tensor([512, 512])]; + int32 var_1250_axis_0 = const()[name = string("op_1250_axis_0"), val = int32(-1)]; + tensor var_1250_cast_fp16_0, tensor var_1250_cast_fp16_1 = split(axis = var_1250_axis_0, split_sizes = var_1250_split_sizes_0, x = linear_32_cast_fp16)[name = string("op_1250_cast_fp16")]; + tensor var_1258 = const()[name = string("op_1258"), val = tensor([1, 256, 8, 64])]; + tensor x_143_cast_fp16 = reshape(shape = var_1258, x = linear_31_cast_fp16)[name = string("x_143_cast_fp16")]; + tensor var_1268 = const()[name = string("op_1268"), val = tensor([1, 256, 8, 64])]; + tensor x_147_cast_fp16 = reshape(shape = var_1268, x = var_1250_cast_fp16_0)[name = string("x_147_cast_fp16")]; + tensor var_1278 = const()[name = string("op_1278"), val = tensor([1, 256, 8, 64])]; + tensor x_151_cast_fp16 = reshape(shape = var_1278, x = var_1250_cast_fp16_1)[name = string("x_151_cast_fp16")]; + tensor var_1280 = const()[name = string("op_1280"), val = tensor([0, 2, 1, 3])]; + bool sim_13_transpose_x_0 = const()[name = string("sim_13_transpose_x_0"), val = bool(false)]; + bool sim_13_transpose_y_0 = const()[name = string("sim_13_transpose_y_0"), val = bool(false)]; + tensor transpose_78_perm_0 = const()[name = string("transpose_78_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_79_perm_0 = const()[name = string("transpose_79_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_79 = transpose(perm = transpose_79_perm_0, x = x_147_cast_fp16)[name = string("transpose_301")]; + tensor transpose_78 = transpose(perm = transpose_78_perm_0, x = x_143_cast_fp16)[name = string("transpose_302")]; + tensor sim_13_cast_fp16 = matmul(transpose_x = sim_13_transpose_x_0, transpose_y = sim_13_transpose_y_0, x = transpose_78, y = transpose_79)[name = string("sim_13_cast_fp16")]; + fp16 var_1284_to_fp16 = const()[name = string("op_1284_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_15_cast_fp16 = mul(x = sim_13_cast_fp16, y = var_1284_to_fp16)[name = string("sim_15_cast_fp16")]; + tensor attn_7_cast_fp16 = softmax(axis = var_801, x = sim_15_cast_fp16)[name = string("attn_7_cast_fp16")]; + bool x_153_transpose_x_0 = const()[name = string("x_153_transpose_x_0"), val = bool(false)]; + bool x_153_transpose_y_0 = const()[name = string("x_153_transpose_y_0"), val = bool(false)]; + tensor v_7_cast_fp16 = transpose(perm = var_1280, x = x_151_cast_fp16)[name = string("transpose_303")]; + tensor x_153_cast_fp16 = matmul(transpose_x = x_153_transpose_x_0, transpose_y = x_153_transpose_y_0, x = attn_7_cast_fp16, y = v_7_cast_fp16)[name = string("x_153_cast_fp16")]; + tensor var_1306 = const()[name = string("op_1306"), val = tensor([0, 2, 1, 3])]; + tensor var_1308 = const()[name = string("op_1308"), val = tensor([1, 256, 512])]; + tensor x_155_cast_fp16 = transpose(perm = var_1306, x = x_153_cast_fp16)[name = string("transpose_300")]; + tensor input_87_cast_fp16 = reshape(shape = var_1308, x = x_155_cast_fp16)[name = string("input_87_cast_fp16")]; + tensor linear_33_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_87_cast_fp16)[name = string("linear_33_cast_fp16")]; + tensor input_89_cast_fp16 = add(x = linear_33_cast_fp16, y = x_127_cast_fp16)[name = string("input_89_cast_fp16")]; + tensor linear_34_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_89_cast_fp16)[name = string("linear_34_cast_fp16")]; + string input_93_mode_0 = const()[name = string("input_93_mode_0"), val = string("EXACT")]; + tensor input_93_cast_fp16 = gelu(mode = input_93_mode_0, x = linear_34_cast_fp16)[name = string("input_93_cast_fp16")]; + tensor linear_35_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_93_cast_fp16)[name = string("linear_35_cast_fp16")]; + tensor x_157_cast_fp16 = add(x = linear_35_cast_fp16, y = input_89_cast_fp16)[name = string("x_157_cast_fp16")]; + tensor x_159_cast_fp16 = add(x = x_157_cast_fp16, y = mapping_7_cast_fp16)[name = string("x_159_cast_fp16")]; + tensor var_1324_split_sizes_0 = const()[name = string("op_1324_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1324_axis_0 = const()[name = string("op_1324_axis_0"), val = int32(1)]; + tensor var_1324_cast_fp16_0, tensor var_1324_cast_fp16_1 = split(axis = var_1324_axis_0, split_sizes = var_1324_split_sizes_0, x = h_11_cast_fp16)[name = string("op_1324_cast_fp16")]; + tensor gamma_35_perm_0 = const()[name = string("gamma_35_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_35_perm_0 = const()[name = string("beta_35_perm_0"), val = tensor([0, -1, 1])]; + tensor x_163_axes_0 = const()[name = string("x_163_axes_0"), val = tensor([-1])]; + tensor x_163_cast_fp16 = layer_norm(axes = x_163_axes_0, epsilon = var_797_to_fp16, x = x_159_cast_fp16)[name = string("x_163_cast_fp16")]; + fp16 var_1330_promoted_to_fp16 = const()[name = string("op_1330_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_35_cast_fp16 = transpose(perm = gamma_35_perm_0, x = var_1324_cast_fp16_0)[name = string("transpose_299")]; + tensor var_1331_cast_fp16 = add(x = gamma_35_cast_fp16, y = var_1330_promoted_to_fp16)[name = string("op_1331_cast_fp16")]; + tensor var_1332_cast_fp16 = mul(x = var_1331_cast_fp16, y = x_163_cast_fp16)[name = string("op_1332_cast_fp16")]; + tensor beta_35_cast_fp16 = transpose(perm = beta_35_perm_0, x = var_1324_cast_fp16_1)[name = string("transpose_298")]; + tensor x_165_cast_fp16 = add(x = var_1332_cast_fp16, y = beta_35_cast_fp16)[name = string("x_165_cast_fp16")]; + tensor var_1343_split_sizes_0 = const()[name = string("op_1343_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1343_axis_0 = const()[name = string("op_1343_axis_0"), val = int32(1)]; + tensor var_1343_cast_fp16_0, tensor var_1343_cast_fp16_1 = split(axis = var_1343_axis_0, split_sizes = var_1343_split_sizes_0, x = h_15_cast_fp16)[name = string("op_1343_cast_fp16")]; + tensor gamma_39_perm_0 = const()[name = string("gamma_39_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_39_perm_0 = const()[name = string("beta_39_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_1349_promoted_to_fp16 = const()[name = string("op_1349_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_39_cast_fp16 = transpose(perm = gamma_39_perm_0, x = var_1343_cast_fp16_0)[name = string("transpose_297")]; + tensor var_1350_cast_fp16 = add(x = gamma_39_cast_fp16, y = var_1349_promoted_to_fp16)[name = string("op_1350_cast_fp16")]; + tensor var_1351_cast_fp16 = mul(x = var_1350_cast_fp16, y = x_163_cast_fp16)[name = string("op_1351_cast_fp16")]; + tensor beta_39_cast_fp16 = transpose(perm = beta_39_perm_0, x = var_1343_cast_fp16_1)[name = string("transpose_296")]; + tensor x_171_cast_fp16 = add(x = var_1351_cast_fp16, y = beta_39_cast_fp16)[name = string("x_171_cast_fp16")]; + tensor linear_38_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_165_cast_fp16)[name = string("linear_38_cast_fp16")]; + tensor linear_39_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_171_cast_fp16)[name = string("linear_39_cast_fp16")]; + tensor var_1357_split_sizes_0 = const()[name = string("op_1357_split_sizes_0"), val = tensor([512, 512])]; + int32 var_1357_axis_0 = const()[name = string("op_1357_axis_0"), val = int32(-1)]; + tensor var_1357_cast_fp16_0, tensor var_1357_cast_fp16_1 = split(axis = var_1357_axis_0, split_sizes = var_1357_split_sizes_0, x = linear_39_cast_fp16)[name = string("op_1357_cast_fp16")]; + tensor var_1365 = const()[name = string("op_1365"), val = tensor([1, 256, 8, 64])]; + tensor x_175_cast_fp16 = reshape(shape = var_1365, x = linear_38_cast_fp16)[name = string("x_175_cast_fp16")]; + tensor var_1375 = const()[name = string("op_1375"), val = tensor([1, 256, 8, 64])]; + tensor x_179_cast_fp16 = reshape(shape = var_1375, x = var_1357_cast_fp16_0)[name = string("x_179_cast_fp16")]; + tensor var_1385 = const()[name = string("op_1385"), val = tensor([1, 256, 8, 64])]; + tensor x_183_cast_fp16 = reshape(shape = var_1385, x = var_1357_cast_fp16_1)[name = string("x_183_cast_fp16")]; + tensor var_1387 = const()[name = string("op_1387"), val = tensor([0, 2, 1, 3])]; + bool sim_17_transpose_x_0 = const()[name = string("sim_17_transpose_x_0"), val = bool(false)]; + bool sim_17_transpose_y_0 = const()[name = string("sim_17_transpose_y_0"), val = bool(false)]; + tensor transpose_80_perm_0 = const()[name = string("transpose_80_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_81_perm_0 = const()[name = string("transpose_81_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_81 = transpose(perm = transpose_81_perm_0, x = x_179_cast_fp16)[name = string("transpose_293")]; + tensor transpose_80 = transpose(perm = transpose_80_perm_0, x = x_175_cast_fp16)[name = string("transpose_294")]; + tensor sim_17_cast_fp16 = matmul(transpose_x = sim_17_transpose_x_0, transpose_y = sim_17_transpose_y_0, x = transpose_80, y = transpose_81)[name = string("sim_17_cast_fp16")]; + fp16 var_1391_to_fp16 = const()[name = string("op_1391_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_19_cast_fp16 = mul(x = sim_17_cast_fp16, y = var_1391_to_fp16)[name = string("sim_19_cast_fp16")]; + tensor attn_9_cast_fp16 = softmax(axis = var_801, x = sim_19_cast_fp16)[name = string("attn_9_cast_fp16")]; + bool x_185_transpose_x_0 = const()[name = string("x_185_transpose_x_0"), val = bool(false)]; + bool x_185_transpose_y_0 = const()[name = string("x_185_transpose_y_0"), val = bool(false)]; + tensor v_9_cast_fp16 = transpose(perm = var_1387, x = x_183_cast_fp16)[name = string("transpose_295")]; + tensor x_185_cast_fp16 = matmul(transpose_x = x_185_transpose_x_0, transpose_y = x_185_transpose_y_0, x = attn_9_cast_fp16, y = v_9_cast_fp16)[name = string("x_185_cast_fp16")]; + tensor var_1413 = const()[name = string("op_1413"), val = tensor([0, 2, 1, 3])]; + tensor var_1415 = const()[name = string("op_1415"), val = tensor([1, 256, 512])]; + tensor x_187_cast_fp16 = transpose(perm = var_1413, x = x_185_cast_fp16)[name = string("transpose_292")]; + tensor input_103_cast_fp16 = reshape(shape = var_1415, x = x_187_cast_fp16)[name = string("input_103_cast_fp16")]; + tensor linear_40_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_103_cast_fp16)[name = string("linear_40_cast_fp16")]; + tensor input_105_cast_fp16 = add(x = linear_40_cast_fp16, y = x_159_cast_fp16)[name = string("input_105_cast_fp16")]; + tensor linear_41_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_105_cast_fp16)[name = string("linear_41_cast_fp16")]; + string input_109_mode_0 = const()[name = string("input_109_mode_0"), val = string("EXACT")]; + tensor input_109_cast_fp16 = gelu(mode = input_109_mode_0, x = linear_41_cast_fp16)[name = string("input_109_cast_fp16")]; + tensor linear_42_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_109_cast_fp16)[name = string("linear_42_cast_fp16")]; + tensor x_189_cast_fp16 = add(x = linear_42_cast_fp16, y = input_105_cast_fp16)[name = string("x_189_cast_fp16")]; + tensor x_191_cast_fp16 = add(x = x_189_cast_fp16, y = mapping_7_cast_fp16)[name = string("x_191_cast_fp16")]; + tensor var_1431_split_sizes_0 = const()[name = string("op_1431_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1431_axis_0 = const()[name = string("op_1431_axis_0"), val = int32(1)]; + tensor var_1431_cast_fp16_0, tensor var_1431_cast_fp16_1 = split(axis = var_1431_axis_0, split_sizes = var_1431_split_sizes_0, x = h_19_cast_fp16)[name = string("op_1431_cast_fp16")]; + tensor gamma_43_perm_0 = const()[name = string("gamma_43_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_43_perm_0 = const()[name = string("beta_43_perm_0"), val = tensor([0, -1, 1])]; + tensor x_195_axes_0 = const()[name = string("x_195_axes_0"), val = tensor([-1])]; + tensor x_195_cast_fp16 = layer_norm(axes = x_195_axes_0, epsilon = var_797_to_fp16, x = x_191_cast_fp16)[name = string("x_195_cast_fp16")]; + fp16 var_1437_promoted_to_fp16 = const()[name = string("op_1437_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_43_cast_fp16 = transpose(perm = gamma_43_perm_0, x = var_1431_cast_fp16_0)[name = string("transpose_291")]; + tensor var_1438_cast_fp16 = add(x = gamma_43_cast_fp16, y = var_1437_promoted_to_fp16)[name = string("op_1438_cast_fp16")]; + tensor var_1439_cast_fp16 = mul(x = var_1438_cast_fp16, y = x_195_cast_fp16)[name = string("op_1439_cast_fp16")]; + tensor beta_43_cast_fp16 = transpose(perm = beta_43_perm_0, x = var_1431_cast_fp16_1)[name = string("transpose_290")]; + tensor x_197_cast_fp16 = add(x = var_1439_cast_fp16, y = beta_43_cast_fp16)[name = string("x_197_cast_fp16")]; + tensor var_1450_split_sizes_0 = const()[name = string("op_1450_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1450_axis_0 = const()[name = string("op_1450_axis_0"), val = int32(1)]; + tensor var_1450_cast_fp16_0, tensor var_1450_cast_fp16_1 = split(axis = var_1450_axis_0, split_sizes = var_1450_split_sizes_0, x = h_23_cast_fp16)[name = string("op_1450_cast_fp16")]; + tensor gamma_47_perm_0 = const()[name = string("gamma_47_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_47_perm_0 = const()[name = string("beta_47_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_1456_promoted_to_fp16 = const()[name = string("op_1456_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_47_cast_fp16 = transpose(perm = gamma_47_perm_0, x = var_1450_cast_fp16_0)[name = string("transpose_289")]; + tensor var_1457_cast_fp16 = add(x = gamma_47_cast_fp16, y = var_1456_promoted_to_fp16)[name = string("op_1457_cast_fp16")]; + tensor var_1458_cast_fp16 = mul(x = var_1457_cast_fp16, y = x_195_cast_fp16)[name = string("op_1458_cast_fp16")]; + tensor beta_47_cast_fp16 = transpose(perm = beta_47_perm_0, x = var_1450_cast_fp16_1)[name = string("transpose_288")]; + tensor x_203_cast_fp16 = add(x = var_1458_cast_fp16, y = beta_47_cast_fp16)[name = string("x_203_cast_fp16")]; + tensor linear_45_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_197_cast_fp16)[name = string("linear_45_cast_fp16")]; + tensor linear_46_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_203_cast_fp16)[name = string("linear_46_cast_fp16")]; + tensor var_1464_split_sizes_0 = const()[name = string("op_1464_split_sizes_0"), val = tensor([512, 512])]; + int32 var_1464_axis_0 = const()[name = string("op_1464_axis_0"), val = int32(-1)]; + tensor var_1464_cast_fp16_0, tensor var_1464_cast_fp16_1 = split(axis = var_1464_axis_0, split_sizes = var_1464_split_sizes_0, x = linear_46_cast_fp16)[name = string("op_1464_cast_fp16")]; + tensor var_1472 = const()[name = string("op_1472"), val = tensor([1, 256, 8, 64])]; + tensor x_207_cast_fp16 = reshape(shape = var_1472, x = linear_45_cast_fp16)[name = string("x_207_cast_fp16")]; + tensor var_1482 = const()[name = string("op_1482"), val = tensor([1, 256, 8, 64])]; + tensor x_211_cast_fp16 = reshape(shape = var_1482, x = var_1464_cast_fp16_0)[name = string("x_211_cast_fp16")]; + tensor var_1492 = const()[name = string("op_1492"), val = tensor([1, 256, 8, 64])]; + tensor x_215_cast_fp16 = reshape(shape = var_1492, x = var_1464_cast_fp16_1)[name = string("x_215_cast_fp16")]; + tensor var_1494 = const()[name = string("op_1494"), val = tensor([0, 2, 1, 3])]; + bool sim_21_transpose_x_0 = const()[name = string("sim_21_transpose_x_0"), val = bool(false)]; + bool sim_21_transpose_y_0 = const()[name = string("sim_21_transpose_y_0"), val = bool(false)]; + tensor transpose_82_perm_0 = const()[name = string("transpose_82_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_83_perm_0 = const()[name = string("transpose_83_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_83 = transpose(perm = transpose_83_perm_0, x = x_211_cast_fp16)[name = string("transpose_285")]; + tensor transpose_82 = transpose(perm = transpose_82_perm_0, x = x_207_cast_fp16)[name = string("transpose_286")]; + tensor sim_21_cast_fp16 = matmul(transpose_x = sim_21_transpose_x_0, transpose_y = sim_21_transpose_y_0, x = transpose_82, y = transpose_83)[name = string("sim_21_cast_fp16")]; + fp16 var_1498_to_fp16 = const()[name = string("op_1498_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_23_cast_fp16 = mul(x = sim_21_cast_fp16, y = var_1498_to_fp16)[name = string("sim_23_cast_fp16")]; + tensor attn_11_cast_fp16 = softmax(axis = var_801, x = sim_23_cast_fp16)[name = string("attn_11_cast_fp16")]; + bool x_217_transpose_x_0 = const()[name = string("x_217_transpose_x_0"), val = bool(false)]; + bool x_217_transpose_y_0 = const()[name = string("x_217_transpose_y_0"), val = bool(false)]; + tensor v_11_cast_fp16 = transpose(perm = var_1494, x = x_215_cast_fp16)[name = string("transpose_287")]; + tensor x_217_cast_fp16 = matmul(transpose_x = x_217_transpose_x_0, transpose_y = x_217_transpose_y_0, x = attn_11_cast_fp16, y = v_11_cast_fp16)[name = string("x_217_cast_fp16")]; + tensor var_1520 = const()[name = string("op_1520"), val = tensor([0, 2, 1, 3])]; + tensor var_1522 = const()[name = string("op_1522"), val = tensor([1, 256, 512])]; + tensor x_219_cast_fp16 = transpose(perm = var_1520, x = x_217_cast_fp16)[name = string("transpose_284")]; + tensor input_119_cast_fp16 = reshape(shape = var_1522, x = x_219_cast_fp16)[name = string("input_119_cast_fp16")]; + tensor linear_47_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_119_cast_fp16)[name = string("linear_47_cast_fp16")]; + tensor input_121_cast_fp16 = add(x = linear_47_cast_fp16, y = x_191_cast_fp16)[name = string("input_121_cast_fp16")]; + tensor linear_48_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_121_cast_fp16)[name = string("linear_48_cast_fp16")]; + string input_125_mode_0 = const()[name = string("input_125_mode_0"), val = string("EXACT")]; + tensor input_125_cast_fp16 = gelu(mode = input_125_mode_0, x = linear_48_cast_fp16)[name = string("input_125_cast_fp16")]; + tensor linear_49_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_125_cast_fp16)[name = string("linear_49_cast_fp16")]; + tensor x_221_cast_fp16 = add(x = linear_49_cast_fp16, y = input_121_cast_fp16)[name = string("x_221_cast_fp16")]; + tensor var_1531_axes_0 = const()[name = string("op_1531_axes_0"), val = tensor([1])]; + bool var_1531_keep_dims_0 = const()[name = string("op_1531_keep_dims_0"), val = bool(false)]; + tensor var_1531_cast_fp16 = reduce_mean(axes = var_1531_axes_0, keep_dims = var_1531_keep_dims_0, x = x_221_cast_fp16)[name = string("op_1531_cast_fp16")]; + tensor x_223_axes_0 = const()[name = string("x_223_axes_0"), val = tensor([1])]; + tensor x_223_cast_fp16 = expand_dims(axes = x_223_axes_0, x = var_1531_cast_fp16)[name = string("x_223_cast_fp16")]; + tensor var_1533 = const()[name = string("op_1533"), val = tensor([0, 2, 1])]; + string x_225_pad_type_0 = const()[name = string("x_225_pad_type_0"), val = string("valid")]; + tensor x_225_strides_0 = const()[name = string("x_225_strides_0"), val = tensor([1])]; + tensor x_225_pad_0 = const()[name = string("x_225_pad_0"), val = tensor([0, 0])]; + tensor x_225_dilations_0 = const()[name = string("x_225_dilations_0"), val = tensor([1])]; + int32 x_225_groups_0 = const()[name = string("x_225_groups_0"), val = int32(1)]; + tensor input_127_cast_fp16 = transpose(perm = var_1533, x = x_223_cast_fp16)[name = string("transpose_283")]; + tensor x_225_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_225_dilations_0, groups = x_225_groups_0, pad = x_225_pad_0, pad_type = x_225_pad_type_0, strides = x_225_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_127_cast_fp16)[name = string("x_225_cast_fp16")]; + tensor x_pred_3_perm_0 = const()[name = string("x_pred_3_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_3_to_fp16 = const()[name = string("c_skip_3_to_fp16"), val = tensor([[[0x1.09cp-6]]])]; + tensor var_1541_cast_fp16 = mul(x = c_skip_3_to_fp16, y = x_noisy_3_cast_fp16)[name = string("op_1541_cast_fp16")]; + tensor c_out_3_to_fp16 = const()[name = string("c_out_3_to_fp16"), val = tensor([[[0x1.94cp-3]]])]; + tensor x_pred_3_cast_fp16 = transpose(perm = x_pred_3_perm_0, x = x_225_cast_fp16)[name = string("transpose_282")]; + tensor var_1542_cast_fp16 = mul(x = c_out_3_to_fp16, y = x_pred_3_cast_fp16)[name = string("op_1542_cast_fp16")]; + tensor x_mid_dn_1_cast_fp16 = add(x = var_1541_cast_fp16, y = var_1542_cast_fp16)[name = string("x_mid_dn_1_cast_fp16")]; + tensor var_1545_cast_fp16 = sub(x = x_noisy_3_cast_fp16, y = x_mid_dn_1_cast_fp16)[name = string("op_1545_cast_fp16")]; + tensor _inversed_d_mid_1_y_0_to_fp16 = const()[name = string("_inversed_d_mid_1_y_0_to_fp16"), val = tensor([0x1.4ap-1])]; + tensor _inversed_d_mid_1_cast_fp16 = mul(x = var_1545_cast_fp16, y = _inversed_d_mid_1_y_0_to_fp16)[name = string("_inversed_d_mid_1_cast_fp16")]; + fp16 var_1554_to_fp16 = const()[name = string("op_1554_to_fp16"), val = fp16(-0x1.72cp+1)]; + tensor var_1555_cast_fp16 = mul(x = _inversed_d_mid_1_cast_fp16, y = var_1554_to_fp16)[name = string("op_1555_cast_fp16")]; + tensor x_227_cast_fp16 = add(x = x_noisy_1_cast_fp16, y = var_1555_cast_fp16)[name = string("x_227_cast_fp16")]; + tensor var_1560_begin_0 = const()[name = string("op_1560_begin_0"), val = tensor([0, 0, 0, 0])]; + tensor var_1560_end_0 = const()[name = string("op_1560_end_0"), val = tensor([1, 1, 1, 256])]; + tensor var_1560_end_mask_0 = const()[name = string("op_1560_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_1560_squeeze_mask_0 = const()[name = string("op_1560_squeeze_mask_0"), val = tensor([true, false, false, false])]; + string noises_aux_to_fp16_dtype_0 = const()[name = string("noises_aux_to_fp16_dtype_0"), val = string("fp16")]; + tensor noises_aux_to_fp16 = cast(dtype = noises_aux_to_fp16_dtype_0, x = noises_aux)[name = string("cast_193")]; + tensor var_1560_cast_fp16 = slice_by_index(begin = var_1560_begin_0, end = var_1560_end_0, end_mask = var_1560_end_mask_0, squeeze_mask = var_1560_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_1560_cast_fp16")]; + fp16 var_1563_to_fp16 = const()[name = string("op_1563_to_fp16"), val = fp16(0x1.18cp-1)]; + tensor var_1564_cast_fp16 = mul(x = var_1560_cast_fp16, y = var_1563_to_fp16)[name = string("op_1564_cast_fp16")]; + tensor x_noisy_5_cast_fp16 = add(x = x_227_cast_fp16, y = var_1564_cast_fp16)[name = string("x_noisy_5_cast_fp16")]; + int32 var_1588 = const()[name = string("op_1588"), val = int32(-1)]; + tensor c_in_5_to_fp16 = const()[name = string("c_in_5_to_fp16"), val = tensor([[[0x1.bp+0]]])]; + tensor x_237_cast_fp16 = mul(x = c_in_5_to_fp16, y = x_noisy_5_cast_fp16)[name = string("x_237_cast_fp16")]; + int32 x_233_axis_0 = const()[name = string("x_233_axis_0"), val = int32(0)]; + tensor var_1974_to_fp16 = const()[name = string("op_1974_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49349184)))]; + tensor x_233_cast_fp16 = stack(axis = x_233_axis_0, values = (var_1974_to_fp16, var_423_cast_fp16))[name = string("x_233_cast_fp16")]; + tensor var_1979 = const()[name = string("op_1979"), val = tensor([1, 2, 0])]; + tensor input_135_axes_0 = const()[name = string("input_135_axes_0"), val = tensor([2])]; + bool input_135_keep_dims_0 = const()[name = string("input_135_keep_dims_0"), val = bool(false)]; + tensor x_235_cast_fp16 = transpose(perm = var_1979, x = x_233_cast_fp16)[name = string("transpose_281")]; + tensor input_135_cast_fp16 = reduce_sum(axes = input_135_axes_0, keep_dims = input_135_keep_dims_0, x = x_235_cast_fp16)[name = string("input_135_cast_fp16")]; + tensor linear_52_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_135_cast_fp16)[name = string("linear_52_cast_fp16")]; + string input_139_mode_0 = const()[name = string("input_139_mode_0"), val = string("EXACT")]; + tensor input_139_cast_fp16 = gelu(mode = input_139_mode_0, x = linear_52_cast_fp16)[name = string("input_139_cast_fp16")]; + tensor linear_53_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_139_cast_fp16)[name = string("linear_53_cast_fp16")]; + string mapping_9_mode_0 = const()[name = string("mapping_9_mode_0"), val = string("EXACT")]; + tensor mapping_9_cast_fp16 = gelu(mode = mapping_9_mode_0, x = linear_53_cast_fp16)[name = string("mapping_9_cast_fp16")]; + tensor var_1989_reps_0 = const()[name = string("op_1989_reps_0"), val = tensor([1, 256, 1])]; + tensor var_1989_cast_fp16 = tile(reps = var_1989_reps_0, x = x_237_cast_fp16)[name = string("op_1989_cast_fp16")]; + bool x_239_interleave_0 = const()[name = string("x_239_interleave_0"), val = bool(false)]; + tensor x_239_cast_fp16 = concat(axis = var_1588, interleave = x_239_interleave_0, values = (var_1989_cast_fp16, embedding_to_fp16))[name = string("x_239_cast_fp16")]; + tensor var_1992_axes_0 = const()[name = string("op_1992_axes_0"), val = tensor([1])]; + tensor var_1992_cast_fp16 = expand_dims(axes = var_1992_axes_0, x = mapping_9_cast_fp16)[name = string("op_1992_cast_fp16")]; + tensor mapping_11_reps_0 = const()[name = string("mapping_11_reps_0"), val = tensor([1, 256, 1])]; + tensor mapping_11_cast_fp16 = tile(reps = mapping_11_reps_0, x = var_1992_cast_fp16)[name = string("mapping_11_cast_fp16")]; + tensor x_241_cast_fp16 = add(x = x_239_cast_fp16, y = mapping_11_cast_fp16)[name = string("x_241_cast_fp16")]; + tensor var_2004_split_sizes_0 = const()[name = string("op_2004_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2004_axis_0 = const()[name = string("op_2004_axis_0"), val = int32(1)]; + tensor var_2004_cast_fp16_0, tensor var_2004_cast_fp16_1 = split(axis = var_2004_axis_0, split_sizes = var_2004_split_sizes_0, x = h_3_cast_fp16)[name = string("op_2004_cast_fp16")]; + tensor gamma_51_perm_0 = const()[name = string("gamma_51_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_51_perm_0 = const()[name = string("beta_51_perm_0"), val = tensor([0, -1, 1])]; + tensor x_245_axes_0 = const()[name = string("x_245_axes_0"), val = tensor([-1])]; + fp16 var_1584_to_fp16 = const()[name = string("op_1584_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_245_cast_fp16 = layer_norm(axes = x_245_axes_0, epsilon = var_1584_to_fp16, x = x_241_cast_fp16)[name = string("x_245_cast_fp16")]; + fp16 var_2010_promoted_to_fp16 = const()[name = string("op_2010_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_51_cast_fp16 = transpose(perm = gamma_51_perm_0, x = var_2004_cast_fp16_0)[name = string("transpose_280")]; + tensor var_2011_cast_fp16 = add(x = gamma_51_cast_fp16, y = var_2010_promoted_to_fp16)[name = string("op_2011_cast_fp16")]; + tensor var_2012_cast_fp16 = mul(x = var_2011_cast_fp16, y = x_245_cast_fp16)[name = string("op_2012_cast_fp16")]; + tensor beta_51_cast_fp16 = transpose(perm = beta_51_perm_0, x = var_2004_cast_fp16_1)[name = string("transpose_279")]; + tensor x_247_cast_fp16 = add(x = var_2012_cast_fp16, y = beta_51_cast_fp16)[name = string("x_247_cast_fp16")]; + tensor var_2023_split_sizes_0 = const()[name = string("op_2023_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2023_axis_0 = const()[name = string("op_2023_axis_0"), val = int32(1)]; + tensor var_2023_cast_fp16_0, tensor var_2023_cast_fp16_1 = split(axis = var_2023_axis_0, split_sizes = var_2023_split_sizes_0, x = h_7_cast_fp16)[name = string("op_2023_cast_fp16")]; + tensor gamma_55_perm_0 = const()[name = string("gamma_55_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_55_perm_0 = const()[name = string("beta_55_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2029_promoted_to_fp16 = const()[name = string("op_2029_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_55_cast_fp16 = transpose(perm = gamma_55_perm_0, x = var_2023_cast_fp16_0)[name = string("transpose_278")]; + tensor var_2030_cast_fp16 = add(x = gamma_55_cast_fp16, y = var_2029_promoted_to_fp16)[name = string("op_2030_cast_fp16")]; + tensor var_2031_cast_fp16 = mul(x = var_2030_cast_fp16, y = x_245_cast_fp16)[name = string("op_2031_cast_fp16")]; + tensor beta_55_cast_fp16 = transpose(perm = beta_55_perm_0, x = var_2023_cast_fp16_1)[name = string("transpose_277")]; + tensor x_253_cast_fp16 = add(x = var_2031_cast_fp16, y = beta_55_cast_fp16)[name = string("x_253_cast_fp16")]; + tensor linear_56_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_247_cast_fp16)[name = string("linear_56_cast_fp16")]; + tensor linear_57_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_253_cast_fp16)[name = string("linear_57_cast_fp16")]; + tensor var_2037_split_sizes_0 = const()[name = string("op_2037_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2037_axis_0 = const()[name = string("op_2037_axis_0"), val = int32(-1)]; + tensor var_2037_cast_fp16_0, tensor var_2037_cast_fp16_1 = split(axis = var_2037_axis_0, split_sizes = var_2037_split_sizes_0, x = linear_57_cast_fp16)[name = string("op_2037_cast_fp16")]; + tensor var_2045 = const()[name = string("op_2045"), val = tensor([1, 256, 8, 64])]; + tensor x_257_cast_fp16 = reshape(shape = var_2045, x = linear_56_cast_fp16)[name = string("x_257_cast_fp16")]; + tensor var_2055 = const()[name = string("op_2055"), val = tensor([1, 256, 8, 64])]; + tensor x_261_cast_fp16 = reshape(shape = var_2055, x = var_2037_cast_fp16_0)[name = string("x_261_cast_fp16")]; + tensor var_2065 = const()[name = string("op_2065"), val = tensor([1, 256, 8, 64])]; + tensor x_265_cast_fp16 = reshape(shape = var_2065, x = var_2037_cast_fp16_1)[name = string("x_265_cast_fp16")]; + tensor var_2067 = const()[name = string("op_2067"), val = tensor([0, 2, 1, 3])]; + bool sim_25_transpose_x_0 = const()[name = string("sim_25_transpose_x_0"), val = bool(false)]; + bool sim_25_transpose_y_0 = const()[name = string("sim_25_transpose_y_0"), val = bool(false)]; + tensor transpose_84_perm_0 = const()[name = string("transpose_84_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_85_perm_0 = const()[name = string("transpose_85_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_85 = transpose(perm = transpose_85_perm_0, x = x_261_cast_fp16)[name = string("transpose_274")]; + tensor transpose_84 = transpose(perm = transpose_84_perm_0, x = x_257_cast_fp16)[name = string("transpose_275")]; + tensor sim_25_cast_fp16 = matmul(transpose_x = sim_25_transpose_x_0, transpose_y = sim_25_transpose_y_0, x = transpose_84, y = transpose_85)[name = string("sim_25_cast_fp16")]; + fp16 var_2071_to_fp16 = const()[name = string("op_2071_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_27_cast_fp16 = mul(x = sim_25_cast_fp16, y = var_2071_to_fp16)[name = string("sim_27_cast_fp16")]; + tensor attn_13_cast_fp16 = softmax(axis = var_1588, x = sim_27_cast_fp16)[name = string("attn_13_cast_fp16")]; + bool x_267_transpose_x_0 = const()[name = string("x_267_transpose_x_0"), val = bool(false)]; + bool x_267_transpose_y_0 = const()[name = string("x_267_transpose_y_0"), val = bool(false)]; + tensor v_13_cast_fp16 = transpose(perm = var_2067, x = x_265_cast_fp16)[name = string("transpose_276")]; + tensor x_267_cast_fp16 = matmul(transpose_x = x_267_transpose_x_0, transpose_y = x_267_transpose_y_0, x = attn_13_cast_fp16, y = v_13_cast_fp16)[name = string("x_267_cast_fp16")]; + tensor var_2093 = const()[name = string("op_2093"), val = tensor([0, 2, 1, 3])]; + tensor var_2095 = const()[name = string("op_2095"), val = tensor([1, 256, 512])]; + tensor x_269_cast_fp16 = transpose(perm = var_2093, x = x_267_cast_fp16)[name = string("transpose_273")]; + tensor input_151_cast_fp16 = reshape(shape = var_2095, x = x_269_cast_fp16)[name = string("input_151_cast_fp16")]; + tensor linear_58_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_151_cast_fp16)[name = string("linear_58_cast_fp16")]; + tensor input_153_cast_fp16 = add(x = linear_58_cast_fp16, y = x_241_cast_fp16)[name = string("input_153_cast_fp16")]; + tensor linear_59_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_153_cast_fp16)[name = string("linear_59_cast_fp16")]; + string input_157_mode_0 = const()[name = string("input_157_mode_0"), val = string("EXACT")]; + tensor input_157_cast_fp16 = gelu(mode = input_157_mode_0, x = linear_59_cast_fp16)[name = string("input_157_cast_fp16")]; + tensor linear_60_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_157_cast_fp16)[name = string("linear_60_cast_fp16")]; + tensor x_271_cast_fp16 = add(x = linear_60_cast_fp16, y = input_153_cast_fp16)[name = string("x_271_cast_fp16")]; + tensor x_273_cast_fp16 = add(x = x_271_cast_fp16, y = mapping_11_cast_fp16)[name = string("x_273_cast_fp16")]; + tensor var_2111_split_sizes_0 = const()[name = string("op_2111_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2111_axis_0 = const()[name = string("op_2111_axis_0"), val = int32(1)]; + tensor var_2111_cast_fp16_0, tensor var_2111_cast_fp16_1 = split(axis = var_2111_axis_0, split_sizes = var_2111_split_sizes_0, x = h_11_cast_fp16)[name = string("op_2111_cast_fp16")]; + tensor gamma_59_perm_0 = const()[name = string("gamma_59_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_59_perm_0 = const()[name = string("beta_59_perm_0"), val = tensor([0, -1, 1])]; + tensor x_277_axes_0 = const()[name = string("x_277_axes_0"), val = tensor([-1])]; + tensor x_277_cast_fp16 = layer_norm(axes = x_277_axes_0, epsilon = var_1584_to_fp16, x = x_273_cast_fp16)[name = string("x_277_cast_fp16")]; + fp16 var_2117_promoted_to_fp16 = const()[name = string("op_2117_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_59_cast_fp16 = transpose(perm = gamma_59_perm_0, x = var_2111_cast_fp16_0)[name = string("transpose_272")]; + tensor var_2118_cast_fp16 = add(x = gamma_59_cast_fp16, y = var_2117_promoted_to_fp16)[name = string("op_2118_cast_fp16")]; + tensor var_2119_cast_fp16 = mul(x = var_2118_cast_fp16, y = x_277_cast_fp16)[name = string("op_2119_cast_fp16")]; + tensor beta_59_cast_fp16 = transpose(perm = beta_59_perm_0, x = var_2111_cast_fp16_1)[name = string("transpose_271")]; + tensor x_279_cast_fp16 = add(x = var_2119_cast_fp16, y = beta_59_cast_fp16)[name = string("x_279_cast_fp16")]; + tensor var_2130_split_sizes_0 = const()[name = string("op_2130_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2130_axis_0 = const()[name = string("op_2130_axis_0"), val = int32(1)]; + tensor var_2130_cast_fp16_0, tensor var_2130_cast_fp16_1 = split(axis = var_2130_axis_0, split_sizes = var_2130_split_sizes_0, x = h_15_cast_fp16)[name = string("op_2130_cast_fp16")]; + tensor gamma_63_perm_0 = const()[name = string("gamma_63_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_63_perm_0 = const()[name = string("beta_63_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2136_promoted_to_fp16 = const()[name = string("op_2136_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_63_cast_fp16 = transpose(perm = gamma_63_perm_0, x = var_2130_cast_fp16_0)[name = string("transpose_270")]; + tensor var_2137_cast_fp16 = add(x = gamma_63_cast_fp16, y = var_2136_promoted_to_fp16)[name = string("op_2137_cast_fp16")]; + tensor var_2138_cast_fp16 = mul(x = var_2137_cast_fp16, y = x_277_cast_fp16)[name = string("op_2138_cast_fp16")]; + tensor beta_63_cast_fp16 = transpose(perm = beta_63_perm_0, x = var_2130_cast_fp16_1)[name = string("transpose_269")]; + tensor x_285_cast_fp16 = add(x = var_2138_cast_fp16, y = beta_63_cast_fp16)[name = string("x_285_cast_fp16")]; + tensor linear_63_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_279_cast_fp16)[name = string("linear_63_cast_fp16")]; + tensor linear_64_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_285_cast_fp16)[name = string("linear_64_cast_fp16")]; + tensor var_2144_split_sizes_0 = const()[name = string("op_2144_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2144_axis_0 = const()[name = string("op_2144_axis_0"), val = int32(-1)]; + tensor var_2144_cast_fp16_0, tensor var_2144_cast_fp16_1 = split(axis = var_2144_axis_0, split_sizes = var_2144_split_sizes_0, x = linear_64_cast_fp16)[name = string("op_2144_cast_fp16")]; + tensor var_2152 = const()[name = string("op_2152"), val = tensor([1, 256, 8, 64])]; + tensor x_289_cast_fp16 = reshape(shape = var_2152, x = linear_63_cast_fp16)[name = string("x_289_cast_fp16")]; + tensor var_2162 = const()[name = string("op_2162"), val = tensor([1, 256, 8, 64])]; + tensor x_293_cast_fp16 = reshape(shape = var_2162, x = var_2144_cast_fp16_0)[name = string("x_293_cast_fp16")]; + tensor var_2172 = const()[name = string("op_2172"), val = tensor([1, 256, 8, 64])]; + tensor x_297_cast_fp16 = reshape(shape = var_2172, x = var_2144_cast_fp16_1)[name = string("x_297_cast_fp16")]; + tensor var_2174 = const()[name = string("op_2174"), val = tensor([0, 2, 1, 3])]; + bool sim_29_transpose_x_0 = const()[name = string("sim_29_transpose_x_0"), val = bool(false)]; + bool sim_29_transpose_y_0 = const()[name = string("sim_29_transpose_y_0"), val = bool(false)]; + tensor transpose_86_perm_0 = const()[name = string("transpose_86_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_87_perm_0 = const()[name = string("transpose_87_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_87 = transpose(perm = transpose_87_perm_0, x = x_293_cast_fp16)[name = string("transpose_266")]; + tensor transpose_86 = transpose(perm = transpose_86_perm_0, x = x_289_cast_fp16)[name = string("transpose_267")]; + tensor sim_29_cast_fp16 = matmul(transpose_x = sim_29_transpose_x_0, transpose_y = sim_29_transpose_y_0, x = transpose_86, y = transpose_87)[name = string("sim_29_cast_fp16")]; + fp16 var_2178_to_fp16 = const()[name = string("op_2178_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_31_cast_fp16 = mul(x = sim_29_cast_fp16, y = var_2178_to_fp16)[name = string("sim_31_cast_fp16")]; + tensor attn_15_cast_fp16 = softmax(axis = var_1588, x = sim_31_cast_fp16)[name = string("attn_15_cast_fp16")]; + bool x_299_transpose_x_0 = const()[name = string("x_299_transpose_x_0"), val = bool(false)]; + bool x_299_transpose_y_0 = const()[name = string("x_299_transpose_y_0"), val = bool(false)]; + tensor v_15_cast_fp16 = transpose(perm = var_2174, x = x_297_cast_fp16)[name = string("transpose_268")]; + tensor x_299_cast_fp16 = matmul(transpose_x = x_299_transpose_x_0, transpose_y = x_299_transpose_y_0, x = attn_15_cast_fp16, y = v_15_cast_fp16)[name = string("x_299_cast_fp16")]; + tensor var_2200 = const()[name = string("op_2200"), val = tensor([0, 2, 1, 3])]; + tensor var_2202 = const()[name = string("op_2202"), val = tensor([1, 256, 512])]; + tensor x_301_cast_fp16 = transpose(perm = var_2200, x = x_299_cast_fp16)[name = string("transpose_265")]; + tensor input_167_cast_fp16 = reshape(shape = var_2202, x = x_301_cast_fp16)[name = string("input_167_cast_fp16")]; + tensor linear_65_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_167_cast_fp16)[name = string("linear_65_cast_fp16")]; + tensor input_169_cast_fp16 = add(x = linear_65_cast_fp16, y = x_273_cast_fp16)[name = string("input_169_cast_fp16")]; + tensor linear_66_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_169_cast_fp16)[name = string("linear_66_cast_fp16")]; + string input_173_mode_0 = const()[name = string("input_173_mode_0"), val = string("EXACT")]; + tensor input_173_cast_fp16 = gelu(mode = input_173_mode_0, x = linear_66_cast_fp16)[name = string("input_173_cast_fp16")]; + tensor linear_67_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_173_cast_fp16)[name = string("linear_67_cast_fp16")]; + tensor x_303_cast_fp16 = add(x = linear_67_cast_fp16, y = input_169_cast_fp16)[name = string("x_303_cast_fp16")]; + tensor x_305_cast_fp16 = add(x = x_303_cast_fp16, y = mapping_11_cast_fp16)[name = string("x_305_cast_fp16")]; + tensor var_2218_split_sizes_0 = const()[name = string("op_2218_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2218_axis_0 = const()[name = string("op_2218_axis_0"), val = int32(1)]; + tensor var_2218_cast_fp16_0, tensor var_2218_cast_fp16_1 = split(axis = var_2218_axis_0, split_sizes = var_2218_split_sizes_0, x = h_19_cast_fp16)[name = string("op_2218_cast_fp16")]; + tensor gamma_67_perm_0 = const()[name = string("gamma_67_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_67_perm_0 = const()[name = string("beta_67_perm_0"), val = tensor([0, -1, 1])]; + tensor x_309_axes_0 = const()[name = string("x_309_axes_0"), val = tensor([-1])]; + tensor x_309_cast_fp16 = layer_norm(axes = x_309_axes_0, epsilon = var_1584_to_fp16, x = x_305_cast_fp16)[name = string("x_309_cast_fp16")]; + fp16 var_2224_promoted_to_fp16 = const()[name = string("op_2224_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_67_cast_fp16 = transpose(perm = gamma_67_perm_0, x = var_2218_cast_fp16_0)[name = string("transpose_264")]; + tensor var_2225_cast_fp16 = add(x = gamma_67_cast_fp16, y = var_2224_promoted_to_fp16)[name = string("op_2225_cast_fp16")]; + tensor var_2226_cast_fp16 = mul(x = var_2225_cast_fp16, y = x_309_cast_fp16)[name = string("op_2226_cast_fp16")]; + tensor beta_67_cast_fp16 = transpose(perm = beta_67_perm_0, x = var_2218_cast_fp16_1)[name = string("transpose_263")]; + tensor x_311_cast_fp16 = add(x = var_2226_cast_fp16, y = beta_67_cast_fp16)[name = string("x_311_cast_fp16")]; + tensor var_2237_split_sizes_0 = const()[name = string("op_2237_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2237_axis_0 = const()[name = string("op_2237_axis_0"), val = int32(1)]; + tensor var_2237_cast_fp16_0, tensor var_2237_cast_fp16_1 = split(axis = var_2237_axis_0, split_sizes = var_2237_split_sizes_0, x = h_23_cast_fp16)[name = string("op_2237_cast_fp16")]; + tensor gamma_71_perm_0 = const()[name = string("gamma_71_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_71_perm_0 = const()[name = string("beta_71_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2243_promoted_to_fp16 = const()[name = string("op_2243_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_71_cast_fp16 = transpose(perm = gamma_71_perm_0, x = var_2237_cast_fp16_0)[name = string("transpose_262")]; + tensor var_2244_cast_fp16 = add(x = gamma_71_cast_fp16, y = var_2243_promoted_to_fp16)[name = string("op_2244_cast_fp16")]; + tensor var_2245_cast_fp16 = mul(x = var_2244_cast_fp16, y = x_309_cast_fp16)[name = string("op_2245_cast_fp16")]; + tensor beta_71_cast_fp16 = transpose(perm = beta_71_perm_0, x = var_2237_cast_fp16_1)[name = string("transpose_261")]; + tensor x_317_cast_fp16 = add(x = var_2245_cast_fp16, y = beta_71_cast_fp16)[name = string("x_317_cast_fp16")]; + tensor linear_70_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_311_cast_fp16)[name = string("linear_70_cast_fp16")]; + tensor linear_71_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_317_cast_fp16)[name = string("linear_71_cast_fp16")]; + tensor var_2251_split_sizes_0 = const()[name = string("op_2251_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2251_axis_0 = const()[name = string("op_2251_axis_0"), val = int32(-1)]; + tensor var_2251_cast_fp16_0, tensor var_2251_cast_fp16_1 = split(axis = var_2251_axis_0, split_sizes = var_2251_split_sizes_0, x = linear_71_cast_fp16)[name = string("op_2251_cast_fp16")]; + tensor var_2259 = const()[name = string("op_2259"), val = tensor([1, 256, 8, 64])]; + tensor x_321_cast_fp16 = reshape(shape = var_2259, x = linear_70_cast_fp16)[name = string("x_321_cast_fp16")]; + tensor var_2269 = const()[name = string("op_2269"), val = tensor([1, 256, 8, 64])]; + tensor x_325_cast_fp16 = reshape(shape = var_2269, x = var_2251_cast_fp16_0)[name = string("x_325_cast_fp16")]; + tensor var_2279 = const()[name = string("op_2279"), val = tensor([1, 256, 8, 64])]; + tensor x_329_cast_fp16 = reshape(shape = var_2279, x = var_2251_cast_fp16_1)[name = string("x_329_cast_fp16")]; + tensor var_2281 = const()[name = string("op_2281"), val = tensor([0, 2, 1, 3])]; + bool sim_33_transpose_x_0 = const()[name = string("sim_33_transpose_x_0"), val = bool(false)]; + bool sim_33_transpose_y_0 = const()[name = string("sim_33_transpose_y_0"), val = bool(false)]; + tensor transpose_88_perm_0 = const()[name = string("transpose_88_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_89_perm_0 = const()[name = string("transpose_89_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_89 = transpose(perm = transpose_89_perm_0, x = x_325_cast_fp16)[name = string("transpose_258")]; + tensor transpose_88 = transpose(perm = transpose_88_perm_0, x = x_321_cast_fp16)[name = string("transpose_259")]; + tensor sim_33_cast_fp16 = matmul(transpose_x = sim_33_transpose_x_0, transpose_y = sim_33_transpose_y_0, x = transpose_88, y = transpose_89)[name = string("sim_33_cast_fp16")]; + fp16 var_2285_to_fp16 = const()[name = string("op_2285_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_35_cast_fp16 = mul(x = sim_33_cast_fp16, y = var_2285_to_fp16)[name = string("sim_35_cast_fp16")]; + tensor attn_17_cast_fp16 = softmax(axis = var_1588, x = sim_35_cast_fp16)[name = string("attn_17_cast_fp16")]; + bool x_331_transpose_x_0 = const()[name = string("x_331_transpose_x_0"), val = bool(false)]; + bool x_331_transpose_y_0 = const()[name = string("x_331_transpose_y_0"), val = bool(false)]; + tensor v_17_cast_fp16 = transpose(perm = var_2281, x = x_329_cast_fp16)[name = string("transpose_260")]; + tensor x_331_cast_fp16 = matmul(transpose_x = x_331_transpose_x_0, transpose_y = x_331_transpose_y_0, x = attn_17_cast_fp16, y = v_17_cast_fp16)[name = string("x_331_cast_fp16")]; + tensor var_2307 = const()[name = string("op_2307"), val = tensor([0, 2, 1, 3])]; + tensor var_2309 = const()[name = string("op_2309"), val = tensor([1, 256, 512])]; + tensor x_333_cast_fp16 = transpose(perm = var_2307, x = x_331_cast_fp16)[name = string("transpose_257")]; + tensor input_183_cast_fp16 = reshape(shape = var_2309, x = x_333_cast_fp16)[name = string("input_183_cast_fp16")]; + tensor linear_72_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_183_cast_fp16)[name = string("linear_72_cast_fp16")]; + tensor input_185_cast_fp16 = add(x = linear_72_cast_fp16, y = x_305_cast_fp16)[name = string("input_185_cast_fp16")]; + tensor linear_73_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_185_cast_fp16)[name = string("linear_73_cast_fp16")]; + string input_189_mode_0 = const()[name = string("input_189_mode_0"), val = string("EXACT")]; + tensor input_189_cast_fp16 = gelu(mode = input_189_mode_0, x = linear_73_cast_fp16)[name = string("input_189_cast_fp16")]; + tensor linear_74_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_189_cast_fp16)[name = string("linear_74_cast_fp16")]; + tensor x_335_cast_fp16 = add(x = linear_74_cast_fp16, y = input_185_cast_fp16)[name = string("x_335_cast_fp16")]; + tensor var_2318_axes_0 = const()[name = string("op_2318_axes_0"), val = tensor([1])]; + bool var_2318_keep_dims_0 = const()[name = string("op_2318_keep_dims_0"), val = bool(false)]; + tensor var_2318_cast_fp16 = reduce_mean(axes = var_2318_axes_0, keep_dims = var_2318_keep_dims_0, x = x_335_cast_fp16)[name = string("op_2318_cast_fp16")]; + tensor x_337_axes_0 = const()[name = string("x_337_axes_0"), val = tensor([1])]; + tensor x_337_cast_fp16 = expand_dims(axes = x_337_axes_0, x = var_2318_cast_fp16)[name = string("x_337_cast_fp16")]; + tensor var_2320 = const()[name = string("op_2320"), val = tensor([0, 2, 1])]; + string x_339_pad_type_0 = const()[name = string("x_339_pad_type_0"), val = string("valid")]; + tensor x_339_strides_0 = const()[name = string("x_339_strides_0"), val = tensor([1])]; + tensor x_339_pad_0 = const()[name = string("x_339_pad_0"), val = tensor([0, 0])]; + tensor x_339_dilations_0 = const()[name = string("x_339_dilations_0"), val = tensor([1])]; + int32 x_339_groups_0 = const()[name = string("x_339_groups_0"), val = int32(1)]; + tensor input_191_cast_fp16 = transpose(perm = var_2320, x = x_337_cast_fp16)[name = string("transpose_256")]; + tensor x_339_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_339_dilations_0, groups = x_339_groups_0, pad = x_339_pad_0, pad_type = x_339_pad_type_0, strides = x_339_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_191_cast_fp16)[name = string("x_339_cast_fp16")]; + tensor x_pred_5_perm_0 = const()[name = string("x_pred_5_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_5_to_fp16 = const()[name = string("c_skip_5_to_fp16"), val = tensor([[[0x1.cf4p-4]]])]; + tensor var_2328_cast_fp16 = mul(x = c_skip_5_to_fp16, y = x_noisy_5_cast_fp16)[name = string("op_2328_cast_fp16")]; + tensor c_out_5_to_fp16 = const()[name = string("c_out_5_to_fp16"), val = tensor([[[0x1.804p-3]]])]; + tensor x_pred_5_cast_fp16 = transpose(perm = x_pred_5_perm_0, x = x_339_cast_fp16)[name = string("transpose_255")]; + tensor var_2329_cast_fp16 = mul(x = c_out_5_to_fp16, y = x_pred_5_cast_fp16)[name = string("op_2329_cast_fp16")]; + tensor x_dn_3_cast_fp16 = add(x = var_2328_cast_fp16, y = var_2329_cast_fp16)[name = string("x_dn_3_cast_fp16")]; + tensor var_2332_cast_fp16 = sub(x = x_noisy_5_cast_fp16, y = x_dn_3_cast_fp16)[name = string("op_2332_cast_fp16")]; + tensor _inversed_d_3_y_0_to_fp16 = const()[name = string("_inversed_d_3_y_0_to_fp16"), val = tensor([0x1.cacp+0])]; + tensor _inversed_d_3_cast_fp16 = mul(x = var_2332_cast_fp16, y = _inversed_d_3_y_0_to_fp16)[name = string("_inversed_d_3_cast_fp16")]; + fp16 var_2341_to_fp16 = const()[name = string("op_2341_to_fp16"), val = fp16(-0x1.19p-2)]; + tensor var_2342_cast_fp16 = mul(x = _inversed_d_3_cast_fp16, y = var_2341_to_fp16)[name = string("op_2342_cast_fp16")]; + tensor x_noisy_7_cast_fp16 = add(x = x_noisy_5_cast_fp16, y = var_2342_cast_fp16)[name = string("x_noisy_7_cast_fp16")]; + int32 var_2354 = const()[name = string("op_2354"), val = int32(-1)]; + tensor c_in_7_to_fp16 = const()[name = string("c_in_7_to_fp16"), val = tensor([[[0x1.718p+1]]])]; + tensor x_349_cast_fp16 = mul(x = c_in_7_to_fp16, y = x_noisy_7_cast_fp16)[name = string("x_349_cast_fp16")]; + int32 x_345_axis_0 = const()[name = string("x_345_axis_0"), val = int32(0)]; + tensor var_2740_to_fp16 = const()[name = string("op_2740_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49351296)))]; + tensor x_345_cast_fp16 = stack(axis = x_345_axis_0, values = (var_2740_to_fp16, var_423_cast_fp16))[name = string("x_345_cast_fp16")]; + tensor var_2745 = const()[name = string("op_2745"), val = tensor([1, 2, 0])]; + tensor input_199_axes_0 = const()[name = string("input_199_axes_0"), val = tensor([2])]; + bool input_199_keep_dims_0 = const()[name = string("input_199_keep_dims_0"), val = bool(false)]; + tensor x_347_cast_fp16 = transpose(perm = var_2745, x = x_345_cast_fp16)[name = string("transpose_254")]; + tensor input_199_cast_fp16 = reduce_sum(axes = input_199_axes_0, keep_dims = input_199_keep_dims_0, x = x_347_cast_fp16)[name = string("input_199_cast_fp16")]; + tensor linear_77_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_199_cast_fp16)[name = string("linear_77_cast_fp16")]; + string input_203_mode_0 = const()[name = string("input_203_mode_0"), val = string("EXACT")]; + tensor input_203_cast_fp16 = gelu(mode = input_203_mode_0, x = linear_77_cast_fp16)[name = string("input_203_cast_fp16")]; + tensor linear_78_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_203_cast_fp16)[name = string("linear_78_cast_fp16")]; + string mapping_13_mode_0 = const()[name = string("mapping_13_mode_0"), val = string("EXACT")]; + tensor mapping_13_cast_fp16 = gelu(mode = mapping_13_mode_0, x = linear_78_cast_fp16)[name = string("mapping_13_cast_fp16")]; + tensor var_2755_reps_0 = const()[name = string("op_2755_reps_0"), val = tensor([1, 256, 1])]; + tensor var_2755_cast_fp16 = tile(reps = var_2755_reps_0, x = x_349_cast_fp16)[name = string("op_2755_cast_fp16")]; + bool x_351_interleave_0 = const()[name = string("x_351_interleave_0"), val = bool(false)]; + tensor x_351_cast_fp16 = concat(axis = var_2354, interleave = x_351_interleave_0, values = (var_2755_cast_fp16, embedding_to_fp16))[name = string("x_351_cast_fp16")]; + tensor var_2758_axes_0 = const()[name = string("op_2758_axes_0"), val = tensor([1])]; + tensor var_2758_cast_fp16 = expand_dims(axes = var_2758_axes_0, x = mapping_13_cast_fp16)[name = string("op_2758_cast_fp16")]; + tensor mapping_15_reps_0 = const()[name = string("mapping_15_reps_0"), val = tensor([1, 256, 1])]; + tensor mapping_15_cast_fp16 = tile(reps = mapping_15_reps_0, x = var_2758_cast_fp16)[name = string("mapping_15_cast_fp16")]; + tensor x_353_cast_fp16 = add(x = x_351_cast_fp16, y = mapping_15_cast_fp16)[name = string("x_353_cast_fp16")]; + tensor var_2770_split_sizes_0 = const()[name = string("op_2770_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2770_axis_0 = const()[name = string("op_2770_axis_0"), val = int32(1)]; + tensor var_2770_cast_fp16_0, tensor var_2770_cast_fp16_1 = split(axis = var_2770_axis_0, split_sizes = var_2770_split_sizes_0, x = h_3_cast_fp16)[name = string("op_2770_cast_fp16")]; + tensor gamma_75_perm_0 = const()[name = string("gamma_75_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_75_perm_0 = const()[name = string("beta_75_perm_0"), val = tensor([0, -1, 1])]; + tensor x_357_axes_0 = const()[name = string("x_357_axes_0"), val = tensor([-1])]; + fp16 var_2350_to_fp16 = const()[name = string("op_2350_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_357_cast_fp16 = layer_norm(axes = x_357_axes_0, epsilon = var_2350_to_fp16, x = x_353_cast_fp16)[name = string("x_357_cast_fp16")]; + fp16 var_2776_promoted_to_fp16 = const()[name = string("op_2776_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_75_cast_fp16 = transpose(perm = gamma_75_perm_0, x = var_2770_cast_fp16_0)[name = string("transpose_253")]; + tensor var_2777_cast_fp16 = add(x = gamma_75_cast_fp16, y = var_2776_promoted_to_fp16)[name = string("op_2777_cast_fp16")]; + tensor var_2778_cast_fp16 = mul(x = var_2777_cast_fp16, y = x_357_cast_fp16)[name = string("op_2778_cast_fp16")]; + tensor beta_75_cast_fp16 = transpose(perm = beta_75_perm_0, x = var_2770_cast_fp16_1)[name = string("transpose_252")]; + tensor x_359_cast_fp16 = add(x = var_2778_cast_fp16, y = beta_75_cast_fp16)[name = string("x_359_cast_fp16")]; + tensor var_2789_split_sizes_0 = const()[name = string("op_2789_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2789_axis_0 = const()[name = string("op_2789_axis_0"), val = int32(1)]; + tensor var_2789_cast_fp16_0, tensor var_2789_cast_fp16_1 = split(axis = var_2789_axis_0, split_sizes = var_2789_split_sizes_0, x = h_7_cast_fp16)[name = string("op_2789_cast_fp16")]; + tensor gamma_79_perm_0 = const()[name = string("gamma_79_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_79_perm_0 = const()[name = string("beta_79_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2795_promoted_to_fp16 = const()[name = string("op_2795_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_79_cast_fp16 = transpose(perm = gamma_79_perm_0, x = var_2789_cast_fp16_0)[name = string("transpose_251")]; + tensor var_2796_cast_fp16 = add(x = gamma_79_cast_fp16, y = var_2795_promoted_to_fp16)[name = string("op_2796_cast_fp16")]; + tensor var_2797_cast_fp16 = mul(x = var_2796_cast_fp16, y = x_357_cast_fp16)[name = string("op_2797_cast_fp16")]; + tensor beta_79_cast_fp16 = transpose(perm = beta_79_perm_0, x = var_2789_cast_fp16_1)[name = string("transpose_250")]; + tensor x_365_cast_fp16 = add(x = var_2797_cast_fp16, y = beta_79_cast_fp16)[name = string("x_365_cast_fp16")]; + tensor linear_81_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_359_cast_fp16)[name = string("linear_81_cast_fp16")]; + tensor linear_82_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_365_cast_fp16)[name = string("linear_82_cast_fp16")]; + tensor var_2803_split_sizes_0 = const()[name = string("op_2803_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2803_axis_0 = const()[name = string("op_2803_axis_0"), val = int32(-1)]; + tensor var_2803_cast_fp16_0, tensor var_2803_cast_fp16_1 = split(axis = var_2803_axis_0, split_sizes = var_2803_split_sizes_0, x = linear_82_cast_fp16)[name = string("op_2803_cast_fp16")]; + tensor var_2811 = const()[name = string("op_2811"), val = tensor([1, 256, 8, 64])]; + tensor x_369_cast_fp16 = reshape(shape = var_2811, x = linear_81_cast_fp16)[name = string("x_369_cast_fp16")]; + tensor var_2821 = const()[name = string("op_2821"), val = tensor([1, 256, 8, 64])]; + tensor x_373_cast_fp16 = reshape(shape = var_2821, x = var_2803_cast_fp16_0)[name = string("x_373_cast_fp16")]; + tensor var_2831 = const()[name = string("op_2831"), val = tensor([1, 256, 8, 64])]; + tensor x_377_cast_fp16 = reshape(shape = var_2831, x = var_2803_cast_fp16_1)[name = string("x_377_cast_fp16")]; + tensor var_2833 = const()[name = string("op_2833"), val = tensor([0, 2, 1, 3])]; + bool sim_37_transpose_x_0 = const()[name = string("sim_37_transpose_x_0"), val = bool(false)]; + bool sim_37_transpose_y_0 = const()[name = string("sim_37_transpose_y_0"), val = bool(false)]; + tensor transpose_90_perm_0 = const()[name = string("transpose_90_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_91_perm_0 = const()[name = string("transpose_91_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_91 = transpose(perm = transpose_91_perm_0, x = x_373_cast_fp16)[name = string("transpose_247")]; + tensor transpose_90 = transpose(perm = transpose_90_perm_0, x = x_369_cast_fp16)[name = string("transpose_248")]; + tensor sim_37_cast_fp16 = matmul(transpose_x = sim_37_transpose_x_0, transpose_y = sim_37_transpose_y_0, x = transpose_90, y = transpose_91)[name = string("sim_37_cast_fp16")]; + fp16 var_2837_to_fp16 = const()[name = string("op_2837_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_39_cast_fp16 = mul(x = sim_37_cast_fp16, y = var_2837_to_fp16)[name = string("sim_39_cast_fp16")]; + tensor attn_19_cast_fp16 = softmax(axis = var_2354, x = sim_39_cast_fp16)[name = string("attn_19_cast_fp16")]; + bool x_379_transpose_x_0 = const()[name = string("x_379_transpose_x_0"), val = bool(false)]; + bool x_379_transpose_y_0 = const()[name = string("x_379_transpose_y_0"), val = bool(false)]; + tensor v_19_cast_fp16 = transpose(perm = var_2833, x = x_377_cast_fp16)[name = string("transpose_249")]; + tensor x_379_cast_fp16 = matmul(transpose_x = x_379_transpose_x_0, transpose_y = x_379_transpose_y_0, x = attn_19_cast_fp16, y = v_19_cast_fp16)[name = string("x_379_cast_fp16")]; + tensor var_2859 = const()[name = string("op_2859"), val = tensor([0, 2, 1, 3])]; + tensor var_2861 = const()[name = string("op_2861"), val = tensor([1, 256, 512])]; + tensor x_381_cast_fp16 = transpose(perm = var_2859, x = x_379_cast_fp16)[name = string("transpose_246")]; + tensor input_215_cast_fp16 = reshape(shape = var_2861, x = x_381_cast_fp16)[name = string("input_215_cast_fp16")]; + tensor linear_83_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_215_cast_fp16)[name = string("linear_83_cast_fp16")]; + tensor input_217_cast_fp16 = add(x = linear_83_cast_fp16, y = x_353_cast_fp16)[name = string("input_217_cast_fp16")]; + tensor linear_84_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_217_cast_fp16)[name = string("linear_84_cast_fp16")]; + string input_221_mode_0 = const()[name = string("input_221_mode_0"), val = string("EXACT")]; + tensor input_221_cast_fp16 = gelu(mode = input_221_mode_0, x = linear_84_cast_fp16)[name = string("input_221_cast_fp16")]; + tensor linear_85_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_221_cast_fp16)[name = string("linear_85_cast_fp16")]; + tensor x_383_cast_fp16 = add(x = linear_85_cast_fp16, y = input_217_cast_fp16)[name = string("x_383_cast_fp16")]; + tensor x_385_cast_fp16 = add(x = x_383_cast_fp16, y = mapping_15_cast_fp16)[name = string("x_385_cast_fp16")]; + tensor var_2877_split_sizes_0 = const()[name = string("op_2877_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2877_axis_0 = const()[name = string("op_2877_axis_0"), val = int32(1)]; + tensor var_2877_cast_fp16_0, tensor var_2877_cast_fp16_1 = split(axis = var_2877_axis_0, split_sizes = var_2877_split_sizes_0, x = h_11_cast_fp16)[name = string("op_2877_cast_fp16")]; + tensor gamma_83_perm_0 = const()[name = string("gamma_83_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_83_perm_0 = const()[name = string("beta_83_perm_0"), val = tensor([0, -1, 1])]; + tensor x_389_axes_0 = const()[name = string("x_389_axes_0"), val = tensor([-1])]; + tensor x_389_cast_fp16 = layer_norm(axes = x_389_axes_0, epsilon = var_2350_to_fp16, x = x_385_cast_fp16)[name = string("x_389_cast_fp16")]; + fp16 var_2883_promoted_to_fp16 = const()[name = string("op_2883_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_83_cast_fp16 = transpose(perm = gamma_83_perm_0, x = var_2877_cast_fp16_0)[name = string("transpose_245")]; + tensor var_2884_cast_fp16 = add(x = gamma_83_cast_fp16, y = var_2883_promoted_to_fp16)[name = string("op_2884_cast_fp16")]; + tensor var_2885_cast_fp16 = mul(x = var_2884_cast_fp16, y = x_389_cast_fp16)[name = string("op_2885_cast_fp16")]; + tensor beta_83_cast_fp16 = transpose(perm = beta_83_perm_0, x = var_2877_cast_fp16_1)[name = string("transpose_244")]; + tensor x_391_cast_fp16 = add(x = var_2885_cast_fp16, y = beta_83_cast_fp16)[name = string("x_391_cast_fp16")]; + tensor var_2896_split_sizes_0 = const()[name = string("op_2896_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2896_axis_0 = const()[name = string("op_2896_axis_0"), val = int32(1)]; + tensor var_2896_cast_fp16_0, tensor var_2896_cast_fp16_1 = split(axis = var_2896_axis_0, split_sizes = var_2896_split_sizes_0, x = h_15_cast_fp16)[name = string("op_2896_cast_fp16")]; + tensor gamma_87_perm_0 = const()[name = string("gamma_87_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_87_perm_0 = const()[name = string("beta_87_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2902_promoted_to_fp16 = const()[name = string("op_2902_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_87_cast_fp16 = transpose(perm = gamma_87_perm_0, x = var_2896_cast_fp16_0)[name = string("transpose_243")]; + tensor var_2903_cast_fp16 = add(x = gamma_87_cast_fp16, y = var_2902_promoted_to_fp16)[name = string("op_2903_cast_fp16")]; + tensor var_2904_cast_fp16 = mul(x = var_2903_cast_fp16, y = x_389_cast_fp16)[name = string("op_2904_cast_fp16")]; + tensor beta_87_cast_fp16 = transpose(perm = beta_87_perm_0, x = var_2896_cast_fp16_1)[name = string("transpose_242")]; + tensor x_397_cast_fp16 = add(x = var_2904_cast_fp16, y = beta_87_cast_fp16)[name = string("x_397_cast_fp16")]; + tensor linear_88_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_391_cast_fp16)[name = string("linear_88_cast_fp16")]; + tensor linear_89_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_397_cast_fp16)[name = string("linear_89_cast_fp16")]; + tensor var_2910_split_sizes_0 = const()[name = string("op_2910_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2910_axis_0 = const()[name = string("op_2910_axis_0"), val = int32(-1)]; + tensor var_2910_cast_fp16_0, tensor var_2910_cast_fp16_1 = split(axis = var_2910_axis_0, split_sizes = var_2910_split_sizes_0, x = linear_89_cast_fp16)[name = string("op_2910_cast_fp16")]; + tensor var_2918 = const()[name = string("op_2918"), val = tensor([1, 256, 8, 64])]; + tensor x_401_cast_fp16 = reshape(shape = var_2918, x = linear_88_cast_fp16)[name = string("x_401_cast_fp16")]; + tensor var_2928 = const()[name = string("op_2928"), val = tensor([1, 256, 8, 64])]; + tensor x_405_cast_fp16 = reshape(shape = var_2928, x = var_2910_cast_fp16_0)[name = string("x_405_cast_fp16")]; + tensor var_2938 = const()[name = string("op_2938"), val = tensor([1, 256, 8, 64])]; + tensor x_409_cast_fp16 = reshape(shape = var_2938, x = var_2910_cast_fp16_1)[name = string("x_409_cast_fp16")]; + tensor var_2940 = const()[name = string("op_2940"), val = tensor([0, 2, 1, 3])]; + bool sim_41_transpose_x_0 = const()[name = string("sim_41_transpose_x_0"), val = bool(false)]; + bool sim_41_transpose_y_0 = const()[name = string("sim_41_transpose_y_0"), val = bool(false)]; + tensor transpose_92_perm_0 = const()[name = string("transpose_92_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_93_perm_0 = const()[name = string("transpose_93_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_93 = transpose(perm = transpose_93_perm_0, x = x_405_cast_fp16)[name = string("transpose_239")]; + tensor transpose_92 = transpose(perm = transpose_92_perm_0, x = x_401_cast_fp16)[name = string("transpose_240")]; + tensor sim_41_cast_fp16 = matmul(transpose_x = sim_41_transpose_x_0, transpose_y = sim_41_transpose_y_0, x = transpose_92, y = transpose_93)[name = string("sim_41_cast_fp16")]; + fp16 var_2944_to_fp16 = const()[name = string("op_2944_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_43_cast_fp16 = mul(x = sim_41_cast_fp16, y = var_2944_to_fp16)[name = string("sim_43_cast_fp16")]; + tensor attn_21_cast_fp16 = softmax(axis = var_2354, x = sim_43_cast_fp16)[name = string("attn_21_cast_fp16")]; + bool x_411_transpose_x_0 = const()[name = string("x_411_transpose_x_0"), val = bool(false)]; + bool x_411_transpose_y_0 = const()[name = string("x_411_transpose_y_0"), val = bool(false)]; + tensor v_21_cast_fp16 = transpose(perm = var_2940, x = x_409_cast_fp16)[name = string("transpose_241")]; + tensor x_411_cast_fp16 = matmul(transpose_x = x_411_transpose_x_0, transpose_y = x_411_transpose_y_0, x = attn_21_cast_fp16, y = v_21_cast_fp16)[name = string("x_411_cast_fp16")]; + tensor var_2966 = const()[name = string("op_2966"), val = tensor([0, 2, 1, 3])]; + tensor var_2968 = const()[name = string("op_2968"), val = tensor([1, 256, 512])]; + tensor x_413_cast_fp16 = transpose(perm = var_2966, x = x_411_cast_fp16)[name = string("transpose_238")]; + tensor input_231_cast_fp16 = reshape(shape = var_2968, x = x_413_cast_fp16)[name = string("input_231_cast_fp16")]; + tensor linear_90_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_231_cast_fp16)[name = string("linear_90_cast_fp16")]; + tensor input_233_cast_fp16 = add(x = linear_90_cast_fp16, y = x_385_cast_fp16)[name = string("input_233_cast_fp16")]; + tensor linear_91_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_233_cast_fp16)[name = string("linear_91_cast_fp16")]; + string input_237_mode_0 = const()[name = string("input_237_mode_0"), val = string("EXACT")]; + tensor input_237_cast_fp16 = gelu(mode = input_237_mode_0, x = linear_91_cast_fp16)[name = string("input_237_cast_fp16")]; + tensor linear_92_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_237_cast_fp16)[name = string("linear_92_cast_fp16")]; + tensor x_415_cast_fp16 = add(x = linear_92_cast_fp16, y = input_233_cast_fp16)[name = string("x_415_cast_fp16")]; + tensor x_417_cast_fp16 = add(x = x_415_cast_fp16, y = mapping_15_cast_fp16)[name = string("x_417_cast_fp16")]; + tensor var_2984_split_sizes_0 = const()[name = string("op_2984_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2984_axis_0 = const()[name = string("op_2984_axis_0"), val = int32(1)]; + tensor var_2984_cast_fp16_0, tensor var_2984_cast_fp16_1 = split(axis = var_2984_axis_0, split_sizes = var_2984_split_sizes_0, x = h_19_cast_fp16)[name = string("op_2984_cast_fp16")]; + tensor gamma_91_perm_0 = const()[name = string("gamma_91_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_91_perm_0 = const()[name = string("beta_91_perm_0"), val = tensor([0, -1, 1])]; + tensor x_421_axes_0 = const()[name = string("x_421_axes_0"), val = tensor([-1])]; + tensor x_421_cast_fp16 = layer_norm(axes = x_421_axes_0, epsilon = var_2350_to_fp16, x = x_417_cast_fp16)[name = string("x_421_cast_fp16")]; + fp16 var_2990_promoted_to_fp16 = const()[name = string("op_2990_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_91_cast_fp16 = transpose(perm = gamma_91_perm_0, x = var_2984_cast_fp16_0)[name = string("transpose_237")]; + tensor var_2991_cast_fp16 = add(x = gamma_91_cast_fp16, y = var_2990_promoted_to_fp16)[name = string("op_2991_cast_fp16")]; + tensor var_2992_cast_fp16 = mul(x = var_2991_cast_fp16, y = x_421_cast_fp16)[name = string("op_2992_cast_fp16")]; + tensor beta_91_cast_fp16 = transpose(perm = beta_91_perm_0, x = var_2984_cast_fp16_1)[name = string("transpose_236")]; + tensor x_423_cast_fp16 = add(x = var_2992_cast_fp16, y = beta_91_cast_fp16)[name = string("x_423_cast_fp16")]; + tensor var_3003_split_sizes_0 = const()[name = string("op_3003_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3003_axis_0 = const()[name = string("op_3003_axis_0"), val = int32(1)]; + tensor var_3003_cast_fp16_0, tensor var_3003_cast_fp16_1 = split(axis = var_3003_axis_0, split_sizes = var_3003_split_sizes_0, x = h_23_cast_fp16)[name = string("op_3003_cast_fp16")]; + tensor gamma_95_perm_0 = const()[name = string("gamma_95_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_95_perm_0 = const()[name = string("beta_95_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3009_promoted_to_fp16 = const()[name = string("op_3009_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_95_cast_fp16 = transpose(perm = gamma_95_perm_0, x = var_3003_cast_fp16_0)[name = string("transpose_235")]; + tensor var_3010_cast_fp16 = add(x = gamma_95_cast_fp16, y = var_3009_promoted_to_fp16)[name = string("op_3010_cast_fp16")]; + tensor var_3011_cast_fp16 = mul(x = var_3010_cast_fp16, y = x_421_cast_fp16)[name = string("op_3011_cast_fp16")]; + tensor beta_95_cast_fp16 = transpose(perm = beta_95_perm_0, x = var_3003_cast_fp16_1)[name = string("transpose_234")]; + tensor x_429_cast_fp16 = add(x = var_3011_cast_fp16, y = beta_95_cast_fp16)[name = string("x_429_cast_fp16")]; + tensor linear_95_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_423_cast_fp16)[name = string("linear_95_cast_fp16")]; + tensor linear_96_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_429_cast_fp16)[name = string("linear_96_cast_fp16")]; + tensor var_3017_split_sizes_0 = const()[name = string("op_3017_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3017_axis_0 = const()[name = string("op_3017_axis_0"), val = int32(-1)]; + tensor var_3017_cast_fp16_0, tensor var_3017_cast_fp16_1 = split(axis = var_3017_axis_0, split_sizes = var_3017_split_sizes_0, x = linear_96_cast_fp16)[name = string("op_3017_cast_fp16")]; + tensor var_3025 = const()[name = string("op_3025"), val = tensor([1, 256, 8, 64])]; + tensor x_433_cast_fp16 = reshape(shape = var_3025, x = linear_95_cast_fp16)[name = string("x_433_cast_fp16")]; + tensor var_3035 = const()[name = string("op_3035"), val = tensor([1, 256, 8, 64])]; + tensor x_437_cast_fp16 = reshape(shape = var_3035, x = var_3017_cast_fp16_0)[name = string("x_437_cast_fp16")]; + tensor var_3045 = const()[name = string("op_3045"), val = tensor([1, 256, 8, 64])]; + tensor x_441_cast_fp16 = reshape(shape = var_3045, x = var_3017_cast_fp16_1)[name = string("x_441_cast_fp16")]; + tensor var_3047 = const()[name = string("op_3047"), val = tensor([0, 2, 1, 3])]; + bool sim_45_transpose_x_0 = const()[name = string("sim_45_transpose_x_0"), val = bool(false)]; + bool sim_45_transpose_y_0 = const()[name = string("sim_45_transpose_y_0"), val = bool(false)]; + tensor transpose_94_perm_0 = const()[name = string("transpose_94_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_95_perm_0 = const()[name = string("transpose_95_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_95 = transpose(perm = transpose_95_perm_0, x = x_437_cast_fp16)[name = string("transpose_231")]; + tensor transpose_94 = transpose(perm = transpose_94_perm_0, x = x_433_cast_fp16)[name = string("transpose_232")]; + tensor sim_45_cast_fp16 = matmul(transpose_x = sim_45_transpose_x_0, transpose_y = sim_45_transpose_y_0, x = transpose_94, y = transpose_95)[name = string("sim_45_cast_fp16")]; + fp16 var_3051_to_fp16 = const()[name = string("op_3051_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_47_cast_fp16 = mul(x = sim_45_cast_fp16, y = var_3051_to_fp16)[name = string("sim_47_cast_fp16")]; + tensor attn_23_cast_fp16 = softmax(axis = var_2354, x = sim_47_cast_fp16)[name = string("attn_23_cast_fp16")]; + bool x_443_transpose_x_0 = const()[name = string("x_443_transpose_x_0"), val = bool(false)]; + bool x_443_transpose_y_0 = const()[name = string("x_443_transpose_y_0"), val = bool(false)]; + tensor v_23_cast_fp16 = transpose(perm = var_3047, x = x_441_cast_fp16)[name = string("transpose_233")]; + tensor x_443_cast_fp16 = matmul(transpose_x = x_443_transpose_x_0, transpose_y = x_443_transpose_y_0, x = attn_23_cast_fp16, y = v_23_cast_fp16)[name = string("x_443_cast_fp16")]; + tensor var_3073 = const()[name = string("op_3073"), val = tensor([0, 2, 1, 3])]; + tensor var_3075 = const()[name = string("op_3075"), val = tensor([1, 256, 512])]; + tensor x_445_cast_fp16 = transpose(perm = var_3073, x = x_443_cast_fp16)[name = string("transpose_230")]; + tensor input_247_cast_fp16 = reshape(shape = var_3075, x = x_445_cast_fp16)[name = string("input_247_cast_fp16")]; + tensor linear_97_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_247_cast_fp16)[name = string("linear_97_cast_fp16")]; + tensor input_249_cast_fp16 = add(x = linear_97_cast_fp16, y = x_417_cast_fp16)[name = string("input_249_cast_fp16")]; + tensor linear_98_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_249_cast_fp16)[name = string("linear_98_cast_fp16")]; + string input_253_mode_0 = const()[name = string("input_253_mode_0"), val = string("EXACT")]; + tensor input_253_cast_fp16 = gelu(mode = input_253_mode_0, x = linear_98_cast_fp16)[name = string("input_253_cast_fp16")]; + tensor linear_99_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_253_cast_fp16)[name = string("linear_99_cast_fp16")]; + tensor x_447_cast_fp16 = add(x = linear_99_cast_fp16, y = input_249_cast_fp16)[name = string("x_447_cast_fp16")]; + tensor var_3084_axes_0 = const()[name = string("op_3084_axes_0"), val = tensor([1])]; + bool var_3084_keep_dims_0 = const()[name = string("op_3084_keep_dims_0"), val = bool(false)]; + tensor var_3084_cast_fp16 = reduce_mean(axes = var_3084_axes_0, keep_dims = var_3084_keep_dims_0, x = x_447_cast_fp16)[name = string("op_3084_cast_fp16")]; + tensor x_449_axes_0 = const()[name = string("x_449_axes_0"), val = tensor([1])]; + tensor x_449_cast_fp16 = expand_dims(axes = x_449_axes_0, x = var_3084_cast_fp16)[name = string("x_449_cast_fp16")]; + tensor var_3086 = const()[name = string("op_3086"), val = tensor([0, 2, 1])]; + string x_451_pad_type_0 = const()[name = string("x_451_pad_type_0"), val = string("valid")]; + tensor x_451_strides_0 = const()[name = string("x_451_strides_0"), val = tensor([1])]; + tensor x_451_pad_0 = const()[name = string("x_451_pad_0"), val = tensor([0, 0])]; + tensor x_451_dilations_0 = const()[name = string("x_451_dilations_0"), val = tensor([1])]; + int32 x_451_groups_0 = const()[name = string("x_451_groups_0"), val = int32(1)]; + tensor input_255_cast_fp16 = transpose(perm = var_3086, x = x_449_cast_fp16)[name = string("transpose_229")]; + tensor x_451_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_451_dilations_0, groups = x_451_groups_0, pad = x_451_pad_0, pad_type = x_451_pad_type_0, strides = x_451_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_255_cast_fp16)[name = string("x_451_cast_fp16")]; + tensor x_pred_7_perm_0 = const()[name = string("x_pred_7_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_7_to_fp16 = const()[name = string("c_skip_7_to_fp16"), val = tensor([[[0x1.528p-2]]])]; + tensor var_3094_cast_fp16 = mul(x = c_skip_7_to_fp16, y = x_noisy_7_cast_fp16)[name = string("op_3094_cast_fp16")]; + tensor c_out_7_to_fp16 = const()[name = string("c_out_7_to_fp16"), val = tensor([[[0x1.4dcp-3]]])]; + tensor x_pred_7_cast_fp16 = transpose(perm = x_pred_7_perm_0, x = x_451_cast_fp16)[name = string("transpose_228")]; + tensor var_3095_cast_fp16 = mul(x = c_out_7_to_fp16, y = x_pred_7_cast_fp16)[name = string("op_3095_cast_fp16")]; + tensor x_mid_dn_3_cast_fp16 = add(x = var_3094_cast_fp16, y = var_3095_cast_fp16)[name = string("x_mid_dn_3_cast_fp16")]; + tensor var_3098_cast_fp16 = sub(x = x_noisy_7_cast_fp16, y = x_mid_dn_3_cast_fp16)[name = string("op_3098_cast_fp16")]; + tensor _inversed_d_mid_3_y_0_to_fp16 = const()[name = string("_inversed_d_mid_3_y_0_to_fp16"), val = tensor([0x1.c3cp+1])]; + tensor _inversed_d_mid_3_cast_fp16 = mul(x = var_3098_cast_fp16, y = _inversed_d_mid_3_y_0_to_fp16)[name = string("_inversed_d_mid_3_cast_fp16")]; + fp16 var_3107_to_fp16 = const()[name = string("op_3107_to_fp16"), val = fp16(-0x1.19p-1)]; + tensor var_3108_cast_fp16 = mul(x = _inversed_d_mid_3_cast_fp16, y = var_3107_to_fp16)[name = string("op_3108_cast_fp16")]; + tensor x_453_cast_fp16 = add(x = x_noisy_5_cast_fp16, y = var_3108_cast_fp16)[name = string("x_453_cast_fp16")]; + tensor var_3113_begin_0 = const()[name = string("op_3113_begin_0"), val = tensor([1, 0, 0, 0])]; + tensor var_3113_end_0 = const()[name = string("op_3113_end_0"), val = tensor([2, 1, 1, 256])]; + tensor var_3113_end_mask_0 = const()[name = string("op_3113_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_3113_squeeze_mask_0 = const()[name = string("op_3113_squeeze_mask_0"), val = tensor([true, false, false, false])]; + tensor var_3113_cast_fp16 = slice_by_index(begin = var_3113_begin_0, end = var_3113_end_0, end_mask = var_3113_end_mask_0, squeeze_mask = var_3113_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_3113_cast_fp16")]; + fp16 var_3116_to_fp16 = const()[name = string("op_3116_to_fp16"), val = fp16(0x1.1ep-4)]; + tensor var_3117_cast_fp16 = mul(x = var_3113_cast_fp16, y = var_3116_to_fp16)[name = string("op_3117_cast_fp16")]; + tensor x_noisy_9_cast_fp16 = add(x = x_453_cast_fp16, y = var_3117_cast_fp16)[name = string("x_noisy_9_cast_fp16")]; + int32 var_3141 = const()[name = string("op_3141"), val = int32(-1)]; + tensor c_in_9_to_fp16 = const()[name = string("c_in_9_to_fp16"), val = tensor([[[0x1.2ecp+2]]])]; + tensor x_463_cast_fp16 = mul(x = c_in_9_to_fp16, y = x_noisy_9_cast_fp16)[name = string("x_463_cast_fp16")]; + int32 x_459_axis_0 = const()[name = string("x_459_axis_0"), val = int32(0)]; + tensor var_3527_to_fp16 = const()[name = string("op_3527_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49353408)))]; + tensor x_459_cast_fp16 = stack(axis = x_459_axis_0, values = (var_3527_to_fp16, var_423_cast_fp16))[name = string("x_459_cast_fp16")]; + tensor var_3532 = const()[name = string("op_3532"), val = tensor([1, 2, 0])]; + tensor input_263_axes_0 = const()[name = string("input_263_axes_0"), val = tensor([2])]; + bool input_263_keep_dims_0 = const()[name = string("input_263_keep_dims_0"), val = bool(false)]; + tensor x_461_cast_fp16 = transpose(perm = var_3532, x = x_459_cast_fp16)[name = string("transpose_227")]; + tensor input_263_cast_fp16 = reduce_sum(axes = input_263_axes_0, keep_dims = input_263_keep_dims_0, x = x_461_cast_fp16)[name = string("input_263_cast_fp16")]; + tensor linear_102_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_263_cast_fp16)[name = string("linear_102_cast_fp16")]; + string input_267_mode_0 = const()[name = string("input_267_mode_0"), val = string("EXACT")]; + tensor input_267_cast_fp16 = gelu(mode = input_267_mode_0, x = linear_102_cast_fp16)[name = string("input_267_cast_fp16")]; + tensor linear_103_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_267_cast_fp16)[name = string("linear_103_cast_fp16")]; + string mapping_17_mode_0 = const()[name = string("mapping_17_mode_0"), val = string("EXACT")]; + tensor mapping_17_cast_fp16 = gelu(mode = mapping_17_mode_0, x = linear_103_cast_fp16)[name = string("mapping_17_cast_fp16")]; + tensor var_3542_reps_0 = const()[name = string("op_3542_reps_0"), val = tensor([1, 256, 1])]; + tensor var_3542_cast_fp16 = tile(reps = var_3542_reps_0, x = x_463_cast_fp16)[name = string("op_3542_cast_fp16")]; + bool x_465_interleave_0 = const()[name = string("x_465_interleave_0"), val = bool(false)]; + tensor x_465_cast_fp16 = concat(axis = var_3141, interleave = x_465_interleave_0, values = (var_3542_cast_fp16, embedding_to_fp16))[name = string("x_465_cast_fp16")]; + tensor var_3545_axes_0 = const()[name = string("op_3545_axes_0"), val = tensor([1])]; + tensor var_3545_cast_fp16 = expand_dims(axes = var_3545_axes_0, x = mapping_17_cast_fp16)[name = string("op_3545_cast_fp16")]; + tensor mapping_19_reps_0 = const()[name = string("mapping_19_reps_0"), val = tensor([1, 256, 1])]; + tensor mapping_19_cast_fp16 = tile(reps = mapping_19_reps_0, x = var_3545_cast_fp16)[name = string("mapping_19_cast_fp16")]; + tensor x_467_cast_fp16 = add(x = x_465_cast_fp16, y = mapping_19_cast_fp16)[name = string("x_467_cast_fp16")]; + tensor var_3557_split_sizes_0 = const()[name = string("op_3557_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3557_axis_0 = const()[name = string("op_3557_axis_0"), val = int32(1)]; + tensor var_3557_cast_fp16_0, tensor var_3557_cast_fp16_1 = split(axis = var_3557_axis_0, split_sizes = var_3557_split_sizes_0, x = h_3_cast_fp16)[name = string("op_3557_cast_fp16")]; + tensor gamma_99_perm_0 = const()[name = string("gamma_99_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_99_perm_0 = const()[name = string("beta_99_perm_0"), val = tensor([0, -1, 1])]; + tensor x_471_axes_0 = const()[name = string("x_471_axes_0"), val = tensor([-1])]; + fp16 var_3137_to_fp16 = const()[name = string("op_3137_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_471_cast_fp16 = layer_norm(axes = x_471_axes_0, epsilon = var_3137_to_fp16, x = x_467_cast_fp16)[name = string("x_471_cast_fp16")]; + fp16 var_3563_promoted_to_fp16 = const()[name = string("op_3563_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_99_cast_fp16 = transpose(perm = gamma_99_perm_0, x = var_3557_cast_fp16_0)[name = string("transpose_226")]; + tensor var_3564_cast_fp16 = add(x = gamma_99_cast_fp16, y = var_3563_promoted_to_fp16)[name = string("op_3564_cast_fp16")]; + tensor var_3565_cast_fp16 = mul(x = var_3564_cast_fp16, y = x_471_cast_fp16)[name = string("op_3565_cast_fp16")]; + tensor beta_99_cast_fp16 = transpose(perm = beta_99_perm_0, x = var_3557_cast_fp16_1)[name = string("transpose_225")]; + tensor x_473_cast_fp16 = add(x = var_3565_cast_fp16, y = beta_99_cast_fp16)[name = string("x_473_cast_fp16")]; + tensor var_3576_split_sizes_0 = const()[name = string("op_3576_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3576_axis_0 = const()[name = string("op_3576_axis_0"), val = int32(1)]; + tensor var_3576_cast_fp16_0, tensor var_3576_cast_fp16_1 = split(axis = var_3576_axis_0, split_sizes = var_3576_split_sizes_0, x = h_7_cast_fp16)[name = string("op_3576_cast_fp16")]; + tensor gamma_103_perm_0 = const()[name = string("gamma_103_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_103_perm_0 = const()[name = string("beta_103_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3582_promoted_to_fp16 = const()[name = string("op_3582_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_103_cast_fp16 = transpose(perm = gamma_103_perm_0, x = var_3576_cast_fp16_0)[name = string("transpose_224")]; + tensor var_3583_cast_fp16 = add(x = gamma_103_cast_fp16, y = var_3582_promoted_to_fp16)[name = string("op_3583_cast_fp16")]; + tensor var_3584_cast_fp16 = mul(x = var_3583_cast_fp16, y = x_471_cast_fp16)[name = string("op_3584_cast_fp16")]; + tensor beta_103_cast_fp16 = transpose(perm = beta_103_perm_0, x = var_3576_cast_fp16_1)[name = string("transpose_223")]; + tensor x_479_cast_fp16 = add(x = var_3584_cast_fp16, y = beta_103_cast_fp16)[name = string("x_479_cast_fp16")]; + tensor linear_106_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_473_cast_fp16)[name = string("linear_106_cast_fp16")]; + tensor linear_107_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_479_cast_fp16)[name = string("linear_107_cast_fp16")]; + tensor var_3590_split_sizes_0 = const()[name = string("op_3590_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3590_axis_0 = const()[name = string("op_3590_axis_0"), val = int32(-1)]; + tensor var_3590_cast_fp16_0, tensor var_3590_cast_fp16_1 = split(axis = var_3590_axis_0, split_sizes = var_3590_split_sizes_0, x = linear_107_cast_fp16)[name = string("op_3590_cast_fp16")]; + tensor var_3598 = const()[name = string("op_3598"), val = tensor([1, 256, 8, 64])]; + tensor x_483_cast_fp16 = reshape(shape = var_3598, x = linear_106_cast_fp16)[name = string("x_483_cast_fp16")]; + tensor var_3608 = const()[name = string("op_3608"), val = tensor([1, 256, 8, 64])]; + tensor x_487_cast_fp16 = reshape(shape = var_3608, x = var_3590_cast_fp16_0)[name = string("x_487_cast_fp16")]; + tensor var_3618 = const()[name = string("op_3618"), val = tensor([1, 256, 8, 64])]; + tensor x_491_cast_fp16 = reshape(shape = var_3618, x = var_3590_cast_fp16_1)[name = string("x_491_cast_fp16")]; + tensor var_3620 = const()[name = string("op_3620"), val = tensor([0, 2, 1, 3])]; + bool sim_49_transpose_x_0 = const()[name = string("sim_49_transpose_x_0"), val = bool(false)]; + bool sim_49_transpose_y_0 = const()[name = string("sim_49_transpose_y_0"), val = bool(false)]; + tensor transpose_96_perm_0 = const()[name = string("transpose_96_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_97_perm_0 = const()[name = string("transpose_97_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_97 = transpose(perm = transpose_97_perm_0, x = x_487_cast_fp16)[name = string("transpose_220")]; + tensor transpose_96 = transpose(perm = transpose_96_perm_0, x = x_483_cast_fp16)[name = string("transpose_221")]; + tensor sim_49_cast_fp16 = matmul(transpose_x = sim_49_transpose_x_0, transpose_y = sim_49_transpose_y_0, x = transpose_96, y = transpose_97)[name = string("sim_49_cast_fp16")]; + fp16 var_3624_to_fp16 = const()[name = string("op_3624_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_51_cast_fp16 = mul(x = sim_49_cast_fp16, y = var_3624_to_fp16)[name = string("sim_51_cast_fp16")]; + tensor attn_25_cast_fp16 = softmax(axis = var_3141, x = sim_51_cast_fp16)[name = string("attn_25_cast_fp16")]; + bool x_493_transpose_x_0 = const()[name = string("x_493_transpose_x_0"), val = bool(false)]; + bool x_493_transpose_y_0 = const()[name = string("x_493_transpose_y_0"), val = bool(false)]; + tensor v_25_cast_fp16 = transpose(perm = var_3620, x = x_491_cast_fp16)[name = string("transpose_222")]; + tensor x_493_cast_fp16 = matmul(transpose_x = x_493_transpose_x_0, transpose_y = x_493_transpose_y_0, x = attn_25_cast_fp16, y = v_25_cast_fp16)[name = string("x_493_cast_fp16")]; + tensor var_3646 = const()[name = string("op_3646"), val = tensor([0, 2, 1, 3])]; + tensor var_3648 = const()[name = string("op_3648"), val = tensor([1, 256, 512])]; + tensor x_495_cast_fp16 = transpose(perm = var_3646, x = x_493_cast_fp16)[name = string("transpose_219")]; + tensor input_279_cast_fp16 = reshape(shape = var_3648, x = x_495_cast_fp16)[name = string("input_279_cast_fp16")]; + tensor linear_108_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_279_cast_fp16)[name = string("linear_108_cast_fp16")]; + tensor input_281_cast_fp16 = add(x = linear_108_cast_fp16, y = x_467_cast_fp16)[name = string("input_281_cast_fp16")]; + tensor linear_109_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_281_cast_fp16)[name = string("linear_109_cast_fp16")]; + string input_285_mode_0 = const()[name = string("input_285_mode_0"), val = string("EXACT")]; + tensor input_285_cast_fp16 = gelu(mode = input_285_mode_0, x = linear_109_cast_fp16)[name = string("input_285_cast_fp16")]; + tensor linear_110_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_285_cast_fp16)[name = string("linear_110_cast_fp16")]; + tensor x_497_cast_fp16 = add(x = linear_110_cast_fp16, y = input_281_cast_fp16)[name = string("x_497_cast_fp16")]; + tensor x_499_cast_fp16 = add(x = x_497_cast_fp16, y = mapping_19_cast_fp16)[name = string("x_499_cast_fp16")]; + tensor var_3664_split_sizes_0 = const()[name = string("op_3664_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3664_axis_0 = const()[name = string("op_3664_axis_0"), val = int32(1)]; + tensor var_3664_cast_fp16_0, tensor var_3664_cast_fp16_1 = split(axis = var_3664_axis_0, split_sizes = var_3664_split_sizes_0, x = h_11_cast_fp16)[name = string("op_3664_cast_fp16")]; + tensor gamma_107_perm_0 = const()[name = string("gamma_107_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_107_perm_0 = const()[name = string("beta_107_perm_0"), val = tensor([0, -1, 1])]; + tensor x_503_axes_0 = const()[name = string("x_503_axes_0"), val = tensor([-1])]; + tensor x_503_cast_fp16 = layer_norm(axes = x_503_axes_0, epsilon = var_3137_to_fp16, x = x_499_cast_fp16)[name = string("x_503_cast_fp16")]; + fp16 var_3670_promoted_to_fp16 = const()[name = string("op_3670_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_107_cast_fp16 = transpose(perm = gamma_107_perm_0, x = var_3664_cast_fp16_0)[name = string("transpose_218")]; + tensor var_3671_cast_fp16 = add(x = gamma_107_cast_fp16, y = var_3670_promoted_to_fp16)[name = string("op_3671_cast_fp16")]; + tensor var_3672_cast_fp16 = mul(x = var_3671_cast_fp16, y = x_503_cast_fp16)[name = string("op_3672_cast_fp16")]; + tensor beta_107_cast_fp16 = transpose(perm = beta_107_perm_0, x = var_3664_cast_fp16_1)[name = string("transpose_217")]; + tensor x_505_cast_fp16 = add(x = var_3672_cast_fp16, y = beta_107_cast_fp16)[name = string("x_505_cast_fp16")]; + tensor var_3683_split_sizes_0 = const()[name = string("op_3683_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3683_axis_0 = const()[name = string("op_3683_axis_0"), val = int32(1)]; + tensor var_3683_cast_fp16_0, tensor var_3683_cast_fp16_1 = split(axis = var_3683_axis_0, split_sizes = var_3683_split_sizes_0, x = h_15_cast_fp16)[name = string("op_3683_cast_fp16")]; + tensor gamma_111_perm_0 = const()[name = string("gamma_111_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_111_perm_0 = const()[name = string("beta_111_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3689_promoted_to_fp16 = const()[name = string("op_3689_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_111_cast_fp16 = transpose(perm = gamma_111_perm_0, x = var_3683_cast_fp16_0)[name = string("transpose_216")]; + tensor var_3690_cast_fp16 = add(x = gamma_111_cast_fp16, y = var_3689_promoted_to_fp16)[name = string("op_3690_cast_fp16")]; + tensor var_3691_cast_fp16 = mul(x = var_3690_cast_fp16, y = x_503_cast_fp16)[name = string("op_3691_cast_fp16")]; + tensor beta_111_cast_fp16 = transpose(perm = beta_111_perm_0, x = var_3683_cast_fp16_1)[name = string("transpose_215")]; + tensor x_511_cast_fp16 = add(x = var_3691_cast_fp16, y = beta_111_cast_fp16)[name = string("x_511_cast_fp16")]; + tensor linear_113_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_505_cast_fp16)[name = string("linear_113_cast_fp16")]; + tensor linear_114_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_511_cast_fp16)[name = string("linear_114_cast_fp16")]; + tensor var_3697_split_sizes_0 = const()[name = string("op_3697_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3697_axis_0 = const()[name = string("op_3697_axis_0"), val = int32(-1)]; + tensor var_3697_cast_fp16_0, tensor var_3697_cast_fp16_1 = split(axis = var_3697_axis_0, split_sizes = var_3697_split_sizes_0, x = linear_114_cast_fp16)[name = string("op_3697_cast_fp16")]; + tensor var_3705 = const()[name = string("op_3705"), val = tensor([1, 256, 8, 64])]; + tensor x_515_cast_fp16 = reshape(shape = var_3705, x = linear_113_cast_fp16)[name = string("x_515_cast_fp16")]; + tensor var_3715 = const()[name = string("op_3715"), val = tensor([1, 256, 8, 64])]; + tensor x_519_cast_fp16 = reshape(shape = var_3715, x = var_3697_cast_fp16_0)[name = string("x_519_cast_fp16")]; + tensor var_3725 = const()[name = string("op_3725"), val = tensor([1, 256, 8, 64])]; + tensor x_523_cast_fp16 = reshape(shape = var_3725, x = var_3697_cast_fp16_1)[name = string("x_523_cast_fp16")]; + tensor var_3727 = const()[name = string("op_3727"), val = tensor([0, 2, 1, 3])]; + bool sim_53_transpose_x_0 = const()[name = string("sim_53_transpose_x_0"), val = bool(false)]; + bool sim_53_transpose_y_0 = const()[name = string("sim_53_transpose_y_0"), val = bool(false)]; + tensor transpose_98_perm_0 = const()[name = string("transpose_98_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_99_perm_0 = const()[name = string("transpose_99_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_99 = transpose(perm = transpose_99_perm_0, x = x_519_cast_fp16)[name = string("transpose_212")]; + tensor transpose_98 = transpose(perm = transpose_98_perm_0, x = x_515_cast_fp16)[name = string("transpose_213")]; + tensor sim_53_cast_fp16 = matmul(transpose_x = sim_53_transpose_x_0, transpose_y = sim_53_transpose_y_0, x = transpose_98, y = transpose_99)[name = string("sim_53_cast_fp16")]; + fp16 var_3731_to_fp16 = const()[name = string("op_3731_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_55_cast_fp16 = mul(x = sim_53_cast_fp16, y = var_3731_to_fp16)[name = string("sim_55_cast_fp16")]; + tensor attn_27_cast_fp16 = softmax(axis = var_3141, x = sim_55_cast_fp16)[name = string("attn_27_cast_fp16")]; + bool x_525_transpose_x_0 = const()[name = string("x_525_transpose_x_0"), val = bool(false)]; + bool x_525_transpose_y_0 = const()[name = string("x_525_transpose_y_0"), val = bool(false)]; + tensor v_27_cast_fp16 = transpose(perm = var_3727, x = x_523_cast_fp16)[name = string("transpose_214")]; + tensor x_525_cast_fp16 = matmul(transpose_x = x_525_transpose_x_0, transpose_y = x_525_transpose_y_0, x = attn_27_cast_fp16, y = v_27_cast_fp16)[name = string("x_525_cast_fp16")]; + tensor var_3753 = const()[name = string("op_3753"), val = tensor([0, 2, 1, 3])]; + tensor var_3755 = const()[name = string("op_3755"), val = tensor([1, 256, 512])]; + tensor x_527_cast_fp16 = transpose(perm = var_3753, x = x_525_cast_fp16)[name = string("transpose_211")]; + tensor input_295_cast_fp16 = reshape(shape = var_3755, x = x_527_cast_fp16)[name = string("input_295_cast_fp16")]; + tensor linear_115_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_295_cast_fp16)[name = string("linear_115_cast_fp16")]; + tensor input_297_cast_fp16 = add(x = linear_115_cast_fp16, y = x_499_cast_fp16)[name = string("input_297_cast_fp16")]; + tensor linear_116_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_297_cast_fp16)[name = string("linear_116_cast_fp16")]; + string input_301_mode_0 = const()[name = string("input_301_mode_0"), val = string("EXACT")]; + tensor input_301_cast_fp16 = gelu(mode = input_301_mode_0, x = linear_116_cast_fp16)[name = string("input_301_cast_fp16")]; + tensor linear_117_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_301_cast_fp16)[name = string("linear_117_cast_fp16")]; + tensor x_529_cast_fp16 = add(x = linear_117_cast_fp16, y = input_297_cast_fp16)[name = string("x_529_cast_fp16")]; + tensor x_531_cast_fp16 = add(x = x_529_cast_fp16, y = mapping_19_cast_fp16)[name = string("x_531_cast_fp16")]; + tensor var_3771_split_sizes_0 = const()[name = string("op_3771_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3771_axis_0 = const()[name = string("op_3771_axis_0"), val = int32(1)]; + tensor var_3771_cast_fp16_0, tensor var_3771_cast_fp16_1 = split(axis = var_3771_axis_0, split_sizes = var_3771_split_sizes_0, x = h_19_cast_fp16)[name = string("op_3771_cast_fp16")]; + tensor gamma_115_perm_0 = const()[name = string("gamma_115_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_115_perm_0 = const()[name = string("beta_115_perm_0"), val = tensor([0, -1, 1])]; + tensor x_535_axes_0 = const()[name = string("x_535_axes_0"), val = tensor([-1])]; + tensor x_535_cast_fp16 = layer_norm(axes = x_535_axes_0, epsilon = var_3137_to_fp16, x = x_531_cast_fp16)[name = string("x_535_cast_fp16")]; + fp16 var_3777_promoted_to_fp16 = const()[name = string("op_3777_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_115_cast_fp16 = transpose(perm = gamma_115_perm_0, x = var_3771_cast_fp16_0)[name = string("transpose_210")]; + tensor var_3778_cast_fp16 = add(x = gamma_115_cast_fp16, y = var_3777_promoted_to_fp16)[name = string("op_3778_cast_fp16")]; + tensor var_3779_cast_fp16 = mul(x = var_3778_cast_fp16, y = x_535_cast_fp16)[name = string("op_3779_cast_fp16")]; + tensor beta_115_cast_fp16 = transpose(perm = beta_115_perm_0, x = var_3771_cast_fp16_1)[name = string("transpose_209")]; + tensor x_537_cast_fp16 = add(x = var_3779_cast_fp16, y = beta_115_cast_fp16)[name = string("x_537_cast_fp16")]; + tensor var_3790_split_sizes_0 = const()[name = string("op_3790_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3790_axis_0 = const()[name = string("op_3790_axis_0"), val = int32(1)]; + tensor var_3790_cast_fp16_0, tensor var_3790_cast_fp16_1 = split(axis = var_3790_axis_0, split_sizes = var_3790_split_sizes_0, x = h_23_cast_fp16)[name = string("op_3790_cast_fp16")]; + tensor gamma_119_perm_0 = const()[name = string("gamma_119_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_119_perm_0 = const()[name = string("beta_119_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3796_promoted_to_fp16 = const()[name = string("op_3796_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_119_cast_fp16 = transpose(perm = gamma_119_perm_0, x = var_3790_cast_fp16_0)[name = string("transpose_208")]; + tensor var_3797_cast_fp16 = add(x = gamma_119_cast_fp16, y = var_3796_promoted_to_fp16)[name = string("op_3797_cast_fp16")]; + tensor var_3798_cast_fp16 = mul(x = var_3797_cast_fp16, y = x_535_cast_fp16)[name = string("op_3798_cast_fp16")]; + tensor beta_119_cast_fp16 = transpose(perm = beta_119_perm_0, x = var_3790_cast_fp16_1)[name = string("transpose_207")]; + tensor x_543_cast_fp16 = add(x = var_3798_cast_fp16, y = beta_119_cast_fp16)[name = string("x_543_cast_fp16")]; + tensor linear_120_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_537_cast_fp16)[name = string("linear_120_cast_fp16")]; + tensor linear_121_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_543_cast_fp16)[name = string("linear_121_cast_fp16")]; + tensor var_3804_split_sizes_0 = const()[name = string("op_3804_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3804_axis_0 = const()[name = string("op_3804_axis_0"), val = int32(-1)]; + tensor var_3804_cast_fp16_0, tensor var_3804_cast_fp16_1 = split(axis = var_3804_axis_0, split_sizes = var_3804_split_sizes_0, x = linear_121_cast_fp16)[name = string("op_3804_cast_fp16")]; + tensor var_3812 = const()[name = string("op_3812"), val = tensor([1, 256, 8, 64])]; + tensor x_547_cast_fp16 = reshape(shape = var_3812, x = linear_120_cast_fp16)[name = string("x_547_cast_fp16")]; + tensor var_3822 = const()[name = string("op_3822"), val = tensor([1, 256, 8, 64])]; + tensor x_551_cast_fp16 = reshape(shape = var_3822, x = var_3804_cast_fp16_0)[name = string("x_551_cast_fp16")]; + tensor var_3832 = const()[name = string("op_3832"), val = tensor([1, 256, 8, 64])]; + tensor x_555_cast_fp16 = reshape(shape = var_3832, x = var_3804_cast_fp16_1)[name = string("x_555_cast_fp16")]; + tensor var_3834 = const()[name = string("op_3834"), val = tensor([0, 2, 1, 3])]; + bool sim_57_transpose_x_0 = const()[name = string("sim_57_transpose_x_0"), val = bool(false)]; + bool sim_57_transpose_y_0 = const()[name = string("sim_57_transpose_y_0"), val = bool(false)]; + tensor transpose_100_perm_0 = const()[name = string("transpose_100_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_101_perm_0 = const()[name = string("transpose_101_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_101 = transpose(perm = transpose_101_perm_0, x = x_551_cast_fp16)[name = string("transpose_204")]; + tensor transpose_100 = transpose(perm = transpose_100_perm_0, x = x_547_cast_fp16)[name = string("transpose_205")]; + tensor sim_57_cast_fp16 = matmul(transpose_x = sim_57_transpose_x_0, transpose_y = sim_57_transpose_y_0, x = transpose_100, y = transpose_101)[name = string("sim_57_cast_fp16")]; + fp16 var_3838_to_fp16 = const()[name = string("op_3838_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_59_cast_fp16 = mul(x = sim_57_cast_fp16, y = var_3838_to_fp16)[name = string("sim_59_cast_fp16")]; + tensor attn_29_cast_fp16 = softmax(axis = var_3141, x = sim_59_cast_fp16)[name = string("attn_29_cast_fp16")]; + bool x_557_transpose_x_0 = const()[name = string("x_557_transpose_x_0"), val = bool(false)]; + bool x_557_transpose_y_0 = const()[name = string("x_557_transpose_y_0"), val = bool(false)]; + tensor v_29_cast_fp16 = transpose(perm = var_3834, x = x_555_cast_fp16)[name = string("transpose_206")]; + tensor x_557_cast_fp16 = matmul(transpose_x = x_557_transpose_x_0, transpose_y = x_557_transpose_y_0, x = attn_29_cast_fp16, y = v_29_cast_fp16)[name = string("x_557_cast_fp16")]; + tensor var_3860 = const()[name = string("op_3860"), val = tensor([0, 2, 1, 3])]; + tensor var_3862 = const()[name = string("op_3862"), val = tensor([1, 256, 512])]; + tensor x_559_cast_fp16 = transpose(perm = var_3860, x = x_557_cast_fp16)[name = string("transpose_203")]; + tensor input_311_cast_fp16 = reshape(shape = var_3862, x = x_559_cast_fp16)[name = string("input_311_cast_fp16")]; + tensor linear_122_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_311_cast_fp16)[name = string("linear_122_cast_fp16")]; + tensor input_313_cast_fp16 = add(x = linear_122_cast_fp16, y = x_531_cast_fp16)[name = string("input_313_cast_fp16")]; + tensor linear_123_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_313_cast_fp16)[name = string("linear_123_cast_fp16")]; + string input_317_mode_0 = const()[name = string("input_317_mode_0"), val = string("EXACT")]; + tensor input_317_cast_fp16 = gelu(mode = input_317_mode_0, x = linear_123_cast_fp16)[name = string("input_317_cast_fp16")]; + tensor linear_124_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_317_cast_fp16)[name = string("linear_124_cast_fp16")]; + tensor x_561_cast_fp16 = add(x = linear_124_cast_fp16, y = input_313_cast_fp16)[name = string("x_561_cast_fp16")]; + tensor var_3871_axes_0 = const()[name = string("op_3871_axes_0"), val = tensor([1])]; + bool var_3871_keep_dims_0 = const()[name = string("op_3871_keep_dims_0"), val = bool(false)]; + tensor var_3871_cast_fp16 = reduce_mean(axes = var_3871_axes_0, keep_dims = var_3871_keep_dims_0, x = x_561_cast_fp16)[name = string("op_3871_cast_fp16")]; + tensor x_563_axes_0 = const()[name = string("x_563_axes_0"), val = tensor([1])]; + tensor x_563_cast_fp16 = expand_dims(axes = x_563_axes_0, x = var_3871_cast_fp16)[name = string("x_563_cast_fp16")]; + tensor var_3873 = const()[name = string("op_3873"), val = tensor([0, 2, 1])]; + string x_565_pad_type_0 = const()[name = string("x_565_pad_type_0"), val = string("valid")]; + tensor x_565_strides_0 = const()[name = string("x_565_strides_0"), val = tensor([1])]; + tensor x_565_pad_0 = const()[name = string("x_565_pad_0"), val = tensor([0, 0])]; + tensor x_565_dilations_0 = const()[name = string("x_565_dilations_0"), val = tensor([1])]; + int32 x_565_groups_0 = const()[name = string("x_565_groups_0"), val = int32(1)]; + tensor input_319_cast_fp16 = transpose(perm = var_3873, x = x_563_cast_fp16)[name = string("transpose_202")]; + tensor x_565_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_565_dilations_0, groups = x_565_groups_0, pad = x_565_pad_0, pad_type = x_565_pad_type_0, strides = x_565_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_319_cast_fp16)[name = string("x_565_cast_fp16")]; + tensor x_pred_9_perm_0 = const()[name = string("x_pred_9_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_9_to_fp16 = const()[name = string("c_skip_9_to_fp16"), val = tensor([[[0x1.c64p-1]]])]; + tensor var_3881_cast_fp16 = mul(x = c_skip_9_to_fp16, y = x_noisy_9_cast_fp16)[name = string("op_3881_cast_fp16")]; + tensor c_out_9_to_fp16 = const()[name = string("c_out_9_to_fp16"), val = tensor([[[0x1.0fcp-4]]])]; + tensor x_pred_9_cast_fp16 = transpose(perm = x_pred_9_perm_0, x = x_565_cast_fp16)[name = string("transpose_201")]; + tensor var_3882_cast_fp16 = mul(x = c_out_9_to_fp16, y = x_pred_9_cast_fp16)[name = string("op_3882_cast_fp16")]; + tensor x_dn_5_cast_fp16 = add(x = var_3881_cast_fp16, y = var_3882_cast_fp16)[name = string("x_dn_5_cast_fp16")]; + tensor var_3885_cast_fp16 = sub(x = x_noisy_9_cast_fp16, y = x_dn_5_cast_fp16)[name = string("op_3885_cast_fp16")]; + tensor _inversed_d_5_y_0_to_fp16 = const()[name = string("_inversed_d_5_y_0_to_fp16"), val = tensor([0x1.c6cp+3])]; + tensor _inversed_d_5_cast_fp16 = mul(x = var_3885_cast_fp16, y = _inversed_d_5_y_0_to_fp16)[name = string("_inversed_d_5_cast_fp16")]; + fp16 var_3894_to_fp16 = const()[name = string("op_3894_to_fp16"), val = fp16(-0x1.1fp-5)]; + tensor var_3895_cast_fp16 = mul(x = _inversed_d_5_cast_fp16, y = var_3894_to_fp16)[name = string("op_3895_cast_fp16")]; + tensor x_noisy_11_cast_fp16 = add(x = x_noisy_9_cast_fp16, y = var_3895_cast_fp16)[name = string("x_noisy_11_cast_fp16")]; + int32 var_3907 = const()[name = string("op_3907"), val = int32(-1)]; + tensor c_in_11_to_fp16 = const()[name = string("c_in_11_to_fp16"), val = tensor([[[0x1.3c4p+2]]])]; + tensor x_575_cast_fp16 = mul(x = c_in_11_to_fp16, y = x_noisy_11_cast_fp16)[name = string("x_575_cast_fp16")]; + int32 x_571_axis_0 = const()[name = string("x_571_axis_0"), val = int32(0)]; + tensor var_4293_to_fp16 = const()[name = string("op_4293_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49355520)))]; + tensor x_571_cast_fp16 = stack(axis = x_571_axis_0, values = (var_4293_to_fp16, var_423_cast_fp16))[name = string("x_571_cast_fp16")]; + tensor var_4298 = const()[name = string("op_4298"), val = tensor([1, 2, 0])]; + tensor input_327_axes_0 = const()[name = string("input_327_axes_0"), val = tensor([2])]; + bool input_327_keep_dims_0 = const()[name = string("input_327_keep_dims_0"), val = bool(false)]; + tensor x_573_cast_fp16 = transpose(perm = var_4298, x = x_571_cast_fp16)[name = string("transpose_200")]; + tensor input_327_cast_fp16 = reduce_sum(axes = input_327_axes_0, keep_dims = input_327_keep_dims_0, x = x_573_cast_fp16)[name = string("input_327_cast_fp16")]; + tensor linear_127_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_327_cast_fp16)[name = string("linear_127_cast_fp16")]; + string input_331_mode_0 = const()[name = string("input_331_mode_0"), val = string("EXACT")]; + tensor input_331_cast_fp16 = gelu(mode = input_331_mode_0, x = linear_127_cast_fp16)[name = string("input_331_cast_fp16")]; + tensor linear_128_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_331_cast_fp16)[name = string("linear_128_cast_fp16")]; + string mapping_21_mode_0 = const()[name = string("mapping_21_mode_0"), val = string("EXACT")]; + tensor mapping_21_cast_fp16 = gelu(mode = mapping_21_mode_0, x = linear_128_cast_fp16)[name = string("mapping_21_cast_fp16")]; + tensor var_4308_reps_0 = const()[name = string("op_4308_reps_0"), val = tensor([1, 256, 1])]; + tensor var_4308_cast_fp16 = tile(reps = var_4308_reps_0, x = x_575_cast_fp16)[name = string("op_4308_cast_fp16")]; + bool x_577_interleave_0 = const()[name = string("x_577_interleave_0"), val = bool(false)]; + tensor x_577_cast_fp16 = concat(axis = var_3907, interleave = x_577_interleave_0, values = (var_4308_cast_fp16, embedding_to_fp16))[name = string("x_577_cast_fp16")]; + tensor var_4311_axes_0 = const()[name = string("op_4311_axes_0"), val = tensor([1])]; + tensor var_4311_cast_fp16 = expand_dims(axes = var_4311_axes_0, x = mapping_21_cast_fp16)[name = string("op_4311_cast_fp16")]; + tensor mapping_23_reps_0 = const()[name = string("mapping_23_reps_0"), val = tensor([1, 256, 1])]; + tensor mapping_23_cast_fp16 = tile(reps = mapping_23_reps_0, x = var_4311_cast_fp16)[name = string("mapping_23_cast_fp16")]; + tensor x_579_cast_fp16 = add(x = x_577_cast_fp16, y = mapping_23_cast_fp16)[name = string("x_579_cast_fp16")]; + tensor var_4323_split_sizes_0 = const()[name = string("op_4323_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4323_axis_0 = const()[name = string("op_4323_axis_0"), val = int32(1)]; + tensor var_4323_cast_fp16_0, tensor var_4323_cast_fp16_1 = split(axis = var_4323_axis_0, split_sizes = var_4323_split_sizes_0, x = h_3_cast_fp16)[name = string("op_4323_cast_fp16")]; + tensor gamma_123_perm_0 = const()[name = string("gamma_123_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_123_perm_0 = const()[name = string("beta_123_perm_0"), val = tensor([0, -1, 1])]; + tensor x_583_axes_0 = const()[name = string("x_583_axes_0"), val = tensor([-1])]; + fp16 var_3903_to_fp16 = const()[name = string("op_3903_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_583_cast_fp16 = layer_norm(axes = x_583_axes_0, epsilon = var_3903_to_fp16, x = x_579_cast_fp16)[name = string("x_583_cast_fp16")]; + fp16 var_4329_promoted_to_fp16 = const()[name = string("op_4329_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_123_cast_fp16 = transpose(perm = gamma_123_perm_0, x = var_4323_cast_fp16_0)[name = string("transpose_199")]; + tensor var_4330_cast_fp16 = add(x = gamma_123_cast_fp16, y = var_4329_promoted_to_fp16)[name = string("op_4330_cast_fp16")]; + tensor var_4331_cast_fp16 = mul(x = var_4330_cast_fp16, y = x_583_cast_fp16)[name = string("op_4331_cast_fp16")]; + tensor beta_123_cast_fp16 = transpose(perm = beta_123_perm_0, x = var_4323_cast_fp16_1)[name = string("transpose_198")]; + tensor x_585_cast_fp16 = add(x = var_4331_cast_fp16, y = beta_123_cast_fp16)[name = string("x_585_cast_fp16")]; + tensor var_4342_split_sizes_0 = const()[name = string("op_4342_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4342_axis_0 = const()[name = string("op_4342_axis_0"), val = int32(1)]; + tensor var_4342_cast_fp16_0, tensor var_4342_cast_fp16_1 = split(axis = var_4342_axis_0, split_sizes = var_4342_split_sizes_0, x = h_7_cast_fp16)[name = string("op_4342_cast_fp16")]; + tensor gamma_127_perm_0 = const()[name = string("gamma_127_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_127_perm_0 = const()[name = string("beta_127_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_4348_promoted_to_fp16 = const()[name = string("op_4348_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_127_cast_fp16 = transpose(perm = gamma_127_perm_0, x = var_4342_cast_fp16_0)[name = string("transpose_197")]; + tensor var_4349_cast_fp16 = add(x = gamma_127_cast_fp16, y = var_4348_promoted_to_fp16)[name = string("op_4349_cast_fp16")]; + tensor var_4350_cast_fp16 = mul(x = var_4349_cast_fp16, y = x_583_cast_fp16)[name = string("op_4350_cast_fp16")]; + tensor beta_127_cast_fp16 = transpose(perm = beta_127_perm_0, x = var_4342_cast_fp16_1)[name = string("transpose_196")]; + tensor x_591_cast_fp16 = add(x = var_4350_cast_fp16, y = beta_127_cast_fp16)[name = string("x_591_cast_fp16")]; + tensor linear_131_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_585_cast_fp16)[name = string("linear_131_cast_fp16")]; + tensor linear_132_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_591_cast_fp16)[name = string("linear_132_cast_fp16")]; + tensor var_4356_split_sizes_0 = const()[name = string("op_4356_split_sizes_0"), val = tensor([512, 512])]; + int32 var_4356_axis_0 = const()[name = string("op_4356_axis_0"), val = int32(-1)]; + tensor var_4356_cast_fp16_0, tensor var_4356_cast_fp16_1 = split(axis = var_4356_axis_0, split_sizes = var_4356_split_sizes_0, x = linear_132_cast_fp16)[name = string("op_4356_cast_fp16")]; + tensor var_4364 = const()[name = string("op_4364"), val = tensor([1, 256, 8, 64])]; + tensor x_595_cast_fp16 = reshape(shape = var_4364, x = linear_131_cast_fp16)[name = string("x_595_cast_fp16")]; + tensor var_4374 = const()[name = string("op_4374"), val = tensor([1, 256, 8, 64])]; + tensor x_599_cast_fp16 = reshape(shape = var_4374, x = var_4356_cast_fp16_0)[name = string("x_599_cast_fp16")]; + tensor var_4384 = const()[name = string("op_4384"), val = tensor([1, 256, 8, 64])]; + tensor x_603_cast_fp16 = reshape(shape = var_4384, x = var_4356_cast_fp16_1)[name = string("x_603_cast_fp16")]; + tensor var_4386 = const()[name = string("op_4386"), val = tensor([0, 2, 1, 3])]; + bool sim_61_transpose_x_0 = const()[name = string("sim_61_transpose_x_0"), val = bool(false)]; + bool sim_61_transpose_y_0 = const()[name = string("sim_61_transpose_y_0"), val = bool(false)]; + tensor transpose_102_perm_0 = const()[name = string("transpose_102_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_103_perm_0 = const()[name = string("transpose_103_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_103 = transpose(perm = transpose_103_perm_0, x = x_599_cast_fp16)[name = string("transpose_193")]; + tensor transpose_102 = transpose(perm = transpose_102_perm_0, x = x_595_cast_fp16)[name = string("transpose_194")]; + tensor sim_61_cast_fp16 = matmul(transpose_x = sim_61_transpose_x_0, transpose_y = sim_61_transpose_y_0, x = transpose_102, y = transpose_103)[name = string("sim_61_cast_fp16")]; + fp16 var_4390_to_fp16 = const()[name = string("op_4390_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_63_cast_fp16 = mul(x = sim_61_cast_fp16, y = var_4390_to_fp16)[name = string("sim_63_cast_fp16")]; + tensor attn_31_cast_fp16 = softmax(axis = var_3907, x = sim_63_cast_fp16)[name = string("attn_31_cast_fp16")]; + bool x_605_transpose_x_0 = const()[name = string("x_605_transpose_x_0"), val = bool(false)]; + bool x_605_transpose_y_0 = const()[name = string("x_605_transpose_y_0"), val = bool(false)]; + tensor v_31_cast_fp16 = transpose(perm = var_4386, x = x_603_cast_fp16)[name = string("transpose_195")]; + tensor x_605_cast_fp16 = matmul(transpose_x = x_605_transpose_x_0, transpose_y = x_605_transpose_y_0, x = attn_31_cast_fp16, y = v_31_cast_fp16)[name = string("x_605_cast_fp16")]; + tensor var_4412 = const()[name = string("op_4412"), val = tensor([0, 2, 1, 3])]; + tensor var_4414 = const()[name = string("op_4414"), val = tensor([1, 256, 512])]; + tensor x_607_cast_fp16 = transpose(perm = var_4412, x = x_605_cast_fp16)[name = string("transpose_192")]; + tensor input_343_cast_fp16 = reshape(shape = var_4414, x = x_607_cast_fp16)[name = string("input_343_cast_fp16")]; + tensor linear_133_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_343_cast_fp16)[name = string("linear_133_cast_fp16")]; + tensor input_345_cast_fp16 = add(x = linear_133_cast_fp16, y = x_579_cast_fp16)[name = string("input_345_cast_fp16")]; + tensor linear_134_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_345_cast_fp16)[name = string("linear_134_cast_fp16")]; + string input_349_mode_0 = const()[name = string("input_349_mode_0"), val = string("EXACT")]; + tensor input_349_cast_fp16 = gelu(mode = input_349_mode_0, x = linear_134_cast_fp16)[name = string("input_349_cast_fp16")]; + tensor linear_135_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_349_cast_fp16)[name = string("linear_135_cast_fp16")]; + tensor x_609_cast_fp16 = add(x = linear_135_cast_fp16, y = input_345_cast_fp16)[name = string("x_609_cast_fp16")]; + tensor x_611_cast_fp16 = add(x = x_609_cast_fp16, y = mapping_23_cast_fp16)[name = string("x_611_cast_fp16")]; + tensor var_4430_split_sizes_0 = const()[name = string("op_4430_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4430_axis_0 = const()[name = string("op_4430_axis_0"), val = int32(1)]; + tensor var_4430_cast_fp16_0, tensor var_4430_cast_fp16_1 = split(axis = var_4430_axis_0, split_sizes = var_4430_split_sizes_0, x = h_11_cast_fp16)[name = string("op_4430_cast_fp16")]; + tensor gamma_131_perm_0 = const()[name = string("gamma_131_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_131_perm_0 = const()[name = string("beta_131_perm_0"), val = tensor([0, -1, 1])]; + tensor x_615_axes_0 = const()[name = string("x_615_axes_0"), val = tensor([-1])]; + tensor x_615_cast_fp16 = layer_norm(axes = x_615_axes_0, epsilon = var_3903_to_fp16, x = x_611_cast_fp16)[name = string("x_615_cast_fp16")]; + fp16 var_4436_promoted_to_fp16 = const()[name = string("op_4436_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_131_cast_fp16 = transpose(perm = gamma_131_perm_0, x = var_4430_cast_fp16_0)[name = string("transpose_191")]; + tensor var_4437_cast_fp16 = add(x = gamma_131_cast_fp16, y = var_4436_promoted_to_fp16)[name = string("op_4437_cast_fp16")]; + tensor var_4438_cast_fp16 = mul(x = var_4437_cast_fp16, y = x_615_cast_fp16)[name = string("op_4438_cast_fp16")]; + tensor beta_131_cast_fp16 = transpose(perm = beta_131_perm_0, x = var_4430_cast_fp16_1)[name = string("transpose_190")]; + tensor x_617_cast_fp16 = add(x = var_4438_cast_fp16, y = beta_131_cast_fp16)[name = string("x_617_cast_fp16")]; + tensor var_4449_split_sizes_0 = const()[name = string("op_4449_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4449_axis_0 = const()[name = string("op_4449_axis_0"), val = int32(1)]; + tensor var_4449_cast_fp16_0, tensor var_4449_cast_fp16_1 = split(axis = var_4449_axis_0, split_sizes = var_4449_split_sizes_0, x = h_15_cast_fp16)[name = string("op_4449_cast_fp16")]; + tensor gamma_135_perm_0 = const()[name = string("gamma_135_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_135_perm_0 = const()[name = string("beta_135_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_4455_promoted_to_fp16 = const()[name = string("op_4455_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_135_cast_fp16 = transpose(perm = gamma_135_perm_0, x = var_4449_cast_fp16_0)[name = string("transpose_189")]; + tensor var_4456_cast_fp16 = add(x = gamma_135_cast_fp16, y = var_4455_promoted_to_fp16)[name = string("op_4456_cast_fp16")]; + tensor var_4457_cast_fp16 = mul(x = var_4456_cast_fp16, y = x_615_cast_fp16)[name = string("op_4457_cast_fp16")]; + tensor beta_135_cast_fp16 = transpose(perm = beta_135_perm_0, x = var_4449_cast_fp16_1)[name = string("transpose_188")]; + tensor x_623_cast_fp16 = add(x = var_4457_cast_fp16, y = beta_135_cast_fp16)[name = string("x_623_cast_fp16")]; + tensor linear_138_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_617_cast_fp16)[name = string("linear_138_cast_fp16")]; + tensor linear_139_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_623_cast_fp16)[name = string("linear_139_cast_fp16")]; + tensor var_4463_split_sizes_0 = const()[name = string("op_4463_split_sizes_0"), val = tensor([512, 512])]; + int32 var_4463_axis_0 = const()[name = string("op_4463_axis_0"), val = int32(-1)]; + tensor var_4463_cast_fp16_0, tensor var_4463_cast_fp16_1 = split(axis = var_4463_axis_0, split_sizes = var_4463_split_sizes_0, x = linear_139_cast_fp16)[name = string("op_4463_cast_fp16")]; + tensor var_4471 = const()[name = string("op_4471"), val = tensor([1, 256, 8, 64])]; + tensor x_627_cast_fp16 = reshape(shape = var_4471, x = linear_138_cast_fp16)[name = string("x_627_cast_fp16")]; + tensor var_4481 = const()[name = string("op_4481"), val = tensor([1, 256, 8, 64])]; + tensor x_631_cast_fp16 = reshape(shape = var_4481, x = var_4463_cast_fp16_0)[name = string("x_631_cast_fp16")]; + tensor var_4491 = const()[name = string("op_4491"), val = tensor([1, 256, 8, 64])]; + tensor x_635_cast_fp16 = reshape(shape = var_4491, x = var_4463_cast_fp16_1)[name = string("x_635_cast_fp16")]; + tensor var_4493 = const()[name = string("op_4493"), val = tensor([0, 2, 1, 3])]; + bool sim_65_transpose_x_0 = const()[name = string("sim_65_transpose_x_0"), val = bool(false)]; + bool sim_65_transpose_y_0 = const()[name = string("sim_65_transpose_y_0"), val = bool(false)]; + tensor transpose_104_perm_0 = const()[name = string("transpose_104_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_105_perm_0 = const()[name = string("transpose_105_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_105 = transpose(perm = transpose_105_perm_0, x = x_631_cast_fp16)[name = string("transpose_185")]; + tensor transpose_104 = transpose(perm = transpose_104_perm_0, x = x_627_cast_fp16)[name = string("transpose_186")]; + tensor sim_65_cast_fp16 = matmul(transpose_x = sim_65_transpose_x_0, transpose_y = sim_65_transpose_y_0, x = transpose_104, y = transpose_105)[name = string("sim_65_cast_fp16")]; + fp16 var_4497_to_fp16 = const()[name = string("op_4497_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_67_cast_fp16 = mul(x = sim_65_cast_fp16, y = var_4497_to_fp16)[name = string("sim_67_cast_fp16")]; + tensor attn_33_cast_fp16 = softmax(axis = var_3907, x = sim_67_cast_fp16)[name = string("attn_33_cast_fp16")]; + bool x_637_transpose_x_0 = const()[name = string("x_637_transpose_x_0"), val = bool(false)]; + bool x_637_transpose_y_0 = const()[name = string("x_637_transpose_y_0"), val = bool(false)]; + tensor v_33_cast_fp16 = transpose(perm = var_4493, x = x_635_cast_fp16)[name = string("transpose_187")]; + tensor x_637_cast_fp16 = matmul(transpose_x = x_637_transpose_x_0, transpose_y = x_637_transpose_y_0, x = attn_33_cast_fp16, y = v_33_cast_fp16)[name = string("x_637_cast_fp16")]; + tensor var_4519 = const()[name = string("op_4519"), val = tensor([0, 2, 1, 3])]; + tensor var_4521 = const()[name = string("op_4521"), val = tensor([1, 256, 512])]; + tensor x_639_cast_fp16 = transpose(perm = var_4519, x = x_637_cast_fp16)[name = string("transpose_184")]; + tensor input_359_cast_fp16 = reshape(shape = var_4521, x = x_639_cast_fp16)[name = string("input_359_cast_fp16")]; + tensor linear_140_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_359_cast_fp16)[name = string("linear_140_cast_fp16")]; + tensor input_361_cast_fp16 = add(x = linear_140_cast_fp16, y = x_611_cast_fp16)[name = string("input_361_cast_fp16")]; + tensor linear_141_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_361_cast_fp16)[name = string("linear_141_cast_fp16")]; + string input_365_mode_0 = const()[name = string("input_365_mode_0"), val = string("EXACT")]; + tensor input_365_cast_fp16 = gelu(mode = input_365_mode_0, x = linear_141_cast_fp16)[name = string("input_365_cast_fp16")]; + tensor linear_142_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_365_cast_fp16)[name = string("linear_142_cast_fp16")]; + tensor x_641_cast_fp16 = add(x = linear_142_cast_fp16, y = input_361_cast_fp16)[name = string("x_641_cast_fp16")]; + tensor x_643_cast_fp16 = add(x = x_641_cast_fp16, y = mapping_23_cast_fp16)[name = string("x_643_cast_fp16")]; + tensor var_4537_split_sizes_0 = const()[name = string("op_4537_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4537_axis_0 = const()[name = string("op_4537_axis_0"), val = int32(1)]; + tensor var_4537_cast_fp16_0, tensor var_4537_cast_fp16_1 = split(axis = var_4537_axis_0, split_sizes = var_4537_split_sizes_0, x = h_19_cast_fp16)[name = string("op_4537_cast_fp16")]; + tensor gamma_139_perm_0 = const()[name = string("gamma_139_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_139_perm_0 = const()[name = string("beta_139_perm_0"), val = tensor([0, -1, 1])]; + tensor x_647_axes_0 = const()[name = string("x_647_axes_0"), val = tensor([-1])]; + tensor x_647_cast_fp16 = layer_norm(axes = x_647_axes_0, epsilon = var_3903_to_fp16, x = x_643_cast_fp16)[name = string("x_647_cast_fp16")]; + fp16 var_4543_promoted_to_fp16 = const()[name = string("op_4543_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_139_cast_fp16 = transpose(perm = gamma_139_perm_0, x = var_4537_cast_fp16_0)[name = string("transpose_183")]; + tensor var_4544_cast_fp16 = add(x = gamma_139_cast_fp16, y = var_4543_promoted_to_fp16)[name = string("op_4544_cast_fp16")]; + tensor var_4545_cast_fp16 = mul(x = var_4544_cast_fp16, y = x_647_cast_fp16)[name = string("op_4545_cast_fp16")]; + tensor beta_139_cast_fp16 = transpose(perm = beta_139_perm_0, x = var_4537_cast_fp16_1)[name = string("transpose_182")]; + tensor x_649_cast_fp16 = add(x = var_4545_cast_fp16, y = beta_139_cast_fp16)[name = string("x_649_cast_fp16")]; + tensor var_4556_split_sizes_0 = const()[name = string("op_4556_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4556_axis_0 = const()[name = string("op_4556_axis_0"), val = int32(1)]; + tensor var_4556_cast_fp16_0, tensor var_4556_cast_fp16_1 = split(axis = var_4556_axis_0, split_sizes = var_4556_split_sizes_0, x = h_23_cast_fp16)[name = string("op_4556_cast_fp16")]; + tensor gamma_143_perm_0 = const()[name = string("gamma_143_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_143_perm_0 = const()[name = string("beta_143_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_4562_promoted_to_fp16 = const()[name = string("op_4562_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_143_cast_fp16 = transpose(perm = gamma_143_perm_0, x = var_4556_cast_fp16_0)[name = string("transpose_181")]; + tensor var_4563_cast_fp16 = add(x = gamma_143_cast_fp16, y = var_4562_promoted_to_fp16)[name = string("op_4563_cast_fp16")]; + tensor var_4564_cast_fp16 = mul(x = var_4563_cast_fp16, y = x_647_cast_fp16)[name = string("op_4564_cast_fp16")]; + tensor beta_143_cast_fp16 = transpose(perm = beta_143_perm_0, x = var_4556_cast_fp16_1)[name = string("transpose_180")]; + tensor x_655_cast_fp16 = add(x = var_4564_cast_fp16, y = beta_143_cast_fp16)[name = string("x_655_cast_fp16")]; + tensor linear_145_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_649_cast_fp16)[name = string("linear_145_cast_fp16")]; + tensor linear_146_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_655_cast_fp16)[name = string("linear_146_cast_fp16")]; + tensor var_4570_split_sizes_0 = const()[name = string("op_4570_split_sizes_0"), val = tensor([512, 512])]; + int32 var_4570_axis_0 = const()[name = string("op_4570_axis_0"), val = int32(-1)]; + tensor var_4570_cast_fp16_0, tensor var_4570_cast_fp16_1 = split(axis = var_4570_axis_0, split_sizes = var_4570_split_sizes_0, x = linear_146_cast_fp16)[name = string("op_4570_cast_fp16")]; + tensor var_4578 = const()[name = string("op_4578"), val = tensor([1, 256, 8, 64])]; + tensor x_659_cast_fp16 = reshape(shape = var_4578, x = linear_145_cast_fp16)[name = string("x_659_cast_fp16")]; + tensor var_4588 = const()[name = string("op_4588"), val = tensor([1, 256, 8, 64])]; + tensor x_663_cast_fp16 = reshape(shape = var_4588, x = var_4570_cast_fp16_0)[name = string("x_663_cast_fp16")]; + tensor var_4598 = const()[name = string("op_4598"), val = tensor([1, 256, 8, 64])]; + tensor x_667_cast_fp16 = reshape(shape = var_4598, x = var_4570_cast_fp16_1)[name = string("x_667_cast_fp16")]; + tensor var_4600 = const()[name = string("op_4600"), val = tensor([0, 2, 1, 3])]; + bool sim_69_transpose_x_0 = const()[name = string("sim_69_transpose_x_0"), val = bool(false)]; + bool sim_69_transpose_y_0 = const()[name = string("sim_69_transpose_y_0"), val = bool(false)]; + tensor transpose_106_perm_0 = const()[name = string("transpose_106_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_107_perm_0 = const()[name = string("transpose_107_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_107 = transpose(perm = transpose_107_perm_0, x = x_663_cast_fp16)[name = string("transpose_177")]; + tensor transpose_106 = transpose(perm = transpose_106_perm_0, x = x_659_cast_fp16)[name = string("transpose_178")]; + tensor sim_69_cast_fp16 = matmul(transpose_x = sim_69_transpose_x_0, transpose_y = sim_69_transpose_y_0, x = transpose_106, y = transpose_107)[name = string("sim_69_cast_fp16")]; + fp16 var_4604_to_fp16 = const()[name = string("op_4604_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_71_cast_fp16 = mul(x = sim_69_cast_fp16, y = var_4604_to_fp16)[name = string("sim_71_cast_fp16")]; + tensor attn_35_cast_fp16 = softmax(axis = var_3907, x = sim_71_cast_fp16)[name = string("attn_35_cast_fp16")]; + bool x_669_transpose_x_0 = const()[name = string("x_669_transpose_x_0"), val = bool(false)]; + bool x_669_transpose_y_0 = const()[name = string("x_669_transpose_y_0"), val = bool(false)]; + tensor v_35_cast_fp16 = transpose(perm = var_4600, x = x_667_cast_fp16)[name = string("transpose_179")]; + tensor x_669_cast_fp16 = matmul(transpose_x = x_669_transpose_x_0, transpose_y = x_669_transpose_y_0, x = attn_35_cast_fp16, y = v_35_cast_fp16)[name = string("x_669_cast_fp16")]; + tensor var_4626 = const()[name = string("op_4626"), val = tensor([0, 2, 1, 3])]; + tensor var_4628 = const()[name = string("op_4628"), val = tensor([1, 256, 512])]; + tensor x_671_cast_fp16 = transpose(perm = var_4626, x = x_669_cast_fp16)[name = string("transpose_176")]; + tensor input_375_cast_fp16 = reshape(shape = var_4628, x = x_671_cast_fp16)[name = string("input_375_cast_fp16")]; + tensor linear_147_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_375_cast_fp16)[name = string("linear_147_cast_fp16")]; + tensor input_377_cast_fp16 = add(x = linear_147_cast_fp16, y = x_643_cast_fp16)[name = string("input_377_cast_fp16")]; + tensor linear_148_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_377_cast_fp16)[name = string("linear_148_cast_fp16")]; + string input_381_mode_0 = const()[name = string("input_381_mode_0"), val = string("EXACT")]; + tensor input_381_cast_fp16 = gelu(mode = input_381_mode_0, x = linear_148_cast_fp16)[name = string("input_381_cast_fp16")]; + tensor linear_149_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_381_cast_fp16)[name = string("linear_149_cast_fp16")]; + tensor x_673_cast_fp16 = add(x = linear_149_cast_fp16, y = input_377_cast_fp16)[name = string("x_673_cast_fp16")]; + tensor var_4637_axes_0 = const()[name = string("op_4637_axes_0"), val = tensor([1])]; + bool var_4637_keep_dims_0 = const()[name = string("op_4637_keep_dims_0"), val = bool(false)]; + tensor var_4637_cast_fp16 = reduce_mean(axes = var_4637_axes_0, keep_dims = var_4637_keep_dims_0, x = x_673_cast_fp16)[name = string("op_4637_cast_fp16")]; + tensor x_675_axes_0 = const()[name = string("x_675_axes_0"), val = tensor([1])]; + tensor x_675_cast_fp16 = expand_dims(axes = x_675_axes_0, x = var_4637_cast_fp16)[name = string("x_675_cast_fp16")]; + tensor var_4639 = const()[name = string("op_4639"), val = tensor([0, 2, 1])]; + string x_677_pad_type_0 = const()[name = string("x_677_pad_type_0"), val = string("valid")]; + tensor x_677_strides_0 = const()[name = string("x_677_strides_0"), val = tensor([1])]; + tensor x_677_pad_0 = const()[name = string("x_677_pad_0"), val = tensor([0, 0])]; + tensor x_677_dilations_0 = const()[name = string("x_677_dilations_0"), val = tensor([1])]; + int32 x_677_groups_0 = const()[name = string("x_677_groups_0"), val = int32(1)]; + tensor input_383_cast_fp16 = transpose(perm = var_4639, x = x_675_cast_fp16)[name = string("transpose_175")]; + tensor x_677_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_677_dilations_0, groups = x_677_groups_0, pad = x_677_pad_0, pad_type = x_677_pad_type_0, strides = x_677_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_383_cast_fp16)[name = string("x_677_cast_fp16")]; + tensor x_pred_11_perm_0 = const()[name = string("x_pred_11_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_11_to_fp16 = const()[name = string("c_skip_11_to_fp16"), val = tensor([[[0x1.ef4p-1]]])]; + tensor var_4647_cast_fp16 = mul(x = c_skip_11_to_fp16, y = x_noisy_11_cast_fp16)[name = string("op_4647_cast_fp16")]; + tensor c_out_11_to_fp16 = const()[name = string("c_out_11_to_fp16"), val = tensor([[[0x1.1dp-5]]])]; + tensor x_pred_11_cast_fp16 = transpose(perm = x_pred_11_perm_0, x = x_677_cast_fp16)[name = string("transpose_174")]; + tensor var_4648_cast_fp16 = mul(x = c_out_11_to_fp16, y = x_pred_11_cast_fp16)[name = string("op_4648_cast_fp16")]; + tensor x_mid_dn_5_cast_fp16 = add(x = var_4647_cast_fp16, y = var_4648_cast_fp16)[name = string("x_mid_dn_5_cast_fp16")]; + tensor var_4651_cast_fp16 = sub(x = x_noisy_11_cast_fp16, y = x_mid_dn_5_cast_fp16)[name = string("op_4651_cast_fp16")]; + tensor _inversed_d_mid_5_y_0_to_fp16 = const()[name = string("_inversed_d_mid_5_y_0_to_fp16"), val = tensor([0x1.c4cp+4])]; + tensor _inversed_d_mid_5_cast_fp16 = mul(x = var_4651_cast_fp16, y = _inversed_d_mid_5_y_0_to_fp16)[name = string("_inversed_d_mid_5_cast_fp16")]; + fp16 var_4660_to_fp16 = const()[name = string("op_4660_to_fp16"), val = fp16(-0x1.1fp-4)]; + tensor var_4661_cast_fp16 = mul(x = _inversed_d_mid_5_cast_fp16, y = var_4660_to_fp16)[name = string("op_4661_cast_fp16")]; + tensor x_679_cast_fp16 = add(x = x_noisy_9_cast_fp16, y = var_4661_cast_fp16)[name = string("x_679_cast_fp16")]; + tensor var_4666_begin_0 = const()[name = string("op_4666_begin_0"), val = tensor([2, 0, 0, 0])]; + tensor var_4666_end_0 = const()[name = string("op_4666_end_0"), val = tensor([3, 1, 1, 256])]; + tensor var_4666_end_mask_0 = const()[name = string("op_4666_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_4666_squeeze_mask_0 = const()[name = string("op_4666_squeeze_mask_0"), val = tensor([true, false, false, false])]; + tensor var_4666_cast_fp16 = slice_by_index(begin = var_4666_begin_0, end = var_4666_end_0, end_mask = var_4666_end_mask_0, squeeze_mask = var_4666_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_4666_cast_fp16")]; + fp16 var_4669_to_fp16 = const()[name = string("op_4669_to_fp16"), val = fp16(0x1.37p-8)]; + tensor var_4670_cast_fp16 = mul(x = var_4666_cast_fp16, y = var_4669_to_fp16)[name = string("op_4670_cast_fp16")]; + tensor x_noisy_13_cast_fp16 = add(x = x_679_cast_fp16, y = var_4670_cast_fp16)[name = string("x_noisy_13_cast_fp16")]; + int32 var_4694 = const()[name = string("op_4694"), val = int32(-1)]; + tensor c_in_13_to_fp16 = const()[name = string("c_in_13_to_fp16"), val = tensor([[[0x1.41p+2]]])]; + tensor x_689_cast_fp16 = mul(x = c_in_13_to_fp16, y = x_noisy_13_cast_fp16)[name = string("x_689_cast_fp16")]; + int32 x_685_axis_0 = const()[name = string("x_685_axis_0"), val = int32(0)]; + tensor var_5080_to_fp16 = const()[name = string("op_5080_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49357632)))]; + tensor x_685_cast_fp16 = stack(axis = x_685_axis_0, values = (var_5080_to_fp16, var_423_cast_fp16))[name = string("x_685_cast_fp16")]; + tensor var_5085 = const()[name = string("op_5085"), val = tensor([1, 2, 0])]; + tensor input_391_axes_0 = const()[name = string("input_391_axes_0"), val = tensor([2])]; + bool input_391_keep_dims_0 = const()[name = string("input_391_keep_dims_0"), val = bool(false)]; + tensor x_687_cast_fp16 = transpose(perm = var_5085, x = x_685_cast_fp16)[name = string("transpose_173")]; + tensor input_391_cast_fp16 = reduce_sum(axes = input_391_axes_0, keep_dims = input_391_keep_dims_0, x = x_687_cast_fp16)[name = string("input_391_cast_fp16")]; + tensor linear_152_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_391_cast_fp16)[name = string("linear_152_cast_fp16")]; + string input_395_mode_0 = const()[name = string("input_395_mode_0"), val = string("EXACT")]; + tensor input_395_cast_fp16 = gelu(mode = input_395_mode_0, x = linear_152_cast_fp16)[name = string("input_395_cast_fp16")]; + tensor linear_153_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_395_cast_fp16)[name = string("linear_153_cast_fp16")]; + string mapping_25_mode_0 = const()[name = string("mapping_25_mode_0"), val = string("EXACT")]; + tensor mapping_25_cast_fp16 = gelu(mode = mapping_25_mode_0, x = linear_153_cast_fp16)[name = string("mapping_25_cast_fp16")]; + tensor var_5095_reps_0 = const()[name = string("op_5095_reps_0"), val = tensor([1, 256, 1])]; + tensor var_5095_cast_fp16 = tile(reps = var_5095_reps_0, x = x_689_cast_fp16)[name = string("op_5095_cast_fp16")]; + bool x_691_interleave_0 = const()[name = string("x_691_interleave_0"), val = bool(false)]; + tensor x_691_cast_fp16 = concat(axis = var_4694, interleave = x_691_interleave_0, values = (var_5095_cast_fp16, embedding_to_fp16))[name = string("x_691_cast_fp16")]; + tensor var_5098_axes_0 = const()[name = string("op_5098_axes_0"), val = tensor([1])]; + tensor var_5098_cast_fp16 = expand_dims(axes = var_5098_axes_0, x = mapping_25_cast_fp16)[name = string("op_5098_cast_fp16")]; + tensor mapping_27_reps_0 = const()[name = string("mapping_27_reps_0"), val = tensor([1, 256, 1])]; + tensor mapping_27_cast_fp16 = tile(reps = mapping_27_reps_0, x = var_5098_cast_fp16)[name = string("mapping_27_cast_fp16")]; + tensor x_693_cast_fp16 = add(x = x_691_cast_fp16, y = mapping_27_cast_fp16)[name = string("x_693_cast_fp16")]; + tensor var_5110_split_sizes_0 = const()[name = string("op_5110_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5110_axis_0 = const()[name = string("op_5110_axis_0"), val = int32(1)]; + tensor var_5110_cast_fp16_0, tensor var_5110_cast_fp16_1 = split(axis = var_5110_axis_0, split_sizes = var_5110_split_sizes_0, x = h_3_cast_fp16)[name = string("op_5110_cast_fp16")]; + tensor gamma_147_perm_0 = const()[name = string("gamma_147_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_147_perm_0 = const()[name = string("beta_147_perm_0"), val = tensor([0, -1, 1])]; + tensor x_697_axes_0 = const()[name = string("x_697_axes_0"), val = tensor([-1])]; + fp16 var_4690_to_fp16 = const()[name = string("op_4690_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_697_cast_fp16 = layer_norm(axes = x_697_axes_0, epsilon = var_4690_to_fp16, x = x_693_cast_fp16)[name = string("x_697_cast_fp16")]; + fp16 var_5116_promoted_to_fp16 = const()[name = string("op_5116_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_147_cast_fp16 = transpose(perm = gamma_147_perm_0, x = var_5110_cast_fp16_0)[name = string("transpose_172")]; + tensor var_5117_cast_fp16 = add(x = gamma_147_cast_fp16, y = var_5116_promoted_to_fp16)[name = string("op_5117_cast_fp16")]; + tensor var_5118_cast_fp16 = mul(x = var_5117_cast_fp16, y = x_697_cast_fp16)[name = string("op_5118_cast_fp16")]; + tensor beta_147_cast_fp16 = transpose(perm = beta_147_perm_0, x = var_5110_cast_fp16_1)[name = string("transpose_171")]; + tensor x_699_cast_fp16 = add(x = var_5118_cast_fp16, y = beta_147_cast_fp16)[name = string("x_699_cast_fp16")]; + tensor var_5129_split_sizes_0 = const()[name = string("op_5129_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5129_axis_0 = const()[name = string("op_5129_axis_0"), val = int32(1)]; + tensor var_5129_cast_fp16_0, tensor var_5129_cast_fp16_1 = split(axis = var_5129_axis_0, split_sizes = var_5129_split_sizes_0, x = h_7_cast_fp16)[name = string("op_5129_cast_fp16")]; + tensor gamma_151_perm_0 = const()[name = string("gamma_151_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_151_perm_0 = const()[name = string("beta_151_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5135_promoted_to_fp16 = const()[name = string("op_5135_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_151_cast_fp16 = transpose(perm = gamma_151_perm_0, x = var_5129_cast_fp16_0)[name = string("transpose_170")]; + tensor var_5136_cast_fp16 = add(x = gamma_151_cast_fp16, y = var_5135_promoted_to_fp16)[name = string("op_5136_cast_fp16")]; + tensor var_5137_cast_fp16 = mul(x = var_5136_cast_fp16, y = x_697_cast_fp16)[name = string("op_5137_cast_fp16")]; + tensor beta_151_cast_fp16 = transpose(perm = beta_151_perm_0, x = var_5129_cast_fp16_1)[name = string("transpose_169")]; + tensor x_705_cast_fp16 = add(x = var_5137_cast_fp16, y = beta_151_cast_fp16)[name = string("x_705_cast_fp16")]; + tensor linear_156_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_699_cast_fp16)[name = string("linear_156_cast_fp16")]; + tensor linear_157_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_705_cast_fp16)[name = string("linear_157_cast_fp16")]; + tensor var_5143_split_sizes_0 = const()[name = string("op_5143_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5143_axis_0 = const()[name = string("op_5143_axis_0"), val = int32(-1)]; + tensor var_5143_cast_fp16_0, tensor var_5143_cast_fp16_1 = split(axis = var_5143_axis_0, split_sizes = var_5143_split_sizes_0, x = linear_157_cast_fp16)[name = string("op_5143_cast_fp16")]; + tensor var_5151 = const()[name = string("op_5151"), val = tensor([1, 256, 8, 64])]; + tensor x_709_cast_fp16 = reshape(shape = var_5151, x = linear_156_cast_fp16)[name = string("x_709_cast_fp16")]; + tensor var_5161 = const()[name = string("op_5161"), val = tensor([1, 256, 8, 64])]; + tensor x_713_cast_fp16 = reshape(shape = var_5161, x = var_5143_cast_fp16_0)[name = string("x_713_cast_fp16")]; + tensor var_5171 = const()[name = string("op_5171"), val = tensor([1, 256, 8, 64])]; + tensor x_717_cast_fp16 = reshape(shape = var_5171, x = var_5143_cast_fp16_1)[name = string("x_717_cast_fp16")]; + tensor var_5173 = const()[name = string("op_5173"), val = tensor([0, 2, 1, 3])]; + bool sim_73_transpose_x_0 = const()[name = string("sim_73_transpose_x_0"), val = bool(false)]; + bool sim_73_transpose_y_0 = const()[name = string("sim_73_transpose_y_0"), val = bool(false)]; + tensor transpose_108_perm_0 = const()[name = string("transpose_108_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_109_perm_0 = const()[name = string("transpose_109_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_109 = transpose(perm = transpose_109_perm_0, x = x_713_cast_fp16)[name = string("transpose_166")]; + tensor transpose_108 = transpose(perm = transpose_108_perm_0, x = x_709_cast_fp16)[name = string("transpose_167")]; + tensor sim_73_cast_fp16 = matmul(transpose_x = sim_73_transpose_x_0, transpose_y = sim_73_transpose_y_0, x = transpose_108, y = transpose_109)[name = string("sim_73_cast_fp16")]; + fp16 var_5177_to_fp16 = const()[name = string("op_5177_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_75_cast_fp16 = mul(x = sim_73_cast_fp16, y = var_5177_to_fp16)[name = string("sim_75_cast_fp16")]; + tensor attn_37_cast_fp16 = softmax(axis = var_4694, x = sim_75_cast_fp16)[name = string("attn_37_cast_fp16")]; + bool x_719_transpose_x_0 = const()[name = string("x_719_transpose_x_0"), val = bool(false)]; + bool x_719_transpose_y_0 = const()[name = string("x_719_transpose_y_0"), val = bool(false)]; + tensor v_37_cast_fp16 = transpose(perm = var_5173, x = x_717_cast_fp16)[name = string("transpose_168")]; + tensor x_719_cast_fp16 = matmul(transpose_x = x_719_transpose_x_0, transpose_y = x_719_transpose_y_0, x = attn_37_cast_fp16, y = v_37_cast_fp16)[name = string("x_719_cast_fp16")]; + tensor var_5199 = const()[name = string("op_5199"), val = tensor([0, 2, 1, 3])]; + tensor var_5201 = const()[name = string("op_5201"), val = tensor([1, 256, 512])]; + tensor x_721_cast_fp16 = transpose(perm = var_5199, x = x_719_cast_fp16)[name = string("transpose_165")]; + tensor input_407_cast_fp16 = reshape(shape = var_5201, x = x_721_cast_fp16)[name = string("input_407_cast_fp16")]; + tensor linear_158_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_407_cast_fp16)[name = string("linear_158_cast_fp16")]; + tensor input_409_cast_fp16 = add(x = linear_158_cast_fp16, y = x_693_cast_fp16)[name = string("input_409_cast_fp16")]; + tensor linear_159_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_409_cast_fp16)[name = string("linear_159_cast_fp16")]; + string input_413_mode_0 = const()[name = string("input_413_mode_0"), val = string("EXACT")]; + tensor input_413_cast_fp16 = gelu(mode = input_413_mode_0, x = linear_159_cast_fp16)[name = string("input_413_cast_fp16")]; + tensor linear_160_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_413_cast_fp16)[name = string("linear_160_cast_fp16")]; + tensor x_723_cast_fp16 = add(x = linear_160_cast_fp16, y = input_409_cast_fp16)[name = string("x_723_cast_fp16")]; + tensor x_725_cast_fp16 = add(x = x_723_cast_fp16, y = mapping_27_cast_fp16)[name = string("x_725_cast_fp16")]; + tensor var_5217_split_sizes_0 = const()[name = string("op_5217_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5217_axis_0 = const()[name = string("op_5217_axis_0"), val = int32(1)]; + tensor var_5217_cast_fp16_0, tensor var_5217_cast_fp16_1 = split(axis = var_5217_axis_0, split_sizes = var_5217_split_sizes_0, x = h_11_cast_fp16)[name = string("op_5217_cast_fp16")]; + tensor gamma_155_perm_0 = const()[name = string("gamma_155_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_155_perm_0 = const()[name = string("beta_155_perm_0"), val = tensor([0, -1, 1])]; + tensor x_729_axes_0 = const()[name = string("x_729_axes_0"), val = tensor([-1])]; + tensor x_729_cast_fp16 = layer_norm(axes = x_729_axes_0, epsilon = var_4690_to_fp16, x = x_725_cast_fp16)[name = string("x_729_cast_fp16")]; + fp16 var_5223_promoted_to_fp16 = const()[name = string("op_5223_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_155_cast_fp16 = transpose(perm = gamma_155_perm_0, x = var_5217_cast_fp16_0)[name = string("transpose_164")]; + tensor var_5224_cast_fp16 = add(x = gamma_155_cast_fp16, y = var_5223_promoted_to_fp16)[name = string("op_5224_cast_fp16")]; + tensor var_5225_cast_fp16 = mul(x = var_5224_cast_fp16, y = x_729_cast_fp16)[name = string("op_5225_cast_fp16")]; + tensor beta_155_cast_fp16 = transpose(perm = beta_155_perm_0, x = var_5217_cast_fp16_1)[name = string("transpose_163")]; + tensor x_731_cast_fp16 = add(x = var_5225_cast_fp16, y = beta_155_cast_fp16)[name = string("x_731_cast_fp16")]; + tensor var_5236_split_sizes_0 = const()[name = string("op_5236_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5236_axis_0 = const()[name = string("op_5236_axis_0"), val = int32(1)]; + tensor var_5236_cast_fp16_0, tensor var_5236_cast_fp16_1 = split(axis = var_5236_axis_0, split_sizes = var_5236_split_sizes_0, x = h_15_cast_fp16)[name = string("op_5236_cast_fp16")]; + tensor gamma_159_perm_0 = const()[name = string("gamma_159_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_159_perm_0 = const()[name = string("beta_159_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5242_promoted_to_fp16 = const()[name = string("op_5242_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_159_cast_fp16 = transpose(perm = gamma_159_perm_0, x = var_5236_cast_fp16_0)[name = string("transpose_162")]; + tensor var_5243_cast_fp16 = add(x = gamma_159_cast_fp16, y = var_5242_promoted_to_fp16)[name = string("op_5243_cast_fp16")]; + tensor var_5244_cast_fp16 = mul(x = var_5243_cast_fp16, y = x_729_cast_fp16)[name = string("op_5244_cast_fp16")]; + tensor beta_159_cast_fp16 = transpose(perm = beta_159_perm_0, x = var_5236_cast_fp16_1)[name = string("transpose_161")]; + tensor x_737_cast_fp16 = add(x = var_5244_cast_fp16, y = beta_159_cast_fp16)[name = string("x_737_cast_fp16")]; + tensor linear_163_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_731_cast_fp16)[name = string("linear_163_cast_fp16")]; + tensor linear_164_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_737_cast_fp16)[name = string("linear_164_cast_fp16")]; + tensor var_5250_split_sizes_0 = const()[name = string("op_5250_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5250_axis_0 = const()[name = string("op_5250_axis_0"), val = int32(-1)]; + tensor var_5250_cast_fp16_0, tensor var_5250_cast_fp16_1 = split(axis = var_5250_axis_0, split_sizes = var_5250_split_sizes_0, x = linear_164_cast_fp16)[name = string("op_5250_cast_fp16")]; + tensor var_5258 = const()[name = string("op_5258"), val = tensor([1, 256, 8, 64])]; + tensor x_741_cast_fp16 = reshape(shape = var_5258, x = linear_163_cast_fp16)[name = string("x_741_cast_fp16")]; + tensor var_5268 = const()[name = string("op_5268"), val = tensor([1, 256, 8, 64])]; + tensor x_745_cast_fp16 = reshape(shape = var_5268, x = var_5250_cast_fp16_0)[name = string("x_745_cast_fp16")]; + tensor var_5278 = const()[name = string("op_5278"), val = tensor([1, 256, 8, 64])]; + tensor x_749_cast_fp16 = reshape(shape = var_5278, x = var_5250_cast_fp16_1)[name = string("x_749_cast_fp16")]; + tensor var_5280 = const()[name = string("op_5280"), val = tensor([0, 2, 1, 3])]; + bool sim_77_transpose_x_0 = const()[name = string("sim_77_transpose_x_0"), val = bool(false)]; + bool sim_77_transpose_y_0 = const()[name = string("sim_77_transpose_y_0"), val = bool(false)]; + tensor transpose_110_perm_0 = const()[name = string("transpose_110_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_111_perm_0 = const()[name = string("transpose_111_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_111 = transpose(perm = transpose_111_perm_0, x = x_745_cast_fp16)[name = string("transpose_158")]; + tensor transpose_110 = transpose(perm = transpose_110_perm_0, x = x_741_cast_fp16)[name = string("transpose_159")]; + tensor sim_77_cast_fp16 = matmul(transpose_x = sim_77_transpose_x_0, transpose_y = sim_77_transpose_y_0, x = transpose_110, y = transpose_111)[name = string("sim_77_cast_fp16")]; + fp16 var_5284_to_fp16 = const()[name = string("op_5284_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_79_cast_fp16 = mul(x = sim_77_cast_fp16, y = var_5284_to_fp16)[name = string("sim_79_cast_fp16")]; + tensor attn_39_cast_fp16 = softmax(axis = var_4694, x = sim_79_cast_fp16)[name = string("attn_39_cast_fp16")]; + bool x_751_transpose_x_0 = const()[name = string("x_751_transpose_x_0"), val = bool(false)]; + bool x_751_transpose_y_0 = const()[name = string("x_751_transpose_y_0"), val = bool(false)]; + tensor v_39_cast_fp16 = transpose(perm = var_5280, x = x_749_cast_fp16)[name = string("transpose_160")]; + tensor x_751_cast_fp16 = matmul(transpose_x = x_751_transpose_x_0, transpose_y = x_751_transpose_y_0, x = attn_39_cast_fp16, y = v_39_cast_fp16)[name = string("x_751_cast_fp16")]; + tensor var_5306 = const()[name = string("op_5306"), val = tensor([0, 2, 1, 3])]; + tensor var_5308 = const()[name = string("op_5308"), val = tensor([1, 256, 512])]; + tensor x_753_cast_fp16 = transpose(perm = var_5306, x = x_751_cast_fp16)[name = string("transpose_157")]; + tensor input_423_cast_fp16 = reshape(shape = var_5308, x = x_753_cast_fp16)[name = string("input_423_cast_fp16")]; + tensor linear_165_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_423_cast_fp16)[name = string("linear_165_cast_fp16")]; + tensor input_425_cast_fp16 = add(x = linear_165_cast_fp16, y = x_725_cast_fp16)[name = string("input_425_cast_fp16")]; + tensor linear_166_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_425_cast_fp16)[name = string("linear_166_cast_fp16")]; + string input_429_mode_0 = const()[name = string("input_429_mode_0"), val = string("EXACT")]; + tensor input_429_cast_fp16 = gelu(mode = input_429_mode_0, x = linear_166_cast_fp16)[name = string("input_429_cast_fp16")]; + tensor linear_167_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_429_cast_fp16)[name = string("linear_167_cast_fp16")]; + tensor x_755_cast_fp16 = add(x = linear_167_cast_fp16, y = input_425_cast_fp16)[name = string("x_755_cast_fp16")]; + tensor x_757_cast_fp16 = add(x = x_755_cast_fp16, y = mapping_27_cast_fp16)[name = string("x_757_cast_fp16")]; + tensor var_5324_split_sizes_0 = const()[name = string("op_5324_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5324_axis_0 = const()[name = string("op_5324_axis_0"), val = int32(1)]; + tensor var_5324_cast_fp16_0, tensor var_5324_cast_fp16_1 = split(axis = var_5324_axis_0, split_sizes = var_5324_split_sizes_0, x = h_19_cast_fp16)[name = string("op_5324_cast_fp16")]; + tensor gamma_163_perm_0 = const()[name = string("gamma_163_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_163_perm_0 = const()[name = string("beta_163_perm_0"), val = tensor([0, -1, 1])]; + tensor x_761_axes_0 = const()[name = string("x_761_axes_0"), val = tensor([-1])]; + tensor x_761_cast_fp16 = layer_norm(axes = x_761_axes_0, epsilon = var_4690_to_fp16, x = x_757_cast_fp16)[name = string("x_761_cast_fp16")]; + fp16 var_5330_promoted_to_fp16 = const()[name = string("op_5330_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_163_cast_fp16 = transpose(perm = gamma_163_perm_0, x = var_5324_cast_fp16_0)[name = string("transpose_156")]; + tensor var_5331_cast_fp16 = add(x = gamma_163_cast_fp16, y = var_5330_promoted_to_fp16)[name = string("op_5331_cast_fp16")]; + tensor var_5332_cast_fp16 = mul(x = var_5331_cast_fp16, y = x_761_cast_fp16)[name = string("op_5332_cast_fp16")]; + tensor beta_163_cast_fp16 = transpose(perm = beta_163_perm_0, x = var_5324_cast_fp16_1)[name = string("transpose_155")]; + tensor x_763_cast_fp16 = add(x = var_5332_cast_fp16, y = beta_163_cast_fp16)[name = string("x_763_cast_fp16")]; + tensor var_5343_split_sizes_0 = const()[name = string("op_5343_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5343_axis_0 = const()[name = string("op_5343_axis_0"), val = int32(1)]; + tensor var_5343_cast_fp16_0, tensor var_5343_cast_fp16_1 = split(axis = var_5343_axis_0, split_sizes = var_5343_split_sizes_0, x = h_23_cast_fp16)[name = string("op_5343_cast_fp16")]; + tensor gamma_167_perm_0 = const()[name = string("gamma_167_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_167_perm_0 = const()[name = string("beta_167_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5349_promoted_to_fp16 = const()[name = string("op_5349_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_167_cast_fp16 = transpose(perm = gamma_167_perm_0, x = var_5343_cast_fp16_0)[name = string("transpose_154")]; + tensor var_5350_cast_fp16 = add(x = gamma_167_cast_fp16, y = var_5349_promoted_to_fp16)[name = string("op_5350_cast_fp16")]; + tensor var_5351_cast_fp16 = mul(x = var_5350_cast_fp16, y = x_761_cast_fp16)[name = string("op_5351_cast_fp16")]; + tensor beta_167_cast_fp16 = transpose(perm = beta_167_perm_0, x = var_5343_cast_fp16_1)[name = string("transpose_153")]; + tensor x_769_cast_fp16 = add(x = var_5351_cast_fp16, y = beta_167_cast_fp16)[name = string("x_769_cast_fp16")]; + tensor linear_170_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_763_cast_fp16)[name = string("linear_170_cast_fp16")]; + tensor linear_171_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_769_cast_fp16)[name = string("linear_171_cast_fp16")]; + tensor var_5357_split_sizes_0 = const()[name = string("op_5357_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5357_axis_0 = const()[name = string("op_5357_axis_0"), val = int32(-1)]; + tensor var_5357_cast_fp16_0, tensor var_5357_cast_fp16_1 = split(axis = var_5357_axis_0, split_sizes = var_5357_split_sizes_0, x = linear_171_cast_fp16)[name = string("op_5357_cast_fp16")]; + tensor var_5365 = const()[name = string("op_5365"), val = tensor([1, 256, 8, 64])]; + tensor x_773_cast_fp16 = reshape(shape = var_5365, x = linear_170_cast_fp16)[name = string("x_773_cast_fp16")]; + tensor var_5375 = const()[name = string("op_5375"), val = tensor([1, 256, 8, 64])]; + tensor x_777_cast_fp16 = reshape(shape = var_5375, x = var_5357_cast_fp16_0)[name = string("x_777_cast_fp16")]; + tensor var_5385 = const()[name = string("op_5385"), val = tensor([1, 256, 8, 64])]; + tensor x_781_cast_fp16 = reshape(shape = var_5385, x = var_5357_cast_fp16_1)[name = string("x_781_cast_fp16")]; + tensor var_5387 = const()[name = string("op_5387"), val = tensor([0, 2, 1, 3])]; + bool sim_81_transpose_x_0 = const()[name = string("sim_81_transpose_x_0"), val = bool(false)]; + bool sim_81_transpose_y_0 = const()[name = string("sim_81_transpose_y_0"), val = bool(false)]; + tensor transpose_112_perm_0 = const()[name = string("transpose_112_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_113_perm_0 = const()[name = string("transpose_113_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_113 = transpose(perm = transpose_113_perm_0, x = x_777_cast_fp16)[name = string("transpose_150")]; + tensor transpose_112 = transpose(perm = transpose_112_perm_0, x = x_773_cast_fp16)[name = string("transpose_151")]; + tensor sim_81_cast_fp16 = matmul(transpose_x = sim_81_transpose_x_0, transpose_y = sim_81_transpose_y_0, x = transpose_112, y = transpose_113)[name = string("sim_81_cast_fp16")]; + fp16 var_5391_to_fp16 = const()[name = string("op_5391_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_83_cast_fp16 = mul(x = sim_81_cast_fp16, y = var_5391_to_fp16)[name = string("sim_83_cast_fp16")]; + tensor attn_41_cast_fp16 = softmax(axis = var_4694, x = sim_83_cast_fp16)[name = string("attn_41_cast_fp16")]; + bool x_783_transpose_x_0 = const()[name = string("x_783_transpose_x_0"), val = bool(false)]; + bool x_783_transpose_y_0 = const()[name = string("x_783_transpose_y_0"), val = bool(false)]; + tensor v_41_cast_fp16 = transpose(perm = var_5387, x = x_781_cast_fp16)[name = string("transpose_152")]; + tensor x_783_cast_fp16 = matmul(transpose_x = x_783_transpose_x_0, transpose_y = x_783_transpose_y_0, x = attn_41_cast_fp16, y = v_41_cast_fp16)[name = string("x_783_cast_fp16")]; + tensor var_5413 = const()[name = string("op_5413"), val = tensor([0, 2, 1, 3])]; + tensor var_5415 = const()[name = string("op_5415"), val = tensor([1, 256, 512])]; + tensor x_785_cast_fp16 = transpose(perm = var_5413, x = x_783_cast_fp16)[name = string("transpose_149")]; + tensor input_439_cast_fp16 = reshape(shape = var_5415, x = x_785_cast_fp16)[name = string("input_439_cast_fp16")]; + tensor linear_172_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_439_cast_fp16)[name = string("linear_172_cast_fp16")]; + tensor input_441_cast_fp16 = add(x = linear_172_cast_fp16, y = x_757_cast_fp16)[name = string("input_441_cast_fp16")]; + tensor linear_173_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_441_cast_fp16)[name = string("linear_173_cast_fp16")]; + string input_445_mode_0 = const()[name = string("input_445_mode_0"), val = string("EXACT")]; + tensor input_445_cast_fp16 = gelu(mode = input_445_mode_0, x = linear_173_cast_fp16)[name = string("input_445_cast_fp16")]; + tensor linear_174_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_445_cast_fp16)[name = string("linear_174_cast_fp16")]; + tensor x_787_cast_fp16 = add(x = linear_174_cast_fp16, y = input_441_cast_fp16)[name = string("x_787_cast_fp16")]; + tensor var_5424_axes_0 = const()[name = string("op_5424_axes_0"), val = tensor([1])]; + bool var_5424_keep_dims_0 = const()[name = string("op_5424_keep_dims_0"), val = bool(false)]; + tensor var_5424_cast_fp16 = reduce_mean(axes = var_5424_axes_0, keep_dims = var_5424_keep_dims_0, x = x_787_cast_fp16)[name = string("op_5424_cast_fp16")]; + tensor x_789_axes_0 = const()[name = string("x_789_axes_0"), val = tensor([1])]; + tensor x_789_cast_fp16 = expand_dims(axes = x_789_axes_0, x = var_5424_cast_fp16)[name = string("x_789_cast_fp16")]; + tensor var_5426 = const()[name = string("op_5426"), val = tensor([0, 2, 1])]; + string x_791_pad_type_0 = const()[name = string("x_791_pad_type_0"), val = string("valid")]; + tensor x_791_strides_0 = const()[name = string("x_791_strides_0"), val = tensor([1])]; + tensor x_791_pad_0 = const()[name = string("x_791_pad_0"), val = tensor([0, 0])]; + tensor x_791_dilations_0 = const()[name = string("x_791_dilations_0"), val = tensor([1])]; + int32 x_791_groups_0 = const()[name = string("x_791_groups_0"), val = int32(1)]; + tensor input_447_cast_fp16 = transpose(perm = var_5426, x = x_789_cast_fp16)[name = string("transpose_148")]; + tensor x_791_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_791_dilations_0, groups = x_791_groups_0, pad = x_791_pad_0, pad_type = x_791_pad_type_0, strides = x_791_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_447_cast_fp16)[name = string("x_791_cast_fp16")]; + tensor x_pred_13_perm_0 = const()[name = string("x_pred_13_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_13_to_fp16 = const()[name = string("c_skip_13_to_fp16"), val = tensor([[[0x1.fe8p-1]]])]; + tensor var_5434_cast_fp16 = mul(x = c_skip_13_to_fp16, y = x_noisy_13_cast_fp16)[name = string("op_5434_cast_fp16")]; + tensor c_out_13_to_fp16 = const()[name = string("c_out_13_to_fp16"), val = tensor([[[0x1.37cp-8]]])]; + tensor x_pred_13_cast_fp16 = transpose(perm = x_pred_13_perm_0, x = x_791_cast_fp16)[name = string("transpose_147")]; + tensor var_5435_cast_fp16 = mul(x = c_out_13_to_fp16, y = x_pred_13_cast_fp16)[name = string("op_5435_cast_fp16")]; + tensor x_dn_cast_fp16 = add(x = var_5434_cast_fp16, y = var_5435_cast_fp16)[name = string("x_dn_cast_fp16")]; + tensor var_5438_cast_fp16 = sub(x = x_noisy_13_cast_fp16, y = x_dn_cast_fp16)[name = string("op_5438_cast_fp16")]; + tensor _inversed_d_y_0_to_fp16 = const()[name = string("_inversed_d_y_0_to_fp16"), val = tensor([0x1.a44p+7])]; + tensor _inversed_d_cast_fp16 = mul(x = var_5438_cast_fp16, y = _inversed_d_y_0_to_fp16)[name = string("_inversed_d_cast_fp16")]; + fp16 var_5447_to_fp16 = const()[name = string("op_5447_to_fp16"), val = fp16(-0x1.37cp-9)]; + tensor var_5448_cast_fp16 = mul(x = _inversed_d_cast_fp16, y = var_5447_to_fp16)[name = string("op_5448_cast_fp16")]; + tensor x_noisy_cast_fp16 = add(x = x_noisy_13_cast_fp16, y = var_5448_cast_fp16)[name = string("x_noisy_cast_fp16")]; + int32 var_5460 = const()[name = string("op_5460"), val = int32(-1)]; + tensor c_in_to_fp16 = const()[name = string("c_in_to_fp16"), val = tensor([[[0x1.414p+2]]])]; + tensor x_801_cast_fp16 = mul(x = c_in_to_fp16, y = x_noisy_cast_fp16)[name = string("x_801_cast_fp16")]; + int32 x_797_axis_0 = const()[name = string("x_797_axis_0"), val = int32(0)]; + tensor var_5846_to_fp16 = const()[name = string("op_5846_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49359744)))]; + tensor x_797_cast_fp16 = stack(axis = x_797_axis_0, values = (var_5846_to_fp16, var_423_cast_fp16))[name = string("x_797_cast_fp16")]; + tensor var_5851 = const()[name = string("op_5851"), val = tensor([1, 2, 0])]; + tensor input_455_axes_0 = const()[name = string("input_455_axes_0"), val = tensor([2])]; + bool input_455_keep_dims_0 = const()[name = string("input_455_keep_dims_0"), val = bool(false)]; + tensor x_799_cast_fp16 = transpose(perm = var_5851, x = x_797_cast_fp16)[name = string("transpose_146")]; + tensor input_455_cast_fp16 = reduce_sum(axes = input_455_axes_0, keep_dims = input_455_keep_dims_0, x = x_799_cast_fp16)[name = string("input_455_cast_fp16")]; + tensor linear_177_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_455_cast_fp16)[name = string("linear_177_cast_fp16")]; + string input_459_mode_0 = const()[name = string("input_459_mode_0"), val = string("EXACT")]; + tensor input_459_cast_fp16 = gelu(mode = input_459_mode_0, x = linear_177_cast_fp16)[name = string("input_459_cast_fp16")]; + tensor linear_178_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_459_cast_fp16)[name = string("linear_178_cast_fp16")]; + string mapping_29_mode_0 = const()[name = string("mapping_29_mode_0"), val = string("EXACT")]; + tensor mapping_29_cast_fp16 = gelu(mode = mapping_29_mode_0, x = linear_178_cast_fp16)[name = string("mapping_29_cast_fp16")]; + tensor var_5861_reps_0 = const()[name = string("op_5861_reps_0"), val = tensor([1, 256, 1])]; + tensor var_5861_cast_fp16 = tile(reps = var_5861_reps_0, x = x_801_cast_fp16)[name = string("op_5861_cast_fp16")]; + bool x_803_interleave_0 = const()[name = string("x_803_interleave_0"), val = bool(false)]; + tensor x_803_cast_fp16 = concat(axis = var_5460, interleave = x_803_interleave_0, values = (var_5861_cast_fp16, embedding_to_fp16))[name = string("x_803_cast_fp16")]; + tensor var_5864_axes_0 = const()[name = string("op_5864_axes_0"), val = tensor([1])]; + tensor var_5864_cast_fp16 = expand_dims(axes = var_5864_axes_0, x = mapping_29_cast_fp16)[name = string("op_5864_cast_fp16")]; + tensor mapping_reps_0 = const()[name = string("mapping_reps_0"), val = tensor([1, 256, 1])]; + tensor mapping_cast_fp16 = tile(reps = mapping_reps_0, x = var_5864_cast_fp16)[name = string("mapping_cast_fp16")]; + tensor x_805_cast_fp16 = add(x = x_803_cast_fp16, y = mapping_cast_fp16)[name = string("x_805_cast_fp16")]; + tensor var_5876_split_sizes_0 = const()[name = string("op_5876_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5876_axis_0 = const()[name = string("op_5876_axis_0"), val = int32(1)]; + tensor var_5876_cast_fp16_0, tensor var_5876_cast_fp16_1 = split(axis = var_5876_axis_0, split_sizes = var_5876_split_sizes_0, x = h_3_cast_fp16)[name = string("op_5876_cast_fp16")]; + tensor gamma_171_perm_0 = const()[name = string("gamma_171_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_171_perm_0 = const()[name = string("beta_171_perm_0"), val = tensor([0, -1, 1])]; + tensor x_809_axes_0 = const()[name = string("x_809_axes_0"), val = tensor([-1])]; + fp16 var_5456_to_fp16 = const()[name = string("op_5456_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_809_cast_fp16 = layer_norm(axes = x_809_axes_0, epsilon = var_5456_to_fp16, x = x_805_cast_fp16)[name = string("x_809_cast_fp16")]; + fp16 var_5882_promoted_to_fp16 = const()[name = string("op_5882_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_171_cast_fp16 = transpose(perm = gamma_171_perm_0, x = var_5876_cast_fp16_0)[name = string("transpose_145")]; + tensor var_5883_cast_fp16 = add(x = gamma_171_cast_fp16, y = var_5882_promoted_to_fp16)[name = string("op_5883_cast_fp16")]; + tensor var_5884_cast_fp16 = mul(x = var_5883_cast_fp16, y = x_809_cast_fp16)[name = string("op_5884_cast_fp16")]; + tensor beta_171_cast_fp16 = transpose(perm = beta_171_perm_0, x = var_5876_cast_fp16_1)[name = string("transpose_144")]; + tensor x_811_cast_fp16 = add(x = var_5884_cast_fp16, y = beta_171_cast_fp16)[name = string("x_811_cast_fp16")]; + tensor var_5895_split_sizes_0 = const()[name = string("op_5895_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5895_axis_0 = const()[name = string("op_5895_axis_0"), val = int32(1)]; + tensor var_5895_cast_fp16_0, tensor var_5895_cast_fp16_1 = split(axis = var_5895_axis_0, split_sizes = var_5895_split_sizes_0, x = h_7_cast_fp16)[name = string("op_5895_cast_fp16")]; + tensor gamma_175_perm_0 = const()[name = string("gamma_175_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_175_perm_0 = const()[name = string("beta_175_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5901_promoted_to_fp16 = const()[name = string("op_5901_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_175_cast_fp16 = transpose(perm = gamma_175_perm_0, x = var_5895_cast_fp16_0)[name = string("transpose_143")]; + tensor var_5902_cast_fp16 = add(x = gamma_175_cast_fp16, y = var_5901_promoted_to_fp16)[name = string("op_5902_cast_fp16")]; + tensor var_5903_cast_fp16 = mul(x = var_5902_cast_fp16, y = x_809_cast_fp16)[name = string("op_5903_cast_fp16")]; + tensor beta_175_cast_fp16 = transpose(perm = beta_175_perm_0, x = var_5895_cast_fp16_1)[name = string("transpose_142")]; + tensor x_817_cast_fp16 = add(x = var_5903_cast_fp16, y = beta_175_cast_fp16)[name = string("x_817_cast_fp16")]; + tensor linear_181_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_811_cast_fp16)[name = string("linear_181_cast_fp16")]; + tensor linear_182_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_817_cast_fp16)[name = string("linear_182_cast_fp16")]; + tensor var_5909_split_sizes_0 = const()[name = string("op_5909_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5909_axis_0 = const()[name = string("op_5909_axis_0"), val = int32(-1)]; + tensor var_5909_cast_fp16_0, tensor var_5909_cast_fp16_1 = split(axis = var_5909_axis_0, split_sizes = var_5909_split_sizes_0, x = linear_182_cast_fp16)[name = string("op_5909_cast_fp16")]; + tensor var_5917 = const()[name = string("op_5917"), val = tensor([1, 256, 8, 64])]; + tensor x_821_cast_fp16 = reshape(shape = var_5917, x = linear_181_cast_fp16)[name = string("x_821_cast_fp16")]; + tensor var_5927 = const()[name = string("op_5927"), val = tensor([1, 256, 8, 64])]; + tensor x_825_cast_fp16 = reshape(shape = var_5927, x = var_5909_cast_fp16_0)[name = string("x_825_cast_fp16")]; + tensor var_5937 = const()[name = string("op_5937"), val = tensor([1, 256, 8, 64])]; + tensor x_829_cast_fp16 = reshape(shape = var_5937, x = var_5909_cast_fp16_1)[name = string("x_829_cast_fp16")]; + tensor var_5939 = const()[name = string("op_5939"), val = tensor([0, 2, 1, 3])]; + bool sim_85_transpose_x_0 = const()[name = string("sim_85_transpose_x_0"), val = bool(false)]; + bool sim_85_transpose_y_0 = const()[name = string("sim_85_transpose_y_0"), val = bool(false)]; + tensor transpose_114_perm_0 = const()[name = string("transpose_114_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_115_perm_0 = const()[name = string("transpose_115_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_115 = transpose(perm = transpose_115_perm_0, x = x_825_cast_fp16)[name = string("transpose_139")]; + tensor transpose_114 = transpose(perm = transpose_114_perm_0, x = x_821_cast_fp16)[name = string("transpose_140")]; + tensor sim_85_cast_fp16 = matmul(transpose_x = sim_85_transpose_x_0, transpose_y = sim_85_transpose_y_0, x = transpose_114, y = transpose_115)[name = string("sim_85_cast_fp16")]; + fp16 var_5943_to_fp16 = const()[name = string("op_5943_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_87_cast_fp16 = mul(x = sim_85_cast_fp16, y = var_5943_to_fp16)[name = string("sim_87_cast_fp16")]; + tensor attn_43_cast_fp16 = softmax(axis = var_5460, x = sim_87_cast_fp16)[name = string("attn_43_cast_fp16")]; + bool x_831_transpose_x_0 = const()[name = string("x_831_transpose_x_0"), val = bool(false)]; + bool x_831_transpose_y_0 = const()[name = string("x_831_transpose_y_0"), val = bool(false)]; + tensor v_43_cast_fp16 = transpose(perm = var_5939, x = x_829_cast_fp16)[name = string("transpose_141")]; + tensor x_831_cast_fp16 = matmul(transpose_x = x_831_transpose_x_0, transpose_y = x_831_transpose_y_0, x = attn_43_cast_fp16, y = v_43_cast_fp16)[name = string("x_831_cast_fp16")]; + tensor var_5965 = const()[name = string("op_5965"), val = tensor([0, 2, 1, 3])]; + tensor var_5967 = const()[name = string("op_5967"), val = tensor([1, 256, 512])]; + tensor x_833_cast_fp16 = transpose(perm = var_5965, x = x_831_cast_fp16)[name = string("transpose_138")]; + tensor input_471_cast_fp16 = reshape(shape = var_5967, x = x_833_cast_fp16)[name = string("input_471_cast_fp16")]; + tensor linear_183_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_471_cast_fp16)[name = string("linear_183_cast_fp16")]; + tensor input_473_cast_fp16 = add(x = linear_183_cast_fp16, y = x_805_cast_fp16)[name = string("input_473_cast_fp16")]; + tensor linear_184_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_473_cast_fp16)[name = string("linear_184_cast_fp16")]; + string input_477_mode_0 = const()[name = string("input_477_mode_0"), val = string("EXACT")]; + tensor input_477_cast_fp16 = gelu(mode = input_477_mode_0, x = linear_184_cast_fp16)[name = string("input_477_cast_fp16")]; + tensor linear_185_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_477_cast_fp16)[name = string("linear_185_cast_fp16")]; + tensor x_835_cast_fp16 = add(x = linear_185_cast_fp16, y = input_473_cast_fp16)[name = string("x_835_cast_fp16")]; + tensor x_837_cast_fp16 = add(x = x_835_cast_fp16, y = mapping_cast_fp16)[name = string("x_837_cast_fp16")]; + tensor var_5983_split_sizes_0 = const()[name = string("op_5983_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5983_axis_0 = const()[name = string("op_5983_axis_0"), val = int32(1)]; + tensor var_5983_cast_fp16_0, tensor var_5983_cast_fp16_1 = split(axis = var_5983_axis_0, split_sizes = var_5983_split_sizes_0, x = h_11_cast_fp16)[name = string("op_5983_cast_fp16")]; + tensor gamma_179_perm_0 = const()[name = string("gamma_179_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_179_perm_0 = const()[name = string("beta_179_perm_0"), val = tensor([0, -1, 1])]; + tensor x_841_axes_0 = const()[name = string("x_841_axes_0"), val = tensor([-1])]; + tensor x_841_cast_fp16 = layer_norm(axes = x_841_axes_0, epsilon = var_5456_to_fp16, x = x_837_cast_fp16)[name = string("x_841_cast_fp16")]; + fp16 var_5989_promoted_to_fp16 = const()[name = string("op_5989_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_179_cast_fp16 = transpose(perm = gamma_179_perm_0, x = var_5983_cast_fp16_0)[name = string("transpose_137")]; + tensor var_5990_cast_fp16 = add(x = gamma_179_cast_fp16, y = var_5989_promoted_to_fp16)[name = string("op_5990_cast_fp16")]; + tensor var_5991_cast_fp16 = mul(x = var_5990_cast_fp16, y = x_841_cast_fp16)[name = string("op_5991_cast_fp16")]; + tensor beta_179_cast_fp16 = transpose(perm = beta_179_perm_0, x = var_5983_cast_fp16_1)[name = string("transpose_136")]; + tensor x_843_cast_fp16 = add(x = var_5991_cast_fp16, y = beta_179_cast_fp16)[name = string("x_843_cast_fp16")]; + tensor var_6002_split_sizes_0 = const()[name = string("op_6002_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_6002_axis_0 = const()[name = string("op_6002_axis_0"), val = int32(1)]; + tensor var_6002_cast_fp16_0, tensor var_6002_cast_fp16_1 = split(axis = var_6002_axis_0, split_sizes = var_6002_split_sizes_0, x = h_15_cast_fp16)[name = string("op_6002_cast_fp16")]; + tensor gamma_183_perm_0 = const()[name = string("gamma_183_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_183_perm_0 = const()[name = string("beta_183_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_6008_promoted_to_fp16 = const()[name = string("op_6008_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_183_cast_fp16 = transpose(perm = gamma_183_perm_0, x = var_6002_cast_fp16_0)[name = string("transpose_135")]; + tensor var_6009_cast_fp16 = add(x = gamma_183_cast_fp16, y = var_6008_promoted_to_fp16)[name = string("op_6009_cast_fp16")]; + tensor var_6010_cast_fp16 = mul(x = var_6009_cast_fp16, y = x_841_cast_fp16)[name = string("op_6010_cast_fp16")]; + tensor beta_183_cast_fp16 = transpose(perm = beta_183_perm_0, x = var_6002_cast_fp16_1)[name = string("transpose_134")]; + tensor x_849_cast_fp16 = add(x = var_6010_cast_fp16, y = beta_183_cast_fp16)[name = string("x_849_cast_fp16")]; + tensor linear_188_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_843_cast_fp16)[name = string("linear_188_cast_fp16")]; + tensor linear_189_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_849_cast_fp16)[name = string("linear_189_cast_fp16")]; + tensor var_6016_split_sizes_0 = const()[name = string("op_6016_split_sizes_0"), val = tensor([512, 512])]; + int32 var_6016_axis_0 = const()[name = string("op_6016_axis_0"), val = int32(-1)]; + tensor var_6016_cast_fp16_0, tensor var_6016_cast_fp16_1 = split(axis = var_6016_axis_0, split_sizes = var_6016_split_sizes_0, x = linear_189_cast_fp16)[name = string("op_6016_cast_fp16")]; + tensor var_6024 = const()[name = string("op_6024"), val = tensor([1, 256, 8, 64])]; + tensor x_853_cast_fp16 = reshape(shape = var_6024, x = linear_188_cast_fp16)[name = string("x_853_cast_fp16")]; + tensor var_6034 = const()[name = string("op_6034"), val = tensor([1, 256, 8, 64])]; + tensor x_857_cast_fp16 = reshape(shape = var_6034, x = var_6016_cast_fp16_0)[name = string("x_857_cast_fp16")]; + tensor var_6044 = const()[name = string("op_6044"), val = tensor([1, 256, 8, 64])]; + tensor x_861_cast_fp16 = reshape(shape = var_6044, x = var_6016_cast_fp16_1)[name = string("x_861_cast_fp16")]; + tensor var_6046 = const()[name = string("op_6046"), val = tensor([0, 2, 1, 3])]; + bool sim_89_transpose_x_0 = const()[name = string("sim_89_transpose_x_0"), val = bool(false)]; + bool sim_89_transpose_y_0 = const()[name = string("sim_89_transpose_y_0"), val = bool(false)]; + tensor transpose_116_perm_0 = const()[name = string("transpose_116_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_117_perm_0 = const()[name = string("transpose_117_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_117 = transpose(perm = transpose_117_perm_0, x = x_857_cast_fp16)[name = string("transpose_131")]; + tensor transpose_116 = transpose(perm = transpose_116_perm_0, x = x_853_cast_fp16)[name = string("transpose_132")]; + tensor sim_89_cast_fp16 = matmul(transpose_x = sim_89_transpose_x_0, transpose_y = sim_89_transpose_y_0, x = transpose_116, y = transpose_117)[name = string("sim_89_cast_fp16")]; + fp16 var_6050_to_fp16 = const()[name = string("op_6050_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_91_cast_fp16 = mul(x = sim_89_cast_fp16, y = var_6050_to_fp16)[name = string("sim_91_cast_fp16")]; + tensor attn_45_cast_fp16 = softmax(axis = var_5460, x = sim_91_cast_fp16)[name = string("attn_45_cast_fp16")]; + bool x_863_transpose_x_0 = const()[name = string("x_863_transpose_x_0"), val = bool(false)]; + bool x_863_transpose_y_0 = const()[name = string("x_863_transpose_y_0"), val = bool(false)]; + tensor v_45_cast_fp16 = transpose(perm = var_6046, x = x_861_cast_fp16)[name = string("transpose_133")]; + tensor x_863_cast_fp16 = matmul(transpose_x = x_863_transpose_x_0, transpose_y = x_863_transpose_y_0, x = attn_45_cast_fp16, y = v_45_cast_fp16)[name = string("x_863_cast_fp16")]; + tensor var_6072 = const()[name = string("op_6072"), val = tensor([0, 2, 1, 3])]; + tensor var_6074 = const()[name = string("op_6074"), val = tensor([1, 256, 512])]; + tensor x_865_cast_fp16 = transpose(perm = var_6072, x = x_863_cast_fp16)[name = string("transpose_130")]; + tensor input_487_cast_fp16 = reshape(shape = var_6074, x = x_865_cast_fp16)[name = string("input_487_cast_fp16")]; + tensor linear_190_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_487_cast_fp16)[name = string("linear_190_cast_fp16")]; + tensor input_489_cast_fp16 = add(x = linear_190_cast_fp16, y = x_837_cast_fp16)[name = string("input_489_cast_fp16")]; + tensor linear_191_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_489_cast_fp16)[name = string("linear_191_cast_fp16")]; + string input_493_mode_0 = const()[name = string("input_493_mode_0"), val = string("EXACT")]; + tensor input_493_cast_fp16 = gelu(mode = input_493_mode_0, x = linear_191_cast_fp16)[name = string("input_493_cast_fp16")]; + tensor linear_192_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_493_cast_fp16)[name = string("linear_192_cast_fp16")]; + tensor x_867_cast_fp16 = add(x = linear_192_cast_fp16, y = input_489_cast_fp16)[name = string("x_867_cast_fp16")]; + tensor x_869_cast_fp16 = add(x = x_867_cast_fp16, y = mapping_cast_fp16)[name = string("x_869_cast_fp16")]; + tensor var_6090_split_sizes_0 = const()[name = string("op_6090_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_6090_axis_0 = const()[name = string("op_6090_axis_0"), val = int32(1)]; + tensor var_6090_cast_fp16_0, tensor var_6090_cast_fp16_1 = split(axis = var_6090_axis_0, split_sizes = var_6090_split_sizes_0, x = h_19_cast_fp16)[name = string("op_6090_cast_fp16")]; + tensor gamma_187_perm_0 = const()[name = string("gamma_187_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_187_perm_0 = const()[name = string("beta_187_perm_0"), val = tensor([0, -1, 1])]; + tensor x_873_axes_0 = const()[name = string("x_873_axes_0"), val = tensor([-1])]; + tensor x_873_cast_fp16 = layer_norm(axes = x_873_axes_0, epsilon = var_5456_to_fp16, x = x_869_cast_fp16)[name = string("x_873_cast_fp16")]; + fp16 var_6096_promoted_to_fp16 = const()[name = string("op_6096_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_187_cast_fp16 = transpose(perm = gamma_187_perm_0, x = var_6090_cast_fp16_0)[name = string("transpose_129")]; + tensor var_6097_cast_fp16 = add(x = gamma_187_cast_fp16, y = var_6096_promoted_to_fp16)[name = string("op_6097_cast_fp16")]; + tensor var_6098_cast_fp16 = mul(x = var_6097_cast_fp16, y = x_873_cast_fp16)[name = string("op_6098_cast_fp16")]; + tensor beta_187_cast_fp16 = transpose(perm = beta_187_perm_0, x = var_6090_cast_fp16_1)[name = string("transpose_128")]; + tensor x_875_cast_fp16 = add(x = var_6098_cast_fp16, y = beta_187_cast_fp16)[name = string("x_875_cast_fp16")]; + tensor var_6109_split_sizes_0 = const()[name = string("op_6109_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_6109_axis_0 = const()[name = string("op_6109_axis_0"), val = int32(1)]; + tensor var_6109_cast_fp16_0, tensor var_6109_cast_fp16_1 = split(axis = var_6109_axis_0, split_sizes = var_6109_split_sizes_0, x = h_23_cast_fp16)[name = string("op_6109_cast_fp16")]; + tensor gamma_perm_0 = const()[name = string("gamma_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_perm_0 = const()[name = string("beta_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_6115_promoted_to_fp16 = const()[name = string("op_6115_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_cast_fp16 = transpose(perm = gamma_perm_0, x = var_6109_cast_fp16_0)[name = string("transpose_127")]; + tensor var_6116_cast_fp16 = add(x = gamma_cast_fp16, y = var_6115_promoted_to_fp16)[name = string("op_6116_cast_fp16")]; + tensor var_6117_cast_fp16 = mul(x = var_6116_cast_fp16, y = x_873_cast_fp16)[name = string("op_6117_cast_fp16")]; + tensor beta_cast_fp16 = transpose(perm = beta_perm_0, x = var_6109_cast_fp16_1)[name = string("transpose_126")]; + tensor x_881_cast_fp16 = add(x = var_6117_cast_fp16, y = beta_cast_fp16)[name = string("x_881_cast_fp16")]; + tensor linear_195_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_875_cast_fp16)[name = string("linear_195_cast_fp16")]; + tensor linear_196_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_881_cast_fp16)[name = string("linear_196_cast_fp16")]; + tensor var_6123_split_sizes_0 = const()[name = string("op_6123_split_sizes_0"), val = tensor([512, 512])]; + int32 var_6123_axis_0 = const()[name = string("op_6123_axis_0"), val = int32(-1)]; + tensor var_6123_cast_fp16_0, tensor var_6123_cast_fp16_1 = split(axis = var_6123_axis_0, split_sizes = var_6123_split_sizes_0, x = linear_196_cast_fp16)[name = string("op_6123_cast_fp16")]; + tensor var_6131 = const()[name = string("op_6131"), val = tensor([1, 256, 8, 64])]; + tensor x_885_cast_fp16 = reshape(shape = var_6131, x = linear_195_cast_fp16)[name = string("x_885_cast_fp16")]; + tensor var_6141 = const()[name = string("op_6141"), val = tensor([1, 256, 8, 64])]; + tensor x_889_cast_fp16 = reshape(shape = var_6141, x = var_6123_cast_fp16_0)[name = string("x_889_cast_fp16")]; + tensor var_6151 = const()[name = string("op_6151"), val = tensor([1, 256, 8, 64])]; + tensor x_893_cast_fp16 = reshape(shape = var_6151, x = var_6123_cast_fp16_1)[name = string("x_893_cast_fp16")]; + tensor var_6153 = const()[name = string("op_6153"), val = tensor([0, 2, 1, 3])]; + bool sim_93_transpose_x_0 = const()[name = string("sim_93_transpose_x_0"), val = bool(false)]; + bool sim_93_transpose_y_0 = const()[name = string("sim_93_transpose_y_0"), val = bool(false)]; + tensor transpose_118_perm_0 = const()[name = string("transpose_118_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_119_perm_0 = const()[name = string("transpose_119_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_119 = transpose(perm = transpose_119_perm_0, x = x_889_cast_fp16)[name = string("transpose_123")]; + tensor transpose_118 = transpose(perm = transpose_118_perm_0, x = x_885_cast_fp16)[name = string("transpose_124")]; + tensor sim_93_cast_fp16 = matmul(transpose_x = sim_93_transpose_x_0, transpose_y = sim_93_transpose_y_0, x = transpose_118, y = transpose_119)[name = string("sim_93_cast_fp16")]; + fp16 var_6157_to_fp16 = const()[name = string("op_6157_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_cast_fp16 = mul(x = sim_93_cast_fp16, y = var_6157_to_fp16)[name = string("sim_cast_fp16")]; + tensor attn_cast_fp16 = softmax(axis = var_5460, x = sim_cast_fp16)[name = string("attn_cast_fp16")]; + bool x_895_transpose_x_0 = const()[name = string("x_895_transpose_x_0"), val = bool(false)]; + bool x_895_transpose_y_0 = const()[name = string("x_895_transpose_y_0"), val = bool(false)]; + tensor v_cast_fp16 = transpose(perm = var_6153, x = x_893_cast_fp16)[name = string("transpose_125")]; + tensor x_895_cast_fp16 = matmul(transpose_x = x_895_transpose_x_0, transpose_y = x_895_transpose_y_0, x = attn_cast_fp16, y = v_cast_fp16)[name = string("x_895_cast_fp16")]; + tensor var_6179 = const()[name = string("op_6179"), val = tensor([0, 2, 1, 3])]; + tensor var_6181 = const()[name = string("op_6181"), val = tensor([1, 256, 512])]; + tensor x_897_cast_fp16 = transpose(perm = var_6179, x = x_895_cast_fp16)[name = string("transpose_122")]; + tensor input_503_cast_fp16 = reshape(shape = var_6181, x = x_897_cast_fp16)[name = string("input_503_cast_fp16")]; + tensor linear_197_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_503_cast_fp16)[name = string("linear_197_cast_fp16")]; + tensor input_505_cast_fp16 = add(x = linear_197_cast_fp16, y = x_869_cast_fp16)[name = string("input_505_cast_fp16")]; + tensor linear_198_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_505_cast_fp16)[name = string("linear_198_cast_fp16")]; + string input_509_mode_0 = const()[name = string("input_509_mode_0"), val = string("EXACT")]; + tensor input_509_cast_fp16 = gelu(mode = input_509_mode_0, x = linear_198_cast_fp16)[name = string("input_509_cast_fp16")]; + tensor linear_199_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_509_cast_fp16)[name = string("linear_199_cast_fp16")]; + tensor x_899_cast_fp16 = add(x = linear_199_cast_fp16, y = input_505_cast_fp16)[name = string("x_899_cast_fp16")]; + tensor var_6190_axes_0 = const()[name = string("op_6190_axes_0"), val = tensor([1])]; + bool var_6190_keep_dims_0 = const()[name = string("op_6190_keep_dims_0"), val = bool(false)]; + tensor var_6190_cast_fp16 = reduce_mean(axes = var_6190_axes_0, keep_dims = var_6190_keep_dims_0, x = x_899_cast_fp16)[name = string("op_6190_cast_fp16")]; + tensor x_901_axes_0 = const()[name = string("x_901_axes_0"), val = tensor([1])]; + tensor x_901_cast_fp16 = expand_dims(axes = x_901_axes_0, x = var_6190_cast_fp16)[name = string("x_901_cast_fp16")]; + tensor var_6192 = const()[name = string("op_6192"), val = tensor([0, 2, 1])]; + string x_903_pad_type_0 = const()[name = string("x_903_pad_type_0"), val = string("valid")]; + tensor x_903_strides_0 = const()[name = string("x_903_strides_0"), val = tensor([1])]; + tensor x_903_pad_0 = const()[name = string("x_903_pad_0"), val = tensor([0, 0])]; + tensor x_903_dilations_0 = const()[name = string("x_903_dilations_0"), val = tensor([1])]; + int32 x_903_groups_0 = const()[name = string("x_903_groups_0"), val = int32(1)]; + tensor input_cast_fp16 = transpose(perm = var_6192, x = x_901_cast_fp16)[name = string("transpose_121")]; + tensor x_903_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_903_dilations_0, groups = x_903_groups_0, pad = x_903_pad_0, pad_type = x_903_pad_type_0, strides = x_903_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_cast_fp16)[name = string("x_903_cast_fp16")]; + tensor x_pred_perm_0 = const()[name = string("x_pred_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_to_fp16 = const()[name = string("c_skip_to_fp16"), val = tensor([[[0x1.fecp-1]]])]; + tensor var_6200_cast_fp16 = mul(x = c_skip_to_fp16, y = x_noisy_cast_fp16)[name = string("op_6200_cast_fp16")]; + tensor c_out_to_fp16 = const()[name = string("c_out_to_fp16"), val = tensor([[[0x1.38p-9]]])]; + tensor x_pred_cast_fp16 = transpose(perm = x_pred_perm_0, x = x_903_cast_fp16)[name = string("transpose_120")]; + tensor var_6201_cast_fp16 = mul(x = c_out_to_fp16, y = x_pred_cast_fp16)[name = string("op_6201_cast_fp16")]; + tensor x_mid_dn_cast_fp16 = add(x = var_6200_cast_fp16, y = var_6201_cast_fp16)[name = string("x_mid_dn_cast_fp16")]; + tensor var_6204_cast_fp16 = sub(x = x_noisy_cast_fp16, y = x_mid_dn_cast_fp16)[name = string("op_6204_cast_fp16")]; + tensor _inversed_d_mid_y_0_to_fp16 = const()[name = string("_inversed_d_mid_y_0_to_fp16"), val = tensor([0x1.a44p+8])]; + tensor _inversed_d_mid_cast_fp16 = mul(x = var_6204_cast_fp16, y = _inversed_d_mid_y_0_to_fp16)[name = string("_inversed_d_mid_cast_fp16")]; + fp16 var_6213_to_fp16 = const()[name = string("op_6213_to_fp16"), val = fp16(-0x1.37cp-8)]; + tensor var_6214_cast_fp16 = mul(x = _inversed_d_mid_cast_fp16, y = var_6213_to_fp16)[name = string("op_6214_cast_fp16")]; + tensor x_cast_fp16 = add(x = x_noisy_13_cast_fp16, y = var_6214_cast_fp16)[name = string("x_cast_fp16")]; + tensor var_6219_begin_0 = const()[name = string("op_6219_begin_0"), val = tensor([3, 0, 0, 0])]; + tensor var_6219_end_0 = const()[name = string("op_6219_end_0"), val = tensor([4, 1, 1, 256])]; + tensor var_6219_end_mask_0 = const()[name = string("op_6219_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_6219_squeeze_mask_0 = const()[name = string("op_6219_squeeze_mask_0"), val = tensor([true, false, false, false])]; + tensor var_6219_cast_fp16 = slice_by_index(begin = var_6219_begin_0, end = var_6219_end_0, end_mask = var_6219_end_mask_0, squeeze_mask = var_6219_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_6219_cast_fp16")]; + fp16 var_6222_to_fp16 = const()[name = string("op_6222_to_fp16"), val = fp16(0x1.a34p-14)]; + tensor var_6223_cast_fp16 = mul(x = var_6219_cast_fp16, y = var_6222_to_fp16)[name = string("op_6223_cast_fp16")]; + tensor var_6225 = add(x = x_cast_fp16, y = var_6223_cast_fp16)[name = string("op_6225_cast_fp16")]; + } -> (var_6225); +} \ No newline at end of file diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/weights/weight.bin b/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..5d524e318268dea3c586fd0de5ce641710361300 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t256.mlmodelc/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f823b5c638d2eb2fd91bf8e4efe4a90b2e1d3d9e2f5ab40e7e93cb03cd212aca +size 49361856 diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/analytics/coremldata.bin b/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/analytics/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..67ed2033b27fbc34ab9832cdd829ba173447d93d --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/analytics/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:554dd7f0ac5139918b06d99c98e202a58621a5783502a2c6585851ae475ca47e +size 243 diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/coremldata.bin b/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/coremldata.bin new file mode 100644 index 0000000000000000000000000000000000000000..134cf404ad4c55fce1b4b78d2e371c1c5af75a36 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/coremldata.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f8eb46b781de148bb4eb0c62576773d51dddbf26741f916f59d5ae2b8778795 +size 461 diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/metadata.json b/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/metadata.json new file mode 100644 index 0000000000000000000000000000000000000000..730b46761e8bca998a306b14cf09c13bf9aa69cd --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/metadata.json @@ -0,0 +1,110 @@ +[ + { + "metadataOutputVersion" : "3.0", + "storagePrecision" : "Float16", + "outputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float16", + "formattedType" : "MultiArray (Float16 1 × 1 × 256)", + "shortDescription" : "", + "shape" : "[1, 1, 256]", + "name" : "var_6225", + "type" : "MultiArray" + } + ], + "modelParameters" : [ + + ], + "specificationVersion" : 9, + "mlProgramOperationTypeHistogram" : { + "Ios18.expandDims" : 16, + "Ios18.softmax" : 24, + "Ios18.mul" : 117, + "Ios18.matmul" : 48, + "Ios16.reduceMean" : 8, + "Split" : 72, + "Tile" : 16, + "Ios18.add" : 188, + "Ios16.reduceSum" : 8, + "Ios18.layerNorm" : 24, + "Ios18.reshape" : 102, + "Ios18.linear" : 143, + "Ios18.conv" : 8, + "Ios18.gelu" : 41, + "Ios18.sub" : 8, + "Ios18.concat" : 8, + "Stack" : 8, + "Ios18.transpose" : 216, + "Ios18.cast" : 4, + "Ios18.sliceByIndex" : 4 + }, + "computePrecision" : "Mixed (Float16, Int32)", + "isUpdatable" : "0", + "stateSchema" : [ + + ], + "availability" : { + "macOS" : "15.0", + "tvOS" : "18.0", + "visionOS" : "2.0", + "watchOS" : "11.0", + "iOS" : "18.0", + "macCatalyst" : "18.0" + }, + "modelType" : { + "name" : "MLModelType_mlProgram" + }, + "userDefinedMetadata" : { + "com.github.apple.coremltools.conversion_date" : "2026-05-08", + "com.github.apple.coremltools.source" : "torch==2.11.0", + "com.github.apple.coremltools.version" : "9.0", + "com.github.apple.coremltools.source_dialect" : "TorchScript" + }, + "inputSchema" : [ + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 1 × 1 × 256)", + "shortDescription" : "", + "shape" : "[1, 1, 256]", + "name" : "noise_init", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 4 × 1 × 1 × 256)", + "shortDescription" : "", + "shape" : "[4, 1, 1, 256]", + "name" : "noises_aux", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 1 × 64 × 768)", + "shortDescription" : "", + "shape" : "[1, 64, 768]", + "name" : "embedding", + "type" : "MultiArray" + }, + { + "hasShapeFlexibility" : "0", + "isOptional" : "0", + "dataType" : "Float32", + "formattedType" : "MultiArray (Float32 1 × 256)", + "shortDescription" : "", + "shape" : "[1, 256]", + "name" : "features", + "type" : "MultiArray" + } + ], + "generatedClassName" : "fused_diffusion_sampler_fp16_t64", + "method" : "predict" + } +] \ No newline at end of file diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/model.mil b/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/model.mil new file mode 100644 index 0000000000000000000000000000000000000000..a313e8219c4c966857839faecd73fbf51b57a6d3 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/model.mil @@ -0,0 +1,2019 @@ +program(1.3) +[buildInfo = dict({{"coremlc-component-MIL", "3520.4.1"}, {"coremlc-version", "3520.5.1"}, {"coremltools-component-torch", "2.11.0"}, {"coremltools-source-dialect", "TorchScript"}, {"coremltools-version", "9.0"}})] +{ + func main(tensor embedding, tensor features, tensor noise_init, tensor noises_aux) { + fp16 var_12_to_fp16 = const()[name = string("op_12_to_fp16"), val = fp16(0x1.8p+1)]; + string noise_init_to_fp16_dtype_0 = const()[name = string("noise_init_to_fp16_dtype_0"), val = string("fp16")]; + tensor noise_init_to_fp16 = cast(dtype = noise_init_to_fp16_dtype_0, x = noise_init)[name = string("cast_196")]; + tensor x_noisy_1_cast_fp16 = mul(x = var_12_to_fp16, y = noise_init_to_fp16)[name = string("x_noisy_1_cast_fp16")]; + int32 var_35 = const()[name = string("op_35"), val = int32(-1)]; + tensor c_in_1_to_fp16 = const()[name = string("c_in_1_to_fp16"), val = tensor([[[0x1.548p-2]]])]; + tensor x_11_cast_fp16 = mul(x = c_in_1_to_fp16, y = x_noisy_1_cast_fp16)[name = string("x_11_cast_fp16")]; + string features_to_fp16_dtype_0 = const()[name = string("features_to_fp16_dtype_0"), val = string("fp16")]; + tensor unet_step_kdiffusion_net_to_features_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_features_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(64)))]; + tensor unet_step_kdiffusion_net_to_features_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_features_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(524416)))]; + tensor features_to_fp16 = cast(dtype = features_to_fp16_dtype_0, x = features)[name = string("cast_195")]; + tensor linear_1_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_features_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_features_0_weight_to_fp16, x = features_to_fp16)[name = string("linear_1_cast_fp16")]; + string var_423_mode_0 = const()[name = string("op_423_mode_0"), val = string("EXACT")]; + tensor var_423_cast_fp16 = gelu(mode = var_423_mode_0, x = linear_1_cast_fp16)[name = string("op_423_cast_fp16")]; + int32 x_7_axis_0 = const()[name = string("x_7_axis_0"), val = int32(0)]; + tensor var_421_to_fp16 = const()[name = string("op_421_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(526528)))]; + tensor x_7_cast_fp16 = stack(axis = x_7_axis_0, values = (var_421_to_fp16, var_423_cast_fp16))[name = string("x_7_cast_fp16")]; + tensor var_426 = const()[name = string("op_426"), val = tensor([1, 2, 0])]; + tensor input_7_axes_0 = const()[name = string("input_7_axes_0"), val = tensor([2])]; + bool input_7_keep_dims_0 = const()[name = string("input_7_keep_dims_0"), val = bool(false)]; + tensor x_9_cast_fp16 = transpose(perm = var_426, x = x_7_cast_fp16)[name = string("transpose_335")]; + tensor input_7_cast_fp16 = reduce_sum(axes = input_7_axes_0, keep_dims = input_7_keep_dims_0, x = x_9_cast_fp16)[name = string("input_7_cast_fp16")]; + tensor unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(528640)))]; + tensor unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2625856)))]; + tensor linear_2_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_7_cast_fp16)[name = string("linear_2_cast_fp16")]; + string input_11_mode_0 = const()[name = string("input_11_mode_0"), val = string("EXACT")]; + tensor input_11_cast_fp16 = gelu(mode = input_11_mode_0, x = linear_2_cast_fp16)[name = string("input_11_cast_fp16")]; + tensor unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(2627968)))]; + tensor unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(4725184)))]; + tensor linear_3_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_11_cast_fp16)[name = string("linear_3_cast_fp16")]; + string mapping_1_mode_0 = const()[name = string("mapping_1_mode_0"), val = string("EXACT")]; + tensor mapping_1_cast_fp16 = gelu(mode = mapping_1_mode_0, x = linear_3_cast_fp16)[name = string("mapping_1_cast_fp16")]; + tensor var_436_reps_0 = const()[name = string("op_436_reps_0"), val = tensor([1, 64, 1])]; + tensor var_436_cast_fp16 = tile(reps = var_436_reps_0, x = x_11_cast_fp16)[name = string("op_436_cast_fp16")]; + bool x_13_interleave_0 = const()[name = string("x_13_interleave_0"), val = bool(false)]; + string embedding_to_fp16_dtype_0 = const()[name = string("embedding_to_fp16_dtype_0"), val = string("fp16")]; + tensor embedding_to_fp16 = cast(dtype = embedding_to_fp16_dtype_0, x = embedding)[name = string("cast_194")]; + tensor x_13_cast_fp16 = concat(axis = var_35, interleave = x_13_interleave_0, values = (var_436_cast_fp16, embedding_to_fp16))[name = string("x_13_cast_fp16")]; + tensor var_439_axes_0 = const()[name = string("op_439_axes_0"), val = tensor([1])]; + tensor var_439_cast_fp16 = expand_dims(axes = var_439_axes_0, x = mapping_1_cast_fp16)[name = string("op_439_cast_fp16")]; + tensor mapping_3_reps_0 = const()[name = string("mapping_3_reps_0"), val = tensor([1, 64, 1])]; + tensor mapping_3_cast_fp16 = tile(reps = mapping_3_reps_0, x = var_439_cast_fp16)[name = string("mapping_3_cast_fp16")]; + tensor x_15_cast_fp16 = add(x = x_13_cast_fp16, y = mapping_3_cast_fp16)[name = string("x_15_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(4727296)))]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5775936)))]; + tensor linear_4_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_norm_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_norm_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_4_cast_fp16")]; + tensor var_449 = const()[name = string("op_449"), val = tensor([1, 2048, 1])]; + tensor h_3_cast_fp16 = reshape(shape = var_449, x = linear_4_cast_fp16)[name = string("h_3_cast_fp16")]; + tensor var_451_split_sizes_0 = const()[name = string("op_451_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_451_axis_0 = const()[name = string("op_451_axis_0"), val = int32(1)]; + tensor var_451_cast_fp16_0, tensor var_451_cast_fp16_1 = split(axis = var_451_axis_0, split_sizes = var_451_split_sizes_0, x = h_3_cast_fp16)[name = string("op_451_cast_fp16")]; + tensor gamma_3_perm_0 = const()[name = string("gamma_3_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_3_perm_0 = const()[name = string("beta_3_perm_0"), val = tensor([0, -1, 1])]; + tensor x_19_axes_0 = const()[name = string("x_19_axes_0"), val = tensor([-1])]; + fp16 var_31_to_fp16 = const()[name = string("op_31_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_19_cast_fp16 = layer_norm(axes = x_19_axes_0, epsilon = var_31_to_fp16, x = x_15_cast_fp16)[name = string("x_19_cast_fp16")]; + fp16 var_457_promoted_to_fp16 = const()[name = string("op_457_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_3_cast_fp16 = transpose(perm = gamma_3_perm_0, x = var_451_cast_fp16_0)[name = string("transpose_334")]; + tensor var_458_cast_fp16 = add(x = gamma_3_cast_fp16, y = var_457_promoted_to_fp16)[name = string("op_458_cast_fp16")]; + tensor var_459_cast_fp16 = mul(x = var_458_cast_fp16, y = x_19_cast_fp16)[name = string("op_459_cast_fp16")]; + tensor beta_3_cast_fp16 = transpose(perm = beta_3_perm_0, x = var_451_cast_fp16_1)[name = string("transpose_333")]; + tensor x_21_cast_fp16 = add(x = var_459_cast_fp16, y = beta_3_cast_fp16)[name = string("x_21_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(5780096)))]; + tensor unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(6828736)))]; + tensor linear_5_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_norm_context_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_5_cast_fp16")]; + tensor var_468 = const()[name = string("op_468"), val = tensor([1, 2048, 1])]; + tensor h_7_cast_fp16 = reshape(shape = var_468, x = linear_5_cast_fp16)[name = string("h_7_cast_fp16")]; + tensor var_470_split_sizes_0 = const()[name = string("op_470_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_470_axis_0 = const()[name = string("op_470_axis_0"), val = int32(1)]; + tensor var_470_cast_fp16_0, tensor var_470_cast_fp16_1 = split(axis = var_470_axis_0, split_sizes = var_470_split_sizes_0, x = h_7_cast_fp16)[name = string("op_470_cast_fp16")]; + tensor gamma_7_perm_0 = const()[name = string("gamma_7_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_7_perm_0 = const()[name = string("beta_7_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_476_promoted_to_fp16 = const()[name = string("op_476_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_7_cast_fp16 = transpose(perm = gamma_7_perm_0, x = var_470_cast_fp16_0)[name = string("transpose_332")]; + tensor var_477_cast_fp16 = add(x = gamma_7_cast_fp16, y = var_476_promoted_to_fp16)[name = string("op_477_cast_fp16")]; + tensor var_478_cast_fp16 = mul(x = var_477_cast_fp16, y = x_19_cast_fp16)[name = string("op_478_cast_fp16")]; + tensor beta_7_cast_fp16 = transpose(perm = beta_7_perm_0, x = var_470_cast_fp16_1)[name = string("transpose_331")]; + tensor x_27_cast_fp16 = add(x = var_478_cast_fp16, y = beta_7_cast_fp16)[name = string("x_27_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(6832896)))]; + tensor linear_6_bias_0_to_fp16 = const()[name = string("linear_6_bias_0_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(7881536)))]; + tensor linear_6_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_21_cast_fp16)[name = string("linear_6_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(7882624)))]; + tensor linear_7_bias_0_to_fp16 = const()[name = string("linear_7_bias_0_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(9979840)))]; + tensor linear_7_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_27_cast_fp16)[name = string("linear_7_cast_fp16")]; + tensor var_484_split_sizes_0 = const()[name = string("op_484_split_sizes_0"), val = tensor([512, 512])]; + int32 var_484_axis_0 = const()[name = string("op_484_axis_0"), val = int32(-1)]; + tensor var_484_cast_fp16_0, tensor var_484_cast_fp16_1 = split(axis = var_484_axis_0, split_sizes = var_484_split_sizes_0, x = linear_7_cast_fp16)[name = string("op_484_cast_fp16")]; + tensor var_492 = const()[name = string("op_492"), val = tensor([1, 64, 8, 64])]; + tensor x_31_cast_fp16 = reshape(shape = var_492, x = linear_6_cast_fp16)[name = string("x_31_cast_fp16")]; + tensor var_502 = const()[name = string("op_502"), val = tensor([1, 64, 8, 64])]; + tensor x_35_cast_fp16 = reshape(shape = var_502, x = var_484_cast_fp16_0)[name = string("x_35_cast_fp16")]; + tensor var_512 = const()[name = string("op_512"), val = tensor([1, 64, 8, 64])]; + tensor x_39_cast_fp16 = reshape(shape = var_512, x = var_484_cast_fp16_1)[name = string("x_39_cast_fp16")]; + tensor var_514 = const()[name = string("op_514"), val = tensor([0, 2, 1, 3])]; + bool sim_1_transpose_x_0 = const()[name = string("sim_1_transpose_x_0"), val = bool(false)]; + bool sim_1_transpose_y_0 = const()[name = string("sim_1_transpose_y_0"), val = bool(false)]; + tensor transpose_72_perm_0 = const()[name = string("transpose_72_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_73_perm_0 = const()[name = string("transpose_73_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_73 = transpose(perm = transpose_73_perm_0, x = x_35_cast_fp16)[name = string("transpose_328")]; + tensor transpose_72 = transpose(perm = transpose_72_perm_0, x = x_31_cast_fp16)[name = string("transpose_329")]; + tensor sim_1_cast_fp16 = matmul(transpose_x = sim_1_transpose_x_0, transpose_y = sim_1_transpose_y_0, x = transpose_72, y = transpose_73)[name = string("sim_1_cast_fp16")]; + fp16 var_518_to_fp16 = const()[name = string("op_518_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_3_cast_fp16 = mul(x = sim_1_cast_fp16, y = var_518_to_fp16)[name = string("sim_3_cast_fp16")]; + tensor attn_1_cast_fp16 = softmax(axis = var_35, x = sim_3_cast_fp16)[name = string("attn_1_cast_fp16")]; + bool x_41_transpose_x_0 = const()[name = string("x_41_transpose_x_0"), val = bool(false)]; + bool x_41_transpose_y_0 = const()[name = string("x_41_transpose_y_0"), val = bool(false)]; + tensor v_1_cast_fp16 = transpose(perm = var_514, x = x_39_cast_fp16)[name = string("transpose_330")]; + tensor x_41_cast_fp16 = matmul(transpose_x = x_41_transpose_x_0, transpose_y = x_41_transpose_y_0, x = attn_1_cast_fp16, y = v_1_cast_fp16)[name = string("x_41_cast_fp16")]; + tensor var_540 = const()[name = string("op_540"), val = tensor([0, 2, 1, 3])]; + tensor var_542 = const()[name = string("op_542"), val = tensor([1, 64, 512])]; + tensor x_43_cast_fp16 = transpose(perm = var_540, x = x_41_cast_fp16)[name = string("transpose_327")]; + tensor input_23_cast_fp16 = reshape(shape = var_542, x = x_43_cast_fp16)[name = string("input_23_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(9981952)))]; + tensor unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11030592)))]; + tensor linear_8_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_23_cast_fp16)[name = string("linear_8_cast_fp16")]; + tensor input_25_cast_fp16 = add(x = linear_8_cast_fp16, y = x_15_cast_fp16)[name = string("input_25_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(11032704)))]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(15227072)))]; + tensor linear_9_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_25_cast_fp16)[name = string("linear_9_cast_fp16")]; + string input_29_mode_0 = const()[name = string("input_29_mode_0"), val = string("EXACT")]; + tensor input_29_cast_fp16 = gelu(mode = input_29_mode_0, x = linear_9_cast_fp16)[name = string("input_29_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(15231232)))]; + tensor unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(19425600)))]; + tensor linear_10_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_29_cast_fp16)[name = string("linear_10_cast_fp16")]; + tensor x_45_cast_fp16 = add(x = linear_10_cast_fp16, y = input_25_cast_fp16)[name = string("x_45_cast_fp16")]; + tensor x_47_cast_fp16 = add(x = x_45_cast_fp16, y = mapping_3_cast_fp16)[name = string("x_47_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(19427712)))]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(20476352)))]; + tensor linear_11_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_norm_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_norm_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_11_cast_fp16")]; + tensor var_556 = const()[name = string("op_556"), val = tensor([1, 2048, 1])]; + tensor h_11_cast_fp16 = reshape(shape = var_556, x = linear_11_cast_fp16)[name = string("h_11_cast_fp16")]; + tensor var_558_split_sizes_0 = const()[name = string("op_558_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_558_axis_0 = const()[name = string("op_558_axis_0"), val = int32(1)]; + tensor var_558_cast_fp16_0, tensor var_558_cast_fp16_1 = split(axis = var_558_axis_0, split_sizes = var_558_split_sizes_0, x = h_11_cast_fp16)[name = string("op_558_cast_fp16")]; + tensor gamma_11_perm_0 = const()[name = string("gamma_11_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_11_perm_0 = const()[name = string("beta_11_perm_0"), val = tensor([0, -1, 1])]; + tensor x_51_axes_0 = const()[name = string("x_51_axes_0"), val = tensor([-1])]; + tensor x_51_cast_fp16 = layer_norm(axes = x_51_axes_0, epsilon = var_31_to_fp16, x = x_47_cast_fp16)[name = string("x_51_cast_fp16")]; + fp16 var_564_promoted_to_fp16 = const()[name = string("op_564_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_11_cast_fp16 = transpose(perm = gamma_11_perm_0, x = var_558_cast_fp16_0)[name = string("transpose_326")]; + tensor var_565_cast_fp16 = add(x = gamma_11_cast_fp16, y = var_564_promoted_to_fp16)[name = string("op_565_cast_fp16")]; + tensor var_566_cast_fp16 = mul(x = var_565_cast_fp16, y = x_51_cast_fp16)[name = string("op_566_cast_fp16")]; + tensor beta_11_cast_fp16 = transpose(perm = beta_11_perm_0, x = var_558_cast_fp16_1)[name = string("transpose_325")]; + tensor x_53_cast_fp16 = add(x = var_566_cast_fp16, y = beta_11_cast_fp16)[name = string("x_53_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(20480512)))]; + tensor unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(21529152)))]; + tensor linear_12_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_norm_context_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_12_cast_fp16")]; + tensor var_575 = const()[name = string("op_575"), val = tensor([1, 2048, 1])]; + tensor h_15_cast_fp16 = reshape(shape = var_575, x = linear_12_cast_fp16)[name = string("h_15_cast_fp16")]; + tensor var_577_split_sizes_0 = const()[name = string("op_577_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_577_axis_0 = const()[name = string("op_577_axis_0"), val = int32(1)]; + tensor var_577_cast_fp16_0, tensor var_577_cast_fp16_1 = split(axis = var_577_axis_0, split_sizes = var_577_split_sizes_0, x = h_15_cast_fp16)[name = string("op_577_cast_fp16")]; + tensor gamma_15_perm_0 = const()[name = string("gamma_15_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_15_perm_0 = const()[name = string("beta_15_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_583_promoted_to_fp16 = const()[name = string("op_583_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_15_cast_fp16 = transpose(perm = gamma_15_perm_0, x = var_577_cast_fp16_0)[name = string("transpose_324")]; + tensor var_584_cast_fp16 = add(x = gamma_15_cast_fp16, y = var_583_promoted_to_fp16)[name = string("op_584_cast_fp16")]; + tensor var_585_cast_fp16 = mul(x = var_584_cast_fp16, y = x_51_cast_fp16)[name = string("op_585_cast_fp16")]; + tensor beta_15_cast_fp16 = transpose(perm = beta_15_perm_0, x = var_577_cast_fp16_1)[name = string("transpose_323")]; + tensor x_59_cast_fp16 = add(x = var_585_cast_fp16, y = beta_15_cast_fp16)[name = string("x_59_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(21533312)))]; + tensor linear_13_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_53_cast_fp16)[name = string("linear_13_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(22581952)))]; + tensor linear_14_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_59_cast_fp16)[name = string("linear_14_cast_fp16")]; + tensor var_591_split_sizes_0 = const()[name = string("op_591_split_sizes_0"), val = tensor([512, 512])]; + int32 var_591_axis_0 = const()[name = string("op_591_axis_0"), val = int32(-1)]; + tensor var_591_cast_fp16_0, tensor var_591_cast_fp16_1 = split(axis = var_591_axis_0, split_sizes = var_591_split_sizes_0, x = linear_14_cast_fp16)[name = string("op_591_cast_fp16")]; + tensor var_599 = const()[name = string("op_599"), val = tensor([1, 64, 8, 64])]; + tensor x_63_cast_fp16 = reshape(shape = var_599, x = linear_13_cast_fp16)[name = string("x_63_cast_fp16")]; + tensor var_609 = const()[name = string("op_609"), val = tensor([1, 64, 8, 64])]; + tensor x_67_cast_fp16 = reshape(shape = var_609, x = var_591_cast_fp16_0)[name = string("x_67_cast_fp16")]; + tensor var_619 = const()[name = string("op_619"), val = tensor([1, 64, 8, 64])]; + tensor x_71_cast_fp16 = reshape(shape = var_619, x = var_591_cast_fp16_1)[name = string("x_71_cast_fp16")]; + tensor var_621 = const()[name = string("op_621"), val = tensor([0, 2, 1, 3])]; + bool sim_5_transpose_x_0 = const()[name = string("sim_5_transpose_x_0"), val = bool(false)]; + bool sim_5_transpose_y_0 = const()[name = string("sim_5_transpose_y_0"), val = bool(false)]; + tensor transpose_74_perm_0 = const()[name = string("transpose_74_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_75_perm_0 = const()[name = string("transpose_75_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_75 = transpose(perm = transpose_75_perm_0, x = x_67_cast_fp16)[name = string("transpose_320")]; + tensor transpose_74 = transpose(perm = transpose_74_perm_0, x = x_63_cast_fp16)[name = string("transpose_321")]; + tensor sim_5_cast_fp16 = matmul(transpose_x = sim_5_transpose_x_0, transpose_y = sim_5_transpose_y_0, x = transpose_74, y = transpose_75)[name = string("sim_5_cast_fp16")]; + fp16 var_625_to_fp16 = const()[name = string("op_625_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_7_cast_fp16 = mul(x = sim_5_cast_fp16, y = var_625_to_fp16)[name = string("sim_7_cast_fp16")]; + tensor attn_3_cast_fp16 = softmax(axis = var_35, x = sim_7_cast_fp16)[name = string("attn_3_cast_fp16")]; + bool x_73_transpose_x_0 = const()[name = string("x_73_transpose_x_0"), val = bool(false)]; + bool x_73_transpose_y_0 = const()[name = string("x_73_transpose_y_0"), val = bool(false)]; + tensor v_3_cast_fp16 = transpose(perm = var_621, x = x_71_cast_fp16)[name = string("transpose_322")]; + tensor x_73_cast_fp16 = matmul(transpose_x = x_73_transpose_x_0, transpose_y = x_73_transpose_y_0, x = attn_3_cast_fp16, y = v_3_cast_fp16)[name = string("x_73_cast_fp16")]; + tensor var_647 = const()[name = string("op_647"), val = tensor([0, 2, 1, 3])]; + tensor var_649 = const()[name = string("op_649"), val = tensor([1, 64, 512])]; + tensor x_75_cast_fp16 = transpose(perm = var_647, x = x_73_cast_fp16)[name = string("transpose_319")]; + tensor input_39_cast_fp16 = reshape(shape = var_649, x = x_75_cast_fp16)[name = string("input_39_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(24679168)))]; + tensor unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(25727808)))]; + tensor linear_15_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_39_cast_fp16)[name = string("linear_15_cast_fp16")]; + tensor input_41_cast_fp16 = add(x = linear_15_cast_fp16, y = x_47_cast_fp16)[name = string("input_41_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(25729920)))]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(29924288)))]; + tensor linear_16_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_41_cast_fp16)[name = string("linear_16_cast_fp16")]; + string input_45_mode_0 = const()[name = string("input_45_mode_0"), val = string("EXACT")]; + tensor input_45_cast_fp16 = gelu(mode = input_45_mode_0, x = linear_16_cast_fp16)[name = string("input_45_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(29928448)))]; + tensor unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(34122816)))]; + tensor linear_17_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_45_cast_fp16)[name = string("linear_17_cast_fp16")]; + tensor x_77_cast_fp16 = add(x = linear_17_cast_fp16, y = input_41_cast_fp16)[name = string("x_77_cast_fp16")]; + tensor x_79_cast_fp16 = add(x = x_77_cast_fp16, y = mapping_3_cast_fp16)[name = string("x_79_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(34124928)))]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(35173568)))]; + tensor linear_18_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_norm_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_norm_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_18_cast_fp16")]; + tensor var_663 = const()[name = string("op_663"), val = tensor([1, 2048, 1])]; + tensor h_19_cast_fp16 = reshape(shape = var_663, x = linear_18_cast_fp16)[name = string("h_19_cast_fp16")]; + tensor var_665_split_sizes_0 = const()[name = string("op_665_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_665_axis_0 = const()[name = string("op_665_axis_0"), val = int32(1)]; + tensor var_665_cast_fp16_0, tensor var_665_cast_fp16_1 = split(axis = var_665_axis_0, split_sizes = var_665_split_sizes_0, x = h_19_cast_fp16)[name = string("op_665_cast_fp16")]; + tensor gamma_19_perm_0 = const()[name = string("gamma_19_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_19_perm_0 = const()[name = string("beta_19_perm_0"), val = tensor([0, -1, 1])]; + tensor x_83_axes_0 = const()[name = string("x_83_axes_0"), val = tensor([-1])]; + tensor x_83_cast_fp16 = layer_norm(axes = x_83_axes_0, epsilon = var_31_to_fp16, x = x_79_cast_fp16)[name = string("x_83_cast_fp16")]; + fp16 var_671_promoted_to_fp16 = const()[name = string("op_671_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_19_cast_fp16 = transpose(perm = gamma_19_perm_0, x = var_665_cast_fp16_0)[name = string("transpose_318")]; + tensor var_672_cast_fp16 = add(x = gamma_19_cast_fp16, y = var_671_promoted_to_fp16)[name = string("op_672_cast_fp16")]; + tensor var_673_cast_fp16 = mul(x = var_672_cast_fp16, y = x_83_cast_fp16)[name = string("op_673_cast_fp16")]; + tensor beta_19_cast_fp16 = transpose(perm = beta_19_perm_0, x = var_665_cast_fp16_1)[name = string("transpose_317")]; + tensor x_85_cast_fp16 = add(x = var_673_cast_fp16, y = beta_19_cast_fp16)[name = string("x_85_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(35177728)))]; + tensor unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(36226368)))]; + tensor linear_19_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_norm_context_fc_weight_to_fp16, x = features_to_fp16)[name = string("linear_19_cast_fp16")]; + tensor var_682 = const()[name = string("op_682"), val = tensor([1, 2048, 1])]; + tensor h_23_cast_fp16 = reshape(shape = var_682, x = linear_19_cast_fp16)[name = string("h_23_cast_fp16")]; + tensor var_684_split_sizes_0 = const()[name = string("op_684_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_684_axis_0 = const()[name = string("op_684_axis_0"), val = int32(1)]; + tensor var_684_cast_fp16_0, tensor var_684_cast_fp16_1 = split(axis = var_684_axis_0, split_sizes = var_684_split_sizes_0, x = h_23_cast_fp16)[name = string("op_684_cast_fp16")]; + tensor gamma_23_perm_0 = const()[name = string("gamma_23_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_23_perm_0 = const()[name = string("beta_23_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_690_promoted_to_fp16 = const()[name = string("op_690_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_23_cast_fp16 = transpose(perm = gamma_23_perm_0, x = var_684_cast_fp16_0)[name = string("transpose_316")]; + tensor var_691_cast_fp16 = add(x = gamma_23_cast_fp16, y = var_690_promoted_to_fp16)[name = string("op_691_cast_fp16")]; + tensor var_692_cast_fp16 = mul(x = var_691_cast_fp16, y = x_83_cast_fp16)[name = string("op_692_cast_fp16")]; + tensor beta_23_cast_fp16 = transpose(perm = beta_23_perm_0, x = var_684_cast_fp16_1)[name = string("transpose_315")]; + tensor x_91_cast_fp16 = add(x = var_692_cast_fp16, y = beta_23_cast_fp16)[name = string("x_91_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(36230528)))]; + tensor linear_20_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_85_cast_fp16)[name = string("linear_20_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(37279168)))]; + tensor linear_21_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_91_cast_fp16)[name = string("linear_21_cast_fp16")]; + tensor var_698_split_sizes_0 = const()[name = string("op_698_split_sizes_0"), val = tensor([512, 512])]; + int32 var_698_axis_0 = const()[name = string("op_698_axis_0"), val = int32(-1)]; + tensor var_698_cast_fp16_0, tensor var_698_cast_fp16_1 = split(axis = var_698_axis_0, split_sizes = var_698_split_sizes_0, x = linear_21_cast_fp16)[name = string("op_698_cast_fp16")]; + tensor var_706 = const()[name = string("op_706"), val = tensor([1, 64, 8, 64])]; + tensor x_95_cast_fp16 = reshape(shape = var_706, x = linear_20_cast_fp16)[name = string("x_95_cast_fp16")]; + tensor var_716 = const()[name = string("op_716"), val = tensor([1, 64, 8, 64])]; + tensor x_99_cast_fp16 = reshape(shape = var_716, x = var_698_cast_fp16_0)[name = string("x_99_cast_fp16")]; + tensor var_726 = const()[name = string("op_726"), val = tensor([1, 64, 8, 64])]; + tensor x_103_cast_fp16 = reshape(shape = var_726, x = var_698_cast_fp16_1)[name = string("x_103_cast_fp16")]; + tensor var_728 = const()[name = string("op_728"), val = tensor([0, 2, 1, 3])]; + bool sim_9_transpose_x_0 = const()[name = string("sim_9_transpose_x_0"), val = bool(false)]; + bool sim_9_transpose_y_0 = const()[name = string("sim_9_transpose_y_0"), val = bool(false)]; + tensor transpose_76_perm_0 = const()[name = string("transpose_76_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_77_perm_0 = const()[name = string("transpose_77_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_77 = transpose(perm = transpose_77_perm_0, x = x_99_cast_fp16)[name = string("transpose_312")]; + tensor transpose_76 = transpose(perm = transpose_76_perm_0, x = x_95_cast_fp16)[name = string("transpose_313")]; + tensor sim_9_cast_fp16 = matmul(transpose_x = sim_9_transpose_x_0, transpose_y = sim_9_transpose_y_0, x = transpose_76, y = transpose_77)[name = string("sim_9_cast_fp16")]; + fp16 var_732_to_fp16 = const()[name = string("op_732_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_11_cast_fp16 = mul(x = sim_9_cast_fp16, y = var_732_to_fp16)[name = string("sim_11_cast_fp16")]; + tensor attn_5_cast_fp16 = softmax(axis = var_35, x = sim_11_cast_fp16)[name = string("attn_5_cast_fp16")]; + bool x_105_transpose_x_0 = const()[name = string("x_105_transpose_x_0"), val = bool(false)]; + bool x_105_transpose_y_0 = const()[name = string("x_105_transpose_y_0"), val = bool(false)]; + tensor v_5_cast_fp16 = transpose(perm = var_728, x = x_103_cast_fp16)[name = string("transpose_314")]; + tensor x_105_cast_fp16 = matmul(transpose_x = x_105_transpose_x_0, transpose_y = x_105_transpose_y_0, x = attn_5_cast_fp16, y = v_5_cast_fp16)[name = string("x_105_cast_fp16")]; + tensor var_754 = const()[name = string("op_754"), val = tensor([0, 2, 1, 3])]; + tensor var_756 = const()[name = string("op_756"), val = tensor([1, 64, 512])]; + tensor x_107_cast_fp16 = transpose(perm = var_754, x = x_105_cast_fp16)[name = string("transpose_311")]; + tensor input_55_cast_fp16 = reshape(shape = var_756, x = x_107_cast_fp16)[name = string("input_55_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(39376384)))]; + tensor unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(40425024)))]; + tensor linear_22_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_55_cast_fp16)[name = string("linear_22_cast_fp16")]; + tensor input_57_cast_fp16 = add(x = linear_22_cast_fp16, y = x_79_cast_fp16)[name = string("input_57_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(40427136)))]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(44621504)))]; + tensor linear_23_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_57_cast_fp16)[name = string("linear_23_cast_fp16")]; + string input_61_mode_0 = const()[name = string("input_61_mode_0"), val = string("EXACT")]; + tensor input_61_cast_fp16 = gelu(mode = input_61_mode_0, x = linear_23_cast_fp16)[name = string("input_61_cast_fp16")]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(44625664)))]; + tensor unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(48820032)))]; + tensor linear_24_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_61_cast_fp16)[name = string("linear_24_cast_fp16")]; + tensor x_109_cast_fp16 = add(x = linear_24_cast_fp16, y = input_57_cast_fp16)[name = string("x_109_cast_fp16")]; + tensor var_765_axes_0 = const()[name = string("op_765_axes_0"), val = tensor([1])]; + bool var_765_keep_dims_0 = const()[name = string("op_765_keep_dims_0"), val = bool(false)]; + tensor var_765_cast_fp16 = reduce_mean(axes = var_765_axes_0, keep_dims = var_765_keep_dims_0, x = x_109_cast_fp16)[name = string("op_765_cast_fp16")]; + tensor x_111_axes_0 = const()[name = string("x_111_axes_0"), val = tensor([1])]; + tensor x_111_cast_fp16 = expand_dims(axes = x_111_axes_0, x = var_765_cast_fp16)[name = string("x_111_cast_fp16")]; + tensor var_767 = const()[name = string("op_767"), val = tensor([0, 2, 1])]; + string x_113_pad_type_0 = const()[name = string("x_113_pad_type_0"), val = string("valid")]; + tensor x_113_strides_0 = const()[name = string("x_113_strides_0"), val = tensor([1])]; + tensor x_113_pad_0 = const()[name = string("x_113_pad_0"), val = tensor([0, 0])]; + tensor x_113_dilations_0 = const()[name = string("x_113_dilations_0"), val = tensor([1])]; + int32 x_113_groups_0 = const()[name = string("x_113_groups_0"), val = int32(1)]; + tensor unet_step_kdiffusion_net_to_out_1_weight_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_out_1_weight_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(48822144)))]; + tensor unet_step_kdiffusion_net_to_out_1_bias_to_fp16 = const()[name = string("unet_step_kdiffusion_net_to_out_1_bias_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49346496)))]; + tensor input_63_cast_fp16 = transpose(perm = var_767, x = x_111_cast_fp16)[name = string("transpose_310")]; + tensor x_113_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_113_dilations_0, groups = x_113_groups_0, pad = x_113_pad_0, pad_type = x_113_pad_type_0, strides = x_113_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_63_cast_fp16)[name = string("x_113_cast_fp16")]; + tensor x_pred_1_perm_0 = const()[name = string("x_pred_1_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_1_to_fp16 = const()[name = string("c_skip_1_to_fp16"), val = tensor([[[0x1.1fcp-8]]])]; + tensor var_775_cast_fp16 = mul(x = c_skip_1_to_fp16, y = x_noisy_1_cast_fp16)[name = string("op_775_cast_fp16")]; + tensor c_out_1_to_fp16 = const()[name = string("c_out_1_to_fp16"), val = tensor([[[0x1.974p-3]]])]; + tensor x_pred_1_cast_fp16 = transpose(perm = x_pred_1_perm_0, x = x_113_cast_fp16)[name = string("transpose_309")]; + tensor var_776_cast_fp16 = mul(x = c_out_1_to_fp16, y = x_pred_1_cast_fp16)[name = string("op_776_cast_fp16")]; + tensor x_dn_1_cast_fp16 = add(x = var_775_cast_fp16, y = var_776_cast_fp16)[name = string("x_dn_1_cast_fp16")]; + tensor var_779_cast_fp16 = sub(x = x_noisy_1_cast_fp16, y = x_dn_1_cast_fp16)[name = string("op_779_cast_fp16")]; + tensor _inversed_d_1_y_0_to_fp16 = const()[name = string("_inversed_d_1_y_0_to_fp16"), val = tensor([0x1.554p-2])]; + tensor _inversed_d_1_cast_fp16 = mul(x = var_779_cast_fp16, y = _inversed_d_1_y_0_to_fp16)[name = string("_inversed_d_1_cast_fp16")]; + fp16 var_788_to_fp16 = const()[name = string("op_788_to_fp16"), val = fp16(-0x1.72cp+0)]; + tensor var_789_cast_fp16 = mul(x = _inversed_d_1_cast_fp16, y = var_788_to_fp16)[name = string("op_789_cast_fp16")]; + tensor x_noisy_3_cast_fp16 = add(x = x_noisy_1_cast_fp16, y = var_789_cast_fp16)[name = string("x_noisy_3_cast_fp16")]; + int32 var_801 = const()[name = string("op_801"), val = int32(-1)]; + tensor c_in_3_to_fp16 = const()[name = string("c_in_3_to_fp16"), val = tensor([[[0x1.474p-1]]])]; + tensor x_123_cast_fp16 = mul(x = c_in_3_to_fp16, y = x_noisy_3_cast_fp16)[name = string("x_123_cast_fp16")]; + int32 x_119_axis_0 = const()[name = string("x_119_axis_0"), val = int32(0)]; + tensor var_1187_to_fp16 = const()[name = string("op_1187_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49347072)))]; + tensor x_119_cast_fp16 = stack(axis = x_119_axis_0, values = (var_1187_to_fp16, var_423_cast_fp16))[name = string("x_119_cast_fp16")]; + tensor var_1192 = const()[name = string("op_1192"), val = tensor([1, 2, 0])]; + tensor input_71_axes_0 = const()[name = string("input_71_axes_0"), val = tensor([2])]; + bool input_71_keep_dims_0 = const()[name = string("input_71_keep_dims_0"), val = bool(false)]; + tensor x_121_cast_fp16 = transpose(perm = var_1192, x = x_119_cast_fp16)[name = string("transpose_308")]; + tensor input_71_cast_fp16 = reduce_sum(axes = input_71_axes_0, keep_dims = input_71_keep_dims_0, x = x_121_cast_fp16)[name = string("input_71_cast_fp16")]; + tensor linear_27_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_71_cast_fp16)[name = string("linear_27_cast_fp16")]; + string input_75_mode_0 = const()[name = string("input_75_mode_0"), val = string("EXACT")]; + tensor input_75_cast_fp16 = gelu(mode = input_75_mode_0, x = linear_27_cast_fp16)[name = string("input_75_cast_fp16")]; + tensor linear_28_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_75_cast_fp16)[name = string("linear_28_cast_fp16")]; + string mapping_5_mode_0 = const()[name = string("mapping_5_mode_0"), val = string("EXACT")]; + tensor mapping_5_cast_fp16 = gelu(mode = mapping_5_mode_0, x = linear_28_cast_fp16)[name = string("mapping_5_cast_fp16")]; + tensor var_1202_reps_0 = const()[name = string("op_1202_reps_0"), val = tensor([1, 64, 1])]; + tensor var_1202_cast_fp16 = tile(reps = var_1202_reps_0, x = x_123_cast_fp16)[name = string("op_1202_cast_fp16")]; + bool x_125_interleave_0 = const()[name = string("x_125_interleave_0"), val = bool(false)]; + tensor x_125_cast_fp16 = concat(axis = var_801, interleave = x_125_interleave_0, values = (var_1202_cast_fp16, embedding_to_fp16))[name = string("x_125_cast_fp16")]; + tensor var_1205_axes_0 = const()[name = string("op_1205_axes_0"), val = tensor([1])]; + tensor var_1205_cast_fp16 = expand_dims(axes = var_1205_axes_0, x = mapping_5_cast_fp16)[name = string("op_1205_cast_fp16")]; + tensor mapping_7_reps_0 = const()[name = string("mapping_7_reps_0"), val = tensor([1, 64, 1])]; + tensor mapping_7_cast_fp16 = tile(reps = mapping_7_reps_0, x = var_1205_cast_fp16)[name = string("mapping_7_cast_fp16")]; + tensor x_127_cast_fp16 = add(x = x_125_cast_fp16, y = mapping_7_cast_fp16)[name = string("x_127_cast_fp16")]; + tensor var_1217_split_sizes_0 = const()[name = string("op_1217_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1217_axis_0 = const()[name = string("op_1217_axis_0"), val = int32(1)]; + tensor var_1217_cast_fp16_0, tensor var_1217_cast_fp16_1 = split(axis = var_1217_axis_0, split_sizes = var_1217_split_sizes_0, x = h_3_cast_fp16)[name = string("op_1217_cast_fp16")]; + tensor gamma_27_perm_0 = const()[name = string("gamma_27_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_27_perm_0 = const()[name = string("beta_27_perm_0"), val = tensor([0, -1, 1])]; + tensor x_131_axes_0 = const()[name = string("x_131_axes_0"), val = tensor([-1])]; + fp16 var_797_to_fp16 = const()[name = string("op_797_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_131_cast_fp16 = layer_norm(axes = x_131_axes_0, epsilon = var_797_to_fp16, x = x_127_cast_fp16)[name = string("x_131_cast_fp16")]; + fp16 var_1223_promoted_to_fp16 = const()[name = string("op_1223_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_27_cast_fp16 = transpose(perm = gamma_27_perm_0, x = var_1217_cast_fp16_0)[name = string("transpose_307")]; + tensor var_1224_cast_fp16 = add(x = gamma_27_cast_fp16, y = var_1223_promoted_to_fp16)[name = string("op_1224_cast_fp16")]; + tensor var_1225_cast_fp16 = mul(x = var_1224_cast_fp16, y = x_131_cast_fp16)[name = string("op_1225_cast_fp16")]; + tensor beta_27_cast_fp16 = transpose(perm = beta_27_perm_0, x = var_1217_cast_fp16_1)[name = string("transpose_306")]; + tensor x_133_cast_fp16 = add(x = var_1225_cast_fp16, y = beta_27_cast_fp16)[name = string("x_133_cast_fp16")]; + tensor var_1236_split_sizes_0 = const()[name = string("op_1236_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1236_axis_0 = const()[name = string("op_1236_axis_0"), val = int32(1)]; + tensor var_1236_cast_fp16_0, tensor var_1236_cast_fp16_1 = split(axis = var_1236_axis_0, split_sizes = var_1236_split_sizes_0, x = h_7_cast_fp16)[name = string("op_1236_cast_fp16")]; + tensor gamma_31_perm_0 = const()[name = string("gamma_31_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_31_perm_0 = const()[name = string("beta_31_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_1242_promoted_to_fp16 = const()[name = string("op_1242_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_31_cast_fp16 = transpose(perm = gamma_31_perm_0, x = var_1236_cast_fp16_0)[name = string("transpose_305")]; + tensor var_1243_cast_fp16 = add(x = gamma_31_cast_fp16, y = var_1242_promoted_to_fp16)[name = string("op_1243_cast_fp16")]; + tensor var_1244_cast_fp16 = mul(x = var_1243_cast_fp16, y = x_131_cast_fp16)[name = string("op_1244_cast_fp16")]; + tensor beta_31_cast_fp16 = transpose(perm = beta_31_perm_0, x = var_1236_cast_fp16_1)[name = string("transpose_304")]; + tensor x_139_cast_fp16 = add(x = var_1244_cast_fp16, y = beta_31_cast_fp16)[name = string("x_139_cast_fp16")]; + tensor linear_31_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_133_cast_fp16)[name = string("linear_31_cast_fp16")]; + tensor linear_32_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_139_cast_fp16)[name = string("linear_32_cast_fp16")]; + tensor var_1250_split_sizes_0 = const()[name = string("op_1250_split_sizes_0"), val = tensor([512, 512])]; + int32 var_1250_axis_0 = const()[name = string("op_1250_axis_0"), val = int32(-1)]; + tensor var_1250_cast_fp16_0, tensor var_1250_cast_fp16_1 = split(axis = var_1250_axis_0, split_sizes = var_1250_split_sizes_0, x = linear_32_cast_fp16)[name = string("op_1250_cast_fp16")]; + tensor var_1258 = const()[name = string("op_1258"), val = tensor([1, 64, 8, 64])]; + tensor x_143_cast_fp16 = reshape(shape = var_1258, x = linear_31_cast_fp16)[name = string("x_143_cast_fp16")]; + tensor var_1268 = const()[name = string("op_1268"), val = tensor([1, 64, 8, 64])]; + tensor x_147_cast_fp16 = reshape(shape = var_1268, x = var_1250_cast_fp16_0)[name = string("x_147_cast_fp16")]; + tensor var_1278 = const()[name = string("op_1278"), val = tensor([1, 64, 8, 64])]; + tensor x_151_cast_fp16 = reshape(shape = var_1278, x = var_1250_cast_fp16_1)[name = string("x_151_cast_fp16")]; + tensor var_1280 = const()[name = string("op_1280"), val = tensor([0, 2, 1, 3])]; + bool sim_13_transpose_x_0 = const()[name = string("sim_13_transpose_x_0"), val = bool(false)]; + bool sim_13_transpose_y_0 = const()[name = string("sim_13_transpose_y_0"), val = bool(false)]; + tensor transpose_78_perm_0 = const()[name = string("transpose_78_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_79_perm_0 = const()[name = string("transpose_79_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_79 = transpose(perm = transpose_79_perm_0, x = x_147_cast_fp16)[name = string("transpose_301")]; + tensor transpose_78 = transpose(perm = transpose_78_perm_0, x = x_143_cast_fp16)[name = string("transpose_302")]; + tensor sim_13_cast_fp16 = matmul(transpose_x = sim_13_transpose_x_0, transpose_y = sim_13_transpose_y_0, x = transpose_78, y = transpose_79)[name = string("sim_13_cast_fp16")]; + fp16 var_1284_to_fp16 = const()[name = string("op_1284_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_15_cast_fp16 = mul(x = sim_13_cast_fp16, y = var_1284_to_fp16)[name = string("sim_15_cast_fp16")]; + tensor attn_7_cast_fp16 = softmax(axis = var_801, x = sim_15_cast_fp16)[name = string("attn_7_cast_fp16")]; + bool x_153_transpose_x_0 = const()[name = string("x_153_transpose_x_0"), val = bool(false)]; + bool x_153_transpose_y_0 = const()[name = string("x_153_transpose_y_0"), val = bool(false)]; + tensor v_7_cast_fp16 = transpose(perm = var_1280, x = x_151_cast_fp16)[name = string("transpose_303")]; + tensor x_153_cast_fp16 = matmul(transpose_x = x_153_transpose_x_0, transpose_y = x_153_transpose_y_0, x = attn_7_cast_fp16, y = v_7_cast_fp16)[name = string("x_153_cast_fp16")]; + tensor var_1306 = const()[name = string("op_1306"), val = tensor([0, 2, 1, 3])]; + tensor var_1308 = const()[name = string("op_1308"), val = tensor([1, 64, 512])]; + tensor x_155_cast_fp16 = transpose(perm = var_1306, x = x_153_cast_fp16)[name = string("transpose_300")]; + tensor input_87_cast_fp16 = reshape(shape = var_1308, x = x_155_cast_fp16)[name = string("input_87_cast_fp16")]; + tensor linear_33_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_87_cast_fp16)[name = string("linear_33_cast_fp16")]; + tensor input_89_cast_fp16 = add(x = linear_33_cast_fp16, y = x_127_cast_fp16)[name = string("input_89_cast_fp16")]; + tensor linear_34_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_89_cast_fp16)[name = string("linear_34_cast_fp16")]; + string input_93_mode_0 = const()[name = string("input_93_mode_0"), val = string("EXACT")]; + tensor input_93_cast_fp16 = gelu(mode = input_93_mode_0, x = linear_34_cast_fp16)[name = string("input_93_cast_fp16")]; + tensor linear_35_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_93_cast_fp16)[name = string("linear_35_cast_fp16")]; + tensor x_157_cast_fp16 = add(x = linear_35_cast_fp16, y = input_89_cast_fp16)[name = string("x_157_cast_fp16")]; + tensor x_159_cast_fp16 = add(x = x_157_cast_fp16, y = mapping_7_cast_fp16)[name = string("x_159_cast_fp16")]; + tensor var_1324_split_sizes_0 = const()[name = string("op_1324_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1324_axis_0 = const()[name = string("op_1324_axis_0"), val = int32(1)]; + tensor var_1324_cast_fp16_0, tensor var_1324_cast_fp16_1 = split(axis = var_1324_axis_0, split_sizes = var_1324_split_sizes_0, x = h_11_cast_fp16)[name = string("op_1324_cast_fp16")]; + tensor gamma_35_perm_0 = const()[name = string("gamma_35_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_35_perm_0 = const()[name = string("beta_35_perm_0"), val = tensor([0, -1, 1])]; + tensor x_163_axes_0 = const()[name = string("x_163_axes_0"), val = tensor([-1])]; + tensor x_163_cast_fp16 = layer_norm(axes = x_163_axes_0, epsilon = var_797_to_fp16, x = x_159_cast_fp16)[name = string("x_163_cast_fp16")]; + fp16 var_1330_promoted_to_fp16 = const()[name = string("op_1330_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_35_cast_fp16 = transpose(perm = gamma_35_perm_0, x = var_1324_cast_fp16_0)[name = string("transpose_299")]; + tensor var_1331_cast_fp16 = add(x = gamma_35_cast_fp16, y = var_1330_promoted_to_fp16)[name = string("op_1331_cast_fp16")]; + tensor var_1332_cast_fp16 = mul(x = var_1331_cast_fp16, y = x_163_cast_fp16)[name = string("op_1332_cast_fp16")]; + tensor beta_35_cast_fp16 = transpose(perm = beta_35_perm_0, x = var_1324_cast_fp16_1)[name = string("transpose_298")]; + tensor x_165_cast_fp16 = add(x = var_1332_cast_fp16, y = beta_35_cast_fp16)[name = string("x_165_cast_fp16")]; + tensor var_1343_split_sizes_0 = const()[name = string("op_1343_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1343_axis_0 = const()[name = string("op_1343_axis_0"), val = int32(1)]; + tensor var_1343_cast_fp16_0, tensor var_1343_cast_fp16_1 = split(axis = var_1343_axis_0, split_sizes = var_1343_split_sizes_0, x = h_15_cast_fp16)[name = string("op_1343_cast_fp16")]; + tensor gamma_39_perm_0 = const()[name = string("gamma_39_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_39_perm_0 = const()[name = string("beta_39_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_1349_promoted_to_fp16 = const()[name = string("op_1349_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_39_cast_fp16 = transpose(perm = gamma_39_perm_0, x = var_1343_cast_fp16_0)[name = string("transpose_297")]; + tensor var_1350_cast_fp16 = add(x = gamma_39_cast_fp16, y = var_1349_promoted_to_fp16)[name = string("op_1350_cast_fp16")]; + tensor var_1351_cast_fp16 = mul(x = var_1350_cast_fp16, y = x_163_cast_fp16)[name = string("op_1351_cast_fp16")]; + tensor beta_39_cast_fp16 = transpose(perm = beta_39_perm_0, x = var_1343_cast_fp16_1)[name = string("transpose_296")]; + tensor x_171_cast_fp16 = add(x = var_1351_cast_fp16, y = beta_39_cast_fp16)[name = string("x_171_cast_fp16")]; + tensor linear_38_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_165_cast_fp16)[name = string("linear_38_cast_fp16")]; + tensor linear_39_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_171_cast_fp16)[name = string("linear_39_cast_fp16")]; + tensor var_1357_split_sizes_0 = const()[name = string("op_1357_split_sizes_0"), val = tensor([512, 512])]; + int32 var_1357_axis_0 = const()[name = string("op_1357_axis_0"), val = int32(-1)]; + tensor var_1357_cast_fp16_0, tensor var_1357_cast_fp16_1 = split(axis = var_1357_axis_0, split_sizes = var_1357_split_sizes_0, x = linear_39_cast_fp16)[name = string("op_1357_cast_fp16")]; + tensor var_1365 = const()[name = string("op_1365"), val = tensor([1, 64, 8, 64])]; + tensor x_175_cast_fp16 = reshape(shape = var_1365, x = linear_38_cast_fp16)[name = string("x_175_cast_fp16")]; + tensor var_1375 = const()[name = string("op_1375"), val = tensor([1, 64, 8, 64])]; + tensor x_179_cast_fp16 = reshape(shape = var_1375, x = var_1357_cast_fp16_0)[name = string("x_179_cast_fp16")]; + tensor var_1385 = const()[name = string("op_1385"), val = tensor([1, 64, 8, 64])]; + tensor x_183_cast_fp16 = reshape(shape = var_1385, x = var_1357_cast_fp16_1)[name = string("x_183_cast_fp16")]; + tensor var_1387 = const()[name = string("op_1387"), val = tensor([0, 2, 1, 3])]; + bool sim_17_transpose_x_0 = const()[name = string("sim_17_transpose_x_0"), val = bool(false)]; + bool sim_17_transpose_y_0 = const()[name = string("sim_17_transpose_y_0"), val = bool(false)]; + tensor transpose_80_perm_0 = const()[name = string("transpose_80_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_81_perm_0 = const()[name = string("transpose_81_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_81 = transpose(perm = transpose_81_perm_0, x = x_179_cast_fp16)[name = string("transpose_293")]; + tensor transpose_80 = transpose(perm = transpose_80_perm_0, x = x_175_cast_fp16)[name = string("transpose_294")]; + tensor sim_17_cast_fp16 = matmul(transpose_x = sim_17_transpose_x_0, transpose_y = sim_17_transpose_y_0, x = transpose_80, y = transpose_81)[name = string("sim_17_cast_fp16")]; + fp16 var_1391_to_fp16 = const()[name = string("op_1391_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_19_cast_fp16 = mul(x = sim_17_cast_fp16, y = var_1391_to_fp16)[name = string("sim_19_cast_fp16")]; + tensor attn_9_cast_fp16 = softmax(axis = var_801, x = sim_19_cast_fp16)[name = string("attn_9_cast_fp16")]; + bool x_185_transpose_x_0 = const()[name = string("x_185_transpose_x_0"), val = bool(false)]; + bool x_185_transpose_y_0 = const()[name = string("x_185_transpose_y_0"), val = bool(false)]; + tensor v_9_cast_fp16 = transpose(perm = var_1387, x = x_183_cast_fp16)[name = string("transpose_295")]; + tensor x_185_cast_fp16 = matmul(transpose_x = x_185_transpose_x_0, transpose_y = x_185_transpose_y_0, x = attn_9_cast_fp16, y = v_9_cast_fp16)[name = string("x_185_cast_fp16")]; + tensor var_1413 = const()[name = string("op_1413"), val = tensor([0, 2, 1, 3])]; + tensor var_1415 = const()[name = string("op_1415"), val = tensor([1, 64, 512])]; + tensor x_187_cast_fp16 = transpose(perm = var_1413, x = x_185_cast_fp16)[name = string("transpose_292")]; + tensor input_103_cast_fp16 = reshape(shape = var_1415, x = x_187_cast_fp16)[name = string("input_103_cast_fp16")]; + tensor linear_40_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_103_cast_fp16)[name = string("linear_40_cast_fp16")]; + tensor input_105_cast_fp16 = add(x = linear_40_cast_fp16, y = x_159_cast_fp16)[name = string("input_105_cast_fp16")]; + tensor linear_41_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_105_cast_fp16)[name = string("linear_41_cast_fp16")]; + string input_109_mode_0 = const()[name = string("input_109_mode_0"), val = string("EXACT")]; + tensor input_109_cast_fp16 = gelu(mode = input_109_mode_0, x = linear_41_cast_fp16)[name = string("input_109_cast_fp16")]; + tensor linear_42_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_109_cast_fp16)[name = string("linear_42_cast_fp16")]; + tensor x_189_cast_fp16 = add(x = linear_42_cast_fp16, y = input_105_cast_fp16)[name = string("x_189_cast_fp16")]; + tensor x_191_cast_fp16 = add(x = x_189_cast_fp16, y = mapping_7_cast_fp16)[name = string("x_191_cast_fp16")]; + tensor var_1431_split_sizes_0 = const()[name = string("op_1431_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1431_axis_0 = const()[name = string("op_1431_axis_0"), val = int32(1)]; + tensor var_1431_cast_fp16_0, tensor var_1431_cast_fp16_1 = split(axis = var_1431_axis_0, split_sizes = var_1431_split_sizes_0, x = h_19_cast_fp16)[name = string("op_1431_cast_fp16")]; + tensor gamma_43_perm_0 = const()[name = string("gamma_43_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_43_perm_0 = const()[name = string("beta_43_perm_0"), val = tensor([0, -1, 1])]; + tensor x_195_axes_0 = const()[name = string("x_195_axes_0"), val = tensor([-1])]; + tensor x_195_cast_fp16 = layer_norm(axes = x_195_axes_0, epsilon = var_797_to_fp16, x = x_191_cast_fp16)[name = string("x_195_cast_fp16")]; + fp16 var_1437_promoted_to_fp16 = const()[name = string("op_1437_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_43_cast_fp16 = transpose(perm = gamma_43_perm_0, x = var_1431_cast_fp16_0)[name = string("transpose_291")]; + tensor var_1438_cast_fp16 = add(x = gamma_43_cast_fp16, y = var_1437_promoted_to_fp16)[name = string("op_1438_cast_fp16")]; + tensor var_1439_cast_fp16 = mul(x = var_1438_cast_fp16, y = x_195_cast_fp16)[name = string("op_1439_cast_fp16")]; + tensor beta_43_cast_fp16 = transpose(perm = beta_43_perm_0, x = var_1431_cast_fp16_1)[name = string("transpose_290")]; + tensor x_197_cast_fp16 = add(x = var_1439_cast_fp16, y = beta_43_cast_fp16)[name = string("x_197_cast_fp16")]; + tensor var_1450_split_sizes_0 = const()[name = string("op_1450_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_1450_axis_0 = const()[name = string("op_1450_axis_0"), val = int32(1)]; + tensor var_1450_cast_fp16_0, tensor var_1450_cast_fp16_1 = split(axis = var_1450_axis_0, split_sizes = var_1450_split_sizes_0, x = h_23_cast_fp16)[name = string("op_1450_cast_fp16")]; + tensor gamma_47_perm_0 = const()[name = string("gamma_47_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_47_perm_0 = const()[name = string("beta_47_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_1456_promoted_to_fp16 = const()[name = string("op_1456_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_47_cast_fp16 = transpose(perm = gamma_47_perm_0, x = var_1450_cast_fp16_0)[name = string("transpose_289")]; + tensor var_1457_cast_fp16 = add(x = gamma_47_cast_fp16, y = var_1456_promoted_to_fp16)[name = string("op_1457_cast_fp16")]; + tensor var_1458_cast_fp16 = mul(x = var_1457_cast_fp16, y = x_195_cast_fp16)[name = string("op_1458_cast_fp16")]; + tensor beta_47_cast_fp16 = transpose(perm = beta_47_perm_0, x = var_1450_cast_fp16_1)[name = string("transpose_288")]; + tensor x_203_cast_fp16 = add(x = var_1458_cast_fp16, y = beta_47_cast_fp16)[name = string("x_203_cast_fp16")]; + tensor linear_45_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_197_cast_fp16)[name = string("linear_45_cast_fp16")]; + tensor linear_46_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_203_cast_fp16)[name = string("linear_46_cast_fp16")]; + tensor var_1464_split_sizes_0 = const()[name = string("op_1464_split_sizes_0"), val = tensor([512, 512])]; + int32 var_1464_axis_0 = const()[name = string("op_1464_axis_0"), val = int32(-1)]; + tensor var_1464_cast_fp16_0, tensor var_1464_cast_fp16_1 = split(axis = var_1464_axis_0, split_sizes = var_1464_split_sizes_0, x = linear_46_cast_fp16)[name = string("op_1464_cast_fp16")]; + tensor var_1472 = const()[name = string("op_1472"), val = tensor([1, 64, 8, 64])]; + tensor x_207_cast_fp16 = reshape(shape = var_1472, x = linear_45_cast_fp16)[name = string("x_207_cast_fp16")]; + tensor var_1482 = const()[name = string("op_1482"), val = tensor([1, 64, 8, 64])]; + tensor x_211_cast_fp16 = reshape(shape = var_1482, x = var_1464_cast_fp16_0)[name = string("x_211_cast_fp16")]; + tensor var_1492 = const()[name = string("op_1492"), val = tensor([1, 64, 8, 64])]; + tensor x_215_cast_fp16 = reshape(shape = var_1492, x = var_1464_cast_fp16_1)[name = string("x_215_cast_fp16")]; + tensor var_1494 = const()[name = string("op_1494"), val = tensor([0, 2, 1, 3])]; + bool sim_21_transpose_x_0 = const()[name = string("sim_21_transpose_x_0"), val = bool(false)]; + bool sim_21_transpose_y_0 = const()[name = string("sim_21_transpose_y_0"), val = bool(false)]; + tensor transpose_82_perm_0 = const()[name = string("transpose_82_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_83_perm_0 = const()[name = string("transpose_83_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_83 = transpose(perm = transpose_83_perm_0, x = x_211_cast_fp16)[name = string("transpose_285")]; + tensor transpose_82 = transpose(perm = transpose_82_perm_0, x = x_207_cast_fp16)[name = string("transpose_286")]; + tensor sim_21_cast_fp16 = matmul(transpose_x = sim_21_transpose_x_0, transpose_y = sim_21_transpose_y_0, x = transpose_82, y = transpose_83)[name = string("sim_21_cast_fp16")]; + fp16 var_1498_to_fp16 = const()[name = string("op_1498_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_23_cast_fp16 = mul(x = sim_21_cast_fp16, y = var_1498_to_fp16)[name = string("sim_23_cast_fp16")]; + tensor attn_11_cast_fp16 = softmax(axis = var_801, x = sim_23_cast_fp16)[name = string("attn_11_cast_fp16")]; + bool x_217_transpose_x_0 = const()[name = string("x_217_transpose_x_0"), val = bool(false)]; + bool x_217_transpose_y_0 = const()[name = string("x_217_transpose_y_0"), val = bool(false)]; + tensor v_11_cast_fp16 = transpose(perm = var_1494, x = x_215_cast_fp16)[name = string("transpose_287")]; + tensor x_217_cast_fp16 = matmul(transpose_x = x_217_transpose_x_0, transpose_y = x_217_transpose_y_0, x = attn_11_cast_fp16, y = v_11_cast_fp16)[name = string("x_217_cast_fp16")]; + tensor var_1520 = const()[name = string("op_1520"), val = tensor([0, 2, 1, 3])]; + tensor var_1522 = const()[name = string("op_1522"), val = tensor([1, 64, 512])]; + tensor x_219_cast_fp16 = transpose(perm = var_1520, x = x_217_cast_fp16)[name = string("transpose_284")]; + tensor input_119_cast_fp16 = reshape(shape = var_1522, x = x_219_cast_fp16)[name = string("input_119_cast_fp16")]; + tensor linear_47_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_119_cast_fp16)[name = string("linear_47_cast_fp16")]; + tensor input_121_cast_fp16 = add(x = linear_47_cast_fp16, y = x_191_cast_fp16)[name = string("input_121_cast_fp16")]; + tensor linear_48_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_121_cast_fp16)[name = string("linear_48_cast_fp16")]; + string input_125_mode_0 = const()[name = string("input_125_mode_0"), val = string("EXACT")]; + tensor input_125_cast_fp16 = gelu(mode = input_125_mode_0, x = linear_48_cast_fp16)[name = string("input_125_cast_fp16")]; + tensor linear_49_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_125_cast_fp16)[name = string("linear_49_cast_fp16")]; + tensor x_221_cast_fp16 = add(x = linear_49_cast_fp16, y = input_121_cast_fp16)[name = string("x_221_cast_fp16")]; + tensor var_1531_axes_0 = const()[name = string("op_1531_axes_0"), val = tensor([1])]; + bool var_1531_keep_dims_0 = const()[name = string("op_1531_keep_dims_0"), val = bool(false)]; + tensor var_1531_cast_fp16 = reduce_mean(axes = var_1531_axes_0, keep_dims = var_1531_keep_dims_0, x = x_221_cast_fp16)[name = string("op_1531_cast_fp16")]; + tensor x_223_axes_0 = const()[name = string("x_223_axes_0"), val = tensor([1])]; + tensor x_223_cast_fp16 = expand_dims(axes = x_223_axes_0, x = var_1531_cast_fp16)[name = string("x_223_cast_fp16")]; + tensor var_1533 = const()[name = string("op_1533"), val = tensor([0, 2, 1])]; + string x_225_pad_type_0 = const()[name = string("x_225_pad_type_0"), val = string("valid")]; + tensor x_225_strides_0 = const()[name = string("x_225_strides_0"), val = tensor([1])]; + tensor x_225_pad_0 = const()[name = string("x_225_pad_0"), val = tensor([0, 0])]; + tensor x_225_dilations_0 = const()[name = string("x_225_dilations_0"), val = tensor([1])]; + int32 x_225_groups_0 = const()[name = string("x_225_groups_0"), val = int32(1)]; + tensor input_127_cast_fp16 = transpose(perm = var_1533, x = x_223_cast_fp16)[name = string("transpose_283")]; + tensor x_225_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_225_dilations_0, groups = x_225_groups_0, pad = x_225_pad_0, pad_type = x_225_pad_type_0, strides = x_225_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_127_cast_fp16)[name = string("x_225_cast_fp16")]; + tensor x_pred_3_perm_0 = const()[name = string("x_pred_3_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_3_to_fp16 = const()[name = string("c_skip_3_to_fp16"), val = tensor([[[0x1.09cp-6]]])]; + tensor var_1541_cast_fp16 = mul(x = c_skip_3_to_fp16, y = x_noisy_3_cast_fp16)[name = string("op_1541_cast_fp16")]; + tensor c_out_3_to_fp16 = const()[name = string("c_out_3_to_fp16"), val = tensor([[[0x1.94cp-3]]])]; + tensor x_pred_3_cast_fp16 = transpose(perm = x_pred_3_perm_0, x = x_225_cast_fp16)[name = string("transpose_282")]; + tensor var_1542_cast_fp16 = mul(x = c_out_3_to_fp16, y = x_pred_3_cast_fp16)[name = string("op_1542_cast_fp16")]; + tensor x_mid_dn_1_cast_fp16 = add(x = var_1541_cast_fp16, y = var_1542_cast_fp16)[name = string("x_mid_dn_1_cast_fp16")]; + tensor var_1545_cast_fp16 = sub(x = x_noisy_3_cast_fp16, y = x_mid_dn_1_cast_fp16)[name = string("op_1545_cast_fp16")]; + tensor _inversed_d_mid_1_y_0_to_fp16 = const()[name = string("_inversed_d_mid_1_y_0_to_fp16"), val = tensor([0x1.4ap-1])]; + tensor _inversed_d_mid_1_cast_fp16 = mul(x = var_1545_cast_fp16, y = _inversed_d_mid_1_y_0_to_fp16)[name = string("_inversed_d_mid_1_cast_fp16")]; + fp16 var_1554_to_fp16 = const()[name = string("op_1554_to_fp16"), val = fp16(-0x1.72cp+1)]; + tensor var_1555_cast_fp16 = mul(x = _inversed_d_mid_1_cast_fp16, y = var_1554_to_fp16)[name = string("op_1555_cast_fp16")]; + tensor x_227_cast_fp16 = add(x = x_noisy_1_cast_fp16, y = var_1555_cast_fp16)[name = string("x_227_cast_fp16")]; + tensor var_1560_begin_0 = const()[name = string("op_1560_begin_0"), val = tensor([0, 0, 0, 0])]; + tensor var_1560_end_0 = const()[name = string("op_1560_end_0"), val = tensor([1, 1, 1, 256])]; + tensor var_1560_end_mask_0 = const()[name = string("op_1560_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_1560_squeeze_mask_0 = const()[name = string("op_1560_squeeze_mask_0"), val = tensor([true, false, false, false])]; + string noises_aux_to_fp16_dtype_0 = const()[name = string("noises_aux_to_fp16_dtype_0"), val = string("fp16")]; + tensor noises_aux_to_fp16 = cast(dtype = noises_aux_to_fp16_dtype_0, x = noises_aux)[name = string("cast_193")]; + tensor var_1560_cast_fp16 = slice_by_index(begin = var_1560_begin_0, end = var_1560_end_0, end_mask = var_1560_end_mask_0, squeeze_mask = var_1560_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_1560_cast_fp16")]; + fp16 var_1563_to_fp16 = const()[name = string("op_1563_to_fp16"), val = fp16(0x1.18cp-1)]; + tensor var_1564_cast_fp16 = mul(x = var_1560_cast_fp16, y = var_1563_to_fp16)[name = string("op_1564_cast_fp16")]; + tensor x_noisy_5_cast_fp16 = add(x = x_227_cast_fp16, y = var_1564_cast_fp16)[name = string("x_noisy_5_cast_fp16")]; + int32 var_1588 = const()[name = string("op_1588"), val = int32(-1)]; + tensor c_in_5_to_fp16 = const()[name = string("c_in_5_to_fp16"), val = tensor([[[0x1.bp+0]]])]; + tensor x_237_cast_fp16 = mul(x = c_in_5_to_fp16, y = x_noisy_5_cast_fp16)[name = string("x_237_cast_fp16")]; + int32 x_233_axis_0 = const()[name = string("x_233_axis_0"), val = int32(0)]; + tensor var_1974_to_fp16 = const()[name = string("op_1974_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49349184)))]; + tensor x_233_cast_fp16 = stack(axis = x_233_axis_0, values = (var_1974_to_fp16, var_423_cast_fp16))[name = string("x_233_cast_fp16")]; + tensor var_1979 = const()[name = string("op_1979"), val = tensor([1, 2, 0])]; + tensor input_135_axes_0 = const()[name = string("input_135_axes_0"), val = tensor([2])]; + bool input_135_keep_dims_0 = const()[name = string("input_135_keep_dims_0"), val = bool(false)]; + tensor x_235_cast_fp16 = transpose(perm = var_1979, x = x_233_cast_fp16)[name = string("transpose_281")]; + tensor input_135_cast_fp16 = reduce_sum(axes = input_135_axes_0, keep_dims = input_135_keep_dims_0, x = x_235_cast_fp16)[name = string("input_135_cast_fp16")]; + tensor linear_52_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_135_cast_fp16)[name = string("linear_52_cast_fp16")]; + string input_139_mode_0 = const()[name = string("input_139_mode_0"), val = string("EXACT")]; + tensor input_139_cast_fp16 = gelu(mode = input_139_mode_0, x = linear_52_cast_fp16)[name = string("input_139_cast_fp16")]; + tensor linear_53_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_139_cast_fp16)[name = string("linear_53_cast_fp16")]; + string mapping_9_mode_0 = const()[name = string("mapping_9_mode_0"), val = string("EXACT")]; + tensor mapping_9_cast_fp16 = gelu(mode = mapping_9_mode_0, x = linear_53_cast_fp16)[name = string("mapping_9_cast_fp16")]; + tensor var_1989_reps_0 = const()[name = string("op_1989_reps_0"), val = tensor([1, 64, 1])]; + tensor var_1989_cast_fp16 = tile(reps = var_1989_reps_0, x = x_237_cast_fp16)[name = string("op_1989_cast_fp16")]; + bool x_239_interleave_0 = const()[name = string("x_239_interleave_0"), val = bool(false)]; + tensor x_239_cast_fp16 = concat(axis = var_1588, interleave = x_239_interleave_0, values = (var_1989_cast_fp16, embedding_to_fp16))[name = string("x_239_cast_fp16")]; + tensor var_1992_axes_0 = const()[name = string("op_1992_axes_0"), val = tensor([1])]; + tensor var_1992_cast_fp16 = expand_dims(axes = var_1992_axes_0, x = mapping_9_cast_fp16)[name = string("op_1992_cast_fp16")]; + tensor mapping_11_reps_0 = const()[name = string("mapping_11_reps_0"), val = tensor([1, 64, 1])]; + tensor mapping_11_cast_fp16 = tile(reps = mapping_11_reps_0, x = var_1992_cast_fp16)[name = string("mapping_11_cast_fp16")]; + tensor x_241_cast_fp16 = add(x = x_239_cast_fp16, y = mapping_11_cast_fp16)[name = string("x_241_cast_fp16")]; + tensor var_2004_split_sizes_0 = const()[name = string("op_2004_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2004_axis_0 = const()[name = string("op_2004_axis_0"), val = int32(1)]; + tensor var_2004_cast_fp16_0, tensor var_2004_cast_fp16_1 = split(axis = var_2004_axis_0, split_sizes = var_2004_split_sizes_0, x = h_3_cast_fp16)[name = string("op_2004_cast_fp16")]; + tensor gamma_51_perm_0 = const()[name = string("gamma_51_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_51_perm_0 = const()[name = string("beta_51_perm_0"), val = tensor([0, -1, 1])]; + tensor x_245_axes_0 = const()[name = string("x_245_axes_0"), val = tensor([-1])]; + fp16 var_1584_to_fp16 = const()[name = string("op_1584_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_245_cast_fp16 = layer_norm(axes = x_245_axes_0, epsilon = var_1584_to_fp16, x = x_241_cast_fp16)[name = string("x_245_cast_fp16")]; + fp16 var_2010_promoted_to_fp16 = const()[name = string("op_2010_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_51_cast_fp16 = transpose(perm = gamma_51_perm_0, x = var_2004_cast_fp16_0)[name = string("transpose_280")]; + tensor var_2011_cast_fp16 = add(x = gamma_51_cast_fp16, y = var_2010_promoted_to_fp16)[name = string("op_2011_cast_fp16")]; + tensor var_2012_cast_fp16 = mul(x = var_2011_cast_fp16, y = x_245_cast_fp16)[name = string("op_2012_cast_fp16")]; + tensor beta_51_cast_fp16 = transpose(perm = beta_51_perm_0, x = var_2004_cast_fp16_1)[name = string("transpose_279")]; + tensor x_247_cast_fp16 = add(x = var_2012_cast_fp16, y = beta_51_cast_fp16)[name = string("x_247_cast_fp16")]; + tensor var_2023_split_sizes_0 = const()[name = string("op_2023_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2023_axis_0 = const()[name = string("op_2023_axis_0"), val = int32(1)]; + tensor var_2023_cast_fp16_0, tensor var_2023_cast_fp16_1 = split(axis = var_2023_axis_0, split_sizes = var_2023_split_sizes_0, x = h_7_cast_fp16)[name = string("op_2023_cast_fp16")]; + tensor gamma_55_perm_0 = const()[name = string("gamma_55_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_55_perm_0 = const()[name = string("beta_55_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2029_promoted_to_fp16 = const()[name = string("op_2029_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_55_cast_fp16 = transpose(perm = gamma_55_perm_0, x = var_2023_cast_fp16_0)[name = string("transpose_278")]; + tensor var_2030_cast_fp16 = add(x = gamma_55_cast_fp16, y = var_2029_promoted_to_fp16)[name = string("op_2030_cast_fp16")]; + tensor var_2031_cast_fp16 = mul(x = var_2030_cast_fp16, y = x_245_cast_fp16)[name = string("op_2031_cast_fp16")]; + tensor beta_55_cast_fp16 = transpose(perm = beta_55_perm_0, x = var_2023_cast_fp16_1)[name = string("transpose_277")]; + tensor x_253_cast_fp16 = add(x = var_2031_cast_fp16, y = beta_55_cast_fp16)[name = string("x_253_cast_fp16")]; + tensor linear_56_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_247_cast_fp16)[name = string("linear_56_cast_fp16")]; + tensor linear_57_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_253_cast_fp16)[name = string("linear_57_cast_fp16")]; + tensor var_2037_split_sizes_0 = const()[name = string("op_2037_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2037_axis_0 = const()[name = string("op_2037_axis_0"), val = int32(-1)]; + tensor var_2037_cast_fp16_0, tensor var_2037_cast_fp16_1 = split(axis = var_2037_axis_0, split_sizes = var_2037_split_sizes_0, x = linear_57_cast_fp16)[name = string("op_2037_cast_fp16")]; + tensor var_2045 = const()[name = string("op_2045"), val = tensor([1, 64, 8, 64])]; + tensor x_257_cast_fp16 = reshape(shape = var_2045, x = linear_56_cast_fp16)[name = string("x_257_cast_fp16")]; + tensor var_2055 = const()[name = string("op_2055"), val = tensor([1, 64, 8, 64])]; + tensor x_261_cast_fp16 = reshape(shape = var_2055, x = var_2037_cast_fp16_0)[name = string("x_261_cast_fp16")]; + tensor var_2065 = const()[name = string("op_2065"), val = tensor([1, 64, 8, 64])]; + tensor x_265_cast_fp16 = reshape(shape = var_2065, x = var_2037_cast_fp16_1)[name = string("x_265_cast_fp16")]; + tensor var_2067 = const()[name = string("op_2067"), val = tensor([0, 2, 1, 3])]; + bool sim_25_transpose_x_0 = const()[name = string("sim_25_transpose_x_0"), val = bool(false)]; + bool sim_25_transpose_y_0 = const()[name = string("sim_25_transpose_y_0"), val = bool(false)]; + tensor transpose_84_perm_0 = const()[name = string("transpose_84_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_85_perm_0 = const()[name = string("transpose_85_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_85 = transpose(perm = transpose_85_perm_0, x = x_261_cast_fp16)[name = string("transpose_274")]; + tensor transpose_84 = transpose(perm = transpose_84_perm_0, x = x_257_cast_fp16)[name = string("transpose_275")]; + tensor sim_25_cast_fp16 = matmul(transpose_x = sim_25_transpose_x_0, transpose_y = sim_25_transpose_y_0, x = transpose_84, y = transpose_85)[name = string("sim_25_cast_fp16")]; + fp16 var_2071_to_fp16 = const()[name = string("op_2071_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_27_cast_fp16 = mul(x = sim_25_cast_fp16, y = var_2071_to_fp16)[name = string("sim_27_cast_fp16")]; + tensor attn_13_cast_fp16 = softmax(axis = var_1588, x = sim_27_cast_fp16)[name = string("attn_13_cast_fp16")]; + bool x_267_transpose_x_0 = const()[name = string("x_267_transpose_x_0"), val = bool(false)]; + bool x_267_transpose_y_0 = const()[name = string("x_267_transpose_y_0"), val = bool(false)]; + tensor v_13_cast_fp16 = transpose(perm = var_2067, x = x_265_cast_fp16)[name = string("transpose_276")]; + tensor x_267_cast_fp16 = matmul(transpose_x = x_267_transpose_x_0, transpose_y = x_267_transpose_y_0, x = attn_13_cast_fp16, y = v_13_cast_fp16)[name = string("x_267_cast_fp16")]; + tensor var_2093 = const()[name = string("op_2093"), val = tensor([0, 2, 1, 3])]; + tensor var_2095 = const()[name = string("op_2095"), val = tensor([1, 64, 512])]; + tensor x_269_cast_fp16 = transpose(perm = var_2093, x = x_267_cast_fp16)[name = string("transpose_273")]; + tensor input_151_cast_fp16 = reshape(shape = var_2095, x = x_269_cast_fp16)[name = string("input_151_cast_fp16")]; + tensor linear_58_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_151_cast_fp16)[name = string("linear_58_cast_fp16")]; + tensor input_153_cast_fp16 = add(x = linear_58_cast_fp16, y = x_241_cast_fp16)[name = string("input_153_cast_fp16")]; + tensor linear_59_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_153_cast_fp16)[name = string("linear_59_cast_fp16")]; + string input_157_mode_0 = const()[name = string("input_157_mode_0"), val = string("EXACT")]; + tensor input_157_cast_fp16 = gelu(mode = input_157_mode_0, x = linear_59_cast_fp16)[name = string("input_157_cast_fp16")]; + tensor linear_60_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_157_cast_fp16)[name = string("linear_60_cast_fp16")]; + tensor x_271_cast_fp16 = add(x = linear_60_cast_fp16, y = input_153_cast_fp16)[name = string("x_271_cast_fp16")]; + tensor x_273_cast_fp16 = add(x = x_271_cast_fp16, y = mapping_11_cast_fp16)[name = string("x_273_cast_fp16")]; + tensor var_2111_split_sizes_0 = const()[name = string("op_2111_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2111_axis_0 = const()[name = string("op_2111_axis_0"), val = int32(1)]; + tensor var_2111_cast_fp16_0, tensor var_2111_cast_fp16_1 = split(axis = var_2111_axis_0, split_sizes = var_2111_split_sizes_0, x = h_11_cast_fp16)[name = string("op_2111_cast_fp16")]; + tensor gamma_59_perm_0 = const()[name = string("gamma_59_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_59_perm_0 = const()[name = string("beta_59_perm_0"), val = tensor([0, -1, 1])]; + tensor x_277_axes_0 = const()[name = string("x_277_axes_0"), val = tensor([-1])]; + tensor x_277_cast_fp16 = layer_norm(axes = x_277_axes_0, epsilon = var_1584_to_fp16, x = x_273_cast_fp16)[name = string("x_277_cast_fp16")]; + fp16 var_2117_promoted_to_fp16 = const()[name = string("op_2117_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_59_cast_fp16 = transpose(perm = gamma_59_perm_0, x = var_2111_cast_fp16_0)[name = string("transpose_272")]; + tensor var_2118_cast_fp16 = add(x = gamma_59_cast_fp16, y = var_2117_promoted_to_fp16)[name = string("op_2118_cast_fp16")]; + tensor var_2119_cast_fp16 = mul(x = var_2118_cast_fp16, y = x_277_cast_fp16)[name = string("op_2119_cast_fp16")]; + tensor beta_59_cast_fp16 = transpose(perm = beta_59_perm_0, x = var_2111_cast_fp16_1)[name = string("transpose_271")]; + tensor x_279_cast_fp16 = add(x = var_2119_cast_fp16, y = beta_59_cast_fp16)[name = string("x_279_cast_fp16")]; + tensor var_2130_split_sizes_0 = const()[name = string("op_2130_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2130_axis_0 = const()[name = string("op_2130_axis_0"), val = int32(1)]; + tensor var_2130_cast_fp16_0, tensor var_2130_cast_fp16_1 = split(axis = var_2130_axis_0, split_sizes = var_2130_split_sizes_0, x = h_15_cast_fp16)[name = string("op_2130_cast_fp16")]; + tensor gamma_63_perm_0 = const()[name = string("gamma_63_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_63_perm_0 = const()[name = string("beta_63_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2136_promoted_to_fp16 = const()[name = string("op_2136_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_63_cast_fp16 = transpose(perm = gamma_63_perm_0, x = var_2130_cast_fp16_0)[name = string("transpose_270")]; + tensor var_2137_cast_fp16 = add(x = gamma_63_cast_fp16, y = var_2136_promoted_to_fp16)[name = string("op_2137_cast_fp16")]; + tensor var_2138_cast_fp16 = mul(x = var_2137_cast_fp16, y = x_277_cast_fp16)[name = string("op_2138_cast_fp16")]; + tensor beta_63_cast_fp16 = transpose(perm = beta_63_perm_0, x = var_2130_cast_fp16_1)[name = string("transpose_269")]; + tensor x_285_cast_fp16 = add(x = var_2138_cast_fp16, y = beta_63_cast_fp16)[name = string("x_285_cast_fp16")]; + tensor linear_63_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_279_cast_fp16)[name = string("linear_63_cast_fp16")]; + tensor linear_64_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_285_cast_fp16)[name = string("linear_64_cast_fp16")]; + tensor var_2144_split_sizes_0 = const()[name = string("op_2144_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2144_axis_0 = const()[name = string("op_2144_axis_0"), val = int32(-1)]; + tensor var_2144_cast_fp16_0, tensor var_2144_cast_fp16_1 = split(axis = var_2144_axis_0, split_sizes = var_2144_split_sizes_0, x = linear_64_cast_fp16)[name = string("op_2144_cast_fp16")]; + tensor var_2152 = const()[name = string("op_2152"), val = tensor([1, 64, 8, 64])]; + tensor x_289_cast_fp16 = reshape(shape = var_2152, x = linear_63_cast_fp16)[name = string("x_289_cast_fp16")]; + tensor var_2162 = const()[name = string("op_2162"), val = tensor([1, 64, 8, 64])]; + tensor x_293_cast_fp16 = reshape(shape = var_2162, x = var_2144_cast_fp16_0)[name = string("x_293_cast_fp16")]; + tensor var_2172 = const()[name = string("op_2172"), val = tensor([1, 64, 8, 64])]; + tensor x_297_cast_fp16 = reshape(shape = var_2172, x = var_2144_cast_fp16_1)[name = string("x_297_cast_fp16")]; + tensor var_2174 = const()[name = string("op_2174"), val = tensor([0, 2, 1, 3])]; + bool sim_29_transpose_x_0 = const()[name = string("sim_29_transpose_x_0"), val = bool(false)]; + bool sim_29_transpose_y_0 = const()[name = string("sim_29_transpose_y_0"), val = bool(false)]; + tensor transpose_86_perm_0 = const()[name = string("transpose_86_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_87_perm_0 = const()[name = string("transpose_87_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_87 = transpose(perm = transpose_87_perm_0, x = x_293_cast_fp16)[name = string("transpose_266")]; + tensor transpose_86 = transpose(perm = transpose_86_perm_0, x = x_289_cast_fp16)[name = string("transpose_267")]; + tensor sim_29_cast_fp16 = matmul(transpose_x = sim_29_transpose_x_0, transpose_y = sim_29_transpose_y_0, x = transpose_86, y = transpose_87)[name = string("sim_29_cast_fp16")]; + fp16 var_2178_to_fp16 = const()[name = string("op_2178_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_31_cast_fp16 = mul(x = sim_29_cast_fp16, y = var_2178_to_fp16)[name = string("sim_31_cast_fp16")]; + tensor attn_15_cast_fp16 = softmax(axis = var_1588, x = sim_31_cast_fp16)[name = string("attn_15_cast_fp16")]; + bool x_299_transpose_x_0 = const()[name = string("x_299_transpose_x_0"), val = bool(false)]; + bool x_299_transpose_y_0 = const()[name = string("x_299_transpose_y_0"), val = bool(false)]; + tensor v_15_cast_fp16 = transpose(perm = var_2174, x = x_297_cast_fp16)[name = string("transpose_268")]; + tensor x_299_cast_fp16 = matmul(transpose_x = x_299_transpose_x_0, transpose_y = x_299_transpose_y_0, x = attn_15_cast_fp16, y = v_15_cast_fp16)[name = string("x_299_cast_fp16")]; + tensor var_2200 = const()[name = string("op_2200"), val = tensor([0, 2, 1, 3])]; + tensor var_2202 = const()[name = string("op_2202"), val = tensor([1, 64, 512])]; + tensor x_301_cast_fp16 = transpose(perm = var_2200, x = x_299_cast_fp16)[name = string("transpose_265")]; + tensor input_167_cast_fp16 = reshape(shape = var_2202, x = x_301_cast_fp16)[name = string("input_167_cast_fp16")]; + tensor linear_65_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_167_cast_fp16)[name = string("linear_65_cast_fp16")]; + tensor input_169_cast_fp16 = add(x = linear_65_cast_fp16, y = x_273_cast_fp16)[name = string("input_169_cast_fp16")]; + tensor linear_66_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_169_cast_fp16)[name = string("linear_66_cast_fp16")]; + string input_173_mode_0 = const()[name = string("input_173_mode_0"), val = string("EXACT")]; + tensor input_173_cast_fp16 = gelu(mode = input_173_mode_0, x = linear_66_cast_fp16)[name = string("input_173_cast_fp16")]; + tensor linear_67_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_173_cast_fp16)[name = string("linear_67_cast_fp16")]; + tensor x_303_cast_fp16 = add(x = linear_67_cast_fp16, y = input_169_cast_fp16)[name = string("x_303_cast_fp16")]; + tensor x_305_cast_fp16 = add(x = x_303_cast_fp16, y = mapping_11_cast_fp16)[name = string("x_305_cast_fp16")]; + tensor var_2218_split_sizes_0 = const()[name = string("op_2218_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2218_axis_0 = const()[name = string("op_2218_axis_0"), val = int32(1)]; + tensor var_2218_cast_fp16_0, tensor var_2218_cast_fp16_1 = split(axis = var_2218_axis_0, split_sizes = var_2218_split_sizes_0, x = h_19_cast_fp16)[name = string("op_2218_cast_fp16")]; + tensor gamma_67_perm_0 = const()[name = string("gamma_67_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_67_perm_0 = const()[name = string("beta_67_perm_0"), val = tensor([0, -1, 1])]; + tensor x_309_axes_0 = const()[name = string("x_309_axes_0"), val = tensor([-1])]; + tensor x_309_cast_fp16 = layer_norm(axes = x_309_axes_0, epsilon = var_1584_to_fp16, x = x_305_cast_fp16)[name = string("x_309_cast_fp16")]; + fp16 var_2224_promoted_to_fp16 = const()[name = string("op_2224_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_67_cast_fp16 = transpose(perm = gamma_67_perm_0, x = var_2218_cast_fp16_0)[name = string("transpose_264")]; + tensor var_2225_cast_fp16 = add(x = gamma_67_cast_fp16, y = var_2224_promoted_to_fp16)[name = string("op_2225_cast_fp16")]; + tensor var_2226_cast_fp16 = mul(x = var_2225_cast_fp16, y = x_309_cast_fp16)[name = string("op_2226_cast_fp16")]; + tensor beta_67_cast_fp16 = transpose(perm = beta_67_perm_0, x = var_2218_cast_fp16_1)[name = string("transpose_263")]; + tensor x_311_cast_fp16 = add(x = var_2226_cast_fp16, y = beta_67_cast_fp16)[name = string("x_311_cast_fp16")]; + tensor var_2237_split_sizes_0 = const()[name = string("op_2237_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2237_axis_0 = const()[name = string("op_2237_axis_0"), val = int32(1)]; + tensor var_2237_cast_fp16_0, tensor var_2237_cast_fp16_1 = split(axis = var_2237_axis_0, split_sizes = var_2237_split_sizes_0, x = h_23_cast_fp16)[name = string("op_2237_cast_fp16")]; + tensor gamma_71_perm_0 = const()[name = string("gamma_71_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_71_perm_0 = const()[name = string("beta_71_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2243_promoted_to_fp16 = const()[name = string("op_2243_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_71_cast_fp16 = transpose(perm = gamma_71_perm_0, x = var_2237_cast_fp16_0)[name = string("transpose_262")]; + tensor var_2244_cast_fp16 = add(x = gamma_71_cast_fp16, y = var_2243_promoted_to_fp16)[name = string("op_2244_cast_fp16")]; + tensor var_2245_cast_fp16 = mul(x = var_2244_cast_fp16, y = x_309_cast_fp16)[name = string("op_2245_cast_fp16")]; + tensor beta_71_cast_fp16 = transpose(perm = beta_71_perm_0, x = var_2237_cast_fp16_1)[name = string("transpose_261")]; + tensor x_317_cast_fp16 = add(x = var_2245_cast_fp16, y = beta_71_cast_fp16)[name = string("x_317_cast_fp16")]; + tensor linear_70_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_311_cast_fp16)[name = string("linear_70_cast_fp16")]; + tensor linear_71_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_317_cast_fp16)[name = string("linear_71_cast_fp16")]; + tensor var_2251_split_sizes_0 = const()[name = string("op_2251_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2251_axis_0 = const()[name = string("op_2251_axis_0"), val = int32(-1)]; + tensor var_2251_cast_fp16_0, tensor var_2251_cast_fp16_1 = split(axis = var_2251_axis_0, split_sizes = var_2251_split_sizes_0, x = linear_71_cast_fp16)[name = string("op_2251_cast_fp16")]; + tensor var_2259 = const()[name = string("op_2259"), val = tensor([1, 64, 8, 64])]; + tensor x_321_cast_fp16 = reshape(shape = var_2259, x = linear_70_cast_fp16)[name = string("x_321_cast_fp16")]; + tensor var_2269 = const()[name = string("op_2269"), val = tensor([1, 64, 8, 64])]; + tensor x_325_cast_fp16 = reshape(shape = var_2269, x = var_2251_cast_fp16_0)[name = string("x_325_cast_fp16")]; + tensor var_2279 = const()[name = string("op_2279"), val = tensor([1, 64, 8, 64])]; + tensor x_329_cast_fp16 = reshape(shape = var_2279, x = var_2251_cast_fp16_1)[name = string("x_329_cast_fp16")]; + tensor var_2281 = const()[name = string("op_2281"), val = tensor([0, 2, 1, 3])]; + bool sim_33_transpose_x_0 = const()[name = string("sim_33_transpose_x_0"), val = bool(false)]; + bool sim_33_transpose_y_0 = const()[name = string("sim_33_transpose_y_0"), val = bool(false)]; + tensor transpose_88_perm_0 = const()[name = string("transpose_88_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_89_perm_0 = const()[name = string("transpose_89_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_89 = transpose(perm = transpose_89_perm_0, x = x_325_cast_fp16)[name = string("transpose_258")]; + tensor transpose_88 = transpose(perm = transpose_88_perm_0, x = x_321_cast_fp16)[name = string("transpose_259")]; + tensor sim_33_cast_fp16 = matmul(transpose_x = sim_33_transpose_x_0, transpose_y = sim_33_transpose_y_0, x = transpose_88, y = transpose_89)[name = string("sim_33_cast_fp16")]; + fp16 var_2285_to_fp16 = const()[name = string("op_2285_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_35_cast_fp16 = mul(x = sim_33_cast_fp16, y = var_2285_to_fp16)[name = string("sim_35_cast_fp16")]; + tensor attn_17_cast_fp16 = softmax(axis = var_1588, x = sim_35_cast_fp16)[name = string("attn_17_cast_fp16")]; + bool x_331_transpose_x_0 = const()[name = string("x_331_transpose_x_0"), val = bool(false)]; + bool x_331_transpose_y_0 = const()[name = string("x_331_transpose_y_0"), val = bool(false)]; + tensor v_17_cast_fp16 = transpose(perm = var_2281, x = x_329_cast_fp16)[name = string("transpose_260")]; + tensor x_331_cast_fp16 = matmul(transpose_x = x_331_transpose_x_0, transpose_y = x_331_transpose_y_0, x = attn_17_cast_fp16, y = v_17_cast_fp16)[name = string("x_331_cast_fp16")]; + tensor var_2307 = const()[name = string("op_2307"), val = tensor([0, 2, 1, 3])]; + tensor var_2309 = const()[name = string("op_2309"), val = tensor([1, 64, 512])]; + tensor x_333_cast_fp16 = transpose(perm = var_2307, x = x_331_cast_fp16)[name = string("transpose_257")]; + tensor input_183_cast_fp16 = reshape(shape = var_2309, x = x_333_cast_fp16)[name = string("input_183_cast_fp16")]; + tensor linear_72_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_183_cast_fp16)[name = string("linear_72_cast_fp16")]; + tensor input_185_cast_fp16 = add(x = linear_72_cast_fp16, y = x_305_cast_fp16)[name = string("input_185_cast_fp16")]; + tensor linear_73_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_185_cast_fp16)[name = string("linear_73_cast_fp16")]; + string input_189_mode_0 = const()[name = string("input_189_mode_0"), val = string("EXACT")]; + tensor input_189_cast_fp16 = gelu(mode = input_189_mode_0, x = linear_73_cast_fp16)[name = string("input_189_cast_fp16")]; + tensor linear_74_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_189_cast_fp16)[name = string("linear_74_cast_fp16")]; + tensor x_335_cast_fp16 = add(x = linear_74_cast_fp16, y = input_185_cast_fp16)[name = string("x_335_cast_fp16")]; + tensor var_2318_axes_0 = const()[name = string("op_2318_axes_0"), val = tensor([1])]; + bool var_2318_keep_dims_0 = const()[name = string("op_2318_keep_dims_0"), val = bool(false)]; + tensor var_2318_cast_fp16 = reduce_mean(axes = var_2318_axes_0, keep_dims = var_2318_keep_dims_0, x = x_335_cast_fp16)[name = string("op_2318_cast_fp16")]; + tensor x_337_axes_0 = const()[name = string("x_337_axes_0"), val = tensor([1])]; + tensor x_337_cast_fp16 = expand_dims(axes = x_337_axes_0, x = var_2318_cast_fp16)[name = string("x_337_cast_fp16")]; + tensor var_2320 = const()[name = string("op_2320"), val = tensor([0, 2, 1])]; + string x_339_pad_type_0 = const()[name = string("x_339_pad_type_0"), val = string("valid")]; + tensor x_339_strides_0 = const()[name = string("x_339_strides_0"), val = tensor([1])]; + tensor x_339_pad_0 = const()[name = string("x_339_pad_0"), val = tensor([0, 0])]; + tensor x_339_dilations_0 = const()[name = string("x_339_dilations_0"), val = tensor([1])]; + int32 x_339_groups_0 = const()[name = string("x_339_groups_0"), val = int32(1)]; + tensor input_191_cast_fp16 = transpose(perm = var_2320, x = x_337_cast_fp16)[name = string("transpose_256")]; + tensor x_339_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_339_dilations_0, groups = x_339_groups_0, pad = x_339_pad_0, pad_type = x_339_pad_type_0, strides = x_339_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_191_cast_fp16)[name = string("x_339_cast_fp16")]; + tensor x_pred_5_perm_0 = const()[name = string("x_pred_5_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_5_to_fp16 = const()[name = string("c_skip_5_to_fp16"), val = tensor([[[0x1.cf4p-4]]])]; + tensor var_2328_cast_fp16 = mul(x = c_skip_5_to_fp16, y = x_noisy_5_cast_fp16)[name = string("op_2328_cast_fp16")]; + tensor c_out_5_to_fp16 = const()[name = string("c_out_5_to_fp16"), val = tensor([[[0x1.804p-3]]])]; + tensor x_pred_5_cast_fp16 = transpose(perm = x_pred_5_perm_0, x = x_339_cast_fp16)[name = string("transpose_255")]; + tensor var_2329_cast_fp16 = mul(x = c_out_5_to_fp16, y = x_pred_5_cast_fp16)[name = string("op_2329_cast_fp16")]; + tensor x_dn_3_cast_fp16 = add(x = var_2328_cast_fp16, y = var_2329_cast_fp16)[name = string("x_dn_3_cast_fp16")]; + tensor var_2332_cast_fp16 = sub(x = x_noisy_5_cast_fp16, y = x_dn_3_cast_fp16)[name = string("op_2332_cast_fp16")]; + tensor _inversed_d_3_y_0_to_fp16 = const()[name = string("_inversed_d_3_y_0_to_fp16"), val = tensor([0x1.cacp+0])]; + tensor _inversed_d_3_cast_fp16 = mul(x = var_2332_cast_fp16, y = _inversed_d_3_y_0_to_fp16)[name = string("_inversed_d_3_cast_fp16")]; + fp16 var_2341_to_fp16 = const()[name = string("op_2341_to_fp16"), val = fp16(-0x1.19p-2)]; + tensor var_2342_cast_fp16 = mul(x = _inversed_d_3_cast_fp16, y = var_2341_to_fp16)[name = string("op_2342_cast_fp16")]; + tensor x_noisy_7_cast_fp16 = add(x = x_noisy_5_cast_fp16, y = var_2342_cast_fp16)[name = string("x_noisy_7_cast_fp16")]; + int32 var_2354 = const()[name = string("op_2354"), val = int32(-1)]; + tensor c_in_7_to_fp16 = const()[name = string("c_in_7_to_fp16"), val = tensor([[[0x1.718p+1]]])]; + tensor x_349_cast_fp16 = mul(x = c_in_7_to_fp16, y = x_noisy_7_cast_fp16)[name = string("x_349_cast_fp16")]; + int32 x_345_axis_0 = const()[name = string("x_345_axis_0"), val = int32(0)]; + tensor var_2740_to_fp16 = const()[name = string("op_2740_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49351296)))]; + tensor x_345_cast_fp16 = stack(axis = x_345_axis_0, values = (var_2740_to_fp16, var_423_cast_fp16))[name = string("x_345_cast_fp16")]; + tensor var_2745 = const()[name = string("op_2745"), val = tensor([1, 2, 0])]; + tensor input_199_axes_0 = const()[name = string("input_199_axes_0"), val = tensor([2])]; + bool input_199_keep_dims_0 = const()[name = string("input_199_keep_dims_0"), val = bool(false)]; + tensor x_347_cast_fp16 = transpose(perm = var_2745, x = x_345_cast_fp16)[name = string("transpose_254")]; + tensor input_199_cast_fp16 = reduce_sum(axes = input_199_axes_0, keep_dims = input_199_keep_dims_0, x = x_347_cast_fp16)[name = string("input_199_cast_fp16")]; + tensor linear_77_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_199_cast_fp16)[name = string("linear_77_cast_fp16")]; + string input_203_mode_0 = const()[name = string("input_203_mode_0"), val = string("EXACT")]; + tensor input_203_cast_fp16 = gelu(mode = input_203_mode_0, x = linear_77_cast_fp16)[name = string("input_203_cast_fp16")]; + tensor linear_78_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_203_cast_fp16)[name = string("linear_78_cast_fp16")]; + string mapping_13_mode_0 = const()[name = string("mapping_13_mode_0"), val = string("EXACT")]; + tensor mapping_13_cast_fp16 = gelu(mode = mapping_13_mode_0, x = linear_78_cast_fp16)[name = string("mapping_13_cast_fp16")]; + tensor var_2755_reps_0 = const()[name = string("op_2755_reps_0"), val = tensor([1, 64, 1])]; + tensor var_2755_cast_fp16 = tile(reps = var_2755_reps_0, x = x_349_cast_fp16)[name = string("op_2755_cast_fp16")]; + bool x_351_interleave_0 = const()[name = string("x_351_interleave_0"), val = bool(false)]; + tensor x_351_cast_fp16 = concat(axis = var_2354, interleave = x_351_interleave_0, values = (var_2755_cast_fp16, embedding_to_fp16))[name = string("x_351_cast_fp16")]; + tensor var_2758_axes_0 = const()[name = string("op_2758_axes_0"), val = tensor([1])]; + tensor var_2758_cast_fp16 = expand_dims(axes = var_2758_axes_0, x = mapping_13_cast_fp16)[name = string("op_2758_cast_fp16")]; + tensor mapping_15_reps_0 = const()[name = string("mapping_15_reps_0"), val = tensor([1, 64, 1])]; + tensor mapping_15_cast_fp16 = tile(reps = mapping_15_reps_0, x = var_2758_cast_fp16)[name = string("mapping_15_cast_fp16")]; + tensor x_353_cast_fp16 = add(x = x_351_cast_fp16, y = mapping_15_cast_fp16)[name = string("x_353_cast_fp16")]; + tensor var_2770_split_sizes_0 = const()[name = string("op_2770_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2770_axis_0 = const()[name = string("op_2770_axis_0"), val = int32(1)]; + tensor var_2770_cast_fp16_0, tensor var_2770_cast_fp16_1 = split(axis = var_2770_axis_0, split_sizes = var_2770_split_sizes_0, x = h_3_cast_fp16)[name = string("op_2770_cast_fp16")]; + tensor gamma_75_perm_0 = const()[name = string("gamma_75_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_75_perm_0 = const()[name = string("beta_75_perm_0"), val = tensor([0, -1, 1])]; + tensor x_357_axes_0 = const()[name = string("x_357_axes_0"), val = tensor([-1])]; + fp16 var_2350_to_fp16 = const()[name = string("op_2350_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_357_cast_fp16 = layer_norm(axes = x_357_axes_0, epsilon = var_2350_to_fp16, x = x_353_cast_fp16)[name = string("x_357_cast_fp16")]; + fp16 var_2776_promoted_to_fp16 = const()[name = string("op_2776_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_75_cast_fp16 = transpose(perm = gamma_75_perm_0, x = var_2770_cast_fp16_0)[name = string("transpose_253")]; + tensor var_2777_cast_fp16 = add(x = gamma_75_cast_fp16, y = var_2776_promoted_to_fp16)[name = string("op_2777_cast_fp16")]; + tensor var_2778_cast_fp16 = mul(x = var_2777_cast_fp16, y = x_357_cast_fp16)[name = string("op_2778_cast_fp16")]; + tensor beta_75_cast_fp16 = transpose(perm = beta_75_perm_0, x = var_2770_cast_fp16_1)[name = string("transpose_252")]; + tensor x_359_cast_fp16 = add(x = var_2778_cast_fp16, y = beta_75_cast_fp16)[name = string("x_359_cast_fp16")]; + tensor var_2789_split_sizes_0 = const()[name = string("op_2789_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2789_axis_0 = const()[name = string("op_2789_axis_0"), val = int32(1)]; + tensor var_2789_cast_fp16_0, tensor var_2789_cast_fp16_1 = split(axis = var_2789_axis_0, split_sizes = var_2789_split_sizes_0, x = h_7_cast_fp16)[name = string("op_2789_cast_fp16")]; + tensor gamma_79_perm_0 = const()[name = string("gamma_79_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_79_perm_0 = const()[name = string("beta_79_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2795_promoted_to_fp16 = const()[name = string("op_2795_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_79_cast_fp16 = transpose(perm = gamma_79_perm_0, x = var_2789_cast_fp16_0)[name = string("transpose_251")]; + tensor var_2796_cast_fp16 = add(x = gamma_79_cast_fp16, y = var_2795_promoted_to_fp16)[name = string("op_2796_cast_fp16")]; + tensor var_2797_cast_fp16 = mul(x = var_2796_cast_fp16, y = x_357_cast_fp16)[name = string("op_2797_cast_fp16")]; + tensor beta_79_cast_fp16 = transpose(perm = beta_79_perm_0, x = var_2789_cast_fp16_1)[name = string("transpose_250")]; + tensor x_365_cast_fp16 = add(x = var_2797_cast_fp16, y = beta_79_cast_fp16)[name = string("x_365_cast_fp16")]; + tensor linear_81_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_359_cast_fp16)[name = string("linear_81_cast_fp16")]; + tensor linear_82_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_365_cast_fp16)[name = string("linear_82_cast_fp16")]; + tensor var_2803_split_sizes_0 = const()[name = string("op_2803_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2803_axis_0 = const()[name = string("op_2803_axis_0"), val = int32(-1)]; + tensor var_2803_cast_fp16_0, tensor var_2803_cast_fp16_1 = split(axis = var_2803_axis_0, split_sizes = var_2803_split_sizes_0, x = linear_82_cast_fp16)[name = string("op_2803_cast_fp16")]; + tensor var_2811 = const()[name = string("op_2811"), val = tensor([1, 64, 8, 64])]; + tensor x_369_cast_fp16 = reshape(shape = var_2811, x = linear_81_cast_fp16)[name = string("x_369_cast_fp16")]; + tensor var_2821 = const()[name = string("op_2821"), val = tensor([1, 64, 8, 64])]; + tensor x_373_cast_fp16 = reshape(shape = var_2821, x = var_2803_cast_fp16_0)[name = string("x_373_cast_fp16")]; + tensor var_2831 = const()[name = string("op_2831"), val = tensor([1, 64, 8, 64])]; + tensor x_377_cast_fp16 = reshape(shape = var_2831, x = var_2803_cast_fp16_1)[name = string("x_377_cast_fp16")]; + tensor var_2833 = const()[name = string("op_2833"), val = tensor([0, 2, 1, 3])]; + bool sim_37_transpose_x_0 = const()[name = string("sim_37_transpose_x_0"), val = bool(false)]; + bool sim_37_transpose_y_0 = const()[name = string("sim_37_transpose_y_0"), val = bool(false)]; + tensor transpose_90_perm_0 = const()[name = string("transpose_90_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_91_perm_0 = const()[name = string("transpose_91_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_91 = transpose(perm = transpose_91_perm_0, x = x_373_cast_fp16)[name = string("transpose_247")]; + tensor transpose_90 = transpose(perm = transpose_90_perm_0, x = x_369_cast_fp16)[name = string("transpose_248")]; + tensor sim_37_cast_fp16 = matmul(transpose_x = sim_37_transpose_x_0, transpose_y = sim_37_transpose_y_0, x = transpose_90, y = transpose_91)[name = string("sim_37_cast_fp16")]; + fp16 var_2837_to_fp16 = const()[name = string("op_2837_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_39_cast_fp16 = mul(x = sim_37_cast_fp16, y = var_2837_to_fp16)[name = string("sim_39_cast_fp16")]; + tensor attn_19_cast_fp16 = softmax(axis = var_2354, x = sim_39_cast_fp16)[name = string("attn_19_cast_fp16")]; + bool x_379_transpose_x_0 = const()[name = string("x_379_transpose_x_0"), val = bool(false)]; + bool x_379_transpose_y_0 = const()[name = string("x_379_transpose_y_0"), val = bool(false)]; + tensor v_19_cast_fp16 = transpose(perm = var_2833, x = x_377_cast_fp16)[name = string("transpose_249")]; + tensor x_379_cast_fp16 = matmul(transpose_x = x_379_transpose_x_0, transpose_y = x_379_transpose_y_0, x = attn_19_cast_fp16, y = v_19_cast_fp16)[name = string("x_379_cast_fp16")]; + tensor var_2859 = const()[name = string("op_2859"), val = tensor([0, 2, 1, 3])]; + tensor var_2861 = const()[name = string("op_2861"), val = tensor([1, 64, 512])]; + tensor x_381_cast_fp16 = transpose(perm = var_2859, x = x_379_cast_fp16)[name = string("transpose_246")]; + tensor input_215_cast_fp16 = reshape(shape = var_2861, x = x_381_cast_fp16)[name = string("input_215_cast_fp16")]; + tensor linear_83_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_215_cast_fp16)[name = string("linear_83_cast_fp16")]; + tensor input_217_cast_fp16 = add(x = linear_83_cast_fp16, y = x_353_cast_fp16)[name = string("input_217_cast_fp16")]; + tensor linear_84_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_217_cast_fp16)[name = string("linear_84_cast_fp16")]; + string input_221_mode_0 = const()[name = string("input_221_mode_0"), val = string("EXACT")]; + tensor input_221_cast_fp16 = gelu(mode = input_221_mode_0, x = linear_84_cast_fp16)[name = string("input_221_cast_fp16")]; + tensor linear_85_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_221_cast_fp16)[name = string("linear_85_cast_fp16")]; + tensor x_383_cast_fp16 = add(x = linear_85_cast_fp16, y = input_217_cast_fp16)[name = string("x_383_cast_fp16")]; + tensor x_385_cast_fp16 = add(x = x_383_cast_fp16, y = mapping_15_cast_fp16)[name = string("x_385_cast_fp16")]; + tensor var_2877_split_sizes_0 = const()[name = string("op_2877_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2877_axis_0 = const()[name = string("op_2877_axis_0"), val = int32(1)]; + tensor var_2877_cast_fp16_0, tensor var_2877_cast_fp16_1 = split(axis = var_2877_axis_0, split_sizes = var_2877_split_sizes_0, x = h_11_cast_fp16)[name = string("op_2877_cast_fp16")]; + tensor gamma_83_perm_0 = const()[name = string("gamma_83_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_83_perm_0 = const()[name = string("beta_83_perm_0"), val = tensor([0, -1, 1])]; + tensor x_389_axes_0 = const()[name = string("x_389_axes_0"), val = tensor([-1])]; + tensor x_389_cast_fp16 = layer_norm(axes = x_389_axes_0, epsilon = var_2350_to_fp16, x = x_385_cast_fp16)[name = string("x_389_cast_fp16")]; + fp16 var_2883_promoted_to_fp16 = const()[name = string("op_2883_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_83_cast_fp16 = transpose(perm = gamma_83_perm_0, x = var_2877_cast_fp16_0)[name = string("transpose_245")]; + tensor var_2884_cast_fp16 = add(x = gamma_83_cast_fp16, y = var_2883_promoted_to_fp16)[name = string("op_2884_cast_fp16")]; + tensor var_2885_cast_fp16 = mul(x = var_2884_cast_fp16, y = x_389_cast_fp16)[name = string("op_2885_cast_fp16")]; + tensor beta_83_cast_fp16 = transpose(perm = beta_83_perm_0, x = var_2877_cast_fp16_1)[name = string("transpose_244")]; + tensor x_391_cast_fp16 = add(x = var_2885_cast_fp16, y = beta_83_cast_fp16)[name = string("x_391_cast_fp16")]; + tensor var_2896_split_sizes_0 = const()[name = string("op_2896_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2896_axis_0 = const()[name = string("op_2896_axis_0"), val = int32(1)]; + tensor var_2896_cast_fp16_0, tensor var_2896_cast_fp16_1 = split(axis = var_2896_axis_0, split_sizes = var_2896_split_sizes_0, x = h_15_cast_fp16)[name = string("op_2896_cast_fp16")]; + tensor gamma_87_perm_0 = const()[name = string("gamma_87_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_87_perm_0 = const()[name = string("beta_87_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_2902_promoted_to_fp16 = const()[name = string("op_2902_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_87_cast_fp16 = transpose(perm = gamma_87_perm_0, x = var_2896_cast_fp16_0)[name = string("transpose_243")]; + tensor var_2903_cast_fp16 = add(x = gamma_87_cast_fp16, y = var_2902_promoted_to_fp16)[name = string("op_2903_cast_fp16")]; + tensor var_2904_cast_fp16 = mul(x = var_2903_cast_fp16, y = x_389_cast_fp16)[name = string("op_2904_cast_fp16")]; + tensor beta_87_cast_fp16 = transpose(perm = beta_87_perm_0, x = var_2896_cast_fp16_1)[name = string("transpose_242")]; + tensor x_397_cast_fp16 = add(x = var_2904_cast_fp16, y = beta_87_cast_fp16)[name = string("x_397_cast_fp16")]; + tensor linear_88_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_391_cast_fp16)[name = string("linear_88_cast_fp16")]; + tensor linear_89_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_397_cast_fp16)[name = string("linear_89_cast_fp16")]; + tensor var_2910_split_sizes_0 = const()[name = string("op_2910_split_sizes_0"), val = tensor([512, 512])]; + int32 var_2910_axis_0 = const()[name = string("op_2910_axis_0"), val = int32(-1)]; + tensor var_2910_cast_fp16_0, tensor var_2910_cast_fp16_1 = split(axis = var_2910_axis_0, split_sizes = var_2910_split_sizes_0, x = linear_89_cast_fp16)[name = string("op_2910_cast_fp16")]; + tensor var_2918 = const()[name = string("op_2918"), val = tensor([1, 64, 8, 64])]; + tensor x_401_cast_fp16 = reshape(shape = var_2918, x = linear_88_cast_fp16)[name = string("x_401_cast_fp16")]; + tensor var_2928 = const()[name = string("op_2928"), val = tensor([1, 64, 8, 64])]; + tensor x_405_cast_fp16 = reshape(shape = var_2928, x = var_2910_cast_fp16_0)[name = string("x_405_cast_fp16")]; + tensor var_2938 = const()[name = string("op_2938"), val = tensor([1, 64, 8, 64])]; + tensor x_409_cast_fp16 = reshape(shape = var_2938, x = var_2910_cast_fp16_1)[name = string("x_409_cast_fp16")]; + tensor var_2940 = const()[name = string("op_2940"), val = tensor([0, 2, 1, 3])]; + bool sim_41_transpose_x_0 = const()[name = string("sim_41_transpose_x_0"), val = bool(false)]; + bool sim_41_transpose_y_0 = const()[name = string("sim_41_transpose_y_0"), val = bool(false)]; + tensor transpose_92_perm_0 = const()[name = string("transpose_92_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_93_perm_0 = const()[name = string("transpose_93_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_93 = transpose(perm = transpose_93_perm_0, x = x_405_cast_fp16)[name = string("transpose_239")]; + tensor transpose_92 = transpose(perm = transpose_92_perm_0, x = x_401_cast_fp16)[name = string("transpose_240")]; + tensor sim_41_cast_fp16 = matmul(transpose_x = sim_41_transpose_x_0, transpose_y = sim_41_transpose_y_0, x = transpose_92, y = transpose_93)[name = string("sim_41_cast_fp16")]; + fp16 var_2944_to_fp16 = const()[name = string("op_2944_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_43_cast_fp16 = mul(x = sim_41_cast_fp16, y = var_2944_to_fp16)[name = string("sim_43_cast_fp16")]; + tensor attn_21_cast_fp16 = softmax(axis = var_2354, x = sim_43_cast_fp16)[name = string("attn_21_cast_fp16")]; + bool x_411_transpose_x_0 = const()[name = string("x_411_transpose_x_0"), val = bool(false)]; + bool x_411_transpose_y_0 = const()[name = string("x_411_transpose_y_0"), val = bool(false)]; + tensor v_21_cast_fp16 = transpose(perm = var_2940, x = x_409_cast_fp16)[name = string("transpose_241")]; + tensor x_411_cast_fp16 = matmul(transpose_x = x_411_transpose_x_0, transpose_y = x_411_transpose_y_0, x = attn_21_cast_fp16, y = v_21_cast_fp16)[name = string("x_411_cast_fp16")]; + tensor var_2966 = const()[name = string("op_2966"), val = tensor([0, 2, 1, 3])]; + tensor var_2968 = const()[name = string("op_2968"), val = tensor([1, 64, 512])]; + tensor x_413_cast_fp16 = transpose(perm = var_2966, x = x_411_cast_fp16)[name = string("transpose_238")]; + tensor input_231_cast_fp16 = reshape(shape = var_2968, x = x_413_cast_fp16)[name = string("input_231_cast_fp16")]; + tensor linear_90_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_231_cast_fp16)[name = string("linear_90_cast_fp16")]; + tensor input_233_cast_fp16 = add(x = linear_90_cast_fp16, y = x_385_cast_fp16)[name = string("input_233_cast_fp16")]; + tensor linear_91_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_233_cast_fp16)[name = string("linear_91_cast_fp16")]; + string input_237_mode_0 = const()[name = string("input_237_mode_0"), val = string("EXACT")]; + tensor input_237_cast_fp16 = gelu(mode = input_237_mode_0, x = linear_91_cast_fp16)[name = string("input_237_cast_fp16")]; + tensor linear_92_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_237_cast_fp16)[name = string("linear_92_cast_fp16")]; + tensor x_415_cast_fp16 = add(x = linear_92_cast_fp16, y = input_233_cast_fp16)[name = string("x_415_cast_fp16")]; + tensor x_417_cast_fp16 = add(x = x_415_cast_fp16, y = mapping_15_cast_fp16)[name = string("x_417_cast_fp16")]; + tensor var_2984_split_sizes_0 = const()[name = string("op_2984_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_2984_axis_0 = const()[name = string("op_2984_axis_0"), val = int32(1)]; + tensor var_2984_cast_fp16_0, tensor var_2984_cast_fp16_1 = split(axis = var_2984_axis_0, split_sizes = var_2984_split_sizes_0, x = h_19_cast_fp16)[name = string("op_2984_cast_fp16")]; + tensor gamma_91_perm_0 = const()[name = string("gamma_91_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_91_perm_0 = const()[name = string("beta_91_perm_0"), val = tensor([0, -1, 1])]; + tensor x_421_axes_0 = const()[name = string("x_421_axes_0"), val = tensor([-1])]; + tensor x_421_cast_fp16 = layer_norm(axes = x_421_axes_0, epsilon = var_2350_to_fp16, x = x_417_cast_fp16)[name = string("x_421_cast_fp16")]; + fp16 var_2990_promoted_to_fp16 = const()[name = string("op_2990_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_91_cast_fp16 = transpose(perm = gamma_91_perm_0, x = var_2984_cast_fp16_0)[name = string("transpose_237")]; + tensor var_2991_cast_fp16 = add(x = gamma_91_cast_fp16, y = var_2990_promoted_to_fp16)[name = string("op_2991_cast_fp16")]; + tensor var_2992_cast_fp16 = mul(x = var_2991_cast_fp16, y = x_421_cast_fp16)[name = string("op_2992_cast_fp16")]; + tensor beta_91_cast_fp16 = transpose(perm = beta_91_perm_0, x = var_2984_cast_fp16_1)[name = string("transpose_236")]; + tensor x_423_cast_fp16 = add(x = var_2992_cast_fp16, y = beta_91_cast_fp16)[name = string("x_423_cast_fp16")]; + tensor var_3003_split_sizes_0 = const()[name = string("op_3003_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3003_axis_0 = const()[name = string("op_3003_axis_0"), val = int32(1)]; + tensor var_3003_cast_fp16_0, tensor var_3003_cast_fp16_1 = split(axis = var_3003_axis_0, split_sizes = var_3003_split_sizes_0, x = h_23_cast_fp16)[name = string("op_3003_cast_fp16")]; + tensor gamma_95_perm_0 = const()[name = string("gamma_95_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_95_perm_0 = const()[name = string("beta_95_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3009_promoted_to_fp16 = const()[name = string("op_3009_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_95_cast_fp16 = transpose(perm = gamma_95_perm_0, x = var_3003_cast_fp16_0)[name = string("transpose_235")]; + tensor var_3010_cast_fp16 = add(x = gamma_95_cast_fp16, y = var_3009_promoted_to_fp16)[name = string("op_3010_cast_fp16")]; + tensor var_3011_cast_fp16 = mul(x = var_3010_cast_fp16, y = x_421_cast_fp16)[name = string("op_3011_cast_fp16")]; + tensor beta_95_cast_fp16 = transpose(perm = beta_95_perm_0, x = var_3003_cast_fp16_1)[name = string("transpose_234")]; + tensor x_429_cast_fp16 = add(x = var_3011_cast_fp16, y = beta_95_cast_fp16)[name = string("x_429_cast_fp16")]; + tensor linear_95_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_423_cast_fp16)[name = string("linear_95_cast_fp16")]; + tensor linear_96_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_429_cast_fp16)[name = string("linear_96_cast_fp16")]; + tensor var_3017_split_sizes_0 = const()[name = string("op_3017_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3017_axis_0 = const()[name = string("op_3017_axis_0"), val = int32(-1)]; + tensor var_3017_cast_fp16_0, tensor var_3017_cast_fp16_1 = split(axis = var_3017_axis_0, split_sizes = var_3017_split_sizes_0, x = linear_96_cast_fp16)[name = string("op_3017_cast_fp16")]; + tensor var_3025 = const()[name = string("op_3025"), val = tensor([1, 64, 8, 64])]; + tensor x_433_cast_fp16 = reshape(shape = var_3025, x = linear_95_cast_fp16)[name = string("x_433_cast_fp16")]; + tensor var_3035 = const()[name = string("op_3035"), val = tensor([1, 64, 8, 64])]; + tensor x_437_cast_fp16 = reshape(shape = var_3035, x = var_3017_cast_fp16_0)[name = string("x_437_cast_fp16")]; + tensor var_3045 = const()[name = string("op_3045"), val = tensor([1, 64, 8, 64])]; + tensor x_441_cast_fp16 = reshape(shape = var_3045, x = var_3017_cast_fp16_1)[name = string("x_441_cast_fp16")]; + tensor var_3047 = const()[name = string("op_3047"), val = tensor([0, 2, 1, 3])]; + bool sim_45_transpose_x_0 = const()[name = string("sim_45_transpose_x_0"), val = bool(false)]; + bool sim_45_transpose_y_0 = const()[name = string("sim_45_transpose_y_0"), val = bool(false)]; + tensor transpose_94_perm_0 = const()[name = string("transpose_94_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_95_perm_0 = const()[name = string("transpose_95_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_95 = transpose(perm = transpose_95_perm_0, x = x_437_cast_fp16)[name = string("transpose_231")]; + tensor transpose_94 = transpose(perm = transpose_94_perm_0, x = x_433_cast_fp16)[name = string("transpose_232")]; + tensor sim_45_cast_fp16 = matmul(transpose_x = sim_45_transpose_x_0, transpose_y = sim_45_transpose_y_0, x = transpose_94, y = transpose_95)[name = string("sim_45_cast_fp16")]; + fp16 var_3051_to_fp16 = const()[name = string("op_3051_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_47_cast_fp16 = mul(x = sim_45_cast_fp16, y = var_3051_to_fp16)[name = string("sim_47_cast_fp16")]; + tensor attn_23_cast_fp16 = softmax(axis = var_2354, x = sim_47_cast_fp16)[name = string("attn_23_cast_fp16")]; + bool x_443_transpose_x_0 = const()[name = string("x_443_transpose_x_0"), val = bool(false)]; + bool x_443_transpose_y_0 = const()[name = string("x_443_transpose_y_0"), val = bool(false)]; + tensor v_23_cast_fp16 = transpose(perm = var_3047, x = x_441_cast_fp16)[name = string("transpose_233")]; + tensor x_443_cast_fp16 = matmul(transpose_x = x_443_transpose_x_0, transpose_y = x_443_transpose_y_0, x = attn_23_cast_fp16, y = v_23_cast_fp16)[name = string("x_443_cast_fp16")]; + tensor var_3073 = const()[name = string("op_3073"), val = tensor([0, 2, 1, 3])]; + tensor var_3075 = const()[name = string("op_3075"), val = tensor([1, 64, 512])]; + tensor x_445_cast_fp16 = transpose(perm = var_3073, x = x_443_cast_fp16)[name = string("transpose_230")]; + tensor input_247_cast_fp16 = reshape(shape = var_3075, x = x_445_cast_fp16)[name = string("input_247_cast_fp16")]; + tensor linear_97_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_247_cast_fp16)[name = string("linear_97_cast_fp16")]; + tensor input_249_cast_fp16 = add(x = linear_97_cast_fp16, y = x_417_cast_fp16)[name = string("input_249_cast_fp16")]; + tensor linear_98_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_249_cast_fp16)[name = string("linear_98_cast_fp16")]; + string input_253_mode_0 = const()[name = string("input_253_mode_0"), val = string("EXACT")]; + tensor input_253_cast_fp16 = gelu(mode = input_253_mode_0, x = linear_98_cast_fp16)[name = string("input_253_cast_fp16")]; + tensor linear_99_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_253_cast_fp16)[name = string("linear_99_cast_fp16")]; + tensor x_447_cast_fp16 = add(x = linear_99_cast_fp16, y = input_249_cast_fp16)[name = string("x_447_cast_fp16")]; + tensor var_3084_axes_0 = const()[name = string("op_3084_axes_0"), val = tensor([1])]; + bool var_3084_keep_dims_0 = const()[name = string("op_3084_keep_dims_0"), val = bool(false)]; + tensor var_3084_cast_fp16 = reduce_mean(axes = var_3084_axes_0, keep_dims = var_3084_keep_dims_0, x = x_447_cast_fp16)[name = string("op_3084_cast_fp16")]; + tensor x_449_axes_0 = const()[name = string("x_449_axes_0"), val = tensor([1])]; + tensor x_449_cast_fp16 = expand_dims(axes = x_449_axes_0, x = var_3084_cast_fp16)[name = string("x_449_cast_fp16")]; + tensor var_3086 = const()[name = string("op_3086"), val = tensor([0, 2, 1])]; + string x_451_pad_type_0 = const()[name = string("x_451_pad_type_0"), val = string("valid")]; + tensor x_451_strides_0 = const()[name = string("x_451_strides_0"), val = tensor([1])]; + tensor x_451_pad_0 = const()[name = string("x_451_pad_0"), val = tensor([0, 0])]; + tensor x_451_dilations_0 = const()[name = string("x_451_dilations_0"), val = tensor([1])]; + int32 x_451_groups_0 = const()[name = string("x_451_groups_0"), val = int32(1)]; + tensor input_255_cast_fp16 = transpose(perm = var_3086, x = x_449_cast_fp16)[name = string("transpose_229")]; + tensor x_451_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_451_dilations_0, groups = x_451_groups_0, pad = x_451_pad_0, pad_type = x_451_pad_type_0, strides = x_451_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_255_cast_fp16)[name = string("x_451_cast_fp16")]; + tensor x_pred_7_perm_0 = const()[name = string("x_pred_7_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_7_to_fp16 = const()[name = string("c_skip_7_to_fp16"), val = tensor([[[0x1.528p-2]]])]; + tensor var_3094_cast_fp16 = mul(x = c_skip_7_to_fp16, y = x_noisy_7_cast_fp16)[name = string("op_3094_cast_fp16")]; + tensor c_out_7_to_fp16 = const()[name = string("c_out_7_to_fp16"), val = tensor([[[0x1.4dcp-3]]])]; + tensor x_pred_7_cast_fp16 = transpose(perm = x_pred_7_perm_0, x = x_451_cast_fp16)[name = string("transpose_228")]; + tensor var_3095_cast_fp16 = mul(x = c_out_7_to_fp16, y = x_pred_7_cast_fp16)[name = string("op_3095_cast_fp16")]; + tensor x_mid_dn_3_cast_fp16 = add(x = var_3094_cast_fp16, y = var_3095_cast_fp16)[name = string("x_mid_dn_3_cast_fp16")]; + tensor var_3098_cast_fp16 = sub(x = x_noisy_7_cast_fp16, y = x_mid_dn_3_cast_fp16)[name = string("op_3098_cast_fp16")]; + tensor _inversed_d_mid_3_y_0_to_fp16 = const()[name = string("_inversed_d_mid_3_y_0_to_fp16"), val = tensor([0x1.c3cp+1])]; + tensor _inversed_d_mid_3_cast_fp16 = mul(x = var_3098_cast_fp16, y = _inversed_d_mid_3_y_0_to_fp16)[name = string("_inversed_d_mid_3_cast_fp16")]; + fp16 var_3107_to_fp16 = const()[name = string("op_3107_to_fp16"), val = fp16(-0x1.19p-1)]; + tensor var_3108_cast_fp16 = mul(x = _inversed_d_mid_3_cast_fp16, y = var_3107_to_fp16)[name = string("op_3108_cast_fp16")]; + tensor x_453_cast_fp16 = add(x = x_noisy_5_cast_fp16, y = var_3108_cast_fp16)[name = string("x_453_cast_fp16")]; + tensor var_3113_begin_0 = const()[name = string("op_3113_begin_0"), val = tensor([1, 0, 0, 0])]; + tensor var_3113_end_0 = const()[name = string("op_3113_end_0"), val = tensor([2, 1, 1, 256])]; + tensor var_3113_end_mask_0 = const()[name = string("op_3113_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_3113_squeeze_mask_0 = const()[name = string("op_3113_squeeze_mask_0"), val = tensor([true, false, false, false])]; + tensor var_3113_cast_fp16 = slice_by_index(begin = var_3113_begin_0, end = var_3113_end_0, end_mask = var_3113_end_mask_0, squeeze_mask = var_3113_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_3113_cast_fp16")]; + fp16 var_3116_to_fp16 = const()[name = string("op_3116_to_fp16"), val = fp16(0x1.1ep-4)]; + tensor var_3117_cast_fp16 = mul(x = var_3113_cast_fp16, y = var_3116_to_fp16)[name = string("op_3117_cast_fp16")]; + tensor x_noisy_9_cast_fp16 = add(x = x_453_cast_fp16, y = var_3117_cast_fp16)[name = string("x_noisy_9_cast_fp16")]; + int32 var_3141 = const()[name = string("op_3141"), val = int32(-1)]; + tensor c_in_9_to_fp16 = const()[name = string("c_in_9_to_fp16"), val = tensor([[[0x1.2ecp+2]]])]; + tensor x_463_cast_fp16 = mul(x = c_in_9_to_fp16, y = x_noisy_9_cast_fp16)[name = string("x_463_cast_fp16")]; + int32 x_459_axis_0 = const()[name = string("x_459_axis_0"), val = int32(0)]; + tensor var_3527_to_fp16 = const()[name = string("op_3527_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49353408)))]; + tensor x_459_cast_fp16 = stack(axis = x_459_axis_0, values = (var_3527_to_fp16, var_423_cast_fp16))[name = string("x_459_cast_fp16")]; + tensor var_3532 = const()[name = string("op_3532"), val = tensor([1, 2, 0])]; + tensor input_263_axes_0 = const()[name = string("input_263_axes_0"), val = tensor([2])]; + bool input_263_keep_dims_0 = const()[name = string("input_263_keep_dims_0"), val = bool(false)]; + tensor x_461_cast_fp16 = transpose(perm = var_3532, x = x_459_cast_fp16)[name = string("transpose_227")]; + tensor input_263_cast_fp16 = reduce_sum(axes = input_263_axes_0, keep_dims = input_263_keep_dims_0, x = x_461_cast_fp16)[name = string("input_263_cast_fp16")]; + tensor linear_102_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_263_cast_fp16)[name = string("linear_102_cast_fp16")]; + string input_267_mode_0 = const()[name = string("input_267_mode_0"), val = string("EXACT")]; + tensor input_267_cast_fp16 = gelu(mode = input_267_mode_0, x = linear_102_cast_fp16)[name = string("input_267_cast_fp16")]; + tensor linear_103_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_267_cast_fp16)[name = string("linear_103_cast_fp16")]; + string mapping_17_mode_0 = const()[name = string("mapping_17_mode_0"), val = string("EXACT")]; + tensor mapping_17_cast_fp16 = gelu(mode = mapping_17_mode_0, x = linear_103_cast_fp16)[name = string("mapping_17_cast_fp16")]; + tensor var_3542_reps_0 = const()[name = string("op_3542_reps_0"), val = tensor([1, 64, 1])]; + tensor var_3542_cast_fp16 = tile(reps = var_3542_reps_0, x = x_463_cast_fp16)[name = string("op_3542_cast_fp16")]; + bool x_465_interleave_0 = const()[name = string("x_465_interleave_0"), val = bool(false)]; + tensor x_465_cast_fp16 = concat(axis = var_3141, interleave = x_465_interleave_0, values = (var_3542_cast_fp16, embedding_to_fp16))[name = string("x_465_cast_fp16")]; + tensor var_3545_axes_0 = const()[name = string("op_3545_axes_0"), val = tensor([1])]; + tensor var_3545_cast_fp16 = expand_dims(axes = var_3545_axes_0, x = mapping_17_cast_fp16)[name = string("op_3545_cast_fp16")]; + tensor mapping_19_reps_0 = const()[name = string("mapping_19_reps_0"), val = tensor([1, 64, 1])]; + tensor mapping_19_cast_fp16 = tile(reps = mapping_19_reps_0, x = var_3545_cast_fp16)[name = string("mapping_19_cast_fp16")]; + tensor x_467_cast_fp16 = add(x = x_465_cast_fp16, y = mapping_19_cast_fp16)[name = string("x_467_cast_fp16")]; + tensor var_3557_split_sizes_0 = const()[name = string("op_3557_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3557_axis_0 = const()[name = string("op_3557_axis_0"), val = int32(1)]; + tensor var_3557_cast_fp16_0, tensor var_3557_cast_fp16_1 = split(axis = var_3557_axis_0, split_sizes = var_3557_split_sizes_0, x = h_3_cast_fp16)[name = string("op_3557_cast_fp16")]; + tensor gamma_99_perm_0 = const()[name = string("gamma_99_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_99_perm_0 = const()[name = string("beta_99_perm_0"), val = tensor([0, -1, 1])]; + tensor x_471_axes_0 = const()[name = string("x_471_axes_0"), val = tensor([-1])]; + fp16 var_3137_to_fp16 = const()[name = string("op_3137_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_471_cast_fp16 = layer_norm(axes = x_471_axes_0, epsilon = var_3137_to_fp16, x = x_467_cast_fp16)[name = string("x_471_cast_fp16")]; + fp16 var_3563_promoted_to_fp16 = const()[name = string("op_3563_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_99_cast_fp16 = transpose(perm = gamma_99_perm_0, x = var_3557_cast_fp16_0)[name = string("transpose_226")]; + tensor var_3564_cast_fp16 = add(x = gamma_99_cast_fp16, y = var_3563_promoted_to_fp16)[name = string("op_3564_cast_fp16")]; + tensor var_3565_cast_fp16 = mul(x = var_3564_cast_fp16, y = x_471_cast_fp16)[name = string("op_3565_cast_fp16")]; + tensor beta_99_cast_fp16 = transpose(perm = beta_99_perm_0, x = var_3557_cast_fp16_1)[name = string("transpose_225")]; + tensor x_473_cast_fp16 = add(x = var_3565_cast_fp16, y = beta_99_cast_fp16)[name = string("x_473_cast_fp16")]; + tensor var_3576_split_sizes_0 = const()[name = string("op_3576_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3576_axis_0 = const()[name = string("op_3576_axis_0"), val = int32(1)]; + tensor var_3576_cast_fp16_0, tensor var_3576_cast_fp16_1 = split(axis = var_3576_axis_0, split_sizes = var_3576_split_sizes_0, x = h_7_cast_fp16)[name = string("op_3576_cast_fp16")]; + tensor gamma_103_perm_0 = const()[name = string("gamma_103_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_103_perm_0 = const()[name = string("beta_103_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3582_promoted_to_fp16 = const()[name = string("op_3582_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_103_cast_fp16 = transpose(perm = gamma_103_perm_0, x = var_3576_cast_fp16_0)[name = string("transpose_224")]; + tensor var_3583_cast_fp16 = add(x = gamma_103_cast_fp16, y = var_3582_promoted_to_fp16)[name = string("op_3583_cast_fp16")]; + tensor var_3584_cast_fp16 = mul(x = var_3583_cast_fp16, y = x_471_cast_fp16)[name = string("op_3584_cast_fp16")]; + tensor beta_103_cast_fp16 = transpose(perm = beta_103_perm_0, x = var_3576_cast_fp16_1)[name = string("transpose_223")]; + tensor x_479_cast_fp16 = add(x = var_3584_cast_fp16, y = beta_103_cast_fp16)[name = string("x_479_cast_fp16")]; + tensor linear_106_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_473_cast_fp16)[name = string("linear_106_cast_fp16")]; + tensor linear_107_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_479_cast_fp16)[name = string("linear_107_cast_fp16")]; + tensor var_3590_split_sizes_0 = const()[name = string("op_3590_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3590_axis_0 = const()[name = string("op_3590_axis_0"), val = int32(-1)]; + tensor var_3590_cast_fp16_0, tensor var_3590_cast_fp16_1 = split(axis = var_3590_axis_0, split_sizes = var_3590_split_sizes_0, x = linear_107_cast_fp16)[name = string("op_3590_cast_fp16")]; + tensor var_3598 = const()[name = string("op_3598"), val = tensor([1, 64, 8, 64])]; + tensor x_483_cast_fp16 = reshape(shape = var_3598, x = linear_106_cast_fp16)[name = string("x_483_cast_fp16")]; + tensor var_3608 = const()[name = string("op_3608"), val = tensor([1, 64, 8, 64])]; + tensor x_487_cast_fp16 = reshape(shape = var_3608, x = var_3590_cast_fp16_0)[name = string("x_487_cast_fp16")]; + tensor var_3618 = const()[name = string("op_3618"), val = tensor([1, 64, 8, 64])]; + tensor x_491_cast_fp16 = reshape(shape = var_3618, x = var_3590_cast_fp16_1)[name = string("x_491_cast_fp16")]; + tensor var_3620 = const()[name = string("op_3620"), val = tensor([0, 2, 1, 3])]; + bool sim_49_transpose_x_0 = const()[name = string("sim_49_transpose_x_0"), val = bool(false)]; + bool sim_49_transpose_y_0 = const()[name = string("sim_49_transpose_y_0"), val = bool(false)]; + tensor transpose_96_perm_0 = const()[name = string("transpose_96_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_97_perm_0 = const()[name = string("transpose_97_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_97 = transpose(perm = transpose_97_perm_0, x = x_487_cast_fp16)[name = string("transpose_220")]; + tensor transpose_96 = transpose(perm = transpose_96_perm_0, x = x_483_cast_fp16)[name = string("transpose_221")]; + tensor sim_49_cast_fp16 = matmul(transpose_x = sim_49_transpose_x_0, transpose_y = sim_49_transpose_y_0, x = transpose_96, y = transpose_97)[name = string("sim_49_cast_fp16")]; + fp16 var_3624_to_fp16 = const()[name = string("op_3624_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_51_cast_fp16 = mul(x = sim_49_cast_fp16, y = var_3624_to_fp16)[name = string("sim_51_cast_fp16")]; + tensor attn_25_cast_fp16 = softmax(axis = var_3141, x = sim_51_cast_fp16)[name = string("attn_25_cast_fp16")]; + bool x_493_transpose_x_0 = const()[name = string("x_493_transpose_x_0"), val = bool(false)]; + bool x_493_transpose_y_0 = const()[name = string("x_493_transpose_y_0"), val = bool(false)]; + tensor v_25_cast_fp16 = transpose(perm = var_3620, x = x_491_cast_fp16)[name = string("transpose_222")]; + tensor x_493_cast_fp16 = matmul(transpose_x = x_493_transpose_x_0, transpose_y = x_493_transpose_y_0, x = attn_25_cast_fp16, y = v_25_cast_fp16)[name = string("x_493_cast_fp16")]; + tensor var_3646 = const()[name = string("op_3646"), val = tensor([0, 2, 1, 3])]; + tensor var_3648 = const()[name = string("op_3648"), val = tensor([1, 64, 512])]; + tensor x_495_cast_fp16 = transpose(perm = var_3646, x = x_493_cast_fp16)[name = string("transpose_219")]; + tensor input_279_cast_fp16 = reshape(shape = var_3648, x = x_495_cast_fp16)[name = string("input_279_cast_fp16")]; + tensor linear_108_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_279_cast_fp16)[name = string("linear_108_cast_fp16")]; + tensor input_281_cast_fp16 = add(x = linear_108_cast_fp16, y = x_467_cast_fp16)[name = string("input_281_cast_fp16")]; + tensor linear_109_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_281_cast_fp16)[name = string("linear_109_cast_fp16")]; + string input_285_mode_0 = const()[name = string("input_285_mode_0"), val = string("EXACT")]; + tensor input_285_cast_fp16 = gelu(mode = input_285_mode_0, x = linear_109_cast_fp16)[name = string("input_285_cast_fp16")]; + tensor linear_110_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_285_cast_fp16)[name = string("linear_110_cast_fp16")]; + tensor x_497_cast_fp16 = add(x = linear_110_cast_fp16, y = input_281_cast_fp16)[name = string("x_497_cast_fp16")]; + tensor x_499_cast_fp16 = add(x = x_497_cast_fp16, y = mapping_19_cast_fp16)[name = string("x_499_cast_fp16")]; + tensor var_3664_split_sizes_0 = const()[name = string("op_3664_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3664_axis_0 = const()[name = string("op_3664_axis_0"), val = int32(1)]; + tensor var_3664_cast_fp16_0, tensor var_3664_cast_fp16_1 = split(axis = var_3664_axis_0, split_sizes = var_3664_split_sizes_0, x = h_11_cast_fp16)[name = string("op_3664_cast_fp16")]; + tensor gamma_107_perm_0 = const()[name = string("gamma_107_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_107_perm_0 = const()[name = string("beta_107_perm_0"), val = tensor([0, -1, 1])]; + tensor x_503_axes_0 = const()[name = string("x_503_axes_0"), val = tensor([-1])]; + tensor x_503_cast_fp16 = layer_norm(axes = x_503_axes_0, epsilon = var_3137_to_fp16, x = x_499_cast_fp16)[name = string("x_503_cast_fp16")]; + fp16 var_3670_promoted_to_fp16 = const()[name = string("op_3670_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_107_cast_fp16 = transpose(perm = gamma_107_perm_0, x = var_3664_cast_fp16_0)[name = string("transpose_218")]; + tensor var_3671_cast_fp16 = add(x = gamma_107_cast_fp16, y = var_3670_promoted_to_fp16)[name = string("op_3671_cast_fp16")]; + tensor var_3672_cast_fp16 = mul(x = var_3671_cast_fp16, y = x_503_cast_fp16)[name = string("op_3672_cast_fp16")]; + tensor beta_107_cast_fp16 = transpose(perm = beta_107_perm_0, x = var_3664_cast_fp16_1)[name = string("transpose_217")]; + tensor x_505_cast_fp16 = add(x = var_3672_cast_fp16, y = beta_107_cast_fp16)[name = string("x_505_cast_fp16")]; + tensor var_3683_split_sizes_0 = const()[name = string("op_3683_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3683_axis_0 = const()[name = string("op_3683_axis_0"), val = int32(1)]; + tensor var_3683_cast_fp16_0, tensor var_3683_cast_fp16_1 = split(axis = var_3683_axis_0, split_sizes = var_3683_split_sizes_0, x = h_15_cast_fp16)[name = string("op_3683_cast_fp16")]; + tensor gamma_111_perm_0 = const()[name = string("gamma_111_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_111_perm_0 = const()[name = string("beta_111_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3689_promoted_to_fp16 = const()[name = string("op_3689_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_111_cast_fp16 = transpose(perm = gamma_111_perm_0, x = var_3683_cast_fp16_0)[name = string("transpose_216")]; + tensor var_3690_cast_fp16 = add(x = gamma_111_cast_fp16, y = var_3689_promoted_to_fp16)[name = string("op_3690_cast_fp16")]; + tensor var_3691_cast_fp16 = mul(x = var_3690_cast_fp16, y = x_503_cast_fp16)[name = string("op_3691_cast_fp16")]; + tensor beta_111_cast_fp16 = transpose(perm = beta_111_perm_0, x = var_3683_cast_fp16_1)[name = string("transpose_215")]; + tensor x_511_cast_fp16 = add(x = var_3691_cast_fp16, y = beta_111_cast_fp16)[name = string("x_511_cast_fp16")]; + tensor linear_113_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_505_cast_fp16)[name = string("linear_113_cast_fp16")]; + tensor linear_114_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_511_cast_fp16)[name = string("linear_114_cast_fp16")]; + tensor var_3697_split_sizes_0 = const()[name = string("op_3697_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3697_axis_0 = const()[name = string("op_3697_axis_0"), val = int32(-1)]; + tensor var_3697_cast_fp16_0, tensor var_3697_cast_fp16_1 = split(axis = var_3697_axis_0, split_sizes = var_3697_split_sizes_0, x = linear_114_cast_fp16)[name = string("op_3697_cast_fp16")]; + tensor var_3705 = const()[name = string("op_3705"), val = tensor([1, 64, 8, 64])]; + tensor x_515_cast_fp16 = reshape(shape = var_3705, x = linear_113_cast_fp16)[name = string("x_515_cast_fp16")]; + tensor var_3715 = const()[name = string("op_3715"), val = tensor([1, 64, 8, 64])]; + tensor x_519_cast_fp16 = reshape(shape = var_3715, x = var_3697_cast_fp16_0)[name = string("x_519_cast_fp16")]; + tensor var_3725 = const()[name = string("op_3725"), val = tensor([1, 64, 8, 64])]; + tensor x_523_cast_fp16 = reshape(shape = var_3725, x = var_3697_cast_fp16_1)[name = string("x_523_cast_fp16")]; + tensor var_3727 = const()[name = string("op_3727"), val = tensor([0, 2, 1, 3])]; + bool sim_53_transpose_x_0 = const()[name = string("sim_53_transpose_x_0"), val = bool(false)]; + bool sim_53_transpose_y_0 = const()[name = string("sim_53_transpose_y_0"), val = bool(false)]; + tensor transpose_98_perm_0 = const()[name = string("transpose_98_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_99_perm_0 = const()[name = string("transpose_99_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_99 = transpose(perm = transpose_99_perm_0, x = x_519_cast_fp16)[name = string("transpose_212")]; + tensor transpose_98 = transpose(perm = transpose_98_perm_0, x = x_515_cast_fp16)[name = string("transpose_213")]; + tensor sim_53_cast_fp16 = matmul(transpose_x = sim_53_transpose_x_0, transpose_y = sim_53_transpose_y_0, x = transpose_98, y = transpose_99)[name = string("sim_53_cast_fp16")]; + fp16 var_3731_to_fp16 = const()[name = string("op_3731_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_55_cast_fp16 = mul(x = sim_53_cast_fp16, y = var_3731_to_fp16)[name = string("sim_55_cast_fp16")]; + tensor attn_27_cast_fp16 = softmax(axis = var_3141, x = sim_55_cast_fp16)[name = string("attn_27_cast_fp16")]; + bool x_525_transpose_x_0 = const()[name = string("x_525_transpose_x_0"), val = bool(false)]; + bool x_525_transpose_y_0 = const()[name = string("x_525_transpose_y_0"), val = bool(false)]; + tensor v_27_cast_fp16 = transpose(perm = var_3727, x = x_523_cast_fp16)[name = string("transpose_214")]; + tensor x_525_cast_fp16 = matmul(transpose_x = x_525_transpose_x_0, transpose_y = x_525_transpose_y_0, x = attn_27_cast_fp16, y = v_27_cast_fp16)[name = string("x_525_cast_fp16")]; + tensor var_3753 = const()[name = string("op_3753"), val = tensor([0, 2, 1, 3])]; + tensor var_3755 = const()[name = string("op_3755"), val = tensor([1, 64, 512])]; + tensor x_527_cast_fp16 = transpose(perm = var_3753, x = x_525_cast_fp16)[name = string("transpose_211")]; + tensor input_295_cast_fp16 = reshape(shape = var_3755, x = x_527_cast_fp16)[name = string("input_295_cast_fp16")]; + tensor linear_115_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_295_cast_fp16)[name = string("linear_115_cast_fp16")]; + tensor input_297_cast_fp16 = add(x = linear_115_cast_fp16, y = x_499_cast_fp16)[name = string("input_297_cast_fp16")]; + tensor linear_116_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_297_cast_fp16)[name = string("linear_116_cast_fp16")]; + string input_301_mode_0 = const()[name = string("input_301_mode_0"), val = string("EXACT")]; + tensor input_301_cast_fp16 = gelu(mode = input_301_mode_0, x = linear_116_cast_fp16)[name = string("input_301_cast_fp16")]; + tensor linear_117_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_301_cast_fp16)[name = string("linear_117_cast_fp16")]; + tensor x_529_cast_fp16 = add(x = linear_117_cast_fp16, y = input_297_cast_fp16)[name = string("x_529_cast_fp16")]; + tensor x_531_cast_fp16 = add(x = x_529_cast_fp16, y = mapping_19_cast_fp16)[name = string("x_531_cast_fp16")]; + tensor var_3771_split_sizes_0 = const()[name = string("op_3771_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3771_axis_0 = const()[name = string("op_3771_axis_0"), val = int32(1)]; + tensor var_3771_cast_fp16_0, tensor var_3771_cast_fp16_1 = split(axis = var_3771_axis_0, split_sizes = var_3771_split_sizes_0, x = h_19_cast_fp16)[name = string("op_3771_cast_fp16")]; + tensor gamma_115_perm_0 = const()[name = string("gamma_115_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_115_perm_0 = const()[name = string("beta_115_perm_0"), val = tensor([0, -1, 1])]; + tensor x_535_axes_0 = const()[name = string("x_535_axes_0"), val = tensor([-1])]; + tensor x_535_cast_fp16 = layer_norm(axes = x_535_axes_0, epsilon = var_3137_to_fp16, x = x_531_cast_fp16)[name = string("x_535_cast_fp16")]; + fp16 var_3777_promoted_to_fp16 = const()[name = string("op_3777_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_115_cast_fp16 = transpose(perm = gamma_115_perm_0, x = var_3771_cast_fp16_0)[name = string("transpose_210")]; + tensor var_3778_cast_fp16 = add(x = gamma_115_cast_fp16, y = var_3777_promoted_to_fp16)[name = string("op_3778_cast_fp16")]; + tensor var_3779_cast_fp16 = mul(x = var_3778_cast_fp16, y = x_535_cast_fp16)[name = string("op_3779_cast_fp16")]; + tensor beta_115_cast_fp16 = transpose(perm = beta_115_perm_0, x = var_3771_cast_fp16_1)[name = string("transpose_209")]; + tensor x_537_cast_fp16 = add(x = var_3779_cast_fp16, y = beta_115_cast_fp16)[name = string("x_537_cast_fp16")]; + tensor var_3790_split_sizes_0 = const()[name = string("op_3790_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_3790_axis_0 = const()[name = string("op_3790_axis_0"), val = int32(1)]; + tensor var_3790_cast_fp16_0, tensor var_3790_cast_fp16_1 = split(axis = var_3790_axis_0, split_sizes = var_3790_split_sizes_0, x = h_23_cast_fp16)[name = string("op_3790_cast_fp16")]; + tensor gamma_119_perm_0 = const()[name = string("gamma_119_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_119_perm_0 = const()[name = string("beta_119_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_3796_promoted_to_fp16 = const()[name = string("op_3796_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_119_cast_fp16 = transpose(perm = gamma_119_perm_0, x = var_3790_cast_fp16_0)[name = string("transpose_208")]; + tensor var_3797_cast_fp16 = add(x = gamma_119_cast_fp16, y = var_3796_promoted_to_fp16)[name = string("op_3797_cast_fp16")]; + tensor var_3798_cast_fp16 = mul(x = var_3797_cast_fp16, y = x_535_cast_fp16)[name = string("op_3798_cast_fp16")]; + tensor beta_119_cast_fp16 = transpose(perm = beta_119_perm_0, x = var_3790_cast_fp16_1)[name = string("transpose_207")]; + tensor x_543_cast_fp16 = add(x = var_3798_cast_fp16, y = beta_119_cast_fp16)[name = string("x_543_cast_fp16")]; + tensor linear_120_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_537_cast_fp16)[name = string("linear_120_cast_fp16")]; + tensor linear_121_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_543_cast_fp16)[name = string("linear_121_cast_fp16")]; + tensor var_3804_split_sizes_0 = const()[name = string("op_3804_split_sizes_0"), val = tensor([512, 512])]; + int32 var_3804_axis_0 = const()[name = string("op_3804_axis_0"), val = int32(-1)]; + tensor var_3804_cast_fp16_0, tensor var_3804_cast_fp16_1 = split(axis = var_3804_axis_0, split_sizes = var_3804_split_sizes_0, x = linear_121_cast_fp16)[name = string("op_3804_cast_fp16")]; + tensor var_3812 = const()[name = string("op_3812"), val = tensor([1, 64, 8, 64])]; + tensor x_547_cast_fp16 = reshape(shape = var_3812, x = linear_120_cast_fp16)[name = string("x_547_cast_fp16")]; + tensor var_3822 = const()[name = string("op_3822"), val = tensor([1, 64, 8, 64])]; + tensor x_551_cast_fp16 = reshape(shape = var_3822, x = var_3804_cast_fp16_0)[name = string("x_551_cast_fp16")]; + tensor var_3832 = const()[name = string("op_3832"), val = tensor([1, 64, 8, 64])]; + tensor x_555_cast_fp16 = reshape(shape = var_3832, x = var_3804_cast_fp16_1)[name = string("x_555_cast_fp16")]; + tensor var_3834 = const()[name = string("op_3834"), val = tensor([0, 2, 1, 3])]; + bool sim_57_transpose_x_0 = const()[name = string("sim_57_transpose_x_0"), val = bool(false)]; + bool sim_57_transpose_y_0 = const()[name = string("sim_57_transpose_y_0"), val = bool(false)]; + tensor transpose_100_perm_0 = const()[name = string("transpose_100_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_101_perm_0 = const()[name = string("transpose_101_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_101 = transpose(perm = transpose_101_perm_0, x = x_551_cast_fp16)[name = string("transpose_204")]; + tensor transpose_100 = transpose(perm = transpose_100_perm_0, x = x_547_cast_fp16)[name = string("transpose_205")]; + tensor sim_57_cast_fp16 = matmul(transpose_x = sim_57_transpose_x_0, transpose_y = sim_57_transpose_y_0, x = transpose_100, y = transpose_101)[name = string("sim_57_cast_fp16")]; + fp16 var_3838_to_fp16 = const()[name = string("op_3838_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_59_cast_fp16 = mul(x = sim_57_cast_fp16, y = var_3838_to_fp16)[name = string("sim_59_cast_fp16")]; + tensor attn_29_cast_fp16 = softmax(axis = var_3141, x = sim_59_cast_fp16)[name = string("attn_29_cast_fp16")]; + bool x_557_transpose_x_0 = const()[name = string("x_557_transpose_x_0"), val = bool(false)]; + bool x_557_transpose_y_0 = const()[name = string("x_557_transpose_y_0"), val = bool(false)]; + tensor v_29_cast_fp16 = transpose(perm = var_3834, x = x_555_cast_fp16)[name = string("transpose_206")]; + tensor x_557_cast_fp16 = matmul(transpose_x = x_557_transpose_x_0, transpose_y = x_557_transpose_y_0, x = attn_29_cast_fp16, y = v_29_cast_fp16)[name = string("x_557_cast_fp16")]; + tensor var_3860 = const()[name = string("op_3860"), val = tensor([0, 2, 1, 3])]; + tensor var_3862 = const()[name = string("op_3862"), val = tensor([1, 64, 512])]; + tensor x_559_cast_fp16 = transpose(perm = var_3860, x = x_557_cast_fp16)[name = string("transpose_203")]; + tensor input_311_cast_fp16 = reshape(shape = var_3862, x = x_559_cast_fp16)[name = string("input_311_cast_fp16")]; + tensor linear_122_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_311_cast_fp16)[name = string("linear_122_cast_fp16")]; + tensor input_313_cast_fp16 = add(x = linear_122_cast_fp16, y = x_531_cast_fp16)[name = string("input_313_cast_fp16")]; + tensor linear_123_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_313_cast_fp16)[name = string("linear_123_cast_fp16")]; + string input_317_mode_0 = const()[name = string("input_317_mode_0"), val = string("EXACT")]; + tensor input_317_cast_fp16 = gelu(mode = input_317_mode_0, x = linear_123_cast_fp16)[name = string("input_317_cast_fp16")]; + tensor linear_124_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_317_cast_fp16)[name = string("linear_124_cast_fp16")]; + tensor x_561_cast_fp16 = add(x = linear_124_cast_fp16, y = input_313_cast_fp16)[name = string("x_561_cast_fp16")]; + tensor var_3871_axes_0 = const()[name = string("op_3871_axes_0"), val = tensor([1])]; + bool var_3871_keep_dims_0 = const()[name = string("op_3871_keep_dims_0"), val = bool(false)]; + tensor var_3871_cast_fp16 = reduce_mean(axes = var_3871_axes_0, keep_dims = var_3871_keep_dims_0, x = x_561_cast_fp16)[name = string("op_3871_cast_fp16")]; + tensor x_563_axes_0 = const()[name = string("x_563_axes_0"), val = tensor([1])]; + tensor x_563_cast_fp16 = expand_dims(axes = x_563_axes_0, x = var_3871_cast_fp16)[name = string("x_563_cast_fp16")]; + tensor var_3873 = const()[name = string("op_3873"), val = tensor([0, 2, 1])]; + string x_565_pad_type_0 = const()[name = string("x_565_pad_type_0"), val = string("valid")]; + tensor x_565_strides_0 = const()[name = string("x_565_strides_0"), val = tensor([1])]; + tensor x_565_pad_0 = const()[name = string("x_565_pad_0"), val = tensor([0, 0])]; + tensor x_565_dilations_0 = const()[name = string("x_565_dilations_0"), val = tensor([1])]; + int32 x_565_groups_0 = const()[name = string("x_565_groups_0"), val = int32(1)]; + tensor input_319_cast_fp16 = transpose(perm = var_3873, x = x_563_cast_fp16)[name = string("transpose_202")]; + tensor x_565_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_565_dilations_0, groups = x_565_groups_0, pad = x_565_pad_0, pad_type = x_565_pad_type_0, strides = x_565_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_319_cast_fp16)[name = string("x_565_cast_fp16")]; + tensor x_pred_9_perm_0 = const()[name = string("x_pred_9_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_9_to_fp16 = const()[name = string("c_skip_9_to_fp16"), val = tensor([[[0x1.c64p-1]]])]; + tensor var_3881_cast_fp16 = mul(x = c_skip_9_to_fp16, y = x_noisy_9_cast_fp16)[name = string("op_3881_cast_fp16")]; + tensor c_out_9_to_fp16 = const()[name = string("c_out_9_to_fp16"), val = tensor([[[0x1.0fcp-4]]])]; + tensor x_pred_9_cast_fp16 = transpose(perm = x_pred_9_perm_0, x = x_565_cast_fp16)[name = string("transpose_201")]; + tensor var_3882_cast_fp16 = mul(x = c_out_9_to_fp16, y = x_pred_9_cast_fp16)[name = string("op_3882_cast_fp16")]; + tensor x_dn_5_cast_fp16 = add(x = var_3881_cast_fp16, y = var_3882_cast_fp16)[name = string("x_dn_5_cast_fp16")]; + tensor var_3885_cast_fp16 = sub(x = x_noisy_9_cast_fp16, y = x_dn_5_cast_fp16)[name = string("op_3885_cast_fp16")]; + tensor _inversed_d_5_y_0_to_fp16 = const()[name = string("_inversed_d_5_y_0_to_fp16"), val = tensor([0x1.c6cp+3])]; + tensor _inversed_d_5_cast_fp16 = mul(x = var_3885_cast_fp16, y = _inversed_d_5_y_0_to_fp16)[name = string("_inversed_d_5_cast_fp16")]; + fp16 var_3894_to_fp16 = const()[name = string("op_3894_to_fp16"), val = fp16(-0x1.1fp-5)]; + tensor var_3895_cast_fp16 = mul(x = _inversed_d_5_cast_fp16, y = var_3894_to_fp16)[name = string("op_3895_cast_fp16")]; + tensor x_noisy_11_cast_fp16 = add(x = x_noisy_9_cast_fp16, y = var_3895_cast_fp16)[name = string("x_noisy_11_cast_fp16")]; + int32 var_3907 = const()[name = string("op_3907"), val = int32(-1)]; + tensor c_in_11_to_fp16 = const()[name = string("c_in_11_to_fp16"), val = tensor([[[0x1.3c4p+2]]])]; + tensor x_575_cast_fp16 = mul(x = c_in_11_to_fp16, y = x_noisy_11_cast_fp16)[name = string("x_575_cast_fp16")]; + int32 x_571_axis_0 = const()[name = string("x_571_axis_0"), val = int32(0)]; + tensor var_4293_to_fp16 = const()[name = string("op_4293_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49355520)))]; + tensor x_571_cast_fp16 = stack(axis = x_571_axis_0, values = (var_4293_to_fp16, var_423_cast_fp16))[name = string("x_571_cast_fp16")]; + tensor var_4298 = const()[name = string("op_4298"), val = tensor([1, 2, 0])]; + tensor input_327_axes_0 = const()[name = string("input_327_axes_0"), val = tensor([2])]; + bool input_327_keep_dims_0 = const()[name = string("input_327_keep_dims_0"), val = bool(false)]; + tensor x_573_cast_fp16 = transpose(perm = var_4298, x = x_571_cast_fp16)[name = string("transpose_200")]; + tensor input_327_cast_fp16 = reduce_sum(axes = input_327_axes_0, keep_dims = input_327_keep_dims_0, x = x_573_cast_fp16)[name = string("input_327_cast_fp16")]; + tensor linear_127_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_327_cast_fp16)[name = string("linear_127_cast_fp16")]; + string input_331_mode_0 = const()[name = string("input_331_mode_0"), val = string("EXACT")]; + tensor input_331_cast_fp16 = gelu(mode = input_331_mode_0, x = linear_127_cast_fp16)[name = string("input_331_cast_fp16")]; + tensor linear_128_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_331_cast_fp16)[name = string("linear_128_cast_fp16")]; + string mapping_21_mode_0 = const()[name = string("mapping_21_mode_0"), val = string("EXACT")]; + tensor mapping_21_cast_fp16 = gelu(mode = mapping_21_mode_0, x = linear_128_cast_fp16)[name = string("mapping_21_cast_fp16")]; + tensor var_4308_reps_0 = const()[name = string("op_4308_reps_0"), val = tensor([1, 64, 1])]; + tensor var_4308_cast_fp16 = tile(reps = var_4308_reps_0, x = x_575_cast_fp16)[name = string("op_4308_cast_fp16")]; + bool x_577_interleave_0 = const()[name = string("x_577_interleave_0"), val = bool(false)]; + tensor x_577_cast_fp16 = concat(axis = var_3907, interleave = x_577_interleave_0, values = (var_4308_cast_fp16, embedding_to_fp16))[name = string("x_577_cast_fp16")]; + tensor var_4311_axes_0 = const()[name = string("op_4311_axes_0"), val = tensor([1])]; + tensor var_4311_cast_fp16 = expand_dims(axes = var_4311_axes_0, x = mapping_21_cast_fp16)[name = string("op_4311_cast_fp16")]; + tensor mapping_23_reps_0 = const()[name = string("mapping_23_reps_0"), val = tensor([1, 64, 1])]; + tensor mapping_23_cast_fp16 = tile(reps = mapping_23_reps_0, x = var_4311_cast_fp16)[name = string("mapping_23_cast_fp16")]; + tensor x_579_cast_fp16 = add(x = x_577_cast_fp16, y = mapping_23_cast_fp16)[name = string("x_579_cast_fp16")]; + tensor var_4323_split_sizes_0 = const()[name = string("op_4323_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4323_axis_0 = const()[name = string("op_4323_axis_0"), val = int32(1)]; + tensor var_4323_cast_fp16_0, tensor var_4323_cast_fp16_1 = split(axis = var_4323_axis_0, split_sizes = var_4323_split_sizes_0, x = h_3_cast_fp16)[name = string("op_4323_cast_fp16")]; + tensor gamma_123_perm_0 = const()[name = string("gamma_123_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_123_perm_0 = const()[name = string("beta_123_perm_0"), val = tensor([0, -1, 1])]; + tensor x_583_axes_0 = const()[name = string("x_583_axes_0"), val = tensor([-1])]; + fp16 var_3903_to_fp16 = const()[name = string("op_3903_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_583_cast_fp16 = layer_norm(axes = x_583_axes_0, epsilon = var_3903_to_fp16, x = x_579_cast_fp16)[name = string("x_583_cast_fp16")]; + fp16 var_4329_promoted_to_fp16 = const()[name = string("op_4329_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_123_cast_fp16 = transpose(perm = gamma_123_perm_0, x = var_4323_cast_fp16_0)[name = string("transpose_199")]; + tensor var_4330_cast_fp16 = add(x = gamma_123_cast_fp16, y = var_4329_promoted_to_fp16)[name = string("op_4330_cast_fp16")]; + tensor var_4331_cast_fp16 = mul(x = var_4330_cast_fp16, y = x_583_cast_fp16)[name = string("op_4331_cast_fp16")]; + tensor beta_123_cast_fp16 = transpose(perm = beta_123_perm_0, x = var_4323_cast_fp16_1)[name = string("transpose_198")]; + tensor x_585_cast_fp16 = add(x = var_4331_cast_fp16, y = beta_123_cast_fp16)[name = string("x_585_cast_fp16")]; + tensor var_4342_split_sizes_0 = const()[name = string("op_4342_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4342_axis_0 = const()[name = string("op_4342_axis_0"), val = int32(1)]; + tensor var_4342_cast_fp16_0, tensor var_4342_cast_fp16_1 = split(axis = var_4342_axis_0, split_sizes = var_4342_split_sizes_0, x = h_7_cast_fp16)[name = string("op_4342_cast_fp16")]; + tensor gamma_127_perm_0 = const()[name = string("gamma_127_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_127_perm_0 = const()[name = string("beta_127_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_4348_promoted_to_fp16 = const()[name = string("op_4348_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_127_cast_fp16 = transpose(perm = gamma_127_perm_0, x = var_4342_cast_fp16_0)[name = string("transpose_197")]; + tensor var_4349_cast_fp16 = add(x = gamma_127_cast_fp16, y = var_4348_promoted_to_fp16)[name = string("op_4349_cast_fp16")]; + tensor var_4350_cast_fp16 = mul(x = var_4349_cast_fp16, y = x_583_cast_fp16)[name = string("op_4350_cast_fp16")]; + tensor beta_127_cast_fp16 = transpose(perm = beta_127_perm_0, x = var_4342_cast_fp16_1)[name = string("transpose_196")]; + tensor x_591_cast_fp16 = add(x = var_4350_cast_fp16, y = beta_127_cast_fp16)[name = string("x_591_cast_fp16")]; + tensor linear_131_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_585_cast_fp16)[name = string("linear_131_cast_fp16")]; + tensor linear_132_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_591_cast_fp16)[name = string("linear_132_cast_fp16")]; + tensor var_4356_split_sizes_0 = const()[name = string("op_4356_split_sizes_0"), val = tensor([512, 512])]; + int32 var_4356_axis_0 = const()[name = string("op_4356_axis_0"), val = int32(-1)]; + tensor var_4356_cast_fp16_0, tensor var_4356_cast_fp16_1 = split(axis = var_4356_axis_0, split_sizes = var_4356_split_sizes_0, x = linear_132_cast_fp16)[name = string("op_4356_cast_fp16")]; + tensor var_4364 = const()[name = string("op_4364"), val = tensor([1, 64, 8, 64])]; + tensor x_595_cast_fp16 = reshape(shape = var_4364, x = linear_131_cast_fp16)[name = string("x_595_cast_fp16")]; + tensor var_4374 = const()[name = string("op_4374"), val = tensor([1, 64, 8, 64])]; + tensor x_599_cast_fp16 = reshape(shape = var_4374, x = var_4356_cast_fp16_0)[name = string("x_599_cast_fp16")]; + tensor var_4384 = const()[name = string("op_4384"), val = tensor([1, 64, 8, 64])]; + tensor x_603_cast_fp16 = reshape(shape = var_4384, x = var_4356_cast_fp16_1)[name = string("x_603_cast_fp16")]; + tensor var_4386 = const()[name = string("op_4386"), val = tensor([0, 2, 1, 3])]; + bool sim_61_transpose_x_0 = const()[name = string("sim_61_transpose_x_0"), val = bool(false)]; + bool sim_61_transpose_y_0 = const()[name = string("sim_61_transpose_y_0"), val = bool(false)]; + tensor transpose_102_perm_0 = const()[name = string("transpose_102_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_103_perm_0 = const()[name = string("transpose_103_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_103 = transpose(perm = transpose_103_perm_0, x = x_599_cast_fp16)[name = string("transpose_193")]; + tensor transpose_102 = transpose(perm = transpose_102_perm_0, x = x_595_cast_fp16)[name = string("transpose_194")]; + tensor sim_61_cast_fp16 = matmul(transpose_x = sim_61_transpose_x_0, transpose_y = sim_61_transpose_y_0, x = transpose_102, y = transpose_103)[name = string("sim_61_cast_fp16")]; + fp16 var_4390_to_fp16 = const()[name = string("op_4390_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_63_cast_fp16 = mul(x = sim_61_cast_fp16, y = var_4390_to_fp16)[name = string("sim_63_cast_fp16")]; + tensor attn_31_cast_fp16 = softmax(axis = var_3907, x = sim_63_cast_fp16)[name = string("attn_31_cast_fp16")]; + bool x_605_transpose_x_0 = const()[name = string("x_605_transpose_x_0"), val = bool(false)]; + bool x_605_transpose_y_0 = const()[name = string("x_605_transpose_y_0"), val = bool(false)]; + tensor v_31_cast_fp16 = transpose(perm = var_4386, x = x_603_cast_fp16)[name = string("transpose_195")]; + tensor x_605_cast_fp16 = matmul(transpose_x = x_605_transpose_x_0, transpose_y = x_605_transpose_y_0, x = attn_31_cast_fp16, y = v_31_cast_fp16)[name = string("x_605_cast_fp16")]; + tensor var_4412 = const()[name = string("op_4412"), val = tensor([0, 2, 1, 3])]; + tensor var_4414 = const()[name = string("op_4414"), val = tensor([1, 64, 512])]; + tensor x_607_cast_fp16 = transpose(perm = var_4412, x = x_605_cast_fp16)[name = string("transpose_192")]; + tensor input_343_cast_fp16 = reshape(shape = var_4414, x = x_607_cast_fp16)[name = string("input_343_cast_fp16")]; + tensor linear_133_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_343_cast_fp16)[name = string("linear_133_cast_fp16")]; + tensor input_345_cast_fp16 = add(x = linear_133_cast_fp16, y = x_579_cast_fp16)[name = string("input_345_cast_fp16")]; + tensor linear_134_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_345_cast_fp16)[name = string("linear_134_cast_fp16")]; + string input_349_mode_0 = const()[name = string("input_349_mode_0"), val = string("EXACT")]; + tensor input_349_cast_fp16 = gelu(mode = input_349_mode_0, x = linear_134_cast_fp16)[name = string("input_349_cast_fp16")]; + tensor linear_135_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_349_cast_fp16)[name = string("linear_135_cast_fp16")]; + tensor x_609_cast_fp16 = add(x = linear_135_cast_fp16, y = input_345_cast_fp16)[name = string("x_609_cast_fp16")]; + tensor x_611_cast_fp16 = add(x = x_609_cast_fp16, y = mapping_23_cast_fp16)[name = string("x_611_cast_fp16")]; + tensor var_4430_split_sizes_0 = const()[name = string("op_4430_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4430_axis_0 = const()[name = string("op_4430_axis_0"), val = int32(1)]; + tensor var_4430_cast_fp16_0, tensor var_4430_cast_fp16_1 = split(axis = var_4430_axis_0, split_sizes = var_4430_split_sizes_0, x = h_11_cast_fp16)[name = string("op_4430_cast_fp16")]; + tensor gamma_131_perm_0 = const()[name = string("gamma_131_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_131_perm_0 = const()[name = string("beta_131_perm_0"), val = tensor([0, -1, 1])]; + tensor x_615_axes_0 = const()[name = string("x_615_axes_0"), val = tensor([-1])]; + tensor x_615_cast_fp16 = layer_norm(axes = x_615_axes_0, epsilon = var_3903_to_fp16, x = x_611_cast_fp16)[name = string("x_615_cast_fp16")]; + fp16 var_4436_promoted_to_fp16 = const()[name = string("op_4436_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_131_cast_fp16 = transpose(perm = gamma_131_perm_0, x = var_4430_cast_fp16_0)[name = string("transpose_191")]; + tensor var_4437_cast_fp16 = add(x = gamma_131_cast_fp16, y = var_4436_promoted_to_fp16)[name = string("op_4437_cast_fp16")]; + tensor var_4438_cast_fp16 = mul(x = var_4437_cast_fp16, y = x_615_cast_fp16)[name = string("op_4438_cast_fp16")]; + tensor beta_131_cast_fp16 = transpose(perm = beta_131_perm_0, x = var_4430_cast_fp16_1)[name = string("transpose_190")]; + tensor x_617_cast_fp16 = add(x = var_4438_cast_fp16, y = beta_131_cast_fp16)[name = string("x_617_cast_fp16")]; + tensor var_4449_split_sizes_0 = const()[name = string("op_4449_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4449_axis_0 = const()[name = string("op_4449_axis_0"), val = int32(1)]; + tensor var_4449_cast_fp16_0, tensor var_4449_cast_fp16_1 = split(axis = var_4449_axis_0, split_sizes = var_4449_split_sizes_0, x = h_15_cast_fp16)[name = string("op_4449_cast_fp16")]; + tensor gamma_135_perm_0 = const()[name = string("gamma_135_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_135_perm_0 = const()[name = string("beta_135_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_4455_promoted_to_fp16 = const()[name = string("op_4455_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_135_cast_fp16 = transpose(perm = gamma_135_perm_0, x = var_4449_cast_fp16_0)[name = string("transpose_189")]; + tensor var_4456_cast_fp16 = add(x = gamma_135_cast_fp16, y = var_4455_promoted_to_fp16)[name = string("op_4456_cast_fp16")]; + tensor var_4457_cast_fp16 = mul(x = var_4456_cast_fp16, y = x_615_cast_fp16)[name = string("op_4457_cast_fp16")]; + tensor beta_135_cast_fp16 = transpose(perm = beta_135_perm_0, x = var_4449_cast_fp16_1)[name = string("transpose_188")]; + tensor x_623_cast_fp16 = add(x = var_4457_cast_fp16, y = beta_135_cast_fp16)[name = string("x_623_cast_fp16")]; + tensor linear_138_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_617_cast_fp16)[name = string("linear_138_cast_fp16")]; + tensor linear_139_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_623_cast_fp16)[name = string("linear_139_cast_fp16")]; + tensor var_4463_split_sizes_0 = const()[name = string("op_4463_split_sizes_0"), val = tensor([512, 512])]; + int32 var_4463_axis_0 = const()[name = string("op_4463_axis_0"), val = int32(-1)]; + tensor var_4463_cast_fp16_0, tensor var_4463_cast_fp16_1 = split(axis = var_4463_axis_0, split_sizes = var_4463_split_sizes_0, x = linear_139_cast_fp16)[name = string("op_4463_cast_fp16")]; + tensor var_4471 = const()[name = string("op_4471"), val = tensor([1, 64, 8, 64])]; + tensor x_627_cast_fp16 = reshape(shape = var_4471, x = linear_138_cast_fp16)[name = string("x_627_cast_fp16")]; + tensor var_4481 = const()[name = string("op_4481"), val = tensor([1, 64, 8, 64])]; + tensor x_631_cast_fp16 = reshape(shape = var_4481, x = var_4463_cast_fp16_0)[name = string("x_631_cast_fp16")]; + tensor var_4491 = const()[name = string("op_4491"), val = tensor([1, 64, 8, 64])]; + tensor x_635_cast_fp16 = reshape(shape = var_4491, x = var_4463_cast_fp16_1)[name = string("x_635_cast_fp16")]; + tensor var_4493 = const()[name = string("op_4493"), val = tensor([0, 2, 1, 3])]; + bool sim_65_transpose_x_0 = const()[name = string("sim_65_transpose_x_0"), val = bool(false)]; + bool sim_65_transpose_y_0 = const()[name = string("sim_65_transpose_y_0"), val = bool(false)]; + tensor transpose_104_perm_0 = const()[name = string("transpose_104_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_105_perm_0 = const()[name = string("transpose_105_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_105 = transpose(perm = transpose_105_perm_0, x = x_631_cast_fp16)[name = string("transpose_185")]; + tensor transpose_104 = transpose(perm = transpose_104_perm_0, x = x_627_cast_fp16)[name = string("transpose_186")]; + tensor sim_65_cast_fp16 = matmul(transpose_x = sim_65_transpose_x_0, transpose_y = sim_65_transpose_y_0, x = transpose_104, y = transpose_105)[name = string("sim_65_cast_fp16")]; + fp16 var_4497_to_fp16 = const()[name = string("op_4497_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_67_cast_fp16 = mul(x = sim_65_cast_fp16, y = var_4497_to_fp16)[name = string("sim_67_cast_fp16")]; + tensor attn_33_cast_fp16 = softmax(axis = var_3907, x = sim_67_cast_fp16)[name = string("attn_33_cast_fp16")]; + bool x_637_transpose_x_0 = const()[name = string("x_637_transpose_x_0"), val = bool(false)]; + bool x_637_transpose_y_0 = const()[name = string("x_637_transpose_y_0"), val = bool(false)]; + tensor v_33_cast_fp16 = transpose(perm = var_4493, x = x_635_cast_fp16)[name = string("transpose_187")]; + tensor x_637_cast_fp16 = matmul(transpose_x = x_637_transpose_x_0, transpose_y = x_637_transpose_y_0, x = attn_33_cast_fp16, y = v_33_cast_fp16)[name = string("x_637_cast_fp16")]; + tensor var_4519 = const()[name = string("op_4519"), val = tensor([0, 2, 1, 3])]; + tensor var_4521 = const()[name = string("op_4521"), val = tensor([1, 64, 512])]; + tensor x_639_cast_fp16 = transpose(perm = var_4519, x = x_637_cast_fp16)[name = string("transpose_184")]; + tensor input_359_cast_fp16 = reshape(shape = var_4521, x = x_639_cast_fp16)[name = string("input_359_cast_fp16")]; + tensor linear_140_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_359_cast_fp16)[name = string("linear_140_cast_fp16")]; + tensor input_361_cast_fp16 = add(x = linear_140_cast_fp16, y = x_611_cast_fp16)[name = string("input_361_cast_fp16")]; + tensor linear_141_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_361_cast_fp16)[name = string("linear_141_cast_fp16")]; + string input_365_mode_0 = const()[name = string("input_365_mode_0"), val = string("EXACT")]; + tensor input_365_cast_fp16 = gelu(mode = input_365_mode_0, x = linear_141_cast_fp16)[name = string("input_365_cast_fp16")]; + tensor linear_142_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_365_cast_fp16)[name = string("linear_142_cast_fp16")]; + tensor x_641_cast_fp16 = add(x = linear_142_cast_fp16, y = input_361_cast_fp16)[name = string("x_641_cast_fp16")]; + tensor x_643_cast_fp16 = add(x = x_641_cast_fp16, y = mapping_23_cast_fp16)[name = string("x_643_cast_fp16")]; + tensor var_4537_split_sizes_0 = const()[name = string("op_4537_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4537_axis_0 = const()[name = string("op_4537_axis_0"), val = int32(1)]; + tensor var_4537_cast_fp16_0, tensor var_4537_cast_fp16_1 = split(axis = var_4537_axis_0, split_sizes = var_4537_split_sizes_0, x = h_19_cast_fp16)[name = string("op_4537_cast_fp16")]; + tensor gamma_139_perm_0 = const()[name = string("gamma_139_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_139_perm_0 = const()[name = string("beta_139_perm_0"), val = tensor([0, -1, 1])]; + tensor x_647_axes_0 = const()[name = string("x_647_axes_0"), val = tensor([-1])]; + tensor x_647_cast_fp16 = layer_norm(axes = x_647_axes_0, epsilon = var_3903_to_fp16, x = x_643_cast_fp16)[name = string("x_647_cast_fp16")]; + fp16 var_4543_promoted_to_fp16 = const()[name = string("op_4543_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_139_cast_fp16 = transpose(perm = gamma_139_perm_0, x = var_4537_cast_fp16_0)[name = string("transpose_183")]; + tensor var_4544_cast_fp16 = add(x = gamma_139_cast_fp16, y = var_4543_promoted_to_fp16)[name = string("op_4544_cast_fp16")]; + tensor var_4545_cast_fp16 = mul(x = var_4544_cast_fp16, y = x_647_cast_fp16)[name = string("op_4545_cast_fp16")]; + tensor beta_139_cast_fp16 = transpose(perm = beta_139_perm_0, x = var_4537_cast_fp16_1)[name = string("transpose_182")]; + tensor x_649_cast_fp16 = add(x = var_4545_cast_fp16, y = beta_139_cast_fp16)[name = string("x_649_cast_fp16")]; + tensor var_4556_split_sizes_0 = const()[name = string("op_4556_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_4556_axis_0 = const()[name = string("op_4556_axis_0"), val = int32(1)]; + tensor var_4556_cast_fp16_0, tensor var_4556_cast_fp16_1 = split(axis = var_4556_axis_0, split_sizes = var_4556_split_sizes_0, x = h_23_cast_fp16)[name = string("op_4556_cast_fp16")]; + tensor gamma_143_perm_0 = const()[name = string("gamma_143_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_143_perm_0 = const()[name = string("beta_143_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_4562_promoted_to_fp16 = const()[name = string("op_4562_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_143_cast_fp16 = transpose(perm = gamma_143_perm_0, x = var_4556_cast_fp16_0)[name = string("transpose_181")]; + tensor var_4563_cast_fp16 = add(x = gamma_143_cast_fp16, y = var_4562_promoted_to_fp16)[name = string("op_4563_cast_fp16")]; + tensor var_4564_cast_fp16 = mul(x = var_4563_cast_fp16, y = x_647_cast_fp16)[name = string("op_4564_cast_fp16")]; + tensor beta_143_cast_fp16 = transpose(perm = beta_143_perm_0, x = var_4556_cast_fp16_1)[name = string("transpose_180")]; + tensor x_655_cast_fp16 = add(x = var_4564_cast_fp16, y = beta_143_cast_fp16)[name = string("x_655_cast_fp16")]; + tensor linear_145_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_649_cast_fp16)[name = string("linear_145_cast_fp16")]; + tensor linear_146_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_655_cast_fp16)[name = string("linear_146_cast_fp16")]; + tensor var_4570_split_sizes_0 = const()[name = string("op_4570_split_sizes_0"), val = tensor([512, 512])]; + int32 var_4570_axis_0 = const()[name = string("op_4570_axis_0"), val = int32(-1)]; + tensor var_4570_cast_fp16_0, tensor var_4570_cast_fp16_1 = split(axis = var_4570_axis_0, split_sizes = var_4570_split_sizes_0, x = linear_146_cast_fp16)[name = string("op_4570_cast_fp16")]; + tensor var_4578 = const()[name = string("op_4578"), val = tensor([1, 64, 8, 64])]; + tensor x_659_cast_fp16 = reshape(shape = var_4578, x = linear_145_cast_fp16)[name = string("x_659_cast_fp16")]; + tensor var_4588 = const()[name = string("op_4588"), val = tensor([1, 64, 8, 64])]; + tensor x_663_cast_fp16 = reshape(shape = var_4588, x = var_4570_cast_fp16_0)[name = string("x_663_cast_fp16")]; + tensor var_4598 = const()[name = string("op_4598"), val = tensor([1, 64, 8, 64])]; + tensor x_667_cast_fp16 = reshape(shape = var_4598, x = var_4570_cast_fp16_1)[name = string("x_667_cast_fp16")]; + tensor var_4600 = const()[name = string("op_4600"), val = tensor([0, 2, 1, 3])]; + bool sim_69_transpose_x_0 = const()[name = string("sim_69_transpose_x_0"), val = bool(false)]; + bool sim_69_transpose_y_0 = const()[name = string("sim_69_transpose_y_0"), val = bool(false)]; + tensor transpose_106_perm_0 = const()[name = string("transpose_106_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_107_perm_0 = const()[name = string("transpose_107_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_107 = transpose(perm = transpose_107_perm_0, x = x_663_cast_fp16)[name = string("transpose_177")]; + tensor transpose_106 = transpose(perm = transpose_106_perm_0, x = x_659_cast_fp16)[name = string("transpose_178")]; + tensor sim_69_cast_fp16 = matmul(transpose_x = sim_69_transpose_x_0, transpose_y = sim_69_transpose_y_0, x = transpose_106, y = transpose_107)[name = string("sim_69_cast_fp16")]; + fp16 var_4604_to_fp16 = const()[name = string("op_4604_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_71_cast_fp16 = mul(x = sim_69_cast_fp16, y = var_4604_to_fp16)[name = string("sim_71_cast_fp16")]; + tensor attn_35_cast_fp16 = softmax(axis = var_3907, x = sim_71_cast_fp16)[name = string("attn_35_cast_fp16")]; + bool x_669_transpose_x_0 = const()[name = string("x_669_transpose_x_0"), val = bool(false)]; + bool x_669_transpose_y_0 = const()[name = string("x_669_transpose_y_0"), val = bool(false)]; + tensor v_35_cast_fp16 = transpose(perm = var_4600, x = x_667_cast_fp16)[name = string("transpose_179")]; + tensor x_669_cast_fp16 = matmul(transpose_x = x_669_transpose_x_0, transpose_y = x_669_transpose_y_0, x = attn_35_cast_fp16, y = v_35_cast_fp16)[name = string("x_669_cast_fp16")]; + tensor var_4626 = const()[name = string("op_4626"), val = tensor([0, 2, 1, 3])]; + tensor var_4628 = const()[name = string("op_4628"), val = tensor([1, 64, 512])]; + tensor x_671_cast_fp16 = transpose(perm = var_4626, x = x_669_cast_fp16)[name = string("transpose_176")]; + tensor input_375_cast_fp16 = reshape(shape = var_4628, x = x_671_cast_fp16)[name = string("input_375_cast_fp16")]; + tensor linear_147_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_375_cast_fp16)[name = string("linear_147_cast_fp16")]; + tensor input_377_cast_fp16 = add(x = linear_147_cast_fp16, y = x_643_cast_fp16)[name = string("input_377_cast_fp16")]; + tensor linear_148_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_377_cast_fp16)[name = string("linear_148_cast_fp16")]; + string input_381_mode_0 = const()[name = string("input_381_mode_0"), val = string("EXACT")]; + tensor input_381_cast_fp16 = gelu(mode = input_381_mode_0, x = linear_148_cast_fp16)[name = string("input_381_cast_fp16")]; + tensor linear_149_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_381_cast_fp16)[name = string("linear_149_cast_fp16")]; + tensor x_673_cast_fp16 = add(x = linear_149_cast_fp16, y = input_377_cast_fp16)[name = string("x_673_cast_fp16")]; + tensor var_4637_axes_0 = const()[name = string("op_4637_axes_0"), val = tensor([1])]; + bool var_4637_keep_dims_0 = const()[name = string("op_4637_keep_dims_0"), val = bool(false)]; + tensor var_4637_cast_fp16 = reduce_mean(axes = var_4637_axes_0, keep_dims = var_4637_keep_dims_0, x = x_673_cast_fp16)[name = string("op_4637_cast_fp16")]; + tensor x_675_axes_0 = const()[name = string("x_675_axes_0"), val = tensor([1])]; + tensor x_675_cast_fp16 = expand_dims(axes = x_675_axes_0, x = var_4637_cast_fp16)[name = string("x_675_cast_fp16")]; + tensor var_4639 = const()[name = string("op_4639"), val = tensor([0, 2, 1])]; + string x_677_pad_type_0 = const()[name = string("x_677_pad_type_0"), val = string("valid")]; + tensor x_677_strides_0 = const()[name = string("x_677_strides_0"), val = tensor([1])]; + tensor x_677_pad_0 = const()[name = string("x_677_pad_0"), val = tensor([0, 0])]; + tensor x_677_dilations_0 = const()[name = string("x_677_dilations_0"), val = tensor([1])]; + int32 x_677_groups_0 = const()[name = string("x_677_groups_0"), val = int32(1)]; + tensor input_383_cast_fp16 = transpose(perm = var_4639, x = x_675_cast_fp16)[name = string("transpose_175")]; + tensor x_677_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_677_dilations_0, groups = x_677_groups_0, pad = x_677_pad_0, pad_type = x_677_pad_type_0, strides = x_677_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_383_cast_fp16)[name = string("x_677_cast_fp16")]; + tensor x_pred_11_perm_0 = const()[name = string("x_pred_11_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_11_to_fp16 = const()[name = string("c_skip_11_to_fp16"), val = tensor([[[0x1.ef4p-1]]])]; + tensor var_4647_cast_fp16 = mul(x = c_skip_11_to_fp16, y = x_noisy_11_cast_fp16)[name = string("op_4647_cast_fp16")]; + tensor c_out_11_to_fp16 = const()[name = string("c_out_11_to_fp16"), val = tensor([[[0x1.1dp-5]]])]; + tensor x_pred_11_cast_fp16 = transpose(perm = x_pred_11_perm_0, x = x_677_cast_fp16)[name = string("transpose_174")]; + tensor var_4648_cast_fp16 = mul(x = c_out_11_to_fp16, y = x_pred_11_cast_fp16)[name = string("op_4648_cast_fp16")]; + tensor x_mid_dn_5_cast_fp16 = add(x = var_4647_cast_fp16, y = var_4648_cast_fp16)[name = string("x_mid_dn_5_cast_fp16")]; + tensor var_4651_cast_fp16 = sub(x = x_noisy_11_cast_fp16, y = x_mid_dn_5_cast_fp16)[name = string("op_4651_cast_fp16")]; + tensor _inversed_d_mid_5_y_0_to_fp16 = const()[name = string("_inversed_d_mid_5_y_0_to_fp16"), val = tensor([0x1.c4cp+4])]; + tensor _inversed_d_mid_5_cast_fp16 = mul(x = var_4651_cast_fp16, y = _inversed_d_mid_5_y_0_to_fp16)[name = string("_inversed_d_mid_5_cast_fp16")]; + fp16 var_4660_to_fp16 = const()[name = string("op_4660_to_fp16"), val = fp16(-0x1.1fp-4)]; + tensor var_4661_cast_fp16 = mul(x = _inversed_d_mid_5_cast_fp16, y = var_4660_to_fp16)[name = string("op_4661_cast_fp16")]; + tensor x_679_cast_fp16 = add(x = x_noisy_9_cast_fp16, y = var_4661_cast_fp16)[name = string("x_679_cast_fp16")]; + tensor var_4666_begin_0 = const()[name = string("op_4666_begin_0"), val = tensor([2, 0, 0, 0])]; + tensor var_4666_end_0 = const()[name = string("op_4666_end_0"), val = tensor([3, 1, 1, 256])]; + tensor var_4666_end_mask_0 = const()[name = string("op_4666_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_4666_squeeze_mask_0 = const()[name = string("op_4666_squeeze_mask_0"), val = tensor([true, false, false, false])]; + tensor var_4666_cast_fp16 = slice_by_index(begin = var_4666_begin_0, end = var_4666_end_0, end_mask = var_4666_end_mask_0, squeeze_mask = var_4666_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_4666_cast_fp16")]; + fp16 var_4669_to_fp16 = const()[name = string("op_4669_to_fp16"), val = fp16(0x1.37p-8)]; + tensor var_4670_cast_fp16 = mul(x = var_4666_cast_fp16, y = var_4669_to_fp16)[name = string("op_4670_cast_fp16")]; + tensor x_noisy_13_cast_fp16 = add(x = x_679_cast_fp16, y = var_4670_cast_fp16)[name = string("x_noisy_13_cast_fp16")]; + int32 var_4694 = const()[name = string("op_4694"), val = int32(-1)]; + tensor c_in_13_to_fp16 = const()[name = string("c_in_13_to_fp16"), val = tensor([[[0x1.41p+2]]])]; + tensor x_689_cast_fp16 = mul(x = c_in_13_to_fp16, y = x_noisy_13_cast_fp16)[name = string("x_689_cast_fp16")]; + int32 x_685_axis_0 = const()[name = string("x_685_axis_0"), val = int32(0)]; + tensor var_5080_to_fp16 = const()[name = string("op_5080_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49357632)))]; + tensor x_685_cast_fp16 = stack(axis = x_685_axis_0, values = (var_5080_to_fp16, var_423_cast_fp16))[name = string("x_685_cast_fp16")]; + tensor var_5085 = const()[name = string("op_5085"), val = tensor([1, 2, 0])]; + tensor input_391_axes_0 = const()[name = string("input_391_axes_0"), val = tensor([2])]; + bool input_391_keep_dims_0 = const()[name = string("input_391_keep_dims_0"), val = bool(false)]; + tensor x_687_cast_fp16 = transpose(perm = var_5085, x = x_685_cast_fp16)[name = string("transpose_173")]; + tensor input_391_cast_fp16 = reduce_sum(axes = input_391_axes_0, keep_dims = input_391_keep_dims_0, x = x_687_cast_fp16)[name = string("input_391_cast_fp16")]; + tensor linear_152_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_391_cast_fp16)[name = string("linear_152_cast_fp16")]; + string input_395_mode_0 = const()[name = string("input_395_mode_0"), val = string("EXACT")]; + tensor input_395_cast_fp16 = gelu(mode = input_395_mode_0, x = linear_152_cast_fp16)[name = string("input_395_cast_fp16")]; + tensor linear_153_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_395_cast_fp16)[name = string("linear_153_cast_fp16")]; + string mapping_25_mode_0 = const()[name = string("mapping_25_mode_0"), val = string("EXACT")]; + tensor mapping_25_cast_fp16 = gelu(mode = mapping_25_mode_0, x = linear_153_cast_fp16)[name = string("mapping_25_cast_fp16")]; + tensor var_5095_reps_0 = const()[name = string("op_5095_reps_0"), val = tensor([1, 64, 1])]; + tensor var_5095_cast_fp16 = tile(reps = var_5095_reps_0, x = x_689_cast_fp16)[name = string("op_5095_cast_fp16")]; + bool x_691_interleave_0 = const()[name = string("x_691_interleave_0"), val = bool(false)]; + tensor x_691_cast_fp16 = concat(axis = var_4694, interleave = x_691_interleave_0, values = (var_5095_cast_fp16, embedding_to_fp16))[name = string("x_691_cast_fp16")]; + tensor var_5098_axes_0 = const()[name = string("op_5098_axes_0"), val = tensor([1])]; + tensor var_5098_cast_fp16 = expand_dims(axes = var_5098_axes_0, x = mapping_25_cast_fp16)[name = string("op_5098_cast_fp16")]; + tensor mapping_27_reps_0 = const()[name = string("mapping_27_reps_0"), val = tensor([1, 64, 1])]; + tensor mapping_27_cast_fp16 = tile(reps = mapping_27_reps_0, x = var_5098_cast_fp16)[name = string("mapping_27_cast_fp16")]; + tensor x_693_cast_fp16 = add(x = x_691_cast_fp16, y = mapping_27_cast_fp16)[name = string("x_693_cast_fp16")]; + tensor var_5110_split_sizes_0 = const()[name = string("op_5110_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5110_axis_0 = const()[name = string("op_5110_axis_0"), val = int32(1)]; + tensor var_5110_cast_fp16_0, tensor var_5110_cast_fp16_1 = split(axis = var_5110_axis_0, split_sizes = var_5110_split_sizes_0, x = h_3_cast_fp16)[name = string("op_5110_cast_fp16")]; + tensor gamma_147_perm_0 = const()[name = string("gamma_147_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_147_perm_0 = const()[name = string("beta_147_perm_0"), val = tensor([0, -1, 1])]; + tensor x_697_axes_0 = const()[name = string("x_697_axes_0"), val = tensor([-1])]; + fp16 var_4690_to_fp16 = const()[name = string("op_4690_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_697_cast_fp16 = layer_norm(axes = x_697_axes_0, epsilon = var_4690_to_fp16, x = x_693_cast_fp16)[name = string("x_697_cast_fp16")]; + fp16 var_5116_promoted_to_fp16 = const()[name = string("op_5116_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_147_cast_fp16 = transpose(perm = gamma_147_perm_0, x = var_5110_cast_fp16_0)[name = string("transpose_172")]; + tensor var_5117_cast_fp16 = add(x = gamma_147_cast_fp16, y = var_5116_promoted_to_fp16)[name = string("op_5117_cast_fp16")]; + tensor var_5118_cast_fp16 = mul(x = var_5117_cast_fp16, y = x_697_cast_fp16)[name = string("op_5118_cast_fp16")]; + tensor beta_147_cast_fp16 = transpose(perm = beta_147_perm_0, x = var_5110_cast_fp16_1)[name = string("transpose_171")]; + tensor x_699_cast_fp16 = add(x = var_5118_cast_fp16, y = beta_147_cast_fp16)[name = string("x_699_cast_fp16")]; + tensor var_5129_split_sizes_0 = const()[name = string("op_5129_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5129_axis_0 = const()[name = string("op_5129_axis_0"), val = int32(1)]; + tensor var_5129_cast_fp16_0, tensor var_5129_cast_fp16_1 = split(axis = var_5129_axis_0, split_sizes = var_5129_split_sizes_0, x = h_7_cast_fp16)[name = string("op_5129_cast_fp16")]; + tensor gamma_151_perm_0 = const()[name = string("gamma_151_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_151_perm_0 = const()[name = string("beta_151_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5135_promoted_to_fp16 = const()[name = string("op_5135_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_151_cast_fp16 = transpose(perm = gamma_151_perm_0, x = var_5129_cast_fp16_0)[name = string("transpose_170")]; + tensor var_5136_cast_fp16 = add(x = gamma_151_cast_fp16, y = var_5135_promoted_to_fp16)[name = string("op_5136_cast_fp16")]; + tensor var_5137_cast_fp16 = mul(x = var_5136_cast_fp16, y = x_697_cast_fp16)[name = string("op_5137_cast_fp16")]; + tensor beta_151_cast_fp16 = transpose(perm = beta_151_perm_0, x = var_5129_cast_fp16_1)[name = string("transpose_169")]; + tensor x_705_cast_fp16 = add(x = var_5137_cast_fp16, y = beta_151_cast_fp16)[name = string("x_705_cast_fp16")]; + tensor linear_156_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_699_cast_fp16)[name = string("linear_156_cast_fp16")]; + tensor linear_157_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_705_cast_fp16)[name = string("linear_157_cast_fp16")]; + tensor var_5143_split_sizes_0 = const()[name = string("op_5143_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5143_axis_0 = const()[name = string("op_5143_axis_0"), val = int32(-1)]; + tensor var_5143_cast_fp16_0, tensor var_5143_cast_fp16_1 = split(axis = var_5143_axis_0, split_sizes = var_5143_split_sizes_0, x = linear_157_cast_fp16)[name = string("op_5143_cast_fp16")]; + tensor var_5151 = const()[name = string("op_5151"), val = tensor([1, 64, 8, 64])]; + tensor x_709_cast_fp16 = reshape(shape = var_5151, x = linear_156_cast_fp16)[name = string("x_709_cast_fp16")]; + tensor var_5161 = const()[name = string("op_5161"), val = tensor([1, 64, 8, 64])]; + tensor x_713_cast_fp16 = reshape(shape = var_5161, x = var_5143_cast_fp16_0)[name = string("x_713_cast_fp16")]; + tensor var_5171 = const()[name = string("op_5171"), val = tensor([1, 64, 8, 64])]; + tensor x_717_cast_fp16 = reshape(shape = var_5171, x = var_5143_cast_fp16_1)[name = string("x_717_cast_fp16")]; + tensor var_5173 = const()[name = string("op_5173"), val = tensor([0, 2, 1, 3])]; + bool sim_73_transpose_x_0 = const()[name = string("sim_73_transpose_x_0"), val = bool(false)]; + bool sim_73_transpose_y_0 = const()[name = string("sim_73_transpose_y_0"), val = bool(false)]; + tensor transpose_108_perm_0 = const()[name = string("transpose_108_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_109_perm_0 = const()[name = string("transpose_109_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_109 = transpose(perm = transpose_109_perm_0, x = x_713_cast_fp16)[name = string("transpose_166")]; + tensor transpose_108 = transpose(perm = transpose_108_perm_0, x = x_709_cast_fp16)[name = string("transpose_167")]; + tensor sim_73_cast_fp16 = matmul(transpose_x = sim_73_transpose_x_0, transpose_y = sim_73_transpose_y_0, x = transpose_108, y = transpose_109)[name = string("sim_73_cast_fp16")]; + fp16 var_5177_to_fp16 = const()[name = string("op_5177_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_75_cast_fp16 = mul(x = sim_73_cast_fp16, y = var_5177_to_fp16)[name = string("sim_75_cast_fp16")]; + tensor attn_37_cast_fp16 = softmax(axis = var_4694, x = sim_75_cast_fp16)[name = string("attn_37_cast_fp16")]; + bool x_719_transpose_x_0 = const()[name = string("x_719_transpose_x_0"), val = bool(false)]; + bool x_719_transpose_y_0 = const()[name = string("x_719_transpose_y_0"), val = bool(false)]; + tensor v_37_cast_fp16 = transpose(perm = var_5173, x = x_717_cast_fp16)[name = string("transpose_168")]; + tensor x_719_cast_fp16 = matmul(transpose_x = x_719_transpose_x_0, transpose_y = x_719_transpose_y_0, x = attn_37_cast_fp16, y = v_37_cast_fp16)[name = string("x_719_cast_fp16")]; + tensor var_5199 = const()[name = string("op_5199"), val = tensor([0, 2, 1, 3])]; + tensor var_5201 = const()[name = string("op_5201"), val = tensor([1, 64, 512])]; + tensor x_721_cast_fp16 = transpose(perm = var_5199, x = x_719_cast_fp16)[name = string("transpose_165")]; + tensor input_407_cast_fp16 = reshape(shape = var_5201, x = x_721_cast_fp16)[name = string("input_407_cast_fp16")]; + tensor linear_158_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_407_cast_fp16)[name = string("linear_158_cast_fp16")]; + tensor input_409_cast_fp16 = add(x = linear_158_cast_fp16, y = x_693_cast_fp16)[name = string("input_409_cast_fp16")]; + tensor linear_159_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_409_cast_fp16)[name = string("linear_159_cast_fp16")]; + string input_413_mode_0 = const()[name = string("input_413_mode_0"), val = string("EXACT")]; + tensor input_413_cast_fp16 = gelu(mode = input_413_mode_0, x = linear_159_cast_fp16)[name = string("input_413_cast_fp16")]; + tensor linear_160_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_413_cast_fp16)[name = string("linear_160_cast_fp16")]; + tensor x_723_cast_fp16 = add(x = linear_160_cast_fp16, y = input_409_cast_fp16)[name = string("x_723_cast_fp16")]; + tensor x_725_cast_fp16 = add(x = x_723_cast_fp16, y = mapping_27_cast_fp16)[name = string("x_725_cast_fp16")]; + tensor var_5217_split_sizes_0 = const()[name = string("op_5217_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5217_axis_0 = const()[name = string("op_5217_axis_0"), val = int32(1)]; + tensor var_5217_cast_fp16_0, tensor var_5217_cast_fp16_1 = split(axis = var_5217_axis_0, split_sizes = var_5217_split_sizes_0, x = h_11_cast_fp16)[name = string("op_5217_cast_fp16")]; + tensor gamma_155_perm_0 = const()[name = string("gamma_155_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_155_perm_0 = const()[name = string("beta_155_perm_0"), val = tensor([0, -1, 1])]; + tensor x_729_axes_0 = const()[name = string("x_729_axes_0"), val = tensor([-1])]; + tensor x_729_cast_fp16 = layer_norm(axes = x_729_axes_0, epsilon = var_4690_to_fp16, x = x_725_cast_fp16)[name = string("x_729_cast_fp16")]; + fp16 var_5223_promoted_to_fp16 = const()[name = string("op_5223_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_155_cast_fp16 = transpose(perm = gamma_155_perm_0, x = var_5217_cast_fp16_0)[name = string("transpose_164")]; + tensor var_5224_cast_fp16 = add(x = gamma_155_cast_fp16, y = var_5223_promoted_to_fp16)[name = string("op_5224_cast_fp16")]; + tensor var_5225_cast_fp16 = mul(x = var_5224_cast_fp16, y = x_729_cast_fp16)[name = string("op_5225_cast_fp16")]; + tensor beta_155_cast_fp16 = transpose(perm = beta_155_perm_0, x = var_5217_cast_fp16_1)[name = string("transpose_163")]; + tensor x_731_cast_fp16 = add(x = var_5225_cast_fp16, y = beta_155_cast_fp16)[name = string("x_731_cast_fp16")]; + tensor var_5236_split_sizes_0 = const()[name = string("op_5236_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5236_axis_0 = const()[name = string("op_5236_axis_0"), val = int32(1)]; + tensor var_5236_cast_fp16_0, tensor var_5236_cast_fp16_1 = split(axis = var_5236_axis_0, split_sizes = var_5236_split_sizes_0, x = h_15_cast_fp16)[name = string("op_5236_cast_fp16")]; + tensor gamma_159_perm_0 = const()[name = string("gamma_159_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_159_perm_0 = const()[name = string("beta_159_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5242_promoted_to_fp16 = const()[name = string("op_5242_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_159_cast_fp16 = transpose(perm = gamma_159_perm_0, x = var_5236_cast_fp16_0)[name = string("transpose_162")]; + tensor var_5243_cast_fp16 = add(x = gamma_159_cast_fp16, y = var_5242_promoted_to_fp16)[name = string("op_5243_cast_fp16")]; + tensor var_5244_cast_fp16 = mul(x = var_5243_cast_fp16, y = x_729_cast_fp16)[name = string("op_5244_cast_fp16")]; + tensor beta_159_cast_fp16 = transpose(perm = beta_159_perm_0, x = var_5236_cast_fp16_1)[name = string("transpose_161")]; + tensor x_737_cast_fp16 = add(x = var_5244_cast_fp16, y = beta_159_cast_fp16)[name = string("x_737_cast_fp16")]; + tensor linear_163_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_731_cast_fp16)[name = string("linear_163_cast_fp16")]; + tensor linear_164_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_737_cast_fp16)[name = string("linear_164_cast_fp16")]; + tensor var_5250_split_sizes_0 = const()[name = string("op_5250_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5250_axis_0 = const()[name = string("op_5250_axis_0"), val = int32(-1)]; + tensor var_5250_cast_fp16_0, tensor var_5250_cast_fp16_1 = split(axis = var_5250_axis_0, split_sizes = var_5250_split_sizes_0, x = linear_164_cast_fp16)[name = string("op_5250_cast_fp16")]; + tensor var_5258 = const()[name = string("op_5258"), val = tensor([1, 64, 8, 64])]; + tensor x_741_cast_fp16 = reshape(shape = var_5258, x = linear_163_cast_fp16)[name = string("x_741_cast_fp16")]; + tensor var_5268 = const()[name = string("op_5268"), val = tensor([1, 64, 8, 64])]; + tensor x_745_cast_fp16 = reshape(shape = var_5268, x = var_5250_cast_fp16_0)[name = string("x_745_cast_fp16")]; + tensor var_5278 = const()[name = string("op_5278"), val = tensor([1, 64, 8, 64])]; + tensor x_749_cast_fp16 = reshape(shape = var_5278, x = var_5250_cast_fp16_1)[name = string("x_749_cast_fp16")]; + tensor var_5280 = const()[name = string("op_5280"), val = tensor([0, 2, 1, 3])]; + bool sim_77_transpose_x_0 = const()[name = string("sim_77_transpose_x_0"), val = bool(false)]; + bool sim_77_transpose_y_0 = const()[name = string("sim_77_transpose_y_0"), val = bool(false)]; + tensor transpose_110_perm_0 = const()[name = string("transpose_110_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_111_perm_0 = const()[name = string("transpose_111_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_111 = transpose(perm = transpose_111_perm_0, x = x_745_cast_fp16)[name = string("transpose_158")]; + tensor transpose_110 = transpose(perm = transpose_110_perm_0, x = x_741_cast_fp16)[name = string("transpose_159")]; + tensor sim_77_cast_fp16 = matmul(transpose_x = sim_77_transpose_x_0, transpose_y = sim_77_transpose_y_0, x = transpose_110, y = transpose_111)[name = string("sim_77_cast_fp16")]; + fp16 var_5284_to_fp16 = const()[name = string("op_5284_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_79_cast_fp16 = mul(x = sim_77_cast_fp16, y = var_5284_to_fp16)[name = string("sim_79_cast_fp16")]; + tensor attn_39_cast_fp16 = softmax(axis = var_4694, x = sim_79_cast_fp16)[name = string("attn_39_cast_fp16")]; + bool x_751_transpose_x_0 = const()[name = string("x_751_transpose_x_0"), val = bool(false)]; + bool x_751_transpose_y_0 = const()[name = string("x_751_transpose_y_0"), val = bool(false)]; + tensor v_39_cast_fp16 = transpose(perm = var_5280, x = x_749_cast_fp16)[name = string("transpose_160")]; + tensor x_751_cast_fp16 = matmul(transpose_x = x_751_transpose_x_0, transpose_y = x_751_transpose_y_0, x = attn_39_cast_fp16, y = v_39_cast_fp16)[name = string("x_751_cast_fp16")]; + tensor var_5306 = const()[name = string("op_5306"), val = tensor([0, 2, 1, 3])]; + tensor var_5308 = const()[name = string("op_5308"), val = tensor([1, 64, 512])]; + tensor x_753_cast_fp16 = transpose(perm = var_5306, x = x_751_cast_fp16)[name = string("transpose_157")]; + tensor input_423_cast_fp16 = reshape(shape = var_5308, x = x_753_cast_fp16)[name = string("input_423_cast_fp16")]; + tensor linear_165_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_423_cast_fp16)[name = string("linear_165_cast_fp16")]; + tensor input_425_cast_fp16 = add(x = linear_165_cast_fp16, y = x_725_cast_fp16)[name = string("input_425_cast_fp16")]; + tensor linear_166_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_425_cast_fp16)[name = string("linear_166_cast_fp16")]; + string input_429_mode_0 = const()[name = string("input_429_mode_0"), val = string("EXACT")]; + tensor input_429_cast_fp16 = gelu(mode = input_429_mode_0, x = linear_166_cast_fp16)[name = string("input_429_cast_fp16")]; + tensor linear_167_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_429_cast_fp16)[name = string("linear_167_cast_fp16")]; + tensor x_755_cast_fp16 = add(x = linear_167_cast_fp16, y = input_425_cast_fp16)[name = string("x_755_cast_fp16")]; + tensor x_757_cast_fp16 = add(x = x_755_cast_fp16, y = mapping_27_cast_fp16)[name = string("x_757_cast_fp16")]; + tensor var_5324_split_sizes_0 = const()[name = string("op_5324_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5324_axis_0 = const()[name = string("op_5324_axis_0"), val = int32(1)]; + tensor var_5324_cast_fp16_0, tensor var_5324_cast_fp16_1 = split(axis = var_5324_axis_0, split_sizes = var_5324_split_sizes_0, x = h_19_cast_fp16)[name = string("op_5324_cast_fp16")]; + tensor gamma_163_perm_0 = const()[name = string("gamma_163_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_163_perm_0 = const()[name = string("beta_163_perm_0"), val = tensor([0, -1, 1])]; + tensor x_761_axes_0 = const()[name = string("x_761_axes_0"), val = tensor([-1])]; + tensor x_761_cast_fp16 = layer_norm(axes = x_761_axes_0, epsilon = var_4690_to_fp16, x = x_757_cast_fp16)[name = string("x_761_cast_fp16")]; + fp16 var_5330_promoted_to_fp16 = const()[name = string("op_5330_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_163_cast_fp16 = transpose(perm = gamma_163_perm_0, x = var_5324_cast_fp16_0)[name = string("transpose_156")]; + tensor var_5331_cast_fp16 = add(x = gamma_163_cast_fp16, y = var_5330_promoted_to_fp16)[name = string("op_5331_cast_fp16")]; + tensor var_5332_cast_fp16 = mul(x = var_5331_cast_fp16, y = x_761_cast_fp16)[name = string("op_5332_cast_fp16")]; + tensor beta_163_cast_fp16 = transpose(perm = beta_163_perm_0, x = var_5324_cast_fp16_1)[name = string("transpose_155")]; + tensor x_763_cast_fp16 = add(x = var_5332_cast_fp16, y = beta_163_cast_fp16)[name = string("x_763_cast_fp16")]; + tensor var_5343_split_sizes_0 = const()[name = string("op_5343_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5343_axis_0 = const()[name = string("op_5343_axis_0"), val = int32(1)]; + tensor var_5343_cast_fp16_0, tensor var_5343_cast_fp16_1 = split(axis = var_5343_axis_0, split_sizes = var_5343_split_sizes_0, x = h_23_cast_fp16)[name = string("op_5343_cast_fp16")]; + tensor gamma_167_perm_0 = const()[name = string("gamma_167_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_167_perm_0 = const()[name = string("beta_167_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5349_promoted_to_fp16 = const()[name = string("op_5349_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_167_cast_fp16 = transpose(perm = gamma_167_perm_0, x = var_5343_cast_fp16_0)[name = string("transpose_154")]; + tensor var_5350_cast_fp16 = add(x = gamma_167_cast_fp16, y = var_5349_promoted_to_fp16)[name = string("op_5350_cast_fp16")]; + tensor var_5351_cast_fp16 = mul(x = var_5350_cast_fp16, y = x_761_cast_fp16)[name = string("op_5351_cast_fp16")]; + tensor beta_167_cast_fp16 = transpose(perm = beta_167_perm_0, x = var_5343_cast_fp16_1)[name = string("transpose_153")]; + tensor x_769_cast_fp16 = add(x = var_5351_cast_fp16, y = beta_167_cast_fp16)[name = string("x_769_cast_fp16")]; + tensor linear_170_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_763_cast_fp16)[name = string("linear_170_cast_fp16")]; + tensor linear_171_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_769_cast_fp16)[name = string("linear_171_cast_fp16")]; + tensor var_5357_split_sizes_0 = const()[name = string("op_5357_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5357_axis_0 = const()[name = string("op_5357_axis_0"), val = int32(-1)]; + tensor var_5357_cast_fp16_0, tensor var_5357_cast_fp16_1 = split(axis = var_5357_axis_0, split_sizes = var_5357_split_sizes_0, x = linear_171_cast_fp16)[name = string("op_5357_cast_fp16")]; + tensor var_5365 = const()[name = string("op_5365"), val = tensor([1, 64, 8, 64])]; + tensor x_773_cast_fp16 = reshape(shape = var_5365, x = linear_170_cast_fp16)[name = string("x_773_cast_fp16")]; + tensor var_5375 = const()[name = string("op_5375"), val = tensor([1, 64, 8, 64])]; + tensor x_777_cast_fp16 = reshape(shape = var_5375, x = var_5357_cast_fp16_0)[name = string("x_777_cast_fp16")]; + tensor var_5385 = const()[name = string("op_5385"), val = tensor([1, 64, 8, 64])]; + tensor x_781_cast_fp16 = reshape(shape = var_5385, x = var_5357_cast_fp16_1)[name = string("x_781_cast_fp16")]; + tensor var_5387 = const()[name = string("op_5387"), val = tensor([0, 2, 1, 3])]; + bool sim_81_transpose_x_0 = const()[name = string("sim_81_transpose_x_0"), val = bool(false)]; + bool sim_81_transpose_y_0 = const()[name = string("sim_81_transpose_y_0"), val = bool(false)]; + tensor transpose_112_perm_0 = const()[name = string("transpose_112_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_113_perm_0 = const()[name = string("transpose_113_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_113 = transpose(perm = transpose_113_perm_0, x = x_777_cast_fp16)[name = string("transpose_150")]; + tensor transpose_112 = transpose(perm = transpose_112_perm_0, x = x_773_cast_fp16)[name = string("transpose_151")]; + tensor sim_81_cast_fp16 = matmul(transpose_x = sim_81_transpose_x_0, transpose_y = sim_81_transpose_y_0, x = transpose_112, y = transpose_113)[name = string("sim_81_cast_fp16")]; + fp16 var_5391_to_fp16 = const()[name = string("op_5391_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_83_cast_fp16 = mul(x = sim_81_cast_fp16, y = var_5391_to_fp16)[name = string("sim_83_cast_fp16")]; + tensor attn_41_cast_fp16 = softmax(axis = var_4694, x = sim_83_cast_fp16)[name = string("attn_41_cast_fp16")]; + bool x_783_transpose_x_0 = const()[name = string("x_783_transpose_x_0"), val = bool(false)]; + bool x_783_transpose_y_0 = const()[name = string("x_783_transpose_y_0"), val = bool(false)]; + tensor v_41_cast_fp16 = transpose(perm = var_5387, x = x_781_cast_fp16)[name = string("transpose_152")]; + tensor x_783_cast_fp16 = matmul(transpose_x = x_783_transpose_x_0, transpose_y = x_783_transpose_y_0, x = attn_41_cast_fp16, y = v_41_cast_fp16)[name = string("x_783_cast_fp16")]; + tensor var_5413 = const()[name = string("op_5413"), val = tensor([0, 2, 1, 3])]; + tensor var_5415 = const()[name = string("op_5415"), val = tensor([1, 64, 512])]; + tensor x_785_cast_fp16 = transpose(perm = var_5413, x = x_783_cast_fp16)[name = string("transpose_149")]; + tensor input_439_cast_fp16 = reshape(shape = var_5415, x = x_785_cast_fp16)[name = string("input_439_cast_fp16")]; + tensor linear_172_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_439_cast_fp16)[name = string("linear_172_cast_fp16")]; + tensor input_441_cast_fp16 = add(x = linear_172_cast_fp16, y = x_757_cast_fp16)[name = string("input_441_cast_fp16")]; + tensor linear_173_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_441_cast_fp16)[name = string("linear_173_cast_fp16")]; + string input_445_mode_0 = const()[name = string("input_445_mode_0"), val = string("EXACT")]; + tensor input_445_cast_fp16 = gelu(mode = input_445_mode_0, x = linear_173_cast_fp16)[name = string("input_445_cast_fp16")]; + tensor linear_174_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_445_cast_fp16)[name = string("linear_174_cast_fp16")]; + tensor x_787_cast_fp16 = add(x = linear_174_cast_fp16, y = input_441_cast_fp16)[name = string("x_787_cast_fp16")]; + tensor var_5424_axes_0 = const()[name = string("op_5424_axes_0"), val = tensor([1])]; + bool var_5424_keep_dims_0 = const()[name = string("op_5424_keep_dims_0"), val = bool(false)]; + tensor var_5424_cast_fp16 = reduce_mean(axes = var_5424_axes_0, keep_dims = var_5424_keep_dims_0, x = x_787_cast_fp16)[name = string("op_5424_cast_fp16")]; + tensor x_789_axes_0 = const()[name = string("x_789_axes_0"), val = tensor([1])]; + tensor x_789_cast_fp16 = expand_dims(axes = x_789_axes_0, x = var_5424_cast_fp16)[name = string("x_789_cast_fp16")]; + tensor var_5426 = const()[name = string("op_5426"), val = tensor([0, 2, 1])]; + string x_791_pad_type_0 = const()[name = string("x_791_pad_type_0"), val = string("valid")]; + tensor x_791_strides_0 = const()[name = string("x_791_strides_0"), val = tensor([1])]; + tensor x_791_pad_0 = const()[name = string("x_791_pad_0"), val = tensor([0, 0])]; + tensor x_791_dilations_0 = const()[name = string("x_791_dilations_0"), val = tensor([1])]; + int32 x_791_groups_0 = const()[name = string("x_791_groups_0"), val = int32(1)]; + tensor input_447_cast_fp16 = transpose(perm = var_5426, x = x_789_cast_fp16)[name = string("transpose_148")]; + tensor x_791_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_791_dilations_0, groups = x_791_groups_0, pad = x_791_pad_0, pad_type = x_791_pad_type_0, strides = x_791_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_447_cast_fp16)[name = string("x_791_cast_fp16")]; + tensor x_pred_13_perm_0 = const()[name = string("x_pred_13_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_13_to_fp16 = const()[name = string("c_skip_13_to_fp16"), val = tensor([[[0x1.fe8p-1]]])]; + tensor var_5434_cast_fp16 = mul(x = c_skip_13_to_fp16, y = x_noisy_13_cast_fp16)[name = string("op_5434_cast_fp16")]; + tensor c_out_13_to_fp16 = const()[name = string("c_out_13_to_fp16"), val = tensor([[[0x1.37cp-8]]])]; + tensor x_pred_13_cast_fp16 = transpose(perm = x_pred_13_perm_0, x = x_791_cast_fp16)[name = string("transpose_147")]; + tensor var_5435_cast_fp16 = mul(x = c_out_13_to_fp16, y = x_pred_13_cast_fp16)[name = string("op_5435_cast_fp16")]; + tensor x_dn_cast_fp16 = add(x = var_5434_cast_fp16, y = var_5435_cast_fp16)[name = string("x_dn_cast_fp16")]; + tensor var_5438_cast_fp16 = sub(x = x_noisy_13_cast_fp16, y = x_dn_cast_fp16)[name = string("op_5438_cast_fp16")]; + tensor _inversed_d_y_0_to_fp16 = const()[name = string("_inversed_d_y_0_to_fp16"), val = tensor([0x1.a44p+7])]; + tensor _inversed_d_cast_fp16 = mul(x = var_5438_cast_fp16, y = _inversed_d_y_0_to_fp16)[name = string("_inversed_d_cast_fp16")]; + fp16 var_5447_to_fp16 = const()[name = string("op_5447_to_fp16"), val = fp16(-0x1.37cp-9)]; + tensor var_5448_cast_fp16 = mul(x = _inversed_d_cast_fp16, y = var_5447_to_fp16)[name = string("op_5448_cast_fp16")]; + tensor x_noisy_cast_fp16 = add(x = x_noisy_13_cast_fp16, y = var_5448_cast_fp16)[name = string("x_noisy_cast_fp16")]; + int32 var_5460 = const()[name = string("op_5460"), val = int32(-1)]; + tensor c_in_to_fp16 = const()[name = string("c_in_to_fp16"), val = tensor([[[0x1.414p+2]]])]; + tensor x_801_cast_fp16 = mul(x = c_in_to_fp16, y = x_noisy_cast_fp16)[name = string("x_801_cast_fp16")]; + int32 x_797_axis_0 = const()[name = string("x_797_axis_0"), val = int32(0)]; + tensor var_5846_to_fp16 = const()[name = string("op_5846_to_fp16"), val = tensor(BLOBFILE(path = string("@model_path/weights/weight.bin"), offset = uint64(49359744)))]; + tensor x_797_cast_fp16 = stack(axis = x_797_axis_0, values = (var_5846_to_fp16, var_423_cast_fp16))[name = string("x_797_cast_fp16")]; + tensor var_5851 = const()[name = string("op_5851"), val = tensor([1, 2, 0])]; + tensor input_455_axes_0 = const()[name = string("input_455_axes_0"), val = tensor([2])]; + bool input_455_keep_dims_0 = const()[name = string("input_455_keep_dims_0"), val = bool(false)]; + tensor x_799_cast_fp16 = transpose(perm = var_5851, x = x_797_cast_fp16)[name = string("transpose_146")]; + tensor input_455_cast_fp16 = reduce_sum(axes = input_455_axes_0, keep_dims = input_455_keep_dims_0, x = x_799_cast_fp16)[name = string("input_455_cast_fp16")]; + tensor linear_177_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_0_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_0_weight_to_fp16, x = input_455_cast_fp16)[name = string("linear_177_cast_fp16")]; + string input_459_mode_0 = const()[name = string("input_459_mode_0"), val = string("EXACT")]; + tensor input_459_cast_fp16 = gelu(mode = input_459_mode_0, x = linear_177_cast_fp16)[name = string("input_459_cast_fp16")]; + tensor linear_178_cast_fp16 = linear(bias = unet_step_kdiffusion_net_to_mapping_2_bias_to_fp16, weight = unet_step_kdiffusion_net_to_mapping_2_weight_to_fp16, x = input_459_cast_fp16)[name = string("linear_178_cast_fp16")]; + string mapping_29_mode_0 = const()[name = string("mapping_29_mode_0"), val = string("EXACT")]; + tensor mapping_29_cast_fp16 = gelu(mode = mapping_29_mode_0, x = linear_178_cast_fp16)[name = string("mapping_29_cast_fp16")]; + tensor var_5861_reps_0 = const()[name = string("op_5861_reps_0"), val = tensor([1, 64, 1])]; + tensor var_5861_cast_fp16 = tile(reps = var_5861_reps_0, x = x_801_cast_fp16)[name = string("op_5861_cast_fp16")]; + bool x_803_interleave_0 = const()[name = string("x_803_interleave_0"), val = bool(false)]; + tensor x_803_cast_fp16 = concat(axis = var_5460, interleave = x_803_interleave_0, values = (var_5861_cast_fp16, embedding_to_fp16))[name = string("x_803_cast_fp16")]; + tensor var_5864_axes_0 = const()[name = string("op_5864_axes_0"), val = tensor([1])]; + tensor var_5864_cast_fp16 = expand_dims(axes = var_5864_axes_0, x = mapping_29_cast_fp16)[name = string("op_5864_cast_fp16")]; + tensor mapping_reps_0 = const()[name = string("mapping_reps_0"), val = tensor([1, 64, 1])]; + tensor mapping_cast_fp16 = tile(reps = mapping_reps_0, x = var_5864_cast_fp16)[name = string("mapping_cast_fp16")]; + tensor x_805_cast_fp16 = add(x = x_803_cast_fp16, y = mapping_cast_fp16)[name = string("x_805_cast_fp16")]; + tensor var_5876_split_sizes_0 = const()[name = string("op_5876_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5876_axis_0 = const()[name = string("op_5876_axis_0"), val = int32(1)]; + tensor var_5876_cast_fp16_0, tensor var_5876_cast_fp16_1 = split(axis = var_5876_axis_0, split_sizes = var_5876_split_sizes_0, x = h_3_cast_fp16)[name = string("op_5876_cast_fp16")]; + tensor gamma_171_perm_0 = const()[name = string("gamma_171_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_171_perm_0 = const()[name = string("beta_171_perm_0"), val = tensor([0, -1, 1])]; + tensor x_809_axes_0 = const()[name = string("x_809_axes_0"), val = tensor([-1])]; + fp16 var_5456_to_fp16 = const()[name = string("op_5456_to_fp16"), val = fp16(0x1.5p-17)]; + tensor x_809_cast_fp16 = layer_norm(axes = x_809_axes_0, epsilon = var_5456_to_fp16, x = x_805_cast_fp16)[name = string("x_809_cast_fp16")]; + fp16 var_5882_promoted_to_fp16 = const()[name = string("op_5882_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_171_cast_fp16 = transpose(perm = gamma_171_perm_0, x = var_5876_cast_fp16_0)[name = string("transpose_145")]; + tensor var_5883_cast_fp16 = add(x = gamma_171_cast_fp16, y = var_5882_promoted_to_fp16)[name = string("op_5883_cast_fp16")]; + tensor var_5884_cast_fp16 = mul(x = var_5883_cast_fp16, y = x_809_cast_fp16)[name = string("op_5884_cast_fp16")]; + tensor beta_171_cast_fp16 = transpose(perm = beta_171_perm_0, x = var_5876_cast_fp16_1)[name = string("transpose_144")]; + tensor x_811_cast_fp16 = add(x = var_5884_cast_fp16, y = beta_171_cast_fp16)[name = string("x_811_cast_fp16")]; + tensor var_5895_split_sizes_0 = const()[name = string("op_5895_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5895_axis_0 = const()[name = string("op_5895_axis_0"), val = int32(1)]; + tensor var_5895_cast_fp16_0, tensor var_5895_cast_fp16_1 = split(axis = var_5895_axis_0, split_sizes = var_5895_split_sizes_0, x = h_7_cast_fp16)[name = string("op_5895_cast_fp16")]; + tensor gamma_175_perm_0 = const()[name = string("gamma_175_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_175_perm_0 = const()[name = string("beta_175_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_5901_promoted_to_fp16 = const()[name = string("op_5901_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_175_cast_fp16 = transpose(perm = gamma_175_perm_0, x = var_5895_cast_fp16_0)[name = string("transpose_143")]; + tensor var_5902_cast_fp16 = add(x = gamma_175_cast_fp16, y = var_5901_promoted_to_fp16)[name = string("op_5902_cast_fp16")]; + tensor var_5903_cast_fp16 = mul(x = var_5902_cast_fp16, y = x_809_cast_fp16)[name = string("op_5903_cast_fp16")]; + tensor beta_175_cast_fp16 = transpose(perm = beta_175_perm_0, x = var_5895_cast_fp16_1)[name = string("transpose_142")]; + tensor x_817_cast_fp16 = add(x = var_5903_cast_fp16, y = beta_175_cast_fp16)[name = string("x_817_cast_fp16")]; + tensor linear_181_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_q_weight_to_fp16, x = x_811_cast_fp16)[name = string("linear_181_cast_fp16")]; + tensor linear_182_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_to_kv_weight_to_fp16, x = x_817_cast_fp16)[name = string("linear_182_cast_fp16")]; + tensor var_5909_split_sizes_0 = const()[name = string("op_5909_split_sizes_0"), val = tensor([512, 512])]; + int32 var_5909_axis_0 = const()[name = string("op_5909_axis_0"), val = int32(-1)]; + tensor var_5909_cast_fp16_0, tensor var_5909_cast_fp16_1 = split(axis = var_5909_axis_0, split_sizes = var_5909_split_sizes_0, x = linear_182_cast_fp16)[name = string("op_5909_cast_fp16")]; + tensor var_5917 = const()[name = string("op_5917"), val = tensor([1, 64, 8, 64])]; + tensor x_821_cast_fp16 = reshape(shape = var_5917, x = linear_181_cast_fp16)[name = string("x_821_cast_fp16")]; + tensor var_5927 = const()[name = string("op_5927"), val = tensor([1, 64, 8, 64])]; + tensor x_825_cast_fp16 = reshape(shape = var_5927, x = var_5909_cast_fp16_0)[name = string("x_825_cast_fp16")]; + tensor var_5937 = const()[name = string("op_5937"), val = tensor([1, 64, 8, 64])]; + tensor x_829_cast_fp16 = reshape(shape = var_5937, x = var_5909_cast_fp16_1)[name = string("x_829_cast_fp16")]; + tensor var_5939 = const()[name = string("op_5939"), val = tensor([0, 2, 1, 3])]; + bool sim_85_transpose_x_0 = const()[name = string("sim_85_transpose_x_0"), val = bool(false)]; + bool sim_85_transpose_y_0 = const()[name = string("sim_85_transpose_y_0"), val = bool(false)]; + tensor transpose_114_perm_0 = const()[name = string("transpose_114_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_115_perm_0 = const()[name = string("transpose_115_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_115 = transpose(perm = transpose_115_perm_0, x = x_825_cast_fp16)[name = string("transpose_139")]; + tensor transpose_114 = transpose(perm = transpose_114_perm_0, x = x_821_cast_fp16)[name = string("transpose_140")]; + tensor sim_85_cast_fp16 = matmul(transpose_x = sim_85_transpose_x_0, transpose_y = sim_85_transpose_y_0, x = transpose_114, y = transpose_115)[name = string("sim_85_cast_fp16")]; + fp16 var_5943_to_fp16 = const()[name = string("op_5943_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_87_cast_fp16 = mul(x = sim_85_cast_fp16, y = var_5943_to_fp16)[name = string("sim_87_cast_fp16")]; + tensor attn_43_cast_fp16 = softmax(axis = var_5460, x = sim_87_cast_fp16)[name = string("attn_43_cast_fp16")]; + bool x_831_transpose_x_0 = const()[name = string("x_831_transpose_x_0"), val = bool(false)]; + bool x_831_transpose_y_0 = const()[name = string("x_831_transpose_y_0"), val = bool(false)]; + tensor v_43_cast_fp16 = transpose(perm = var_5939, x = x_829_cast_fp16)[name = string("transpose_141")]; + tensor x_831_cast_fp16 = matmul(transpose_x = x_831_transpose_x_0, transpose_y = x_831_transpose_y_0, x = attn_43_cast_fp16, y = v_43_cast_fp16)[name = string("x_831_cast_fp16")]; + tensor var_5965 = const()[name = string("op_5965"), val = tensor([0, 2, 1, 3])]; + tensor var_5967 = const()[name = string("op_5967"), val = tensor([1, 64, 512])]; + tensor x_833_cast_fp16 = transpose(perm = var_5965, x = x_831_cast_fp16)[name = string("transpose_138")]; + tensor input_471_cast_fp16 = reshape(shape = var_5967, x = x_833_cast_fp16)[name = string("input_471_cast_fp16")]; + tensor linear_183_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_attention_attention_to_out_weight_to_fp16, x = input_471_cast_fp16)[name = string("linear_183_cast_fp16")]; + tensor input_473_cast_fp16 = add(x = linear_183_cast_fp16, y = x_805_cast_fp16)[name = string("input_473_cast_fp16")]; + tensor linear_184_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_0_weight_to_fp16, x = input_473_cast_fp16)[name = string("linear_184_cast_fp16")]; + string input_477_mode_0 = const()[name = string("input_477_mode_0"), val = string("EXACT")]; + tensor input_477_cast_fp16 = gelu(mode = input_477_mode_0, x = linear_184_cast_fp16)[name = string("input_477_cast_fp16")]; + tensor linear_185_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_0_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_0_feed_forward_2_weight_to_fp16, x = input_477_cast_fp16)[name = string("linear_185_cast_fp16")]; + tensor x_835_cast_fp16 = add(x = linear_185_cast_fp16, y = input_473_cast_fp16)[name = string("x_835_cast_fp16")]; + tensor x_837_cast_fp16 = add(x = x_835_cast_fp16, y = mapping_cast_fp16)[name = string("x_837_cast_fp16")]; + tensor var_5983_split_sizes_0 = const()[name = string("op_5983_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_5983_axis_0 = const()[name = string("op_5983_axis_0"), val = int32(1)]; + tensor var_5983_cast_fp16_0, tensor var_5983_cast_fp16_1 = split(axis = var_5983_axis_0, split_sizes = var_5983_split_sizes_0, x = h_11_cast_fp16)[name = string("op_5983_cast_fp16")]; + tensor gamma_179_perm_0 = const()[name = string("gamma_179_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_179_perm_0 = const()[name = string("beta_179_perm_0"), val = tensor([0, -1, 1])]; + tensor x_841_axes_0 = const()[name = string("x_841_axes_0"), val = tensor([-1])]; + tensor x_841_cast_fp16 = layer_norm(axes = x_841_axes_0, epsilon = var_5456_to_fp16, x = x_837_cast_fp16)[name = string("x_841_cast_fp16")]; + fp16 var_5989_promoted_to_fp16 = const()[name = string("op_5989_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_179_cast_fp16 = transpose(perm = gamma_179_perm_0, x = var_5983_cast_fp16_0)[name = string("transpose_137")]; + tensor var_5990_cast_fp16 = add(x = gamma_179_cast_fp16, y = var_5989_promoted_to_fp16)[name = string("op_5990_cast_fp16")]; + tensor var_5991_cast_fp16 = mul(x = var_5990_cast_fp16, y = x_841_cast_fp16)[name = string("op_5991_cast_fp16")]; + tensor beta_179_cast_fp16 = transpose(perm = beta_179_perm_0, x = var_5983_cast_fp16_1)[name = string("transpose_136")]; + tensor x_843_cast_fp16 = add(x = var_5991_cast_fp16, y = beta_179_cast_fp16)[name = string("x_843_cast_fp16")]; + tensor var_6002_split_sizes_0 = const()[name = string("op_6002_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_6002_axis_0 = const()[name = string("op_6002_axis_0"), val = int32(1)]; + tensor var_6002_cast_fp16_0, tensor var_6002_cast_fp16_1 = split(axis = var_6002_axis_0, split_sizes = var_6002_split_sizes_0, x = h_15_cast_fp16)[name = string("op_6002_cast_fp16")]; + tensor gamma_183_perm_0 = const()[name = string("gamma_183_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_183_perm_0 = const()[name = string("beta_183_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_6008_promoted_to_fp16 = const()[name = string("op_6008_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_183_cast_fp16 = transpose(perm = gamma_183_perm_0, x = var_6002_cast_fp16_0)[name = string("transpose_135")]; + tensor var_6009_cast_fp16 = add(x = gamma_183_cast_fp16, y = var_6008_promoted_to_fp16)[name = string("op_6009_cast_fp16")]; + tensor var_6010_cast_fp16 = mul(x = var_6009_cast_fp16, y = x_841_cast_fp16)[name = string("op_6010_cast_fp16")]; + tensor beta_183_cast_fp16 = transpose(perm = beta_183_perm_0, x = var_6002_cast_fp16_1)[name = string("transpose_134")]; + tensor x_849_cast_fp16 = add(x = var_6010_cast_fp16, y = beta_183_cast_fp16)[name = string("x_849_cast_fp16")]; + tensor linear_188_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_q_weight_to_fp16, x = x_843_cast_fp16)[name = string("linear_188_cast_fp16")]; + tensor linear_189_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_to_kv_weight_to_fp16, x = x_849_cast_fp16)[name = string("linear_189_cast_fp16")]; + tensor var_6016_split_sizes_0 = const()[name = string("op_6016_split_sizes_0"), val = tensor([512, 512])]; + int32 var_6016_axis_0 = const()[name = string("op_6016_axis_0"), val = int32(-1)]; + tensor var_6016_cast_fp16_0, tensor var_6016_cast_fp16_1 = split(axis = var_6016_axis_0, split_sizes = var_6016_split_sizes_0, x = linear_189_cast_fp16)[name = string("op_6016_cast_fp16")]; + tensor var_6024 = const()[name = string("op_6024"), val = tensor([1, 64, 8, 64])]; + tensor x_853_cast_fp16 = reshape(shape = var_6024, x = linear_188_cast_fp16)[name = string("x_853_cast_fp16")]; + tensor var_6034 = const()[name = string("op_6034"), val = tensor([1, 64, 8, 64])]; + tensor x_857_cast_fp16 = reshape(shape = var_6034, x = var_6016_cast_fp16_0)[name = string("x_857_cast_fp16")]; + tensor var_6044 = const()[name = string("op_6044"), val = tensor([1, 64, 8, 64])]; + tensor x_861_cast_fp16 = reshape(shape = var_6044, x = var_6016_cast_fp16_1)[name = string("x_861_cast_fp16")]; + tensor var_6046 = const()[name = string("op_6046"), val = tensor([0, 2, 1, 3])]; + bool sim_89_transpose_x_0 = const()[name = string("sim_89_transpose_x_0"), val = bool(false)]; + bool sim_89_transpose_y_0 = const()[name = string("sim_89_transpose_y_0"), val = bool(false)]; + tensor transpose_116_perm_0 = const()[name = string("transpose_116_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_117_perm_0 = const()[name = string("transpose_117_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_117 = transpose(perm = transpose_117_perm_0, x = x_857_cast_fp16)[name = string("transpose_131")]; + tensor transpose_116 = transpose(perm = transpose_116_perm_0, x = x_853_cast_fp16)[name = string("transpose_132")]; + tensor sim_89_cast_fp16 = matmul(transpose_x = sim_89_transpose_x_0, transpose_y = sim_89_transpose_y_0, x = transpose_116, y = transpose_117)[name = string("sim_89_cast_fp16")]; + fp16 var_6050_to_fp16 = const()[name = string("op_6050_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_91_cast_fp16 = mul(x = sim_89_cast_fp16, y = var_6050_to_fp16)[name = string("sim_91_cast_fp16")]; + tensor attn_45_cast_fp16 = softmax(axis = var_5460, x = sim_91_cast_fp16)[name = string("attn_45_cast_fp16")]; + bool x_863_transpose_x_0 = const()[name = string("x_863_transpose_x_0"), val = bool(false)]; + bool x_863_transpose_y_0 = const()[name = string("x_863_transpose_y_0"), val = bool(false)]; + tensor v_45_cast_fp16 = transpose(perm = var_6046, x = x_861_cast_fp16)[name = string("transpose_133")]; + tensor x_863_cast_fp16 = matmul(transpose_x = x_863_transpose_x_0, transpose_y = x_863_transpose_y_0, x = attn_45_cast_fp16, y = v_45_cast_fp16)[name = string("x_863_cast_fp16")]; + tensor var_6072 = const()[name = string("op_6072"), val = tensor([0, 2, 1, 3])]; + tensor var_6074 = const()[name = string("op_6074"), val = tensor([1, 64, 512])]; + tensor x_865_cast_fp16 = transpose(perm = var_6072, x = x_863_cast_fp16)[name = string("transpose_130")]; + tensor input_487_cast_fp16 = reshape(shape = var_6074, x = x_865_cast_fp16)[name = string("input_487_cast_fp16")]; + tensor linear_190_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_attention_attention_to_out_weight_to_fp16, x = input_487_cast_fp16)[name = string("linear_190_cast_fp16")]; + tensor input_489_cast_fp16 = add(x = linear_190_cast_fp16, y = x_837_cast_fp16)[name = string("input_489_cast_fp16")]; + tensor linear_191_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_0_weight_to_fp16, x = input_489_cast_fp16)[name = string("linear_191_cast_fp16")]; + string input_493_mode_0 = const()[name = string("input_493_mode_0"), val = string("EXACT")]; + tensor input_493_cast_fp16 = gelu(mode = input_493_mode_0, x = linear_191_cast_fp16)[name = string("input_493_cast_fp16")]; + tensor linear_192_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_1_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_1_feed_forward_2_weight_to_fp16, x = input_493_cast_fp16)[name = string("linear_192_cast_fp16")]; + tensor x_867_cast_fp16 = add(x = linear_192_cast_fp16, y = input_489_cast_fp16)[name = string("x_867_cast_fp16")]; + tensor x_869_cast_fp16 = add(x = x_867_cast_fp16, y = mapping_cast_fp16)[name = string("x_869_cast_fp16")]; + tensor var_6090_split_sizes_0 = const()[name = string("op_6090_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_6090_axis_0 = const()[name = string("op_6090_axis_0"), val = int32(1)]; + tensor var_6090_cast_fp16_0, tensor var_6090_cast_fp16_1 = split(axis = var_6090_axis_0, split_sizes = var_6090_split_sizes_0, x = h_19_cast_fp16)[name = string("op_6090_cast_fp16")]; + tensor gamma_187_perm_0 = const()[name = string("gamma_187_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_187_perm_0 = const()[name = string("beta_187_perm_0"), val = tensor([0, -1, 1])]; + tensor x_873_axes_0 = const()[name = string("x_873_axes_0"), val = tensor([-1])]; + tensor x_873_cast_fp16 = layer_norm(axes = x_873_axes_0, epsilon = var_5456_to_fp16, x = x_869_cast_fp16)[name = string("x_873_cast_fp16")]; + fp16 var_6096_promoted_to_fp16 = const()[name = string("op_6096_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_187_cast_fp16 = transpose(perm = gamma_187_perm_0, x = var_6090_cast_fp16_0)[name = string("transpose_129")]; + tensor var_6097_cast_fp16 = add(x = gamma_187_cast_fp16, y = var_6096_promoted_to_fp16)[name = string("op_6097_cast_fp16")]; + tensor var_6098_cast_fp16 = mul(x = var_6097_cast_fp16, y = x_873_cast_fp16)[name = string("op_6098_cast_fp16")]; + tensor beta_187_cast_fp16 = transpose(perm = beta_187_perm_0, x = var_6090_cast_fp16_1)[name = string("transpose_128")]; + tensor x_875_cast_fp16 = add(x = var_6098_cast_fp16, y = beta_187_cast_fp16)[name = string("x_875_cast_fp16")]; + tensor var_6109_split_sizes_0 = const()[name = string("op_6109_split_sizes_0"), val = tensor([1024, 1024])]; + int32 var_6109_axis_0 = const()[name = string("op_6109_axis_0"), val = int32(1)]; + tensor var_6109_cast_fp16_0, tensor var_6109_cast_fp16_1 = split(axis = var_6109_axis_0, split_sizes = var_6109_split_sizes_0, x = h_23_cast_fp16)[name = string("op_6109_cast_fp16")]; + tensor gamma_perm_0 = const()[name = string("gamma_perm_0"), val = tensor([0, -1, 1])]; + tensor beta_perm_0 = const()[name = string("beta_perm_0"), val = tensor([0, -1, 1])]; + fp16 var_6115_promoted_to_fp16 = const()[name = string("op_6115_promoted_to_fp16"), val = fp16(0x1p+0)]; + tensor gamma_cast_fp16 = transpose(perm = gamma_perm_0, x = var_6109_cast_fp16_0)[name = string("transpose_127")]; + tensor var_6116_cast_fp16 = add(x = gamma_cast_fp16, y = var_6115_promoted_to_fp16)[name = string("op_6116_cast_fp16")]; + tensor var_6117_cast_fp16 = mul(x = var_6116_cast_fp16, y = x_873_cast_fp16)[name = string("op_6117_cast_fp16")]; + tensor beta_cast_fp16 = transpose(perm = beta_perm_0, x = var_6109_cast_fp16_1)[name = string("transpose_126")]; + tensor x_881_cast_fp16 = add(x = var_6117_cast_fp16, y = beta_cast_fp16)[name = string("x_881_cast_fp16")]; + tensor linear_195_cast_fp16 = linear(bias = linear_6_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_q_weight_to_fp16, x = x_875_cast_fp16)[name = string("linear_195_cast_fp16")]; + tensor linear_196_cast_fp16 = linear(bias = linear_7_bias_0_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_to_kv_weight_to_fp16, x = x_881_cast_fp16)[name = string("linear_196_cast_fp16")]; + tensor var_6123_split_sizes_0 = const()[name = string("op_6123_split_sizes_0"), val = tensor([512, 512])]; + int32 var_6123_axis_0 = const()[name = string("op_6123_axis_0"), val = int32(-1)]; + tensor var_6123_cast_fp16_0, tensor var_6123_cast_fp16_1 = split(axis = var_6123_axis_0, split_sizes = var_6123_split_sizes_0, x = linear_196_cast_fp16)[name = string("op_6123_cast_fp16")]; + tensor var_6131 = const()[name = string("op_6131"), val = tensor([1, 64, 8, 64])]; + tensor x_885_cast_fp16 = reshape(shape = var_6131, x = linear_195_cast_fp16)[name = string("x_885_cast_fp16")]; + tensor var_6141 = const()[name = string("op_6141"), val = tensor([1, 64, 8, 64])]; + tensor x_889_cast_fp16 = reshape(shape = var_6141, x = var_6123_cast_fp16_0)[name = string("x_889_cast_fp16")]; + tensor var_6151 = const()[name = string("op_6151"), val = tensor([1, 64, 8, 64])]; + tensor x_893_cast_fp16 = reshape(shape = var_6151, x = var_6123_cast_fp16_1)[name = string("x_893_cast_fp16")]; + tensor var_6153 = const()[name = string("op_6153"), val = tensor([0, 2, 1, 3])]; + bool sim_93_transpose_x_0 = const()[name = string("sim_93_transpose_x_0"), val = bool(false)]; + bool sim_93_transpose_y_0 = const()[name = string("sim_93_transpose_y_0"), val = bool(false)]; + tensor transpose_118_perm_0 = const()[name = string("transpose_118_perm_0"), val = tensor([0, 2, -3, -1])]; + tensor transpose_119_perm_0 = const()[name = string("transpose_119_perm_0"), val = tensor([0, 2, -1, -3])]; + tensor transpose_119 = transpose(perm = transpose_119_perm_0, x = x_889_cast_fp16)[name = string("transpose_123")]; + tensor transpose_118 = transpose(perm = transpose_118_perm_0, x = x_885_cast_fp16)[name = string("transpose_124")]; + tensor sim_93_cast_fp16 = matmul(transpose_x = sim_93_transpose_x_0, transpose_y = sim_93_transpose_y_0, x = transpose_118, y = transpose_119)[name = string("sim_93_cast_fp16")]; + fp16 var_6157_to_fp16 = const()[name = string("op_6157_to_fp16"), val = fp16(0x1p-3)]; + tensor sim_cast_fp16 = mul(x = sim_93_cast_fp16, y = var_6157_to_fp16)[name = string("sim_cast_fp16")]; + tensor attn_cast_fp16 = softmax(axis = var_5460, x = sim_cast_fp16)[name = string("attn_cast_fp16")]; + bool x_895_transpose_x_0 = const()[name = string("x_895_transpose_x_0"), val = bool(false)]; + bool x_895_transpose_y_0 = const()[name = string("x_895_transpose_y_0"), val = bool(false)]; + tensor v_cast_fp16 = transpose(perm = var_6153, x = x_893_cast_fp16)[name = string("transpose_125")]; + tensor x_895_cast_fp16 = matmul(transpose_x = x_895_transpose_x_0, transpose_y = x_895_transpose_y_0, x = attn_cast_fp16, y = v_cast_fp16)[name = string("x_895_cast_fp16")]; + tensor var_6179 = const()[name = string("op_6179"), val = tensor([0, 2, 1, 3])]; + tensor var_6181 = const()[name = string("op_6181"), val = tensor([1, 64, 512])]; + tensor x_897_cast_fp16 = transpose(perm = var_6179, x = x_895_cast_fp16)[name = string("transpose_122")]; + tensor input_503_cast_fp16 = reshape(shape = var_6181, x = x_897_cast_fp16)[name = string("input_503_cast_fp16")]; + tensor linear_197_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_attention_attention_to_out_weight_to_fp16, x = input_503_cast_fp16)[name = string("linear_197_cast_fp16")]; + tensor input_505_cast_fp16 = add(x = linear_197_cast_fp16, y = x_869_cast_fp16)[name = string("input_505_cast_fp16")]; + tensor linear_198_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_0_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_0_weight_to_fp16, x = input_505_cast_fp16)[name = string("linear_198_cast_fp16")]; + string input_509_mode_0 = const()[name = string("input_509_mode_0"), val = string("EXACT")]; + tensor input_509_cast_fp16 = gelu(mode = input_509_mode_0, x = linear_198_cast_fp16)[name = string("input_509_cast_fp16")]; + tensor linear_199_cast_fp16 = linear(bias = unet_step_kdiffusion_net_blocks_2_feed_forward_2_bias_to_fp16, weight = unet_step_kdiffusion_net_blocks_2_feed_forward_2_weight_to_fp16, x = input_509_cast_fp16)[name = string("linear_199_cast_fp16")]; + tensor x_899_cast_fp16 = add(x = linear_199_cast_fp16, y = input_505_cast_fp16)[name = string("x_899_cast_fp16")]; + tensor var_6190_axes_0 = const()[name = string("op_6190_axes_0"), val = tensor([1])]; + bool var_6190_keep_dims_0 = const()[name = string("op_6190_keep_dims_0"), val = bool(false)]; + tensor var_6190_cast_fp16 = reduce_mean(axes = var_6190_axes_0, keep_dims = var_6190_keep_dims_0, x = x_899_cast_fp16)[name = string("op_6190_cast_fp16")]; + tensor x_901_axes_0 = const()[name = string("x_901_axes_0"), val = tensor([1])]; + tensor x_901_cast_fp16 = expand_dims(axes = x_901_axes_0, x = var_6190_cast_fp16)[name = string("x_901_cast_fp16")]; + tensor var_6192 = const()[name = string("op_6192"), val = tensor([0, 2, 1])]; + string x_903_pad_type_0 = const()[name = string("x_903_pad_type_0"), val = string("valid")]; + tensor x_903_strides_0 = const()[name = string("x_903_strides_0"), val = tensor([1])]; + tensor x_903_pad_0 = const()[name = string("x_903_pad_0"), val = tensor([0, 0])]; + tensor x_903_dilations_0 = const()[name = string("x_903_dilations_0"), val = tensor([1])]; + int32 x_903_groups_0 = const()[name = string("x_903_groups_0"), val = int32(1)]; + tensor input_cast_fp16 = transpose(perm = var_6192, x = x_901_cast_fp16)[name = string("transpose_121")]; + tensor x_903_cast_fp16 = conv(bias = unet_step_kdiffusion_net_to_out_1_bias_to_fp16, dilations = x_903_dilations_0, groups = x_903_groups_0, pad = x_903_pad_0, pad_type = x_903_pad_type_0, strides = x_903_strides_0, weight = unet_step_kdiffusion_net_to_out_1_weight_to_fp16, x = input_cast_fp16)[name = string("x_903_cast_fp16")]; + tensor x_pred_perm_0 = const()[name = string("x_pred_perm_0"), val = tensor([0, -1, -2])]; + tensor c_skip_to_fp16 = const()[name = string("c_skip_to_fp16"), val = tensor([[[0x1.fecp-1]]])]; + tensor var_6200_cast_fp16 = mul(x = c_skip_to_fp16, y = x_noisy_cast_fp16)[name = string("op_6200_cast_fp16")]; + tensor c_out_to_fp16 = const()[name = string("c_out_to_fp16"), val = tensor([[[0x1.38p-9]]])]; + tensor x_pred_cast_fp16 = transpose(perm = x_pred_perm_0, x = x_903_cast_fp16)[name = string("transpose_120")]; + tensor var_6201_cast_fp16 = mul(x = c_out_to_fp16, y = x_pred_cast_fp16)[name = string("op_6201_cast_fp16")]; + tensor x_mid_dn_cast_fp16 = add(x = var_6200_cast_fp16, y = var_6201_cast_fp16)[name = string("x_mid_dn_cast_fp16")]; + tensor var_6204_cast_fp16 = sub(x = x_noisy_cast_fp16, y = x_mid_dn_cast_fp16)[name = string("op_6204_cast_fp16")]; + tensor _inversed_d_mid_y_0_to_fp16 = const()[name = string("_inversed_d_mid_y_0_to_fp16"), val = tensor([0x1.a44p+8])]; + tensor _inversed_d_mid_cast_fp16 = mul(x = var_6204_cast_fp16, y = _inversed_d_mid_y_0_to_fp16)[name = string("_inversed_d_mid_cast_fp16")]; + fp16 var_6213_to_fp16 = const()[name = string("op_6213_to_fp16"), val = fp16(-0x1.37cp-8)]; + tensor var_6214_cast_fp16 = mul(x = _inversed_d_mid_cast_fp16, y = var_6213_to_fp16)[name = string("op_6214_cast_fp16")]; + tensor x_cast_fp16 = add(x = x_noisy_13_cast_fp16, y = var_6214_cast_fp16)[name = string("x_cast_fp16")]; + tensor var_6219_begin_0 = const()[name = string("op_6219_begin_0"), val = tensor([3, 0, 0, 0])]; + tensor var_6219_end_0 = const()[name = string("op_6219_end_0"), val = tensor([4, 1, 1, 256])]; + tensor var_6219_end_mask_0 = const()[name = string("op_6219_end_mask_0"), val = tensor([false, true, true, true])]; + tensor var_6219_squeeze_mask_0 = const()[name = string("op_6219_squeeze_mask_0"), val = tensor([true, false, false, false])]; + tensor var_6219_cast_fp16 = slice_by_index(begin = var_6219_begin_0, end = var_6219_end_0, end_mask = var_6219_end_mask_0, squeeze_mask = var_6219_squeeze_mask_0, x = noises_aux_to_fp16)[name = string("op_6219_cast_fp16")]; + fp16 var_6222_to_fp16 = const()[name = string("op_6222_to_fp16"), val = fp16(0x1.a34p-14)]; + tensor var_6223_cast_fp16 = mul(x = var_6219_cast_fp16, y = var_6222_to_fp16)[name = string("op_6223_cast_fp16")]; + tensor var_6225 = add(x = x_cast_fp16, y = var_6223_cast_fp16)[name = string("op_6225_cast_fp16")]; + } -> (var_6225); +} \ No newline at end of file diff --git a/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/weights/weight.bin b/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..5d524e318268dea3c586fd0de5ce641710361300 --- /dev/null +++ b/iteration_3/compiled/fused_diffusion_sampler_fp16_t64.mlmodelc/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f823b5c638d2eb2fd91bf8e4efe4a90b2e1d3d9e2f5ab40e7e93cb03cd212aca +size 49361856 diff --git a/iteration_3/packages/.DS_Store b/iteration_3/packages/.DS_Store index 9a9c3c5175eefc17adffba9c51a3e971025bd103..e4c4b1e057870a38d3ae62e267c4aee64c5ed004 100644 Binary files a/iteration_3/packages/.DS_Store and b/iteration_3/packages/.DS_Store differ diff --git a/iteration_3/packages/bert_fp16_t128.mlpackage/Data/com.apple.CoreML/model.mlmodel b/iteration_3/packages/bert_fp16_t128.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..ce4695f34a2759aabc0143fc4af5a949c5b2a871 --- /dev/null +++ b/iteration_3/packages/bert_fp16_t128.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:86a52d9bd4b18a4a30aeb17118ca75cd57c729e45db4f4555f9b9745d33fa48a +size 85782 diff --git a/iteration_3/packages/bert_fp16_t128.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/iteration_3/packages/bert_fp16_t128.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..df96cc13a5400d026fdae28981166d7389f06921 --- /dev/null +++ b/iteration_3/packages/bert_fp16_t128.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ff3ca8fac0332427ddfe5e78954382359d26516284113001a7484b60455eb10 +size 12126848 diff --git a/iteration_3/packages/bert_fp16_t128.mlpackage/Manifest.json b/iteration_3/packages/bert_fp16_t128.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..db634719e141dbc985ffce03bca1628c0fcaa803 --- /dev/null +++ b/iteration_3/packages/bert_fp16_t128.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "8196AB77-8508-4F1F-8211-7AC9A128F5E9": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + }, + "CBF6095A-75FA-4003-8CB6-EB68D8C98F8F": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + } + }, + "rootModelIdentifier": "CBF6095A-75FA-4003-8CB6-EB68D8C98F8F" +} diff --git a/iteration_3/packages/bert_fp16_t256.mlpackage/Data/com.apple.CoreML/model.mlmodel b/iteration_3/packages/bert_fp16_t256.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..f2ed89092495a82287d6db362c90e97f0828e53e --- /dev/null +++ b/iteration_3/packages/bert_fp16_t256.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:96c9d6993382797abc86b9e8fae92f3af3a1cf0dca68c77eedbe5842f7f56706 +size 85782 diff --git a/iteration_3/packages/bert_fp16_t256.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/iteration_3/packages/bert_fp16_t256.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..ee82062c83de4d5127acc3df81c91bfb2c075860 --- /dev/null +++ b/iteration_3/packages/bert_fp16_t256.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7569b53a68b9664e246fda171851daa1dd5f01f64aa31533dfcaf40f4034fee3 +size 12192384 diff --git a/iteration_3/packages/bert_fp16_t256.mlpackage/Manifest.json b/iteration_3/packages/bert_fp16_t256.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..dc52f55de14da2337cdbbb15130e05391bba3b25 --- /dev/null +++ b/iteration_3/packages/bert_fp16_t256.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "0DB32ECF-03B7-442F-883B-BAD01666A6EB": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + }, + "D3449315-4391-4FD3-A386-218E0755ACD4": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + } + }, + "rootModelIdentifier": "0DB32ECF-03B7-442F-883B-BAD01666A6EB" +} diff --git a/iteration_3/packages/bert_fp16_t64.mlpackage/Data/com.apple.CoreML/model.mlmodel b/iteration_3/packages/bert_fp16_t64.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..21e19b3056b8344e1b50a44e0c57afa5f2a21cb7 --- /dev/null +++ b/iteration_3/packages/bert_fp16_t64.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3ce83515fe05aeb2fab52063ca9a04aa700254ca393d0202e259ab3f60c0b99c +size 85458 diff --git a/iteration_3/packages/bert_fp16_t64.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/iteration_3/packages/bert_fp16_t64.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..af96b10f5f63bf47c6b0a8eeabea8ce8d0913aed --- /dev/null +++ b/iteration_3/packages/bert_fp16_t64.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:629b75e73fbceeb89e92b0b85548bde59918208b424b8d6467202d72d82629b2 +size 12094080 diff --git a/iteration_3/packages/bert_fp16_t64.mlpackage/Manifest.json b/iteration_3/packages/bert_fp16_t64.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..4de9804cf6c2fb902bfba3d0c852beb17cef399d --- /dev/null +++ b/iteration_3/packages/bert_fp16_t64.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "7919D85D-6DA9-40DE-8836-373256963D46": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + }, + "C58FFFDE-2C63-420D-A908-5E1FD94373FB": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + } + }, + "rootModelIdentifier": "C58FFFDE-2C63-420D-A908-5E1FD94373FB" +} diff --git a/iteration_3/packages/fused_diffusion_sampler_fp16_t128.mlpackage/Data/com.apple.CoreML/model.mlmodel b/iteration_3/packages/fused_diffusion_sampler_fp16_t128.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..aa89c8bd3dabb780405621b6680dc91c33ec3c09 --- /dev/null +++ b/iteration_3/packages/fused_diffusion_sampler_fp16_t128.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ec9d3ede7bc94cf7ea72a865f1997159b7916d7549e9639b08a50ecb0ccee874 +size 312430 diff --git a/iteration_3/packages/fused_diffusion_sampler_fp16_t128.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/iteration_3/packages/fused_diffusion_sampler_fp16_t128.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..5d524e318268dea3c586fd0de5ce641710361300 --- /dev/null +++ b/iteration_3/packages/fused_diffusion_sampler_fp16_t128.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f823b5c638d2eb2fd91bf8e4efe4a90b2e1d3d9e2f5ab40e7e93cb03cd212aca +size 49361856 diff --git a/iteration_3/packages/fused_diffusion_sampler_fp16_t128.mlpackage/Manifest.json b/iteration_3/packages/fused_diffusion_sampler_fp16_t128.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..2eb4f41dd65153b5d7540f220835c1f915ef31d9 --- /dev/null +++ b/iteration_3/packages/fused_diffusion_sampler_fp16_t128.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "0AD329AC-53BC-4290-912D-058CBA5BA5D7": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + }, + "9C5F32F4-A05E-4BF3-9782-D67784A8D9D5": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + } + }, + "rootModelIdentifier": "0AD329AC-53BC-4290-912D-058CBA5BA5D7" +} diff --git a/iteration_3/packages/fused_diffusion_sampler_fp16_t256.mlpackage/Data/com.apple.CoreML/model.mlmodel b/iteration_3/packages/fused_diffusion_sampler_fp16_t256.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..fc84d65ae382443ae45c9735d90409f52449e7a1 --- /dev/null +++ b/iteration_3/packages/fused_diffusion_sampler_fp16_t256.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ab926a4ef86b086e32276632d7b6232c2b8b31566641a82a9cf5d51dfa92431f +size 312430 diff --git a/iteration_3/packages/fused_diffusion_sampler_fp16_t256.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/iteration_3/packages/fused_diffusion_sampler_fp16_t256.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..5d524e318268dea3c586fd0de5ce641710361300 --- /dev/null +++ b/iteration_3/packages/fused_diffusion_sampler_fp16_t256.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f823b5c638d2eb2fd91bf8e4efe4a90b2e1d3d9e2f5ab40e7e93cb03cd212aca +size 49361856 diff --git a/iteration_3/packages/fused_diffusion_sampler_fp16_t256.mlpackage/Manifest.json b/iteration_3/packages/fused_diffusion_sampler_fp16_t256.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..351e76d5763c90206392a7ef2a8b9fb1a48aae3f --- /dev/null +++ b/iteration_3/packages/fused_diffusion_sampler_fp16_t256.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "09308FB4-D32A-41FF-B79D-FAA53069F9CC": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + }, + "76E947D9-F207-4B3D-93D4-708641940DCF": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + } + }, + "rootModelIdentifier": "09308FB4-D32A-41FF-B79D-FAA53069F9CC" +} diff --git a/iteration_3/packages/fused_diffusion_sampler_fp16_t64.mlpackage/Data/com.apple.CoreML/model.mlmodel b/iteration_3/packages/fused_diffusion_sampler_fp16_t64.mlpackage/Data/com.apple.CoreML/model.mlmodel new file mode 100644 index 0000000000000000000000000000000000000000..54554c780e36d08867da9ddee782baf2dd4d63b0 --- /dev/null +++ b/iteration_3/packages/fused_diffusion_sampler_fp16_t64.mlpackage/Data/com.apple.CoreML/model.mlmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e8aed8b7b3017488f499400131aa62942687ab060d5a2bf628e1fc9054b50569 +size 311547 diff --git a/iteration_3/packages/fused_diffusion_sampler_fp16_t64.mlpackage/Data/com.apple.CoreML/weights/weight.bin b/iteration_3/packages/fused_diffusion_sampler_fp16_t64.mlpackage/Data/com.apple.CoreML/weights/weight.bin new file mode 100644 index 0000000000000000000000000000000000000000..5d524e318268dea3c586fd0de5ce641710361300 --- /dev/null +++ b/iteration_3/packages/fused_diffusion_sampler_fp16_t64.mlpackage/Data/com.apple.CoreML/weights/weight.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f823b5c638d2eb2fd91bf8e4efe4a90b2e1d3d9e2f5ab40e7e93cb03cd212aca +size 49361856 diff --git a/iteration_3/packages/fused_diffusion_sampler_fp16_t64.mlpackage/Manifest.json b/iteration_3/packages/fused_diffusion_sampler_fp16_t64.mlpackage/Manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..766891face4dc871561ad2789d8197164d64a98b --- /dev/null +++ b/iteration_3/packages/fused_diffusion_sampler_fp16_t64.mlpackage/Manifest.json @@ -0,0 +1,18 @@ +{ + "fileFormatVersion": "1.0.0", + "itemInfoEntries": { + "A21D9DAC-9F9F-42D8-9FDB-E328454A27D7": { + "author": "com.apple.CoreML", + "description": "CoreML Model Specification", + "name": "model.mlmodel", + "path": "com.apple.CoreML/model.mlmodel" + }, + "CB405F91-933D-4224-B8F2-7397D38BB0CD": { + "author": "com.apple.CoreML", + "description": "CoreML Model Weights", + "name": "weights", + "path": "com.apple.CoreML/weights" + } + }, + "rootModelIdentifier": "A21D9DAC-9F9F-42D8-9FDB-E328454A27D7" +}