README.md

---
license: other
license_name: mixed-see-model-card
pipeline_tag: image-to-image
library_name: diffusers
tags:
  - sar-to-eo
  - remote-sensing
  - flow-matching
  - image-to-image
  - synthetic-aperture-radar
base_model: black-forest-labs/FLUX.2-klein-base-4B
---

# ReFlowSET

SAR-to-EO image translation with a conditional flow-matching transformer trained
from scratch inside a frozen high-fidelity autoencoder latent space.

- **Paper:** https://arxiv.org/abs/2609.00968
- **Code:** https://github.com/KAIST-VICLab/ReFlowSET
- **Project page:** https://kaist-viclab.github.io/ReFlowSET_site/
- **Comparison-method weights:** [`baselines/`](https://huggingface.co/JeonghyeokDo/ReFlowSET/tree/main/baselines) in this repository — all fifteen prior methods, both datasets

## Checkpoints

| Subfolder | Dataset | Resolution | Training | Deployed parameters |
|---|---|---|---|---|
| `qxs-saropt` | QXS-SAROPT | 256×256 | 40,000 steps × global batch 64 (2.56 M samples) | 509,324,417 + 84,046,115 frozen autoencoder |
| `sar2opt` | SAR2Opt | 512×512 | 20,000 steps × global batch 32 (640 k samples) | same |

Each subfolder is a complete `diffusers` pipeline: `transformer/`, `vae/`,
`scheduler/` and `model_index.json`. The two arms share the same architecture and
the same frozen autoencoder; they differ only in resolution, batch size and step
count. SAR2Opt stops at 20,000 steps to hold a comparable sample budget on a
1,450-image training set.

The published weights are the **EMA** parameters. The training-only REPA projector
is not included.

## Usage

```python
import torch
from PIL import Image
from diffusers import DiffusionPipeline
from huggingface_hub import snapshot_download

# Both arms live in this one repository, one per subfolder. `DiffusionPipeline`
# has no `subfolder` argument, so fetch the arm and load it as a local pipeline.
ARM = "qxs-saropt"                                   # or "sar2opt"
root = snapshot_download("JeonghyeokDo/ReFlowSET", allow_patterns=[f"{ARM}/*"])
pipe = DiffusionPipeline.from_pretrained(
    f"{root}/{ARM}", custom_pipeline=f"{root}/{ARM}", torch_dtype=torch.float32,
).to("cuda")

sar = Image.open("sar.png")                       # 1-channel SAR, 8-bit PNG
eo = pipe(sar, num_inference_steps=50, guidance_scale=1.5,
          generator=torch.Generator("cuda").manual_seed(2024)).images[0]
eo.save("eo.png")
```

`custom_pipeline` points at the same directory because the pipeline, transformer,
autoencoder and scheduler classes ship with the checkpoint rather than living in
`diffusers`. The classes are also on GitHub under `src/reflowset/`.

**Sampling settings are part of the reported result, not free knobs.** The paper's
main table is NFE 50 with guidance scale 1.5. NFE 4 samples **11× faster at 256²**
(163 ms vs 1824 ms) and **13× faster at 512²** (371 ms vs 4807 ms), batch 1 on one
B200, and trades distribution metrics against pixel metrics; do not mix the two in
one comparison.

The SAR input is read without a colour conversion, collapsed to one channel,
center-cropped (never resized), scaled by `x / 127.5 − 1`, replicated to three
channels, and encoded by the same frozen autoencoder that defines the EO latent
space. The pipeline does all of this; feed it the raw PNG.

## Results

Scored on the same test items as fifteen prior methods that we retrained under one
protocol, by a single evaluator.

| Dataset | n | FID↓ | DISTS↓ | LPIPS↓ | SSIM↑ | PSNR↑ |
|---|---|---|---|---|---|---|
| QXS-SAROPT @256 | 3,999 | 19.1 | **0.2310** | 0.5344 | 0.3554 | 16.09 |
| SAR2Opt @512 | 627 | **66.3** | **0.1847** | **0.5217** | 0.2871 | 16.06 |

Bold marks the best value among all sixteen methods in the paper's main table.
The full table, with every comparison method's weights and licence, is in
[`MODEL_ZOO.md`](https://github.com/KAIST-VICLab/ReFlowSET/blob/main/MODEL_ZOO.md).

> **These numbers are not comparable with the ones printed in the source papers.**
> Splits, resolutions and evaluator conventions differ. In particular **LPIPS has
> two conventions in this literature that differ by ~0.05**: we feed `x*2−1` to
> the LPIPS network, while several released evaluators feed `[0,1]` with
> `normalize=False` and obtain a systematically lower number.

## Architecture

A DiT with hidden size 1024 and depth 24 — eight double-stream blocks that give
the EO and SAR streams their own projections and joint attention, then sixteen
single-stream blocks over the concatenated token sequence — with 16 heads of
dimension 64 and 2-D RoPE over axes (32, 32).

There is **no separate SAR encoder**: the SAR image goes through the same frozen
autoencoder as the EO image. Training defines a linear bridge
`z_t = (1−t)·ε + t·z_e` and regresses the velocity `u* = z_e − ε` conditioned on
the SAR latent; sampling starts from `N(0, I)` and integrates `t: 0 → 1` with an
explicit Euler step. Classifier-free guidance was trained by zeroing the SAR
condition on 10 % of rows.

## Training data

QXS-SAROPT and SAR2Opt only, one dataset per arm, trained from scratch. No
pretraining corpus. Neither dataset is redistributed. QXS-SAROPT requires citing
arXiv:2103.08259 for research use.

## Comparison methods, in this same repository

`baselines/` holds the **fifteen prior methods of the paper's main table**,
retrained by us on the same splits and scored by the same evaluator — thirty
checkpoints, both datasets, each with its own card, its measured row, its
upstream repository and its licence. Start at
[`baselines/README.md`](https://huggingface.co/JeonghyeokDo/ReFlowSET/blob/main/baselines/README.md).

```python
from diffusers import UNet2DConditionModel
unet = UNet2DConditionModel.from_pretrained(
    "JeonghyeokDo/ReFlowSET", subfolder="baselines/qxs-saropt/cdiffset")
```

## Licence and provenance

**This repository is mixed-licence, so the Hub tag is `other`.** ReFlowSET's own
weights are **CC BY-NC 4.0** and the code is **Apache-2.0**. Everything under
`baselines/` is a third-party method carrying its own terms — MIT, BSD-3-Clause,
Apache-2.0, CreativeML-OpenRAIL-M, CC BY-NC-SA 4.0, S-Lab 1.0, and four with no
upstream licence file at all. The per-method cards state each one, and the texts
that must travel are in `baselines/licenses/`. Check the method you intend to
use; the repo-level tag is not a substitute.

The frozen autoencoder bundled in `vae/` is the **Apache-2.0** autoencoder from
[`black-forest-labs/FLUX.2-klein-base-4B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-4B),
re-serialised to the upstream layout and cast to bfloat16 — tensors paired by
value, not by an assumed rename table, and bit-identical to that source through a
full encode/decode.

**It is a substitution, and here is exactly what was substituted.** Both arms were
*trained and evaluated* with the `FLUX.2-dev` serialisation of the same network;
that file is under the FLUX Non-Commercial License, whose §4(a)(iii) forbids
"research and development related to surveillance" and whose §1(a) makes the
restriction inherit permanently. The two serialisations are the same autoencoder
— 250 of 251 tensors pair by value, worst absolute deviation 7.8e-03 (bfloat16
rounding) — and swapping the Apache file into the released checkpoints changes
QXS-SAROPT PSNR by **less than 0.004 dB in absolute value**. Four independent
measurements at different guidance scales and sample sets land between −0.004 and
+0.002 dB, so the sign is not resolved and only the magnitude is meaningful.
Changing only the evaluation seed moves the same number by +0.395 dB. Nothing
reported here changes.

Apache-2.0 §6 withholds trademark rights. This model is named ReFlowSET; it is not
a FLUX product and is not endorsed by Black Forest Labs.

**Why this autoencoder.** A latent generator cannot beat its codec's round trip,
so the codec is a ceiling on every row of a latent-model comparison.
[`vae_audit/`](https://github.com/KAIST-VICLab/ReFlowSET/tree/main/vae_audit)
measures that ceiling for six autoencoders — SD2.1, SDXL, SD3.0, SD3.5, FLUX.1
and FLUX.2 — on four SAR/EO benchmarks, EO and SAR scored separately. It ships as
code with download links and licence terms; no imagery and no third-party
autoencoder weights are redistributed.

Training used a frozen **DINOv3** ViT-L/16 (LVD-1689M) as a representation-
alignment teacher, acknowledged here as the DINOv3 License §1(b)(ii) requires. The
teacher is not loaded at inference and **no DINOv3 weights are redistributed**;
obtain them from [Meta's release](https://github.com/facebookresearch/dinov3)
under its own terms if you intend to retrain.

See [`LICENSE-WEIGHTS.md`](https://github.com/KAIST-VICLab/ReFlowSET/blob/main/LICENSE-WEIGHTS.md)
for the full reasoning, including one open question about the datasets' optical
imagery that we flag rather than resolve.

## Citation

```bibtex
@article{do2026reflowset,
  title   = {ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation},
  author  = {Do, Jeonghyeok and Lee, Seungchul and Kim, Munchurl},
  journal = {arXiv preprint arXiv:2609.00968},
  year    = {2026}
}
```

Our earlier SAR-to-EO work, which ReFlowSET builds on and compares against:

```bibtex
@article{do2026cdiffset,
  title   = {C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generation},
  author  = {Do, Jeonghyeok and Lee, Jaehyup and Lee, Seungchul and Kim, Munchurl},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year    = {2026}
}
```