分布指标

图像与视频集合级指标 — FID、FVD、JEDi、CMMD、Clean-FID 及特征数组距离。

Markdown 源码

最后更新: 2026年7月11日

比较参考集与生成集的集合级指标。

跳转到指标23 个 id · 点击展开

指标原理与计算

本节说明各 registry 指标的原理与计算约定，目标是让用户无需先读实现代码即可理解每个指标。

下文符号约定：

$X = \{x_i\}$ 表示参考样本集。
$Y = \{y_i\}$ 表示生成样本集。
$f(\cdot)$ 表示特征提取器。
$\mu_X, \Sigma_X$ 与 $\mu_Y, \Sigma_Y$ 表示特征空间中的经验均值与协方差。
$\uparrow$ 表示越高越好。
$\downarrow$ 表示越低越好。

若涉及 benchmark 特有的聚合、过滤或官方榜单归一化，请在对应 benchmark 页面记录协议，而不要在此处重定义可复用指标。

图像分布指标

`fid` (↓)

原理： Fréchet Inception Distance compares reference and generated image distributions in a neural feature space. The standard computation extracts Inception features, fits a Gaussian to each set, and computes the 2-Wasserstein distance between the two Gaussians:

\mathrm{FID}(X, Y) = \lVert \mu_X - \mu_Y \rVert^2 + \mathrm{Tr}\!\left(\Sigma_X + \Sigma_Y - 2\left(\Sigma_X \Sigma_Y\right)^{1/2}\right)

Lower values indicate closer generated and reference feature distributions. FID is useful for set-level image generation evaluation, but it is sensitive to feature extractor choice, preprocessing, sample count, and reference statistics. WorldFoundry aliases such as clip_fid, swav_fid, scene_fid, object_crop_fid, and fid_vid keep the same Fréchet-distance idea but change the feature extractor or the sample construction.

用法: 比较两个图像文件夹（reference vs generated）。

参考: Inception（或 CLIP/SwAV）特征分布间的 Fréchet 距离。论文: Heusel et al. 仓库: toshas/torch-fidelity

资产： Torch-fidelity InceptionV3 / CLIP / SwAV 权重，经 torch.hub 与 torchvision cache 加载。离线可传 feature_extractor_weights_path 或提前放入 Torch cache。

from worldfoundry.evaluation.tasks.metrics import compute_fid

ref_dir = "/data/coco_val"
gen_dir = "/data/my_model_samples"

fid = compute_fid(ref_dir, gen_dir)
print(fid)  # lower is better

clip_fid = compute_fid(ref_dir, gen_dir, feature_extractor="clip-vit-b-32")
swav_fid = compute_fid(ref_dir, gen_dir, feature_extractor="swav-resnet50")

返回: 浮点 FID；越低越好。

`scene_fid` (↓)

\mathrm{FID}(X, Y) = \lVert \mu_X - \mu_Y \rVert^2 + \mathrm{Tr}\!\left(\Sigma_X + \Sigma_Y - 2\left(\Sigma_X \Sigma_Y\right)^{1/2}\right)

用法: 带 bbox 物体裁剪的 SceneFID。

from worldfoundry.evaluation.tasks.metrics import compute_scene_fid

score = compute_scene_fid(
    "/data/ref",
    "/data/gen",
    reference_bboxes_json="/data/ref_bboxes.json",
)
print(score)

返回: Scene 级 FID 浮点数。

`inception_score` (↑)

原理： Inception Score estimates whether generated images are both classifiable and diverse. The original formulation feeds generated images into an Inception classifier, obtains p(y | x), compares each conditional label distribution with the marginal label distribution p(y), and exponentiates the expected KL divergence:

\mathrm{IS} = \exp\!\left(\mathbb{E}_x\left[\mathrm{KL}\!\left(p(y|x)\,\|\,p(y)\right)\right]\right)

A high score means individual images produce confident predictions while the whole generated set covers many predicted classes. It does not use reference images and can be misleading for domains that do not match the classifier's training distribution.

用法: 用 Inception 分类器衡量生成集质量/多样性。

参考: 生成图像在 Inception 分类器上的置信度与标签熵。论文: Salimans et al. 仓库: toshas/torch-fidelity（vendored）

资产： Torch-fidelity InceptionV3，经 torch.hub / torchvision cache 加载。

from worldfoundry.evaluation.tasks.metrics import compute_inception_score

result = compute_inception_score("/data/generated_only")
print(result)  # dict with mean/std IS

返回: 含 IS mean/std 的 dict；mean 越高越好。

`kid` (↓)

原理： Kernel Inception Distance compares reference and generated image sets using squared Maximum Mean Discrepancy over Inception features. A common implementation uses a polynomial kernel:

\mathrm{KID}(X, Y) = \mathrm{MMD}^2\!\left(f(X), f(Y)\right)

Unlike FID, KID does not fit Gaussian distributions and its estimator can be unbiased. Lower scores indicate closer feature distributions. It is often useful when sample counts are smaller or when users want an MMD-based complement to FID.

用法: 两图像集合间的 Kernel Inception Distance。

参考: Kernel Inception Distance（Inception 特征上多项式核 MMD）。论文: Bińkowski et al. 仓库: toshas/torch-fidelity

资产： Torch-fidelity Inception feature，经 torch.hub / torchvision cache 加载。

from worldfoundry.evaluation.tasks.metrics import compute_kid

kid = compute_kid("/data/ref", "/data/gen")
print(kid["kernel_inception_distance_mean"])

返回: KID 统计 dict；mean 越低越好。

`precision_recall` (↑)

原理： Distribution precision and recall separate two failure modes that a single scalar such as FID can hide. Precision measures how much of the generated distribution lies on or near the real data manifold, while recall measures how much of the real data manifold is covered by generated samples. Implementations typically embed real and generated samples, estimate manifolds or support regions, then compute the fraction of samples falling inside the other distribution's support. Higher precision means more realistic samples; higher recall means better coverage.

用法: 流形 precision/recall（覆盖度与保真度）。

参考: 生成模型覆盖度/保真度的流形 precision 与 recall。论文: Kynkäänniemi et al. 仓库: toshas/torch-fidelity

资产： Torch-fidelity Inception feature，经 torch.hub / torchvision cache 加载。

from worldfoundry.evaluation.tasks.metrics import compute_precision_recall

pr = compute_precision_recall("/data/ref", "/data/gen")
print(pr)

返回: 含 precision、recall 等键的 dict。

`improved_precision_recall` (↑)

原理： Improved Precision and Recall uses non-parametric manifold estimates in feature space. For each real and generated sample, the method estimates a local radius using nearest neighbors, then checks cross-membership between real and generated manifolds. Precision is the fraction of generated samples that lie inside the real manifold; recall is the fraction of real samples covered by the generated manifold. Higher values are better. This metric is useful for distinguishing fidelity failure from coverage failure.

用法: α-precision、β-recall、realism score（k-NN 半径）。

参考: 特征空间 k-NN 半径的 α-precision、β-recall 与 realism score。论文: Kynkä et al. 仓库: kynkaat/improved-precision-and-recall-metric

资产： Torchvision VGG16 feature extractor（TORCH_HOME / torch cache）。

from worldfoundry.evaluation.tasks.metrics import (
    compute_improved_precision_recall,
    compute_realism_score,
)

ipr = compute_improved_precision_recall("/data/ref", "/data/gen")
realism = compute_realism_score("/data/ref", "/data/gen")
print(ipr, realism)

返回: IPR 指标 dict；precision/recall/realism 越高越好。

`fwd / cmmd / clean_fid / mind / trend` (↓)

`fwd`

原理： Fréchet Wavelet Distance is a domain-agnostic alternative to neural-feature Fréchet metrics. Instead of extracting Inception features, it projects images into wavelet-packet coefficient space and computes a Fréchet distance between the resulting coefficient distributions. The core computation is:

\mathrm{FWD}(X, Y) = d_{\text{Fréchet}}\!\left(W_p(X), W_p(Y)\right)

where Wp is the wavelet packet transform. Lower values indicate closer distributions. Because the representation is frequency-based rather than classifier-based, FWD can be useful for domains where ImageNet-pretrained features are poorly matched.

`cmmd`

原理： CMMD, or CLIP Maximum Mean Discrepancy, compares image sets in CLIP embedding space using MMD, usually with a Gaussian RBF kernel. The computation is:

\mathrm{CMMD}(X, Y) = \mathrm{MMD}^2\!\left(\mathrm{CLIP}_{\mathrm{img}}(X), \mathrm{CLIP}_{\mathrm{img}}(Y)\right)

Lower values indicate that generated and reference image distributions are closer in CLIP's semantic feature space. CMMD was proposed as an alternative to FID for modern image generation, especially text-to-image settings, because it avoids the Gaussian assumption used by FID and uses richer CLIP features.

`clean_fid`

原理： Clean-FID keeps the FID principle but standardizes preprocessing details such as resizing, quantization, and reference statistics. The computation is still a Fréchet distance between feature Gaussians, but the implementation aims to reduce score drift caused by inconsistent image-processing libraries. Use clean_fid when you need FID values that are more comparable across papers and codebases. Do not mix Clean-FID and non-Clean-FID values in the same leaderboard unless explicitly documented.

`mind`

原理： Monge Inception Distance compares generated and reference distributions using sliced Wasserstein distance over Inception-style features. Instead of estimating high-dimensional Gaussian moments as FID does, MIND projects features onto many one-dimensional directions, sorts projected samples, computes one-dimensional optimal transport distances, and averages them. Lower values indicate closer distributions. The motivation is better sample efficiency and robustness compared with moment-matching metrics such as FID.

`trend`

原理： trend / trend_jsd compares reference and generated trend distributions, usually through a divergence such as Jensen-Shannon divergence. Given two categorical or binned distributions P and Q, the common computation is:

\mathrm{JSD}(P, Q) = \tfrac{1}{2}\,\mathrm{KL}(P \parallel M) + \tfrac{1}{2}\,\mathrm{KL}(Q \parallel M), \quad M = \tfrac{1}{2}(P + Q)

Lower values indicate that generated outputs preserve the reference trend distribution more closely. In WorldFoundry, this is best used when the benchmark defines explicit attributes, categories, temporal bins, or other trend statistics to compare.

用法: 图像文件夹上的分布距离（不同特征后端）。

参考：

fwd: 图像分布间的 Fréchet Wavelet Distance。论文: Karras et al. 仓库: WorldFoundry（vendored pytorchfwd）
cmmd: 图像集合间的 CLIP Maximum Mean Discrepancy。论文: Jayasumana et al. 仓库: WorldFoundry（改编自 CMMD repo）
clean_fid: 改进 Inception 预处理的 Clean-FID。论文: Parmar et al. 仓库: GaParmar/clean-fid
mind: 图像集合间的 Monge Inception Distance。论文: John et al. 仓库: WorldFoundry（torch-fidelity 集成）
trend: 特征直方图上的截断熵 / trend divergence。论文: Han et al. 仓库: WorldFoundry（按论文复现）

资产：

fwd: Vendored distribution helper / wavelet 后端权重，经 torch cache 加载。
cmmd: CLIP vision encoder openai/clip-vit-large-patch14-336，Hugging Face transformers cache（WORLDFOUNDRY_HFD_ROOT）。
clean_fid: Clean-FID Inception 网络与可选 reference statistics，放在 cleanfid runtime cache。
mind: Torch-fidelity feature extractor，经 torch.hub / torchvision cache 加载。
trend: 调用方提供 API 要求的统计量或 feature 时，无需模型 checkpoint。

from worldfoundry.evaluation.tasks.metrics import (
    compute_fwd,
    compute_cmmd,
    compute_clean_fid,
    compute_mind,
    compute_trend,
)

ref, gen = "/data/ref", "/data/gen"
print("fwd", compute_fwd(ref, gen))
print("cmmd", compute_cmmd(ref, gen))
print("clean_fid", compute_clean_fid(ref, gen, dataset_name="cifar10", mode="clean"))
print("mind", compute_mind(ref, gen))
print("trend", compute_trend(ref, gen))

返回: 各 metric 返回 float 或 dict；这些 id 越低越好。

`ppl` (↓)

原理： Perceptual Path Length measures smoothness of a generator's latent space. The metric samples two latent codes, interpolates between them, takes a small step along the interpolation path, generates the corresponding images, and measures perceptual distance between the two nearby generated images. A simplified form is:

\mathrm{PPL} = \mathbb{E}\left[\frac{d\!\left(G(\mathrm{interp}(z_1, z_2, t)),\, G(\mathrm{interp}(z_1, z_2, t + \varepsilon))\right)}{\varepsilon^2}\right]

Lower values indicate smoother perceptual changes under latent interpolation. PPL is meaningful only when the model exposes a valid latent-space interpolation interface.

用法: StyleGAN 潜空间感知路径长度（平滑度）。

参考: StyleGAN 潜空间感知路径长度（平滑度）。论文: Karras et al. 仓库: toshas/torch-fidelity

资产： Torch-fidelity latent / perceptual path 权重，经 torch.hub / torchvision cache 加载。

from worldfoundry.evaluation.tasks.metrics import compute_ppl

ppl = compute_ppl("/path/to/generated_or_latent_inputs")
print(ppl)

返回: PPL 统计量；越低生成器越平滑。

`vendi_score / rke / rnd` (↑)

`vendi_score`

原理： Vendi Score measures diversity from a similarity matrix. Given samples and a similarity function, construct a normalized similarity matrix K, compute its eigenvalues λ, and return the exponential Shannon entropy:

\mathrm{Vendi}(X) = \exp\!\left(-\sum_i \lambda_i \log \lambda_i\right)

The score can be interpreted as an effective number of distinct samples under the chosen similarity function. Higher values indicate greater diversity. Because the metric is reference-free, it should be paired with a fidelity metric when evaluating generation quality.

`rke`

原理： Rényi Kernel Entropy evaluates diversity through entropy in a kernel-induced feature space. Given a positive semidefinite kernel matrix, the method computes a Rényi-entropy-style functional over its eigenvalues. Higher scores indicate more effective modes or greater diversity under the chosen kernel. Because the metric is reference-free, it should be reported with the kernel, feature representation, and any normalization used.

`rnd`

原理： Random Network Distillation score measures diversity or novelty using prediction error against random-network features. The method passes samples through a fixed random target network and a predictor or feature-matching path; greater prediction error or feature spread can indicate greater diversity depending on the implementation. Higher values are generally interpreted as more diversity or novelty. It is best treated as a diagnostic metric and should be paired with fidelity metrics.

用法: 由特征矩阵或图像目录计算多样性。

参考：

vendi_score: 由 feature 矩阵或图像 embedding 计算的 Vendi 多样性分数。论文: Friedman & Dieng 仓库: WorldFoundry（改编）
rke: feature embedding 上 Rényi 核熵模式计数（多样性）。论文: Jalali et al. (NeurIPS 2023) 仓库: mjalali/renyi-kernel-entropy
rnd: feature 向量上的 Random Network Distillation 新颖度分数。论文: Burda et al. 仓库: WorldFoundry（按论文复现）

资产：

vendi_score: 图像 helper：torchvision InceptionV3。文本 helper：roberta-base 与 princeton-nlp/unsup-simcse-roberta-base（WORLDFOUNDRY_HFD_ROOT）。feature 数组模式无需 extractor。
rke: 调用方提供 feature embedding 时，无需模型 checkpoint。
rnd: 调用方提供 feature embedding 时，无需模型 checkpoint。

import numpy as np
from worldfoundry.evaluation.tasks.metrics import (
    compute_vendi_score,
    compute_rke,
    compute_rrke,
    compute_rnd,
    compute_rnd_from_images,
)

features = np.load("embeddings.npy")  # shape (N, D)
print("vendi", compute_vendi_score(features))
print("rke", compute_rke(features))
print("rrke", compute_rrke(ref_features, gen_features))
print("rnd", compute_rnd(features))
print("rnd_images", compute_rnd_from_images("/data/generated"))

返回: 多样性浮点分数；vendi/rke/rnd 越高越多样。

`rarity_score` (↑)

原理： Rarity Score measures how uncommon each generated image is relative to a reference distribution. The method embeds images, uses nearest-neighbor distances to model how sparse the local region is, and assigns higher rarity to samples that lie in less crowded regions of feature space. A set-level score is usually an average or distribution summary over individual rarity scores. Higher rarity can indicate novelty, but not necessarily quality: rare samples may also be unrealistic or out-of-distribution.

用法: 生成样本相对真实特征库的罕见程度。

参考: 到 k 近邻真实样本的平均距离（越高越罕见）。论文: Han et al. (ICLR 2023) 仓库: hichoe95/Rarity-Score

资产： 由真实图像构建 feature bank；从媒体抽取时会使用 metric 配置的 feature extractor cache。

import numpy as np
from worldfoundry.evaluation.tasks.metrics import compute_mean_rarity_score

real_feats = np.load("real.npy")
gen_feats = np.load("gen.npy")
print(compute_mean_rarity_score(real_feats, gen_feats, k=5))

返回: 平均 rarity；越高越罕见。

`fld` (↓)

原理： Feature Likelihood Divergence evaluates fidelity, diversity, and novelty using density estimation in feature space. The method fits or estimates feature likelihoods for train, test, and generated samples, then measures whether generated samples look like unseen test data rather than memorized training data. Lower values indicate better generalization under the FLD formulation. This metric is useful when overfitting or memorization is a concern and when train/test/generated feature sets are available.

适用场景：在预计算特征数组上计算 Feature Likelihood Divergence。

参考： 真实与生成特征分布之间的 Feature Likelihood Divergence。论文：Richter & Rhodes (ICML 2022) 仓库：WorldFoundry（论文复现）

资产： 调用方提供 train/test/generated 特征数组时无需 checkpoint。

import numpy as np
from worldfoundry.evaluation.tasks.metrics import compute_fld

train = np.load("train_feats.npy")
test = np.load("test_feats.npy")
gen = np.load("gen_feats.npy")
print(compute_fld(train, test, gen))

返回：浮点 FLD；越低越好。

`multimodal_mid` (↑)

原理： Mutual Information Divergence is a multimodal generation metric based on CLIP-style image-text features. It estimates cross-modal mutual information behavior between paired modalities and uses a negative Gaussian cross-mutual-information formulation to score alignment. Higher WorldFoundry scores should indicate stronger multimodal consistency when the implementation exposes MID as a positive alignment score. Use it for paired multimodal generation such as text-to-image or image captioning, and document the exact feature model.

适用场景：在配对图文 embedding 上计算 Mutual Information Divergence。

参考： 比较联合图文 embedding 分布的 Mutual Information Divergence。论文：Dhaka et al. (NeurIPS 2021) 仓库：WorldFoundry（论文复现）

资产： 调用方提供配对特征数组时无需 checkpoint。

from worldfoundry.evaluation.tasks.metrics import compute_multimodal_mid

print(compute_multimodal_mid(image_feats, text_feats))

返回：浮点 MID；越高越好。

`fjd` (↓)

原理： Fréchet Joint Distance evaluates conditional generation by comparing joint distributions of generated content and conditioning variables. Instead of computing Fréchet distance only on image features, FJD builds joint embeddings, for example image features concatenated or combined with class, text, mask, or layout-condition features, then computes a Fréchet distance between real and generated joint distributions. Lower values indicate better joint fidelity, condition consistency, and intra-condition diversity.

适用场景：在配对图文联合 embedding 上计算 Fréchet Joint Distance。

参考： 配对图文联合 embedding 的 Fréchet Joint Distance。论文：Lee et al. (ICCV 2023) 仓库：WorldFoundry（论文复现）

资产： 调用方提供联合 embedding 时无需 checkpoint。

from worldfoundry.evaluation.tasks.metrics import compute_fjd_from_joint_embeddings

print(compute_fjd_from_joint_embeddings(real_joint, gen_joint))

返回：浮点 FJD；越低越好。

`crosslid` (↓)

原理： CrossLID evaluates how well generated and real manifolds coincide using local intrinsic dimensionality. The method estimates the local intrinsic dimensionality of real samples with respect to neighborhoods formed by generated samples. Lower CrossLID indicates that generated samples better match the local structure of the real data manifold. It is useful for diagnosing mode collapse and distribution mismatch beyond global moment statistics.

适用场景：用 Cross local intrinsic dimensionality 评估多样性。

参考： 用于多样性评估的 Cross local intrinsic dimensionality。论文：Barua et al. (ICLR 2019) 仓库：sukarnabarua/CrossLID

资产： 调用方提供特征数组时无需 checkpoint。

from worldfoundry.evaluation.tasks.metrics import compute_crosslid

print(compute_crosslid(features, k=5))

返回：浮点 CrossLID；越低表示多样性越低。

`cfid` (↓)

原理： Conditional FID generalizes FID to conditional distributions. Instead of comparing only unconditional generated and real image distributions, CFID measures discrepancies between conditional distributions, making it more sensitive to cases where outputs are realistic but unrelated to inputs. The closed-form version follows a conditional Gaussian/Wasserstein setup. Lower values indicate better conditional fidelity.

适用场景：从条件/无条件配对 embedding 计算 Conditional FID。

参考： 从条件/无条件配对 embedding 计算的 Conditional FID。论文：Benny & Wolf (NeurIPS 2020) 仓库：WorldFoundry（论文复现）

资产： 调用方提供条件特征数组时无需 checkpoint。

from worldfoundry.evaluation.tasks.metrics import compute_cfid

print(compute_cfid(ref_pairs, gen_pairs))

返回：浮点 cFID；越低越好。

`ssd` (↓)

原理： Semantic Similarity Distance is a CLIP-based metric for text-image consistency. It compares generated images and text prompts from a distributional semantic-alignment perspective, combining text-image similarity and semantic variation differences. Lower values indicate better text-image semantic consistency. Use it for text-to-image benchmarks where prompt faithfulness matters more than pixel-level similarity.

适用场景：文本与图像 embedding 之间的 Semantic Similarity Distance。

参考： 文本与图像 embedding 之间的 Semantic Similarity Distance。论文：Tschannen et al. (ICLR 2018) 仓库：WorldFoundry（论文复现）

资产： 调用方提供文本/图像 embedding 时无需 checkpoint。

from worldfoundry.evaluation.tasks.metrics import compute_ssd

print(compute_ssd(text_embeds, image_embeds))

返回：浮点 SSD；越低越好。

`linear_separability` (↑)

原理： Linear separability measures disentanglement in a latent or attribute space. The StyleGAN formulation asks whether latent-space samples corresponding to a binary attribute can be separated by a linear hyperplane. In WorldFoundry, the public helper computes a score from a confusion matrix, so the effective computation depends on classifier outcomes:

\text{linear\_separability} = \mathrm{score}(\mathbf{C})

Higher values indicate that the attribute or factor is more linearly separable. This metric is meaningful only when the upstream classifier, labels, and attribute definitions are stable.

用法: 由 StyleGAN 潜空间 SVM 混淆矩阵计算线性可分性。

参考: StyleGAN 潜空间 SVM 线性可分性（混淆矩阵）。论文: Karras et al. 仓库: NVlabs/stylegan

资产： 无需模型 checkpoint — 传入 API 要求的 confusion matrix / separability 输入。

import numpy as np
from worldfoundry.evaluation.tasks.metrics import compute_linear_separability

confusion = np.load("svm_confusion.npy")
print(compute_linear_separability(confusion))

返回: 可分性分数；越高潜变量越解耦。

`fdd` (↓)

原理： Fréchet Denoised Distance replaces the Inception feature extractor in FID with a denoising-autoencoder representation. The metric encodes real and generated images with a DAE, fits Gaussian statistics in the denoised latent space, and computes a Fréchet distance. Lower values indicate that generated images are closer to reference images in a structure-biased denoised feature space. FDD is especially relevant when structural plausibility matters more than low-level visual artifacts.

用法: DAE 潜特征上的 Fréchet Denoised Distance。

参考: 去噪自编码器潜特征分布的 Fréchet 距离（结构合理性）。论文: Fan et al. (ECCV 2024) 仓库: jiajie96/FDD_pytorch

资产： WORLDFOUNDRY_FDD_DAE_CKPT，或通过 gdown 自动下载（Google Drive id 1j7MVFWYfRNZLQ3uChe7TGQG8L1Oaf9Gt）。

from worldfoundry.evaluation.tasks.metrics import compute_fdd

# export WORLDFOUNDRY_FDD_DAE_CKPT=/path/to/dae.pt
score = compute_fdd("/data/ref", "/data/gen")
print(score)

返回: FDD 浮点；越低结构越合理。

`cis` (↑)

原理： Conditional Inception Score generalizes Inception Score to class-conditional generation. It decomposes quality and diversity by conditioning class, using within-class and between-class components derived from classifier predictions. Higher values indicate better class-conditional quality and diversity. Use CIS only when generated samples have known class conditions and classifier predictions are meaningful for those classes.

用法: 由类概率 batch 计算 Conditional Inception Score。

参考: 条件 Inception Score：按类分桶 softmax 的 BCIS × WCIS。论文: Benny et al. (IJCV 2020) 仓库: WorldFoundry（按论文复现）

资产： 调用方提供 class-probability batch 时，无需模型 checkpoint。

import numpy as np
from worldfoundry.evaluation.tasks.metrics import compute_cis, compute_cis_from_predictions

# probs: list of (N, num_classes) arrays, one bucket per condition
print(compute_cis(class_probs))
print(compute_cis_from_predictions(predictions, num_classes=1000))

返回: 含 BCIS、WCIS 与组合 CIS 的 dict。

`attribute_sad / attribute_pad` (↓)

`attribute_sad`

原理： Attribute SaD is an interpretable attribute-distribution metric. Given high-level concept scores for real and generated images and a text attribute list, it measures how far the generated attribute distribution deviates from the real attribute distribution. Lower values indicate better preservation of marginal attribute statistics. This metric is useful when benchmark users need to know which semantic attributes are mismatched rather than only receiving a scalar realism score.

`attribute_pad`

原理： Attribute PaD is the paired or relationship-sensitive counterpart to SaD. It evaluates whether generated samples preserve attribute relationships or paired attribute behavior relative to the reference set. Lower values indicate better attribute-pair consistency. Use it when attribute co-occurrence or implausible attribute relationships are important evaluation targets.

用法: 由预计算 HCS 直方图计算 SadPaD 属性散度。

参考：

attribute_sad: 属性直方图间的 KL / JS / W1 散度（attribute SAD）。论文: Ramesh et al. 仓库: WorldFoundry（SadPaD 复现）
attribute_pad: 文本属性的 attribute-aware Padé approximant divergence。论文: Ramesh et al. 仓库: WorldFoundry（SadPaD 复现）

资产：

attribute_sad: 无需模型 checkpoint — 传入预计算 HCS histogram（hcs_real、hcs_gen、text_list）。
attribute_pad: 无需模型 checkpoint — 传入预计算 HCS histogram（hcs_real、hcs_gen、text_list）。

from worldfoundry.evaluation.tasks.metrics import compute_attribute_sad, compute_attribute_pad

text_list = ["smiling", "young", "eyeglasses"]
sad = compute_attribute_sad(hcs_real, hcs_gen, text_list)
pad = compute_attribute_pad(hcs_real, hcs_gen, text_list)
print(sad, pad)

返回: 含各属性散度的 dict；越低越接近真实分布。

视频分布指标

`fvd` (↓)

原理： Fréchet Video Distance extends the FID idea to videos. It extracts spatiotemporal video features, commonly from an I3D network, fits Gaussian statistics to reference and generated video features, and computes the same Fréchet Gaussian distance used by FID. Lower values indicate closer reference and generated video distributions. FVD captures both appearance and temporal information better than frame-only metrics, but later work shows that it can still be biased toward frame quality and may be insensitive to some temporal corruptions.

用法: 真实/生成视频集合间的 Fréchet Video Distance。

参考: I3D 视频特征分布间的 Fréchet 距离。论文: Unterthiner et al. 仓库: WorldFoundry（改编自 StyleGAN-V / MiraBench）

资产： Inception I3D i3d_pretrained_400.pt：显式参数 → WORLDFOUNDRY_FVD_I3D_CKPT → WORLDFOUNDRY_MIRABENCH_FVD_I3D_CKPT → WORLDFOUNDRY_CKPT_DIR。

import numpy as np
from worldfoundry.evaluation.tasks.metrics import (
    compute_fvd_from_numpy,
    compute_fvd_from_frame_dirs,
)

# uint8 arrays shaped (N, T, H, W, C)
real = np.load("real_videos.npy")
gen = np.load("gen_videos.npy")
print(compute_fvd_from_numpy(real, gen, device="cuda"))

print(
    compute_fvd_from_frame_dirs(
        reference_frame_dirs=["/data/ref_frames/vid0"],
        generated_frame_dirs=["/data/gen_frames/vid0"],
        device="cuda",
    )
)

返回: FVD 浮点；必要时设置 WORLDFOUNDRY_FVD_I3D_CKPT。

`fvmd` (↓)

原理： Fréchet Video Motion Distance focuses on motion consistency. It extracts explicit motion features, for example through keypoint tracking, fits reference and generated motion-feature distributions, and computes a Fréchet distance between them. Lower values indicate closer motion distributions. FVMD is useful when a video model produces sharp frames but unrealistic temporal dynamics.

用法: 视频目录间基于运动特征的 Fréchet 距离。

参考: PIPs++ 关键点速度/加速度运动特征的 Fréchet 距离。论文: Liu et al. 仓库: ljh0v0/FVMD-frechet-video-motion-distance

资产： FVMD release 的 PIPs2 tracker pips2_weights.pth，经 torch hub cache 加载。

from worldfoundry.evaluation.tasks.metrics import compute_fvmd

score = compute_fvmd("/data/ref_videos", "/data/gen_videos")
print(score)

返回: FVMD 浮点；越低运动分布越接近。

`jedi` (↓)

原理： JEDi, or JEPA Embedding Distance, evaluates video generation using V-JEPA-style video representations and an MMD distance, commonly with a polynomial kernel. The computation extracts video embeddings, then compares generated and reference embedding sets through MMD:

\mathrm{JEDi}(X, Y) = \mathrm{MMD}^2\!\left(\mathrm{VJEPA}(X), \mathrm{VJEPA}(Y)\right)

Lower values indicate closer video distributions. JEDi was proposed to address limitations of FVD, including non-Gaussian feature assumptions, sample inefficiency, and weak temporal sensitivity.

用法: V-JEPA 特征 MMD 衡量视频分布质量。

参考: V-JEPA embedding 上带多项式核的 MMD（视频分布质量）。论文: Luo et al. 仓库: oooolga/JEDi

资产： 实时路径：V-JEPA 目录（WORLDFOUNDRY_JEDI_MODEL_DIR / WORLDFOUNDRY_JEDI_VJEPA_DIR / WORLDFOUNDRY_VJEPA_MODEL_DIR，可选 WORLDFOUNDRY_JEDI_CONFIG_PATH）。feature 模式：仅预计算 train/test 数组（WORLDFOUNDRY_JEDI_FEATURE_PATH）。

import numpy as np
from worldfoundry.evaluation.tasks.metrics import (
    compute_jedi_from_features,
    compute_mock_jedi,
)

train_feats = np.load("jedi_train.npy")
test_feats = np.load("jedi_test.npy")
print(compute_jedi_from_features(train_feats, test_feats))

# smoke test without checkpoints
print(compute_mock_jedi())

返回: JEDi MMD 浮点；live backend 需配置 WORLDFOUNDRY_JEDI_*。

一次跑多个 torch-fidelity 指标

from worldfoundry.evaluation.tasks.metrics import compute_distribution_metrics

scores = compute_distribution_metrics(
    "/data/ref", "/data/gen", metrics=("fid", "kid", "prc")
)
print(scores)

分布指标

指标原理与计算

图像分布指标

fid (↓)

scene_fid (↓)

inception_score (↑)

kid (↓)

precision_recall (↑)

improved_precision_recall (↑)

fwd / cmmd / clean_fid / mind / trend (↓)

fwd

cmmd

clean_fid

mind

trend

ppl (↓)

vendi_score / rke / rnd (↑)

vendi_score

rke

rnd

rarity_score (↑)

fld (↓)

multimodal_mid (↑)

fjd (↓)

crosslid (↓)

cfid (↓)

ssd (↓)

linear_separability (↑)

fdd (↓)

cis (↑)

attribute_sad / attribute_pad (↓)

attribute_sad

attribute_pad

视频分布指标

fvd (↓)

fvmd (↓)

jedi (↓)

一次跑多个 torch-fidelity 指标

`fid` (↓)

`scene_fid` (↓)

`inception_score` (↑)

`kid` (↓)

`precision_recall` (↑)

`improved_precision_recall` (↑)

`fwd / cmmd / clean_fid / mind / trend` (↓)

`fwd`

`cmmd`

`clean_fid`

`mind`

`trend`

`ppl` (↓)

`vendi_score / rke / rnd` (↑)

`vendi_score`

`rke`

`rnd`

`rarity_score` (↑)

`fld` (↓)

`multimodal_mid` (↑)

`fjd` (↓)

`crosslid` (↓)

`cfid` (↓)

`ssd` (↓)

`linear_separability` (↑)

`fdd` (↓)

`cis` (↑)

`attribute_sad / attribute_pad` (↓)

`attribute_sad`

`attribute_pad`

`fvd` (↓)

`fvmd` (↓)

`jedi` (↓)