BlendFace: Re-designing Identity Encoders for Face-Swapping

1 The University of Tokyo  2 CyberAgent AI Lab
International Conference on Computer Vision (ICCV) 2023
Banner image

The direct use of face recognition models (e.g., ArcFace) for face-swapping causes attribute leakage. Our novel identity encoder BlendFace provides well-disentangled identity features for face-swapping.


The great advancements of generative adversarial networks and face recognition models in computer vision have made it possible to swap identities on images from single sources. Although a lot of studies seems to have proposed almost satisfactory solutions, we notice previous methods still suffer from an identity-attribute entanglement that causes undesired attributes swapping because widely used identity encoders, eg, ArcFace, have some crucial attribute biases owing to their pretraining on face recognition tasks. To address this issue, we design BlendFace, a novel identity encoder for face-swapping. The key idea behind BlendFace is training face recognition models on blended images whose attributes are replaced with those of another mitigates inter-personal biases such as hairsyles. BlendFace feeds disentangled identity features into generators and guides generators properly as an identity loss function. Extensive experiments demonstrate that BlendFace improves the identity-attribute disentanglement in face-swapping models, maintaining a comparable quantitative performance to previous methods.

Attribute leakage problem in face recognition models

We examine identity similarity distributions between anchor and positive, negative, and swapped. The result shows that ArcFace underestimates the similarity for swapped images, resulting in the excessive identity guidance on face-swapping. We solve this problem by simply training ArcFace with swapped images to bridge the gap between positive and swapped.

Banner image

Training face-swapping models with BlendFace

After pretraining of BlendFace, we replace ArcFace with BlendFace to train better-disentangled face-swapping models.

Banner image

Comparison with previous models

Previous ArcFace-based face-swapping models generate inconsistent face boundaries between the swapped region and the original region. BlendFace boosts the consistency of the swapped faces while keeping the fidelity of the identity similarity.

Banner image


  author    = {Shiohara, Kaede and Yang, Xingchao and Taketomi, Takafumi},
  title     = {BlendFace: Re-designing Identity Encoders for Face-Swapping},
  booktitle   = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  year      = {2023},