research-article

Domain Generalization via Frequency-domain-based Feature Disentanglement and Interaction

Authors:

Dongliang Chang,

Kongming Liang, and

Zhanyu MaAuthors Info & Claims

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

October 2022

Pages 4821 - 4829

https://doi.org/10.1145/3503161.3548267

Published: 10 October 2022 Publication History

Abstract

Adaptation to out-of-distribution data is a meta-challenge for all statistical learning algorithms that strongly rely on the i.i.d. assumption. It leads to unavoidable labor costs and confidence crises in realistic applications. For that, domain generalization aims at mining domain-irrelevant knowledge from multiple source domains that can generalize to unseen target domains. In this paper, by leveraging the frequency domain of an image, we uniquely work with two key observations: (i) the high-frequency information of an image depicts object edge structure, which preserves high-level semantic information of the object is naturally consistent across different domains, and (ii) the low-frequency component retains object smooth structure, while this information is susceptible to domain shifts. Motivated by the above observations, we introduce (i) an encoder-decoder structure to disentangle high- and low-frequency features of an image, (ii) an information interaction mechanism to ensure the helpful knowledge from both two parts can cooperate effectively, and (iii) a novel data augmentation technique that works on the frequency domain to encourage the robustness of frequency-wise feature disentangling. The proposed method obtains state-of-the-art performance on three widely used domain generalization benchmarks (Digit-DG, Office-Home, and PACS).

Supplementary Material

MP4 File (MM22-mmfp2217.mp4)

Presentation video

Download
372.52 MB

References

[1]

Yogesh Balaji, Swami Sankaranarayanan, and Rama Chellappa. 2018. Metareg: Towards domain generalization using meta-regularization. Advances in neural information processing systems (2018).

[2]

Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. 2006. Analysis of representations for domain adaptation. Advances in neural information processing systems (2006).

[3]

Fabio M Carlucci, Antonio D'Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. 2019. Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2229--2238.

[4]

Dongliang Chang, Yifeng Ding, Jiyang Xie, Ayan Kumar Bhunia, Xiaoxu Li, Zhanyu Ma, Ming Wu, Jun Guo, and Yi-Zhe Song. 2020. The devil is in the channels: Mutual-channel loss for fine-grained image classification. IEEE Transactions on Image Processing (2020), 4683--4695.

[5]

Dongliang Chang, Kaiyue Pang, Yixiao Zheng, Zhanyu Ma, Yi-Zhe Song, and Jun Guo. 2021. Your" Flamingo" is My" Bird": Fine-Grained, or Not. In Proceedings of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 11476--11485.

[6]

Qi Dou, Daniel Coelho de Castro, Konstantinos Kamnitsas, and Ben Glocker. 2019. Domain generalization via model-agnostic learning of semantic features. Advances in Neural Information Processing Systems (2019).

[7]

Ruoyi Du, Jiyang Xie, Zhanyu Ma, Dongliang Chang, Yi-Zhe Song, and Jun Guo. 2021b. Progressive Learning of Category-Consistent Multi-Granularity Features for Fine-Grained Visual Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).

[8]

Zhekai Du, Jingjing Li, Ke Lu, Lei Zhu, and Zi Huang. 2021a. Learning Transferrable and Interpretable Representations for Domain Generalization. In Proceedings of the 29th ACM Int'l Conf. on Multimedia. 3340--3349.

Digital Library

[9]

Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In Int'l Conf. on Machine Learning. PMLR, 1180--1189.

[10]

Abel Gonzalez-Garcia, Joost Van De Weijer, and Yoshua Bengio. 2018. Image-to-image translation for cross-domain disentanglement. Advances in neural information processing systems (2018).

[11]

Melvyn A Goodale and A David Milner. 1992. Separate visual pathways for perception and action. Trends in neurosciences 1 (1992), 20--25.

[12]

Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu. 2021. Fsdr: Frequency space domain randomization for domain generalization. In Proceedings of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 6891--6902.

[13]

Zeyi Huang, Haohan Wang, Eric P Xing, and Dong Huang. 2020. Self-challenging improves cross-domain generalization. In European Conf. on Computer Vision. Springer, 124--140.

Digital Library

[14]

Seogkyu Jeon, Kibeom Hong, Pilhyeon Lee, Jewook Lee, and Hyeran Byun. 2021. Feature stylization and domain-aware contrastive learning for domain generalization. In Proceedings of the 29th ACM Int'l Conf. on Multimedia. 22--31.

Digital Library

[15]

Mamta Juneja and Parvinder Singh Sandhu. 2009. Performance evaluation of edge detection techniques for images in spatial domain. Int'l Journal of Computer Theory and Engineering 5 (2009), 614.

[16]

Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. 2017. Deeper, broader and artier domain generalization. In Proceedings of the IEEE Int'l Conf. on Computer Vision. 5542--5550.

[17]

Da Li, Jianshu Zhang, Yongxin Yang, Cong Liu, Yi-Zhe Song, and Timothy M Hospedales. 2019. Episodic training for domain generalization. In Proceedings of the IEEE/CVF Int'l Conf. on Computer Vision. 1446--1455.

[18]

Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. 2018. Domain generalization with adversarial feature learning. In Proceedings of the IEEE conf. on Computer Vision and Pattern Recognition. 5400--5409.

[19]

Pan Li, Da Li, Wei Li, Shaogang Gong, Yanwei Fu, and Timothy M Hospedales. 2021. A simple feature augmentation for domain generalization. In Proceedings of the IEEE/CVF Int'l Conf. on Computer Vision. 8886--8895.

[20]

Tsung-Yu Lin, Aruni RoyChowdhury, and Subhransu Maji. 2015. Bilinear cnn models for fine-grained visual recognition. In Proceedings of the IEEE Int'l Conf. on Computer Vision. 1449--1457.

Digital Library

[21]

Alexander H Liu, Yen-Cheng Liu, Yu-Ying Yeh, and Yu-Chiang Frank Wang. 2018. A unified feature disentangler for multi-domain image translation and manipulation. Advances in Neural Information Processing Systems (2018).

[22]

Chang Liu, Lichen Wang, Kai Li, and Yun Fu. 2021. Domain Generalization via Feature Variation Decorrelation. In Proceedings of the 29th ACM Int'l Conf. on Multimedia. 1683--1691.

Digital Library

[23]

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. 2015. Learning transferable features with deep adaptation networks. In Int'l Conf. on Machine Learning. PMLR, 97--105.

[24]

Divyat Mahajan, Shruti Tople, and Amit Sharma. 2021. Domain generalization using causal matching. In Int'l Conf. on Machine Learning. PMLR, 7313--7324.

[25]

Toshihiko Matsuura and Tatsuya Harada. 2020. Domain generalization using a mixture of multiple latent domains. In Proceedings of the AAAI Conf. on Artificial Intelligence. 11749--11756.

[26]

Saeid Motiian, Marco Piccirilli, Donald A Adjeroh, and Gianfranco Doretto. 2017. Unified deep supervised domain adaptation and generalization. In Proceedings of the IEEE Int'l Conf. on Computer Vision. 5715--5725.

[27]

Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. 2019a. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF Int'l Conf. on Computer Vision. 1406--1415.

[28]

Xingchao Peng, Zijun Huang, Ximeng Sun, and Kate Saenko. 2019b. Domain agnostic learning with disentangled representations. In Int'l Conf. on Machine Learning. PMLR, 5102--5112.

[29]

Vihari Piratla, Praneeth Netrapalli, and Sunita Sarawagi. 2020. Efficient domain generalization via common-specific low-rank decomposition. In Int'l Conf. on Machine Learning. PMLR, 7728--7738.

[30]

Fengchun Qiao and Xi Peng. 2021. Uncertainty-guided model generalization to unseen domains. In Proceedings of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 6790--6800.

[31]

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conf. on computer vision and pattern recognition. 779--788.

[32]

Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. 2020. Distributionally robust neural networks. In Int'l Conf. on Learning Representations.

[33]

Shiv Shankar, Vihari Piratla, Soumen Chakrabarti, Siddhartha Chaudhuri, Preethi Jyothi, and Sunita Sarawagi. 2018a. Generalizing across domains via cross-gradient training. Int'l Conf. on Learning Representations (2018).

[34]

Shiv Shankar, Vihari Piratla, Soumen Chakrabarti, Siddhartha Chaudhuri, Preethi Jyothi, and Sunita Sarawagi. 2018b. Generalizing across domains via cross-gradient training. arXiv preprint arXiv:1804.10745 (2018).

[35]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[36]

Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. 2017. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition. 5018--5027.

[37]

Guoqing Wang, Hu Han, Shiguang Shan, and Xilin Chen. 2020a. Cross-domain face presentation attack detection via multi-domain disentangled representation learning. In Proceedings of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 6678--6687.

[38]

Hao Wang, Hao He, and Dina Katabi. 2020b. Continuously indexed domain adaptation. arXiv preprint arXiv:2007.01807 (2020).

[39]

Jindong Wang, Wenjie Feng, Yiqiang Chen, Han Yu, Meiyu Huang, and Philip S Yu. 2018. Visual domain adaptation with manifold embedded distribution alignment. In Proceedings of the 26th ACM Int'l Conf. on Multimedia. 402--410.

Digital Library

[40]

Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Wenjun Zeng, and Tao Qin. 2021. Generalizing to unseen domains: A survey on domain generalization. arXiv preprint arXiv:2103.03097 (2021).

[41]

Shujun Wang, Lequan Yu, Caizi Li, Chi-Wing Fu, and Pheng-Ann Heng. 2020c. Learning from extrinsic and intrinsic supervisions for domain generalization. In European Conf. on Computer Vision. Springer, 159--176.

Digital Library

[42]

Kun Wei, Cheng Deng, Xu Yang, et al. 2020. Lifelong Zero-Shot Learning. In Int'l Joint Conf. on Artificial Intelligence. 551--557.

[43]

Kun Wei, Muli Yang, Hao Wang, Cheng Deng, and Xianglong Liu. 2019. Adversarial fine-grained composition learning for unseen attribute-object recognition. In Proceedings of the IEEE/CVF Int'l Conf. on Computer Vision. 3741--3749.

[44]

Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. 2018. Cbam: Convolutional block attention module. In Proceedings of the European Conf. on Computer Vision. 3--19.

[45]

Jiyang Xie, Zhanyu Ma, Dongliang Chang, Guoqiang Zhang, and Jun Guo. 2021. GPCA: A probabilistic framework for gaussian process embedded channel attention. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).

[46]

Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian. 2021. A fourier-based framework for domain generalization. In Proceedings of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 14383--14392.

[47]

Yanchao Yang, Dong Lao, Ganesh Sundaramoorthi, and Stefano Soatto. 2020. Phase consistent ecological domain adaptation. In Proceedings of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 9011--9020.

[48]

Yanchao Yang and Stefano Soatto. 2020. Fda: Fourier domain adaptation for semantic segmentation. In Proceedings of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 4085--4095.

[49]

Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. 2020a. Deep domain-adversarial image generation for domain generalisation. In Proceedings of the AAAI Conf. on Artificial Intelligence. 13025--13032.

[50]

Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. 2020b. Learning to generate novel domains for domain generalization. In European Conf. on Computer Vision. Springer, 561--578.

Digital Library

[51]

Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. 2021. Domain generalization with mixstyle. arXiv preprint arXiv:2104.02008 (2021).

Cited By

Zhang ZXue WLiu QZhang KChen S(2024)Learnable Diffusion-Based Amplitude Feature Augmentation for Object Tracking in Intelligent VehiclesIEEE Transactions on Intelligent Vehicles10.1109/TIV.2023.33275019:4(4749-4758)Online publication date: Apr-2024
https://doi.org/10.1109/TIV.2023.3327501
Chen CLi BCai RZeng JHuang J(2024)Distortion Model-Based Spectral Augmentation for Generalized Recaptured Document DetectionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.333354819(1283-1298)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIFS.2023.3333548
Pan ZZhang WYu XZhang MGao Y(2024)Pseudo-set Frequency Refinement architecture for fine-grained few-shot class-incremental learningPattern Recognition10.1016/j.patcog.2024.110686155(110686)Online publication date: Nov-2024
https://doi.org/10.1016/j.patcog.2024.110686
Show More Cited By

Index Terms

Domain Generalization via Frequency-domain-based Feature Disentanglement and Interaction
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Image representations

Recommendations

Domain Generalization via Feature Variation Decorrelation
MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Domain generalization aims to learn a model that generalizes to unseen target domains from multiple source domains. Various approaches have been proposed to address this problem by adversarial learning, meta-learning, and data augmentation. However, ...
Read More
Feature Stylization and Domain-aware Contrastive Learning for Domain Generalization
MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Domain generalization aims to enhance the model robustness against domain shift without accessing the target domain. Since the available source domains for training are limited, recent approaches focus on generating samples of novel domains. ...
Read More
Category-Stitch Learning for Union Domain Generalization
Domain generalization aims at generalizing the network trained on multiple domains to unknown but related domains. Under the assumption that different domains share the same classes, previous works can build relationships across domains. However, in ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

October 2022

7537 pages

ISBN:9781450392037

DOI:10.1145/3503161

General Chairs:
João Magalhães
NOVA University of Lisbon, Portugal
,
Alberto del Bimbo
University of Florence, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Nicu Sebe
University of Trento, Italy
,
Program Chairs:
Xavier Alameda-Pineda
Inria, Grenoble, France
,
Qin Jin
Renmin University of China, China
,
Vincent Oria
New Jersey Institute of Technology, USA
,
Laura Toni
University College London, UK

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Beijing Natural Science Foundation Project
National Natural Science Foundation of China (NSFC)

Conference

MM '22

Sponsor:

SIGMM

MM '22: The 30th ACM International Conference on Multimedia

October 10 - 14, 2022

Lisboa, Portugal

Acceptance Rates

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
326
Total Downloads

Downloads (Last 12 months)124
Downloads (Last 6 weeks)21

Other Metrics

View Author Metrics

Citations

Cited By

Zhang ZXue WLiu QZhang KChen S(2024)Learnable Diffusion-Based Amplitude Feature Augmentation for Object Tracking in Intelligent VehiclesIEEE Transactions on Intelligent Vehicles10.1109/TIV.2023.33275019:4(4749-4758)Online publication date: Apr-2024
https://doi.org/10.1109/TIV.2023.3327501
Chen CLi BCai RZeng JHuang J(2024)Distortion Model-Based Spectral Augmentation for Generalized Recaptured Document DetectionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.333354819(1283-1298)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIFS.2023.3333548
Pan ZZhang WYu XZhang MGao Y(2024)Pseudo-set Frequency Refinement architecture for fine-grained few-shot class-incremental learningPattern Recognition10.1016/j.patcog.2024.110686155(110686)Online publication date: Nov-2024
https://doi.org/10.1016/j.patcog.2024.110686
Liu YZou YLi RLi Y(2024)Spectral Decomposition and Transformation for Cross-domain few-shot LearningNeural Networks10.1016/j.neunet.2024.106536(106536)Online publication date: Jul-2024
https://doi.org/10.1016/j.neunet.2024.106536
Sun WZhang XLu HChen YGe YHuang XYuan JChen YEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Resolve Domain Conflicts for Generalizable Remote Physiological MeasurementProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612265(8214-8224)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612265
Yu YZhao JHuang CZhao X(2023)Treat Noise as Domain Shift: Noise Feature Disentanglement for Underwater Perception and Maritime Surveys in Side-Scan Sonar ImagesIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2023.332278761(1-15)Online publication date: 2023
https://doi.org/10.1109/TGRS.2023.3322787
Chattopadhyay PSarangmath KVijaykumar VHoffman J(2023)Pasta: Proportional Amplitude Spectrum Training Augmentation for Syn-to-Real Domain Generalization2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.01767(19231-19243)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.01767
Hu CZhang CLee G(2023)Unsupervised Feature Representation Learning for Domain-generalized Cross-domain Image Retrieval2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.01011(10982-10991)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.01011
Guo JWang NQi LShi Y(2023)ALOFT: A Lightweight MLP-Like Architecture with Dynamic Low-Frequency Transform for Domain Generalization2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.02311(24132-24141)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.02311
Lee SBae JKim H(2023)Decompose, Adjust, Compose: Effective Normalization by Playing with Frequency for Domain Generalization2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01133(11776-11785)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01133
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents