Abstract
Contactless and efficient systems are implemented rapidly to advocate preventive methods in the fight against the COVID-19 pandemic. Despite the positive benefits of such systems, there is potential for exploitation by invading user privacy. In this work, we analyse the privacy invasiveness of face biometric systems by predicting privacy-sensitive soft-biometrics using masked face images. We train and apply a CNN based on the ResNet-50 architecture with 20,003 synthetic masked images and measure the privacy invasiveness. Despite the popular belief of the privacy benefits of wearing a mask among people, we show that there is no significant difference to privacy invasiveness when a mask is worn. In our experiments we were able to accurately predict sex (94.7%), race (83.1%) and age (MAE 6.21 and RMSE 8.33) from masked face images. Our proposed approach can serve as a baseline utility to evaluate the privacy-invasiveness of artificial intelligence systems that make use of privacy-sensitive information. We open-source all contributions for reproducibility and broader use by the research community.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahmed, M.A., Choudhury, R.D., Kashyap, K.: Race estimation with deep networks. J. King Saud Univ.-Comput. Inf. Sci. (2020)
Alonso-Fernandez, F., Diaz, K.H., Ramis, S., Perales, F.J., Bigun, J.: Soft-biometrics estimation in the era of facial masks. In: 2020 International Conference of the Biometrics Special Interest Group (BIOSIG), pp. 1–6 (2020)
Badawi, A.M., Mahfouz, M., Tadross, R., Jantz, R.: Fingerprint-based gender classification. IPCV 6, 41–46 (2006)
Boutros, F., et al.: MFR 2021: masked face recognition competition. In: 2021 IEEE International Joint Conference on Biometrics (IJCB), pp. 1–10. IEEE (2021)
Buckley, O., Nurse, J.R.: The language of biometrics: analysing public perceptions. J. Inf. Secur. Appl. 47, 112–119 (2019). https://doi.org/10.1016/j.jisa.2019.05.001
Cabani, A., Hammoudi, K., Benhabiles, H., Melkemi, M.: MaskedFace-Net - a dataset of correctly/incorrectly masked face images in the context of COVID-19. Smart Health 19(October 2020), 100144 (2021). https://doi.org/10.1016/j.smhl.2020.100144
Carlaw, S.: Impact on biometrics of COVID-19. Biom. Technol. Today 2020(4), 8–9 (2020). https://doi.org/10.1016/S0969-4765(20)30050-3. https://www.sciencedirect.com/science/article/pii/S0969476520300503
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations (2020)
Chen, X., Fan, H., Girshick, R., He, K.: Improved baselines with momentum contrastive learning (2020)
Childers, D.G., Wu, K.: Gender recognition from speech. PART II: fine analysis. J. Acoust. Soc. Am. 90(4), 1841–1856 (1991)
Cubuk, E.D., Zoph, B., Shlens, J., Le, Q.V.: Randaugment: practical automated data augmentation with a reduced search space (2019)
Damer, N., Grebe, J.H., Chen, C., Boutros, F., Kirchbuchner, F., Kuijper, A.: The effect of wearing a mask on face recognition performance: an exploratory study. In: BIOSIG 2020 - Proceedings of the 19th International Conference of the Biometrics Special Interest Group (2020)
Das, A., Dantcheva, A., Bremond, F.: Mitigating bias in gender, age and ethnicity classification: a multi-task convolution neural network approach. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops (2018)
Guo, G., Fu, Y., Huang, T.S., Dyer, C.R.: Locally adjusted robust regression for human age estimation. In: 2008 IEEE Workshop on Applications of Computer Vision, pp. 1–6. IEEE (2008)
Haggag, O., Haggag, S., Grundy, J., Abdelrazek, M.: COVID-19 vs social media apps: does privacy really matter? In: 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS), pp. 48–57 (2021). https://doi.org/10.1109/ICSE-SEIS52602.2021.00014
Hariri, W.: Efficient Masked Face Recognition Method during the COVID-19 Pandemic (2021). https://doi.org/10.21203/rs.3.rs-39289/v1. http://arxiv.org/abs/2105.03026
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning (2020)
Kim, J., Choi, J., Yi, J., Turk, M.: Effective representation using ICA for face recognition robust to local distortion and partial occlusion. IEEE Trans. Pattern Anal. Mach. Intell. 27(12), 1977–1981 (2005). https://doi.org/10.1109/TPAMI.2005.242
Ar, M.L.A., Nugraha, Y., Ernesto, A., Kanggrawan, J.I., Suherman, A.L.: A computer vision-based object detection and counting for COVID-19 protocol compliance: a case study of Jakarta. In: 2020 International Conference on ICT for Smart Society (ICISS), vol. CFP2013V-ART, pp. 1–5 (2020). https://doi.org/10.1109/ICISS50791.2020.9307594
Liu, Y.l., Yan, W., Hu, B.: Resistance to facial recognition payment in China: the influence of privacy-related factors. Telecommun. Policy 45(5), 102155 (2021). https://doi.org/10.1016/j.telpol.2021.102155
Loey, M., Manogaran, G., Taha, M.H.N., Khalifa, N.E.M.: A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic. Meas.: J. Int. Meas. Confed. 167(May 2020), 108288 (2021). https://doi.org/10.1016/j.measurement.2020.108288. https://doi.org/10.1016/j.measurement.2020.108288
Ngan, M.L., Grother, P.J., Hanaoka, K.K.: Ongoing face recognition vendor test (FRVT) part 6B: face recognition accuracy with face masks using post-COVID-19 algorithms (2020)
Noyes, E., Davis, J.P., Petrov, N., Gray, K.L., Ritchie, K.L.: The effect of face masks and sunglasses on identity and expression recognition with super-recognizers and typical observers. R. Soc. Open Sci. 8(3), 201169 (2021). https://doi.org/10.1098/rsos.201169
Rasnayaka, S., Sim, T.: Who wants continuous authentication on mobile devices? In: 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp. 1–9 (2018). https://doi.org/10.1109/BTAS.2018.8698599
Rasnayaka, S., Sim, T.: Your tattletale gait privacy invasiveness of IMU gait data. In: 2020 IEEE International Joint Conference on Biometrics (IJCB), pp. 1–10 (2020). https://doi.org/10.1109/IJCB48548.2020.9304922
Savchenko, A.V.: Efficient facial representations for age, gender and identity recognition in organizing photo albums using multi-output convnet. PeerJ Comput. Sci. 5, e197 (2019)
Seneviratne, S., Kasthuriaarachchi, N., Rasnayaka, S.: Multi-dataset benchmarks for masked identification using contrastive representation learning (2021)
Thomas, V., Chawla, N.V., Bowyer, K.W., Flynn, P.J.: Learning to predict gender from iris images. In: 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems, pp. 1–5. IEEE (2007)
Wang, Z., et al.: Masked Face Recognition Dataset and Application, pp. 1–3 (2020). http://arxiv.org/abs/2003.09093
Weng, R., Lu, J., Tan, Y.P.: Robust point set matching for partial face recognition. IEEE Trans. Image Process. 25(3), 1163–1176 (2016). https://doi.org/10.1109/TIP.2016.2515987. http://ieeexplore.ieee.org/document/7377089/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Seneviratne, S., Kasthuriarachchi, N., Rasnayaka, S., Hettiachchi, D., Shariffdeen, R. (2022). Does a Face Mask Protect My Privacy?: Deep Learning to Predict Protected Attributes from Masked Face Images. In: Long, G., Yu, X., Wang, S. (eds) AI 2021: Advances in Artificial Intelligence. AI 2022. Lecture Notes in Computer Science(), vol 13151. Springer, Cham. https://doi.org/10.1007/978-3-030-97546-3_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-97546-3_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-97545-6
Online ISBN: 978-3-030-97546-3
eBook Packages: Computer ScienceComputer Science (R0)