Repositorio UVigo

Adam and the ants: on the influence of the optimization algorithm on the detectability of DNN watermarks

Investigo Repository

Adam and the ants: on the influence of the optimization algorithm on the detectability of DNN watermarks

Cortiñas Lorenzo, Betty; Perez Gonzalez, Fernando
 
DATE : 2020-12-06
UNIVERSAL IDENTIFIER : http://hdl.handle.net/11093/1899
UNESCO SUBJECT : 3325 Tecnología de las Telecomunicaciones ; 1203.17 Informática ; 2209.90 Tratamiento Digital Imágenes
DOCUMENT TYPE : article

ABSTRACT :

As training Deep Neural Networks (DNNs) becomes more expensive, the interest in protecting the ownership of the models with watermarking techniques increases. Uchida et al. proposed a digital watermarking algorithm that embeds the secret message into the model coefficients. However, despite its appeal, in this paper, we show that its efficacy can be compromised by the optimization algorithm being used. In particular, we found through a theoretical analysis that, as opposed to Stochastic Gradient Descent (SGD), the update direction given by Adam optimization strongly depends on the sign of a combination of columns of the projection matrix used for watermarking. Consequently, as observed in the empirical results, this makes the coefficients move in unison giving rise to heavily spiked weight distributions that can be easily detected by adversaries. As a way to solve this problem, we propose a new method called Block-Orthonormal Projections (BOP) that allows one to combine watermarking with Adam optimization with a minor impact on the detectability of the watermark and an increased ... [+]
As training Deep Neural Networks (DNNs) becomes more expensive, the interest in protecting the ownership of the models with watermarking techniques increases. Uchida et al. proposed a digital watermarking algorithm that embeds the secret message into the model coefficients. However, despite its appeal, in this paper, we show that its efficacy can be compromised by the optimization algorithm being used. In particular, we found through a theoretical analysis that, as opposed to Stochastic Gradient Descent (SGD), the update direction given by Adam optimization strongly depends on the sign of a combination of columns of the projection matrix used for watermarking. Consequently, as observed in the empirical results, this makes the coefficients move in unison giving rise to heavily spiked weight distributions that can be easily detected by adversaries. As a way to solve this problem, we propose a new method called Block-Orthonormal Projections (BOP) that allows one to combine watermarking with Adam optimization with a minor impact on the detectability of the watermark and an increased robustness. [-]

Show full item record



Files in this item

Creative Commons Attribution
(CC BY) license Except where otherwise noted, this item's license is described as Creative Commons Attribution (CC BY) license
2013 Universidade de Vigo, Todos los derechos reservados
Calidad So9001