Fast and robust deep neural networks design

In the past few years, we have witnessed a rapid development of deep neural networks in computer vision, from basic image classiffcation tasks to some more advanced applications e.g. object detection and semantic segmentation. Inspire of its great success, there exists two challenges of deep neural networks real-world applications: its computational cost and vulnerability. Thus we are aimed to deal with these two problems in this thesis. To speed up deep networks, we propose a L1-Norm based low-rank approximation method to reduce oat operations based on the alternating direction method (ADM) in Chapter 2. Our experimental results on public datasets, including CIFAR-10 and ImageNet, demonstrate that this new decomposition scheme outperforms the recently developed L2-norm based nonlinear decomposition method. To defend against adversarial examples, we develop a novel pre-processing alogrithm based on image restoration to remove adversarial attack noise in Chapter 3. We detect high-sensitivity which have signiffcant contributions to the image classiffcation performance. Then we partition the image pixels into the two groups: high-sensitivity and low-sensitivity keypoints. For the low-sensitivity pixels, we use the existing total variation (TV) norm-based image smoothing. For the high-sensitivity pixels, we develop a structure-preserving low-rank image completion methods. Based on matrix analysis and optimization, we have derived an iterative solution for this optimization problem. This high-sensitivity points detection helps us to improve the defense against white-box attack BPDA. However, in our keypoints defense we only remove and recover a few part of pixels, which indicates there are still many perturbation over the whole image. In Chapter 4, we propose a novel image completion algorithm structure-preserving progressive lowrank image completion (SPLIC ) based on smoothed rank function (SRF) in which we can reconstruct a image with over 50 percent removed pixels. In SPLIC, we randomly remove over 50 percent pixels on the image and then do matrix completion by low-rank approximation to remain the global structure of the image. Differ from other lowrank methods, we replace nuclear norm by smoothed rank function (SRF) for its closer rank function approximation. We introduce total variance (TV) regularization to improve image reconstruction, and then combine total variance (TV) norm de-noising to further remove the perturbation over the whole image. Then we train the network on the SPLIC images. The experimental results show our SPLIC outperforms other pre-processing methods in image reconstruction, gray-box and black-box scenario.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.