Pruning Overview

Most neural networks are typically over-parameterized, with significant redundancy to achieve a certain accuracy. “Pruning” is the process of eliminating redundant weights while keeping the accuracy loss as low as possible.

Figure 1: Pruning Methods

The simplest form of pruning is called “fine-grained pruning” and results in sparse weight matrices. The Vitis AI pruner employs the “coarse-grained pruning” method, which eliminates neurons that do not contribute significantly to the network’s accuracy. For convolutional layers, “coarse-grained pruning” prunes the entire 3D kernel and so is also called channel pruning.

Pruning always reduces the accuracy of the original model. Retraining (finetuning) adjusts the remaining weights to recover accuracy.

Iterative Pruning

The Vitis AI pruner is designed to reduce the number of model parameters while minimizing the accuracy loss. This is done in an iterative process as shown in the following figure. Pruning results in accuracy loss and retraining recovers accuracy. Pruning, followed by retraining, forms one iteration. In the first iteration of pruning, the input model is the baseline model, and it is pruned and fine-tuned. In subsequent iterations, the fine-tuned model obtained from previous iterations is used to prune again. This process is usually repeated several times until a desired sparse model is obtained. A model cannot be pruned to a smaller size at once. Once too many parameters are removed from the model, the performance of the model is reduced and it is challenging to restore the model.

IMPORTANT: The reduction parameter is gradually increased in every iteration, to help better recover accuracy during the finetune stage.

Following the process of iterative pruning, higher pruning rates can be achieved without significant loss of model performance.

Figure 2: Iterative Process of Pruning

The four primary tasks in the Vitis AI pruner are as follows:

  1. Analysis (ana): Perform a sensitivity analysis on the model to determine the optimal pruning strategy.
  2. Pruning (prune): Reduce the number of computations in the input model.
  3. Fine-tuning (finetune): Retrain the pruned model to recover accuracy.
  4. Transformation (transform): Generate a dense model with reduced weights.

Follow these steps to prune a model. The steps are also shown in the following figure.

  1. Analyze the original baseline model.
  2. Prune the model.
  3. Finetune the pruned model.
  4. Repeat steps 2 and 3 several times.
  5. Transform the pruned sparse model to a final dense model.
Figure 3: Pruning Workflow

Guidelines for Better Pruning Results

The following is a list of suggestions for better pruning results, higher pruning rate, and smaller accuracy loss.

  1. Use as much data as possible to perform a model analysis. Ideally, you should use all the data in the validation dataset, which can be time consuming. You can also use partial validation set data, but you need to make sure at least half of the data set is used.
  2. During the finetuning stage, experiment with a few parameters, including the initial learning rate, the learning rate decay policy and use the best result as the input to the next round of pruning.
  3. The data used in fine-tuning should be the same as the data used to train the baseline.
  4. If the accuracy does not improve sufficiently after several finetuning experiments, try reducing the pruning rate and then re-run pruning and finetuning.