Evaluate on 224 x 224 RGB Image

[ck-amrahd]

Hi
I have my model trained on [3 x 224 x 224] images. But when I modify your code [for the parts to reshape into 3 x 224 x 224] and run on the model I get infidelity values around 1-2, I think that's in a good range. But I get a very high value of sensitivity [in range of thousands]. Am I missing something?

Thank you.
Regards,
Dharma KC

[chihkuanyeh]

Hi, you can check the infidelity and sensitivity numbers reported in our paper. For imagenet we increased the number of samples, and added a small diagonal matrix before inverse to ensure stability. These both improve the sensitivity to a stable range I think. I'll try to update our code for a larger image size.

[ck-amrahd]

Hi,
Ya, that would be great. I tried on [3x224x224] image and during sensitivity calculation, I saw you are calculating the max (../norm) and as the norm is around 10*-2, the quantity goes quite high if we loop around 10 times. That was giving me sensitivity in the range of 10k.

[chihkuanyeh]

Hi,
Ya, that would be great. I tried on [3x224x224] image and during sensitivity calculation, I saw you are calculating the max (../norm) and as the norm is around 10*-2, the quantity goes quite high if we loop around 10 times. That was giving me sensitivity in the range of 10k.

Hi, the main difference between MNIST and Imagenet is that it becomes a global explanation (see 2.5 in the paper). In the code, binary_I should be true in the function evaluate_infid_sen(), and also pert variable should be set to 'Square'. In this case, the norm for Imagenet data is around 3.5. I believe you might have used Gaussian perturbation for Imagenet data since your norm value is way too small.