std and mean for image normalization different from ImageNet

[dimitry12]

torchvision model-zoo's image normalization is:

mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]

CLIP's is:

mean=[0.48145466, 0.4578275, 0.40821073], std=[0.26862954, 0.26130258, 0.27577711]

what's the story behind the difference? Are CLIP's normalization parameters re-calculated on WebImageText?

[jongwook]

Are CLIP's normalization parameters re-calculated on WebImageText?

Yes! Meanwhile, we empirically found that the model is quite robust to slightly different normalization constants

[dimitry12]

Thank you!

[meigaoms]

Are CLIP's normalization parameters re-calculated on WebImageText?

Yes! Meanwhile, we empirically found that the model is quite robust to slightly different normalization constants

Do you also use CLIP's normalization parameters for evaluation on public datasets? Thanks!

[jongwook]

@meigaoms Yes, we treat those normalization parameters as model-specific rather than dataset-specific.

[meigaoms]

@meigaoms Yes, we treat those normalization parameters as model-specific rather than dataset-specific.

Thanks for the prompt reply!
One more question for clarification. When you perform linear probe on other models, say ResNet50 in table 10, do you use CLIP normalization parameters? or other normalization parameters?

[jongwook]

That's correct; for ResNet{50,101,152} in Table 10, we used the torchvision implementation and followed their normalization constants which are at the top of this thread as well. Many other models taken from timm use the same numbers, while big_transfer and vision_transformer use 0.5s all the way.

PS - we identified an error in the STL-10 column of Table 10 and will update the corresponding table/figures shortly.

[pj-ms]

Hi, @jongwook, just wondering if those results for ViT coming from the models from timm as well? Is there a clear mapping (like vit-b/16 = timm vit_base_patch16_224) at hand that can be shared? Thanks!