Should I only be interested the saliency of my image/design?

This depends on the purpose of the image/display.

A work of art in a gallery will not work in the same in store display in a busy shop. We would recommend that you consider the context as well, and test the whole display layout if necessary.