Your proficiency, competence, and excellence as an executive are based far more on your work style than on your clothing style. If people look at your appearance on a casual day and somehow misperceive your role, stature, or status in the organization, that is their problem. It is important that you do not let it become your problem.
If you regard yourself as less significant or influential when you are casually attired, you may actually turn this self-perception into a self-fulfilling prophecy. When people believe that they are not effective, their performance tends to slip and fall into line with this expectation.
In recent years, there have been some studies that have noted the differences you mentioned in the way that males vs. females are perceived on casual dress days. This perception is clearly on the endangered stereotype list and should be extinct fairly soon. The irony is that when people do not dress casually on casual days, that can point to questionable confidence, excessive formality, and even indifference to the company culture.
At this point, your company has a policy of casual dress days, and you should take advantage of it if you want to. Your strength as a leader comes from within, and just because your attire is casual does not mean that your professionalism is casual too.
| 
