It's only considered derogatory because idiots tell us it is.
An excellent secretary is one of the most valuable employees an organization can have. "Secretary" is a real, meaningful function--most of the US cabinet members hold that title. Assistant, by definition, implies you're just there to help out 😃.