Perhaps the best way to interpret saying like this is to define the terms we may not know and them fir them back in to discern a word pattern.
"A golden mind" would be one that is highly valued, rare, and gifted. Dross refers to something that is of low value or held in low regard.
So this means that a mind that is gifted, and held in high esteem, someone who is very smart and wise does not not waste its time considering or becoming involved of things that are of low regard or low value.
