Let's explore what this line does if it breaks the meter!
A poet will use a formal meter not only because it is stylistically appropriate, but also because it communicates something to the reader. When the verse flows, that says something about the language. For example, in Romeo and Juliet, Romeo and Juliet's opening lines form a perfectly iambic sonnet. This communicates that from the first moment they meet, they are perfectly aligned.
What you are noticing here is called a pyrrhic foot. If an iamb is a typical set of beats (I'll indicate a long beat with a / and a short beat with a >):
> /
A pyrrhic foot is two unstressed beats:
> >
You won't find a pyrrhic foot as the star of the show; it doesn't really go anywhere. However, it can add speed to a line, or emphasize monotony.
Because this line is the second to last line of the stanza, one interpretation of this could be that Hardy is trying to emphasize how "fervorless" he feels. It also may be a commentary on how eerie it feels, out there in the cloudy-canopied crypt of the Century's corpse.
However, as the other stanzas largely scan in perfect iambic, he could also have been enjoying the words so much that he thought forcing a little "SPI-rit U-pon EARTH" wouldn't hurt anyone. I think that this is also relatively likely, considering that "Earth" is such a lovely syllable to stress at the end of a stanza, and it would be unusual to force the final syllable to be / >, rather than > /.
Basically, we can guess at the poet's intentions, but when it comes to poetry the real question is whether or not you can make an argument for what you interpret. I think you are doing a wonderful job of closely reading text and asking questions about the details that stand out to you. That, more that the specifics of metrical feet, shows that you are really getting into the poetry-reading and text-analyzing mindset!