Prior to the American Revolution, the British considered the colonies eventually forming the United States and the eventual provinces of Canada to be one single entity - British North America. These colonies all had different charters for their governments, but the British did not see one particular colony and more important than the other.
There are lots of reasons the eventual Canadian provinces did not join with the Americans. First, think of the size. The British just finished beating the French in the French and Indian War. Quebec was their new province and just like today, it was massive and occupied bordered many of the New England states. Furthermore, Benedict Arnold's ill-fated campaign to take Quebec at the outset of the war did little more than give his soldiers smallpox.
The other barrier is natural rivalries of the colonies. During the American Revolution, Massachusetts was a hub of privateer activity and many raids took place off the coast of Nova Scotia. This would have been a natural colony to side with the Americans and indeed almost did. However, despite cultural familiarity - New Englanders were essentially imported to replace the French Acadians removed by the British - the privateer action horribly disrupted commercial activity at Halifax and earned New Englanders little sympathy in Nova Scotia.
The part of your question regarding New Brunswick is also interesting. New Brunswick wasn't established as a colony until 1784 - a year after Cornwallis surrendered at Yorktown. Prior to that, New Brunswick was split between Quebec and Nova Scotia. Nova Scotia also was in possession of the island today known as Prince Edward's Island. So, if history were to change around a little bit with the Nova Scotians having a greater predisposition to being a 14th colony, then the current United States' boundaries would extend all the way to the mouth of the St. Lawrence River.
Also, one last interesting linguistic point of interest is this: the island of Great Britain is divided into three specific regions: England, Scotland, and Wales. Prior to the American Revolution, the colonies northeast of New York were collectively known as New England, and Nova Scotia is Latin for New Scotland. This is an interesting way to see how the British replicated the geography of Europe with the geography of the new continent.
