The most significant source of carbon in your compost heap will be plant material. Think of what goes into a compost bin - food scraps, leaves, some types of paper. All of this will contain carbon, because it comes from plants.
Does that make sense? Let me know if you need some more explanation :)