Project Cost Management is one of the 10 Knowledge Areas identified by the Guide to the PMBOK (Guide to the Project Management Body of Knowledge).
Past performance (of your company or others) gives a wealth of information about costs. Ask any home builder how much a typical middle-class, 3000-square-foot home would cost to build and they can give you a range of prices. Ask them how much it would cost to build an underground bunker and they might say that this is "more complicated than is the norm for our organization," and may even decline to build it. [note: This is why many contractors will not bid on government projects if they are of the firm-fixed-price type.]
So, you can use:
Analogous estimates (also called top-down estimates) that similar projects use -- and, use lots of expert judgement. For example, if your IT company has developed lots of projects using Java, but this one requires the use of C++, costs of purchasing software, retraining, and converting old algorithms could be estimated. A chart relating C++ to Java ranging from small project to large projects is quite helpful (even though it is an estimate). The first estimate is often called a "SWAG" (look it up) because it is not very accurate. Thus, a stepwise refinement development process, like the Evolutionary Spiral Model or the Agile Model, decreases the error and the risk.
Bottom-Up Estimates (also called Activity-Based Estimates) take the estimates for individual work items, or activities, and add them up to get a total. Even though the project may be more complicated, use new technology, be for a new customer, etc., there are ways to estimate number of people needed (thus, labor cost), list of tools needed (thus, equipment costs), etc. Collecting item cost data is very important if this method is to be successful.
It is good to make multiple iterations of top-down vs. bottom-up and to attempt get them close to each other.
Parametric estimates use project characteristics (parameters) in a mathematical model to estimate project costs. In the house example, the number of square feet is a parameter, the number of bedrooms is a parameter, the number of bathrooms is a parameter, etc. The more parameters you have, the better the cost estimate should be.
Note, again, that actual data must be fed back into any data base to keep parameters current.
It is also a best practice for companies to keep a data base of "actuals" for the purpose of planning. Often, constructing a prototype (or small proof-of-concept) model will be important -- even though this step costs extra, it usually saves money in the long run because potential risks are discovered and mitigated early in the life cycle. As a rule, I said, "We always, always build a prototype (made to throw away) before we perform the actual project." Are you aware of model construction (now with computers), wind tunnels, simulations, etc.?? These can be used to help estimate costs.