Mortgage Bankers

Mortgage Bankers are lenders that are large enough to originate loans and create pools of loans which they sell directly to Fannie Mae, Freddie Mac, Ginnie Mae, jumbo loan investors, and others. Any company that does this is considered to be a mortgage banker.

Some companies don't sell directly to those major investors, but sell their loans to the mortgage bankers. They often refer to themselves as mortgage bankers as well. Since they are actually engaging in the selling of loans, there is some justification for using this label. The point is that you cannot reliably determine the size or strength of a particular lender based on whether or not they identify themselves as a mortgage banker.

Portfolio lenders

An institution which is lending their own money and originating loans for itself is called a "portfolio lender." This is because they are lending for their own portfolio of loans and not worried about being able to immediately sell them on the secondary market. Because of this, they don't have to obey Fannie/Freddie guidelines and can create their own rules for determining credit worthiness. . Usually these institutions are larger banks and savings & loans.

Quite often only a portion of their loan programs are "portfolio" product. If they are offering fixed rate loans or government loans, they are certainly engaging in mortgage banking as well as portfolio lending.

Once a borrower has made the payments on a portfolio loan for over a year without any late payments, the loan is considered to be "seasoned." Once a loan has a track history of timely payments it becomes marketable, even if it does not meet Freddie/Fannie guidelines.

Selling these "seasoned" loans frees up more money for the "portfolio" lender to make more loans. If they are sold, they are packaged into pools and sold on the secondary market. You will probably not even realize your loan is sold because, quite likely, you will still make your loan payments to the same lender, which has now become your "servicer."

Direct Lenders

Lenders are considered to be direct lenders if they fund their own loans. A "direct lender" can range anywhere from the biggest lender to a very tiny one. Banks and savings & loans obviously have deposits they can use to fund loans with, but they usually use "warehouse lines of credit" from which they draw the money to fund the loans. Smaller institutions also have warehouse lines of credit from which they draw money to fund loans.

Direct lenders usually fit into the category of mortgage bankers or portfolio lenders, but not always.

One way you used to be able to distinguish a direct lender was from the fact that the loan documents were drawn up in their name, but this is no longer the case. Even the tiniest mortgage broker can make arrangements to fund loans in their own name nowadays.

Correspondents

Correspondent is usually a term that refers to a company which originates and closes home loans in their own name, then sells them individually to a larger lender, called a sponsor. The sponsor acts as the mortgage banker, re-selling the loan to Ginnie Mae, Fannie Mae, or Freddie Mac as part of a pool.
The correspondent may fund the loans themselves or funding may take place from the larger company. Either way, the loan is usually underwritten by the sponsor.

It is almost like being a mortgage broker, except that there is usually a very strong relationship between the correspondent and their sponsor.

Mortgage Brokers

Mortgage Brokers are companies that originate loans with the intention of brokering them to lending institutions. A broker has established relationships with these companies. Underwriting and funding takes place at the larger institutions. Many mortgage brokers are also correspondents.

Mortgage brokers deal with lending institutions that have a wholesale loan department.

Wholesale Lenders

Most mortgage bankers and portfolio lenders also act as wholesale lenders, catering to mortgage brokers for loan origination. Some wholesale lenders do not even have their own retail branches, relying solely on mortgage brokers for their loans.
These wholesale divisions offer loans to mortgage brokers at a lower cost than their retail branches offer them to the general public. The mortgage broker then adds on his fee. The result for the borrower is that the loan costs about the same as if he obtained a loan directly from a retail branch of the wholesale lender.

Banks and Savings & Loans - Banks and savings & loans usually operate as portfolio lenders, mortgage bankers, or some combination of both.

Credit Unions - Credit Unions usually seem to operate as correspondents, although a large one could act as a portfolio lender or a mortgage banker.