Madison was the oldest of twelve children. He had seven brothers and four sisters, but only six of his siblings would live to adulthood. His father was a tobacco planter who grew up on a plantation called Mount Pleasant, which he had inherited from his father. The plantation was operated by approximately 100 enslaved persons, and was 5,000 acres in size, making Madison's father was the largest landowner in the Piedmont region. In those days land holdings was the measure of status. Madison's maternal grandfather was also a prominent planter and tobacco merchant. In the early 1760s, the Madison family moved into a newly built house, which they named Montpelier. It was where James Madison would breathe his last breath in 1836.
The family connections for the Madison family were significant. The clergyman who presided over his baptism was a relative. So was his namesake, Bishop James Madison, the President of the College of William and Mary. As a child, James Jr. was called Jemmy, to distinguish him from his father, James Sr. His father's influence as the largest landholder in the region and patriarch of the family, resulted in his holding the offices of Justice of the Peace, vestryman of the church, and commander of the county militia.
From age 11 to 16, Madison was tutored by Donald Robertson, a Scottish instructor who also tutored the children of a number of prominent planter families. Madison was taught mathematics, geography, and modern and classical languages. He was said to have excelled at Latin. When he turned 16, Madison returned to Montpelier, where he began a two-year course of study under the Reverend Thomas Martin, to prepare him for college. As a child he experienced a number of health problems, and this affected his choice of school. Most children of prominent Virginians at this time were sent to attend the College of William and Mary. But his father was worried that the lowland Williamsburg climate might be a home infectious disease of the day such as cholera or scarlet fever. Instead, Madison was sent to the College of New Jersey (now Princeton University) in 1769. There he studied Latin, Greek, theology, the works of the Enlightenment scholars, oratory and debate.
At Princeton, Madison became a leading member of the American Whig Society. His closest friend was future Attorney General William Bradford. Madison was able to complete Princeton's three-year bachelor of arts degree in just two years, graduating in 1771. Now at a crossroads in his career, Madison considered entering into either the clergy or the legal profession. Instead, he remained at Princeton to study Hebrew and political philosophy under President John Witherspoon before returning home to Montpelier in early 1772. Witherspoon had a profound influence on Madison's ideas on philosophy and morality. This was said to be where the seeds of Madison's passion for liberty were planted.
After returning to Montpelier, Madison tutored his younger siblings. Probably as a result of Witherspoon's influence, Madison appeared to have made his choice among prospective careers. He began to study law books on his own in 1773. He asked his Princeton friend William Bradford, who was now studying law under Edward Shippen in Philadelphia, to send him a written plan on reading law books. But by 1783, though he had studied a lot of law books, Madison still had not joined the bar or practiced law. He never did.
In 1765, Great Britain had passed the Stamp Act, and the British-American colonies of North America began to demand proper representation and independence. King George III declined to grant the British-American colonies representation in Parliament or independence. The British cited the high cost of aiding the colonists in their fight of the French and Indian War. By the early 1770s the relationship between the British-American colonies and Britain deteriorated over the issue of British taxation, and the American Revolutionary War began in 1775. The American colonists split between two factions, the Loyalists to King George III, and the Patriots. Madison joined the latter, and insisted that those in his county take a loyalty oath to their cause. Madison argued that Parliament had exceeded its authority by imposing taxation on the British-American colonies, without giving them representation in Parliament. He also favored de-establishing the Anglican Church in Virginia, arguing that an established religion was detrimental because it encouraged closed-mindedness and unquestioning obedience to the authority of the state.
In 1774, Madison, took a seat on the local Committee of Safety, a pro-revolution group that oversaw the local Patriot militia. In October 1775, he was commissioned as the colonel of the Orange County militia, serving as his father's second-in-command. This lasted until he was elected as a delegate to the Fifth Virginia Convention, which was charged with producing Virginia's first constitution. Madison was frequently in poor health, and he never saw battle in the Revolutionary War. But he was recognized for his service to his state and to his country as a wartime leader.
At the Virginia constitutional convention of 1776, Madison was able to convince delegates to alter the Virginia Declaration of Rights to provide for "equal entitlement," rather than mere "tolerance," in the exercise of religion. With the passage of the Virginia constitution, Madison was elected to the Virginia House of Delegates, and he was subsequently elected to the Virginia governor's Council of State. In that role, he worked closely with Governor Thomas Jefferson.
On July 4, 1776, the Declaration of Independence was published formally declaring 13 American states an independent nation, no longer under the Crown or British rule. Madison served on the Council of State from 1777 to 1779, when he was elected to the Second Continental Congress, the governing body of the United States. The Continental Congress faced many difficult challenges: war against Great Britain, the world's leading military power, runaway inflation, lack of revenue to finance the war, and lack of cooperation between the different levels of government. Madison soon gained expertise on financial issues, as well as a master of parliamentary procedure. He was frustrated by the failure of the states to supply needed requisitions, and he proposed to amend the Articles of Confederation to grant Congress the power to independently raise revenue through tariffs on foreign imports. General George Washington, Congressman Alexander Hamilton, and many other leading figures favored the amendment, it was defeated because it failed to win the ratification of all thirteen states.
Madison was an ardent supporter of a close alliance between the United States and France, and, as an advocate of westward expansion. He strongly believed that the United States had the right to navigation on the Mississippi River and control of all lands east of it. After serving Congress from 1780 to 1783, Madison won election to the Virginia House of Delegates in 1784.