Isle of Palms is part of Jacksonville Beach which is located in Duval County. Duval County was formed in 1822 from part of St. Johns County, which was named after William Pope Duval, Territorial Governor of Florida from 1822 to 1834. In 1968 the government of Duval County merged with the government of the city of Jacksonville. The Duval County cities of Atlantic Beach, Baldwin, Jacksonville Beach and Neptune Beach are not included in the governmental limits of Jacksonville and maintain their own municipal governments.
The first European settlers in what is now Duval County were French Huguenots who started a colony called Fort Caroline in 1562. Spanish, English and the local Timucaua Indians fought for control of the area. The British took possession of the land in 1763 when Spain traded Florida for Havana. When the War for Independence ended, Britain returned Florida to Spain. American expansion and the War of 1812 caused Spain to relinquish Florida, along with Oregon, to the U.S. in exchange for Texas. The deal was struck in 1821 and settlers came to Florida by the thousands. The arrival of the railroad and wealthy tourists after the Civil War created growth for the area that continues today. Florida became a state in 1845.