For over 130 years, the Brown family lived in Liberty Hall. The first residents were Margaretta and John Brown and the last resident was Mary Mason Scott. Four generations of family members lived at the home. Women family members became leaders in their church, civic organizations, and historic preservation, while others became writers or artists. Several of the men in the family became lawyers and some had political careers, still others became farmers.
First Generation: John and Margaretta Brown
Portrait miniature of John Brown, attributed to Robert Field, watercolor on ivory, 1800. LHHS Collections.
John Brown (1757-1837) served Kentucky for over twenty years as a lawyer, congressman, and United States Senator. John grew up in Virginia, attended college at Princeton University (then the College of New Jersey) and graduated in 1780 from the College of William and Mary with a law degree. In 1782, he studied law with Thomas Jefferson. John Brown and Jefferson remained friends and corresponded throughout their lives. The two Virginians were also both Democratic-Republicans.
Shortly after moving west in 1784 he began representing the District of Kentucky in the Virginia Legislature. At this time, a movement amongst many Kentuckians to separate from Virginia was growing. Kentucky’s settlers felt the distance from Virginia’s capital, Richmond, was too great. They also felt that Virginia had failed to provide adequate protection from the conflicts with Native Americans. Kentuckians also wanted free navigation of the Mississippi River, an economic necessity, then controlled by Spain. John’s connections in the East and West proved advantageous in the statehood debate. His political allies in the East included Virginians James Madison and Thomas Jefferson. His membership in Danville’s Political Club introduced him to Kentucky’s early leaders. In 1792, after eight years and several petitions, Congress approved separation and Kentucky was admitted to the Union as the 15th state. Brown became one of the state’s first U.S. Senators, chosen by the newly formed Kentucky Legislature, he served for three terms from 1792-1805.
When John married Margaretta Mason he was 42 years old and she was 26. Soon after their marriage, they had their first son, Mason. In 1801, the family moved into Liberty Hall and their second son, Orlando was born. They had three more children, none of whom survived into adulthood: Alfred (1803-1804), Alfred (1804-1804), and Euphemia (1807-1814).
Portrait miniature of Margaretta Mason Brown by Pierre Henri, watercolor on ivory, 1800. LHHS Collections. Raised in a religious household, Margaretta Mason Brown (1772-1838) grew up in New York City. Her father, Reverend John Mason was a Presbyterian minister. She attended a school run by Isabella Graham, a noted educator. She was also taught by Graham’s daughter, Joanna Bethune, at Graham’s school.
Margaretta was an avid reader and writer. She began writing poetry at age thirteen and continued throughout her life. As an adult, Margaretta became a devoted letter-writer. Through correspondence with friends and family, she expressed strong political and religious views and demonstrated her knowledge of current events.
With John away in Washington for extended periods, Margaretta’s new life in Kentucky was challenging. In 1802, she wrote to John, “I do not feel as helpless as I thought I should—when people are obliged to depend upon themselves, they frequently find, that they possess resources of which they were ignorant…”
The Browns helped to establish a Sabbath school for boys in 1810 and Frankfort’s First Presbyterian church in 1816. In 1819, Margaretta became the first superintendent of a Sabbath school for girls and served as its treasurer. Margaretta was also a teacher at the Sunday school and early classes met in the home of fellow teacher and neighbor, Elizabeth Love, and at Liberty Hall. In 1824, Margaretta is thought to have written a dictionary of religious terms for her students entitled Food for Lambs or Familiar Explanations of some Religious Terms.
John and Margaretta Brown enslaved at least fourteen individuals who lived at Liberty Hall. To learn more about the enslaved at Liberty Hall and the Browns’ views on slavery, click here.
Second Generation: Mason and Orlando Brown (for Orlando Brown family history, click here.)
Portrait of Mason Brown by Chester Harding, oil on canvas, ca. 1840. LHHS Collections.
Mason Brown (1799-1867) was born in Philadelphia and grew up in Frankfort. Mason attended Yale University and graduated in 1820. After college, Mason followed in his father’s footsteps by beginning a law career. In 1823, he was practicing law in Maysville. Mason was a Kentucky Circuit Court Judge from 1839-1849. In 1849, he started a private law practice. His law partner was Charles S. Morehead with whom he had co-authored, A Digest of the Statute Laws of Kentucky (1834). When Morehead became Kentucky’s governor, Judge Mason Brown served as Secretary of State from 1855-1859.
Mason Brown, along with his brother Orlando, and seven other men were the first owners of the Frankfort Cemetery in 1844, the second incorporated cemetery in the U.S. Mason married Judith Ann Bledsoe of Lexington in 1825. In 1826, they had one son together named Benjamin Gratz Brown. The next year, Judith Ann died from a prolonged illness, described as “water in the lungs.” She may have had pneumonia. Mason was deeply despondent and Gratz was raised by his grandmother, Margaretta Brown. In a letter to Orlando, Margaretta wrote about Judith Ann’s death, “She has left little Gratz to me; he is a fine boy, and very healthy…” Mason married again in 1835 to Mary Yoder (1810-1881) of 
Cabinet card photograph of Mary Yoder Brown, ca. 1875. LHHS Collections. Spencer County. Margaretta Brown approved of the match and described Mary in a letter in both flattering and unflattering terms, “My heart overflows with gratitude to Providence for this union. Miss Mary Yoder though not decidedly handsome, is very agreeable both in feature and person, intelligent, wealthy, and above all remarkably pious…” The couple had six children: John Mason, Margaretta Mason, Mary Yoder, Yoder, Eliza Eloise, and Knox. When John Brown died, Mason inherited Liberty Hall, his childhood home.
Third Generation: Benjamin Gratz Brown, John Mason Brown, Mary Yoder Brown Scott, Margaretta Brown Barret & Knox Brown
Like his father and grandfather, Benjamin Gratz Brown (1826-1885) became a 
Portrait of Benjamin Gratz Brown by Oliver Frazer, oil on canvas, ca. 1845. LHHS Collections. lawyer. Brown’s family referred to him as “Gratz” not Benjamin. Gratz attended Transylvania University in Lexington and Yale University. After graduating from Yale in 1847, he moved back to Kentucky to practice law with his father and attend the Louisville Law School. In 1849 he moved to St. Louis, Missouri, and joined the law office of his cousins, the politically prominent Frank and Montgomery Blair. In the mid-1850s, Gratz was the editor of the "Missouri Democrat." While a member of the Missouri House of Representatives, Gratz spoke to that body in 1857 on the subject of gradual emancipation in Missouri. His position on slavery evolved from gradual emancipation to abolitionism and after the U.S. Civil War, he advocated for universal suffrage.
Gratz was a U.S. Senator from Missouri and the Governor of Missouri (1871-1873). He was an unsuccessful Democratic candidate for Vice-President with Horace Greely in 1872. He married Mary Gunn and they had 10 children, 8 lived to adulthood.
Another one of Mason Brown’s sons, John Mason Brown (1837-1890), also became a lawyer and lived in Louisville. Mason’s son, Knox Brown (1845-1939) became a farmer and for a time he lived and farmed on land previously owned by Mason in Owen County. In the late 19th century, Knox Brown grew tobacco for sale and corn for personal use. 
Mary Mason Scott with her mother Mary Yoder Brown Scott (on right) in the Liberty Hall Gardens, ca. 1905. LHHS Collections.The second oldest daughter of Mary Yoder and Mason Brown, Mary Yoder Brown Scott (1841-1916) inherited Liberty Hall after her mother’s death in 1881. In a letter to her sister-in-law Mary Owen Brown, Mary wrote, “There is no formal assignment of anything but the real estate after Mama’s death.” An arrangement between the siblings allowed Mary Yoder Brown Scott to take ownership of Liberty Hall as long as the furniture remained. By this time, Mary had lost her husband William T. Scott and her nine-year-old son, William Jr. Her two remaining children, Mary Mason Scott and John Matthew Scott lived with her at Liberty Hall.
Mary Yoder Brown Scott was married at Liberty Hall in 1865 and her wedding gown and invitation remain in the LHHS Collections. Mary Yoder Brown Scott traveled to Europe in 1908 with her niece Lillian Mason Brown and again in 1911 with her daughter Mary Mason Scott.
Mary was passionate about her garden and a newspaper article after her death stated that her “greatest pride was her flower garden in the yard of her home. Even in her late years Mrs. Scott worked among her flowers, and her garden was considered one of the best in Kentucky.” Mary seemed particularly proud of her roses. She wrote to her sister-in-law about them, “I wish you could see our roses today. The hedge is one mass of pink, and the ‘perpetuals’ are magnificient.”
She also kept a commonplace book about gardening that included planting and care instructions for various flowering plants and vegetables. See Garden History.
Margaretta Brown Barret (1839-1920), was the eldest of three daughters of Mason and Mary Yoder Brown. Margaretta was involved in preservation and civic activities. She became an early member of the National Society of The Colonial Dames of America (NSCDA) in Kentucky in 1908. The NSCDA is a patriotic organization committed to preserving the history of the United States through preservation, restoration and the interpretation of historic sites. Margaretta was also a Vice-Regent representing Kentucky in the Mount Vernon Ladies Association from 1902-1920. George Washington’s home Mt. Vernon and its surrounding grounds were purchased by the Mount Vernon Ladies Association in 1859 in order to preserve the decaying property. The Vice-Regents from each state were charged with raising awareness and funds, and even collecting donations of historic items to restore and furnish Mt. Vernon. For example, in 1903, Margaretta Barret spoke to women’s clubs in Kentucky and raised $200 to purchase two mules for Mount Vernon.
Margaretta was also a member of the Kentucky Equal Rights Association and she was the first President of the Woman’s Club of Frankfort. She also served as treasurer of the First Presbyterian Church in Frankfort, beginning in 1909.
Margaretta married William Francis Barret in 1871 and they had two boys together, Mason (1872-1918) and Frank Barret (1874-1902). William was a lawyer and the family lived in Louisville. William Barret died in 1882. Margaretta Barret outlived both of her children and moved back to Liberty Hall after her husband’s death.
Fourth Generation: Mary Mason Scott and Lillian Mason Brown
The daughter of Mary Yoder Brown and William T. Scott, Mary Mason "Mame" Scott (1867-1934) grew up at Liberty Hall and lived there until her death in 1934. After her mother’s death in 1916, Mame inherited Liberty Hall and was the last Brown family resident to live in the home. Mame, like her Aunt Margaretta Barret, served as the Vice-Regent from Kentucky in the Mount Vernon Ladies Association in the 1920s-1930s. Not only was she interested in the preservation of Mount Vernon, she also wanted to preserve Liberty Hall for future generations. Mame became a member of the National Society of The Colonial Dames of America (NSCDA) in Kentucky in 1921. During the last decade of her life Mame corresponded with the Colonial Dames in Kentucky in the hopes that the organization would preserve her family home after she and her brother John Matthew died. In a letter to the President of the NSCDA, Scott wrote, “We wish the house to be kept as it is. Knowing that if placed in your hands you will maintain it as it should be, it is our desire on our death to leave the grounds, the house and contents to the Colonial Dames.” Neither Mame nor her brother John Matthew married or had children to inherit Liberty Hall.
In her will, Mame left the home and its contents to her brother John Matthew Scott (1875-1956). She specified that in the event that her brother was not living at the time of her death, Liberty Hall should be left to the Society of the
John Matthew Scott, ca. 1902. LHHS Collections. Colonial Dames of America in the State of Kentucky. In 1937, John Matthew Scott sold Liberty Hall to a group of concerned citizens (which included Colonial Dames) who created Liberty Hall Association. It soon became Liberty Hall, Inc. and six Colonial Dames served on the board. The Colonial Dames of Kentucky began managing the property and opened it as a museum. In 2018, the NSCDA-KY who had been managing Liberty Hall without directly owning it took over ownership of the property. (The NSCDA-KY owned and operated the Orlando Brown House since 1955-see Orlando Brown family page).
Mame also sought to document Liberty Hall’s architectural and cultural importance. In a letter to Mame, Rexford Newcomb, a Professor of Architecture from the University of Illinois, wrote, “…by all means Liberty Hall should go into any representative book on Kentucky architecture and it certainly will not be brought into controversy.” Newcomb published a book, Old Kentucky Architecture, in 1940 that included Liberty Hall. Mame offered tours of Liberty Hall in the 1920s and communicated with librarians and art historians at the Frick Art Reference Library in New York, NY about portraits of her grandfather John Brown and his brothers.
Mary Mason Scott also was one of the first family members to see the Gray Lady Ghost of Liberty Hall. Click here to learn more.
Lillian Mason Brown (1859-1924) was Benjamin Gratz and Mary Gunn Brown’s oldest child. She became an artist and a high school art teacher in St. Louis, Missouri. Lillian represented Missouri on the Board of Lady Managers for the Chicago International Exposition of 1893. The board was tasked with figuring out how women would be involved in the fair, a Woman’s Building was ultimately part of the exposition where there was exhibition space for woman artists. Lillian never married. She traveled to Europe with Mary Yoder Brown Scott in 1908 and again in 1910 to attend art school. Tragically, Lillian died when she was struck by a car in St. Louis.
Fifth Generation: Margaret Wise Brown
Margaret Wise Brown by Consuelo Kanaga, ca. 1940. Brooklyn Museum. Margaret Wise Brown (1910-1952) was a great-granddaughter of Mason Brown and while she did not live at Liberty Hall or grow up in Kentucky, she is probably the most widely known Brown in the family. A children’s author, she wrote Goodnight Moon (1942) and The Runaway Bunny (1947). Margaret wrote over one hundred volumes in her brief life; she died unexpectedly of an embolism following a routine operation at the age of 42. She was a pioneer in her field. An early author of picture books, Margaret was one of the first to write for very young children.
Margaret was the daughter of Robert Bruce and Margaret Brown. Robert Bruce Brown was one of Benjamin Gratz Brown’s sons. While Margaret grew up on Long Island, New York, she had some interest in her Kentucky roots, even using the pen name, Kaintuck Brown. Margaret also refers to her Kentucky relatives in The Days Before Now: An Autobiographical Note by Margaret Wise Brown, 1994, adapted by Joan W. Blos:
“My mother and father came from Virginia and Kentucky and Missouri and I always heard of those places. Once my great-aunts from Kentucky who, I had been told, were giants-they were very tall and beautiful-arrived in our house for dinner and I came down in front of the fire to meet them. They were going across the ocean on a boat next morning.”
Margaret visited Liberty Hall in the late 1920s. Margaret had several romantic relationships and in 1952 became engaged to James Stillman Rockefeller Jr. She died later that same year.