Penrose Library was built in 1957 and was named after Stephen Penrose, the president of Whitman College between 1894 and 1934. Before its establishment, the college library was housed in numerous locations, including College Hall, the Memorial Building, an annex to the Memorial Building referred to as “the Library Shack,” and then Reynolds Hall.

Whitman’s initial libraries were founded in 1882, during the tenure of Alexander Anderson, the Whitman president from 1882 to 1891. Anderson wanted the college to have its own established library, asserting it was "an educating force.” With the combined efforts of Myron Eells, who donated ten volumes, fifteen pamphlets, and $25, either for the start of a library endowment or for immediate book purchases, and Elvira Cobleigh, the matron of Women's Hall at Whitman College, Whitman was able to develop and strengthen its first-ever libraries. Other notable contributors to the college’s initial library include Mark Twain, who donated $20 worth of books.

In 1957, the college finally decided to construct a building dedicated to the school library, naming it after the former Whitman president Penrose, who, along with his wife, Mary, requested a new library as a form of memorial for them. When it was first built, the college cancelled its classes so students could help move books into the new building. The library had further additions and renovations in 1974 and 1999, constructing the library entrance as we know it today.