The neighborhood began in the mid-1800s as an unincorporated community adjacent to San Jose, California. By the 1860s the small unincorporated community needed its first school, and Willow Glen Elementary School was founded in 1863. Willow Glen continued as an unincorporated community until the 1920s, when the City of San Jose ordered the Southern Pacific Railroad to re-route the Southern Pacific Railroad trunk line which at that time was going down Fourth Street. The Southern Pacific then proposed to re-route down Lincoln Avenue. In order to forestall that attempt, Willow Glen was incorporated as a city in 1927. The railroad was instead re-routed to its current route through a then-unincorporated area now known as North Willow Glen, where its principal user is now Caltrain.
Lincoln Avenue was renamed from "Willow Glen Road" in 1865, shortly after the assassination of President Abraham Lincoln.