Francis Fan Lee, former MIT EECS professor and interdisciplinary speech processing inventor, dies at 97.

Francis Fan Lee, a former professor of MIT’s Department of Electrical Engineering and Computer Science, died on Jan. 12, 2024. He was approximately 97 years old. Born in 1927 in Nanjing, China, to educators Prof. Li Rumian and Zhou Huizhan, Lee learned English from his father, a member of the English department’s faculty at the University of Wuhan. Lee’s mastery of the language led to an interpreter position at the US Office of Strategic Services, and eventually a passport and permission from the Chinese Government to study in the United States. Lee left China via steamship in 1948 to pursue his undergraduate education at MIT. He earned his Bachelor’s and his Master’s degree in electrical engineering in 1950 and 1951, respectively, before going into industry. Around this time, he became reacquainted with a friend he’d known in China, who had since emigrated; he married Teresa Jen Lee, and the two welcomed children Franklin, Elizabeth, Gloria, and Roberta over the next decade. During his ten-year industrial career, Lee distinguished himself in roles at Ultrasonic (where he worked on instrument type servomechanisms, circuit design, and a missile simulator), R.C.A. Camden (where he worked on an experimental time-shared digital processor for department store point-of-sale interactions), and Univac (where he held a variety of roles, culminating in a stint planning next-generation computing systems in Philadelphia.) 

Lee returned to MIT to earn his PhD in 1966, after which he joined the then-Department of Electrical Engineering as an Associate Professor with Tenure, affiliated with the Research Laboratory of Electronics (RLE). There, he pursued the subject of his doctoral research: the development of a machine that would read printed text, outloud–a tremendously ambitious and complex goal for the time. 

Work on the “RLE reading machine”, as it was called, was inherently interdisciplinary, and Lee drew upon the influences of multiple contemporaries, including linguists Morris Halle and Noam Chomsky, and engineer Kenneth Stevens (whose quantal theory of speech production and recognition broke down human speech into discrete, and limited, combinations of sound.) One of Lee’s greatest contributions to the machine, which he co-built with Donald Troxel, was a clever and efficient storage system which used root words, prefixes, and suffixes to make the real-time synthesis of half a million English words possible, while only requiring about 32,000 words’ worth of storage. The solution was emblematic of Lee’s creative approach to solving complex research problems, an approach which earned him respect and admiration from his colleagues and contemporaries.

In reflection of Lee’s remarkable accomplishments in both industry and building the reading machine, he was promoted to full Professor in 1969, a mere 3 years after he earned his PhD. Many awards and other recognitions followed, including the IEEE Fellowship in 1971 and the Audio Engineering Society Best Paper Award in 1972. Additionally, Lee occupied several important roles within the department, including over a decade spent as the Undergraduate Advisor. He consistently supported and advocated for more funding to go to ongoing professional education for faculty members, especially those who were no longer junior faculty, identifying ongoing development as an important, but often-overlooked, priority. 

Lee’s research work continued to straddle both novel inquiry and practical, commercial application–in 1969, together with Charles Bagnaschi, he founded American Data Sciences, later changing the company’s name to Lexicon, Inc. The company specialized in producing devices which expanded on Lee’s work in digital signal compression and expansion: for example, the first commercially available speech compressor and pitch shifter, which was marketed as an educational tool for blind students and those with speech processing disorders. The device, called Varispeech, allowed students to speed up written material without losing pitch–much as modern audiobook listeners speed up their chapters to absorb books at their preferred rate. Later innovations of Lee’s included the Time Compressor Model 1200, which added a film and video component to the speeding-up process, allowing television producers to subtly speed up a movie, sitcom, or advertisement to precisely fill a limited timeslot, without having to resort to making cuts. For this work, he received an Emmy Award for technical contributions to editing. 

In the mid-to-late 1980’s, Lee’s influential academic career was brought to a close by a series of deeply personal tragedies, including the 1984 murder of his daughter Roberta, and the subsequent and sudden deaths of his wife, Theresa, and his son, Franklin. Reeling from his losses, Lee ultimately decided to take an early retirement, dedicating his energy to healing. For the next two decades, he would explore the world extensively, a nomadic second chapter that included multiple road trips across the United States in a VW camper van. He eventually settled in California, where he met his last wife Ellen, and where his lively intellectual life persisted despite diagnoses of deafness and dementia; as his family recalled, he enjoyed playing games of Scrabble until his final weeks. He is survived by his wife Ellen Li; his daughters Elizabeth Lee (David Goya) and Gloria Lee (Matthew Lynaugh); his grandsons Alex, Benjamin, Mason and Sam; his sister Li Zhong (Lei Tongshen); and family friend Angelique Agbigay. His family have asked that gifts honoring Francis Fan Lee’s life be directed to the Hertz Foundation.