Constantly transferring schools since childhood, Kaoru Nishimi has always been seen as a social outcast when he transfers due to his intellectual nature and wealthy background. But after moving to Kyushu for his first year of high school, things begin to change when Kaoru befriends the class president Ritsuko Mukae and, unexpectedly, the notorious school delinquent Sentarou Kawabuchi. While Kaoru is an honors student, Sentarou is constantly getting into fights and skipping class. However, these two opposites find a common ground in one thing—music.

One day, Kaoru is invited by Ritsuko to visit her family's record store, where he finds Sentarou in the basement showcasing his skill in jazz drumming. Kaoru, being a classical pianist, knows nothing about jazz. But inspired by the performance, he begins to open his mind to the genre alongside Sentarou. The duo begin to practice in after-school jazz sessions in the record store's basement. As he spends more time improving his skills in jazz, Kaoru finds a person whom he can truly call his friend, and comes to learn the joys of playing music. Sakamichi no Apollon follows their story of friendship and love brought together by jazz.