Well I'm a third-generation musician. My Grandfather's a musician and my father and mother were both musicians and so I'm a musician. It was just natural that I should be a musician 'cause I was born into the family.

He appeared on Blue Note recordings by Tony Williams Andrew Hill and Larry Young. The first of his Blue Note albums Fuchsia Swing Song (1964) adopts an approach sometimes called "inside-outside". It was located on Bond Street in Lower Manhattan and was originally opened as a public performance space as part of the first New York Musicians Festival in 1970.

Active in jazz since the early 1950s he earned wider attention during the mid-1960s spread of free jazz. Rivers was born in Enid Oklahoma. He performed on soprano and tenor saxophones bass clarinet flute harmonica and piano.