In 1928 Rudy joined Gene Rodger's Revellers. By the end of the year he was playing with Cliff Jackson's Krazy Kats, where he remained a fixture until 1931. From 1934 to 1937 he played with Fats Waller's group. He also played in Fletcher Henderson's band, Cab Calloway and Jimmy Rushing's Kansas City Seven. Later in his career he played with Ray Charles.