Short rates of interest are considered within the term structure model of Heath-Jarrow-Morton with deterministic volatility function driven by a Levy process. It is shown that they are Markovian if and only if the volatility function factorizes. This extends reseluts of Caverhill for the Wiener process and of Eberlein, Raible for Levy processes with a restricting property to the most general class of Levy processes being possible in this model. As new examples compound Poisson processes and bilateral gamma processes are included, in particular variance gamma processes in the sense of Madan and Madan&Seneta.