In this paper we consider the discretization in space and time of parabolic differential equations where we use the so-called space-time sparse grid technique. It employs the tensor product of a one-dimensional multilevel basis in time and a proper multilevel basis in space. This way, the additional order of complexity of a direct space-time discretization can be avoided, provided that the solution fulfills a certain smoothness assumption in space-time, namely that its mixed space-time derivatives are bounded. This holds in many applications due to the smoothing properties of the propagator of the parabolic PDE (heat kernel). In the more general case, the space-time sparse grid approach can be employed together with adaptive refinement in space and time and then leads to similar approximation rates as the non-adaptive method for smooth functions. We analyze the properties of different space-time sparse grid discretizations for parabolic differential equations from both, the theoretical and practical point of view, discuss their implementational aspects and report on the results of numerical experiments.