Frequent supernova explosions in compact starburst regions are a main shaper of these regions' interstellar media (ISM). In most starbursts, the supernova remnants blast open a hot phase that fills the regions and launches a superwind. Denser starbursts are too overpressured for hot wind formation, but supernovae still stir up the ISM. I argue that supernovae power ubiquitous turbulence through each of the starburst ISM phases, including the hot wind, and that a fluctuation dynamo amplifies magnetic fields until they are in equipartition with the turbulence. Supernovae can drive turbulence with speeds of ~1000 km/s in the hot wind and ~20 km/s in the cold molecular gas, depending on the outer scale. I predict magnetic field strengths of 70 muG in the Galactic Center starburst, 300 muG in M82 and NGC 253, and 2 mG in Arp 220's nuclei. The mean magnetic field strengths are a few times stronger in molecular gas than in hot winds, but do not vary strongly with density within a starburst. I explain how the dominance of supernova-driven turbulence leads to near equipartition between the components of starburst ISM. I also consider implications for cosmic ray (CR) diffusion in starbursts. The high amounts of power cascading to small scales could confine CRs very effectively in starbursts, so much that CR transport is dominated by the flow of gas rather than diffusion through the gas. In addition, I discuss the effects of turbulence on X-ray line width, the far-infrared--radio correlation, observed radio polarization, and Faraday rotation measures. Finally, I discuss the many questions raised regarding the physics of turbulence in starbursts.