The role of tunneling for two proton-transfer steps in the reactions catalyzed by triosephosphate isomerase (TIM) has been studied. One step is the rate-limiting proton transfer from Calpha in the substrate to Glu 165, and the other is an intrasubstrate proton transfer proposed for the isomerization of the enediolate intermediate. The latter, which is not important in the wild-type enzyme but is a useful model system because of its simplicity, has also been examined in the gas phase and in solution. Variational transition-state theory with semiclassical ground-state tunneling was used for the calculation with potential energy surface determined by an AM1 method specifically parametrized for the TIM system. The effect of tunneling on the reaction rate was found to be less than a factor of 10 at room temperature; the tunneling becomes more important at lower temperature, as expected. The imaginary frequency (barrier) mode and modes that have large contributions to the reaction path curvature are localized on the atoms in the active site, within 4 A of the substrate. This suggests that only a small number of atoms that are close to the substrate and their motions (e.g., donor-acceptor vibration) directly determine the magnitude of tunneling. Atoms that are farther away influence the effect of tunneling indirectly by modulating the energetics of the proton transfer. For the intramolecular proton transfer, tunneling was found to be most important in the gas phase, to be similar in the enzyme, and to be the smallest in water. The major reason for this trend is that the barrier frequency is substantially lower in solution than in the gas phase and enzyme; the broader solution barrier is caused by the strong electrostatic interaction between the highly charged solute and the polar solvent molecules. Analysis of isotope effects showed that the conventional Arrenhius parameters are more useful as experimental criteria for determining the magnitude of tunneling than the widely used Swain-Schaad exponent (SSE). For the primary SSE, although values larger than the transition-state theory limit (3.3) occur when tunneling is included, there is no clear relationship between the calculated magnitudes of tunneling and the SSE. Also, the temperature dependence of the primary SSE is rather complex; the value of SSE tends to decrease as the temperature is lowered (i.e., when tunneling becomes more significant). For the secondary SSE, the results suggest that it is more relevant for evaluating the "coupled motion" between the secondary hydrogen and the reaction coordinate than the magnitude of tunneling. Although tunneling makes a significant contribution to the rate of proton transfer, it appears not to be a major aspect of the catalysis by TIM at room temperature; i.e., the tunneling factor of 10 is "small" relative to the overall rate acceleration by 10(9). For the intramolecular proton transfer, the tunneling in the enzyme is larger by a factor of 5 than in solution.
