The realization of a scalable architecture for quantum information processing is a major challenge of quantum science. A promising approach is based on emitters in nanostructures that are coupled by light. Here, we show that erbium dopants in high-purity silicon-on-insulator chips combine the ease of low-loss waveguide fabrication with < 1 GHz inhomogeneous broadening, strong optical transitions, and an outstanding optical coherence even at temperatures of 8 K, with an upper bound to the homogeneous linewidth of 20 kHz. Our study thus introduces a promising materials platform for the implementation of on-chip quantum memories, microwave-to-optical conversion, and distributed quantum information processing.