To study diet before and after diagnosis of breast and colorectal cancers compared with diet in cancer-free women in the Norwegian Women and Cancer study. This paper reports dietary changes from a data collection in 1996–1999 to another in 2002–2005. A total of 43,847 cancer-free women aged 41–70 years answered the baseline questionnaire on diet and lifestyle, 130 women developed colorectal cancer and 563 breast cancer. Dietary change in the three groups was compared, for breast cancer a comparison was made according to stage and time since diagnosis. Breast cancer survivors increased fruit and vegetable consumption with 81 g compared to 42 g in colorectal cancer survivors and 50 g in cancer-free women (p difference in change <0.001). Milk consumption decreased among cancer-free women, but not among colorectal cancer survivors (p = 0.007). Significantly more cancer survivors quit smoking (p < 0.001). There were no differences in change of alcohol consumption or BMI. In breast cancer survivors, differences increased with time since diagnosis, and stage II survivors made larger changes than stage I survivors. Cancer survivors showed little change toward cancer-preventive guidelines, although more advanced stage and being more than 2.4 years post diagnosis was associated with greater change in diet and smoking behaviors.