Seven experiments examined the spatial reference systems used in memory to represent the locations of objects in the environment. Participants learned the locations of common objects in a room and then made judgments of relative direction using their memories of the layout (e.g., "Imagine you are standing at the shoe, facing the lamp; point to the clock"). The experiments manipulated the number of views that observers were allowed to experience, the presence or absence of local and global reference systems (e.g., a rectangular mat on which objects were placed and the walls of the room, respectively), and the congruence of local and global reference systems. Judgments of relative direction were more accurate for imagined headings parallel to study views than for imagined headings parallel to novel views, even with up to three study views. However, study views misaligned with salient reference systems in the environment were not strongly represented if they were experienced in the context of aligned views. Novel views aligned with a local reference system were, under certain conditions, easier to imagine than were novel views misaligned with the local reference system. We propose that learning and remembering the spatial structure of the surrounding environment involves interpreting the layout in terms of a spatial reference system. This reference system is imposed on the environment but defined by egocentric experience. Copyright 2001 Academic Press.