The memories contained in each Memory Stone come from a certain character, which are based on the color of the stone. Purple for Mei Amanohokosaka, red for Kuon Glamred Stroheim, green for Hinata Himezuru and black for Touya Kagari.
Riddle prizes. There are four kinds: purple, red, green, and black. They each have a memory contained within them. The memories that are released can be observed by going to “Memories” in the “information” menu option. 
- XBlaze Lost: Memories, TIPS, #15