This paper argues that the caste of memory is an unexplored phenomenon in memory studies. The paper outlines two possible epistemological categories to advance caste-based understanding of memory. The first category suggests that existing studies of memory can be grouped into Memory of Caste. The memory of caste as an epistemological category group is the work which explores the role of collective memory in consolidating the caste-based group identity. The second category is termed as Caste of Memory- this category lacks existing research- for which the paper contributes and argues that individual memory in the caste system’s context is not naïve or free of caste’s influence. In this endeavour, the work analyses lived experience of the Dalit individual as narrated in autobiography. The paper concludes that memory should not be considered caste neutral and should be accounted in the analysis of memory.