Whose memories does the Giver hold?

In Ch. 10, the Giver explains to Jonas what memories he holds, although it is difficult for Jonas to understand this. The Giver is the repository for the memories of a bygone world and its citizens. He holds all of the memories for everyone in all the world for all of time.

"My job is to transmit to you all of the memories I have within me. Memories of the past."

At first, Jonas thinks this means the Giver's own memories, and quickly the Giver realizes he is not adequately explaining himself.

"Im not being clear. It's not my past, not my childhood that I must transmit to you. It's the memories of the whole world. Before you, before me, before the previous Receiver, and generations before him."

Jonas is unaware that there is a "whole world" outside of the community in which he lives because he has never left the community and everywhere outside of it is simply referred to as "Elsewhere." So all of this is really hard for him to comprehend. Also, the idea of there being memories of the past generations, not just the Giver's individual childhood, is foreign to him as well. All of the individuals he knows, him included, only have memories of their own childhoods. They are not taught about the community's history prior to their own lives, as that information is not needed by individuals.