A covenant is an agreement or pact entered into by two parties, generally of equal standing, and typically confirmed by some kind of solemn ceremony like an oath or religious service. In the Bible, besides God's covenants, you can find covenants between men like the one between David and Jonathan. In the King James Version, the word covenant is also translated as the word testament. The Latin word for covenant is testamentum so easy to see how this could have been. A testament typically defines the will of a dying person. The early Bible codifiers divided the Bible into Old and New Testaments to show that the Christian truly inherits everything by the death of Christ and decree of God. Furthermore the word testament places the priority on God in the covenant relationship with man which is absolutely the case. Thus, the idea of a covenant goes right to the root of God's relationship with man and the division of the Bible.
The covenant of grace that God made for the benefit of man is first delineated in Genesis 3:14,15 although in its most cryptic and rudimentary form (the term protoevangelic is sometimes used). God first promises that there will be enmity between the seed of the woman and the seed of the Serpent. Man cannot be saved while he is a friend of the devil. God promises to give power and grace to fight the devil and the world as the "enmity" indicates. Finally, there is the promise of the triumph of God and His way. Christ would be bruised on the heel -- see other prophecy and Christ's direct fulfillment of this in John 13:18. It is the gospel in a nutshell but not fully known or revealed.
Genesis 3:14-15
14: And the LORD God said unto the serpent, Because thou hast done this, thou art cursed above all cattle, and above every beast of the field; upon thy belly shalt thou go, and dust shalt thou eat all the days of thy life: 15: And I will put enmity between thee and the woman, and between thy seed and her seed; it shall bruise thy head, and thou shalt bruise his heel.
Jeremiah 31:33
33: But this shall be the covenant that I will make with the house of Israel; After those days, saith the LORD, I will put my law in their inward parts, and write it in their hearts; and will be their God, and they shall be my people.
Jeremiah 32:38-41
38: And they shall be my people, and I will be their God: 39: And I will give them one heart, and one way, that they may fear me for ever, for the good of them, and of their children after them: 40: And I will make an everlasting covenant with them, that I will not turn away from them, to do them good; but I will put my fear in their hearts, that they shall not depart from me. 41: Yea, I will rejoice over them to do them good, and I will plant them in this land assuredly with my whole heart and with my whole soul.
Ezekial 34:23-5
23: And I will set up one shepherd over them, and he shall feed them, even my servant David; he shall feed them, and he shall be their shepherd. 24: And I the LORD will be their God, and my servant David a prince among them; I the LORD have spoken it. 25: And I will make with them a covenant of peace, and will cause the evil beasts to cease out of the land: and they shall dwell safely in the wilderness, and sleep in the woods.
Ezekial 36:25-28
25: Then will I sprinkle clean water upon you, and ye shall be clean: from all your filthiness, and from all your idols, will I cleanse you. 26: A new heart also will I give you, and a new spirit will I put within you: and I will take away the stony heart out of your flesh, and I will give you an heart of flesh. 27: And I will put my spirit within you, and cause you to walk in my statutes, and ye shall keep my judgments, and do them. 28: And ye shall dwell in the land that I gave to your fathers; and ye shall be my people, and I will be your God.
Ezekial 37:26-7
26: Moreover I will make a covenant of peace with them; it shall be an everlasting covenant with them: and I will place them, and multiply them, and will set my sanctuary in the midst of them for evermore. 27: My tabernacle also shall be with them: yea, I will be their God, and they shall be my people.
II Corinthians 6:16-18
16: And what agreement hath the temple of God with idols? for ye are the temple of the living God; as God hath said, I will dwell in them, and walk in them; and I will be their God, and they shall be my people. 17: Wherefore come out from among them, and be ye separate, saith the Lord, and touch not the unclean thing; and I will receive you, 18: And will be a Father unto you, and ye shall be my sons and daughters, saith the Lord Almighty.
Hebrews 8:10
10: For this is the covenant that I will make with the house of Israel after those days, saith the Lord; I will put my laws into their mind, and write them in their hearts: and I will be to them a God, and they shall be to me a people: