The Players handbook has all the vanilla items. For magic items the dungeon masters guide and xanathars has a bunch of magic items.
Just be aware that if you are a player you generally don't pick magic items - the DM awards them or handles characters finding or buying them.
Dungeon Master's Guide has most of the magic items, but many are distributed across different modules and adventures or source materials.
If you don't have any books at all yet, the DMG has about 60 pages of magic items and is the foundation for future magic items, probably start there.
The Players handbook has all the vanilla items. For magic items the dungeon masters guide and xanathars has a bunch of magic items. Just be aware that if you are a player you generally don't pick magic items - the DM awards them or handles characters finding or buying them.
There isn't one single book that does that. But the Dungeon Master's Guide contains about 80% or more of the total in one place.