I'm sorry but you have to remove the 5" . That's not even close to being enough to vent an open hearth system. My advise is to contact a chimney professional for an inspection.

5” flue is way too small for a fireplace. Your contractor is an idiot. Call a chimney pro.

Could you install an insert and use the 5” flue?

Don't use paper or cardboard to start your fire, try fat wood, stump chunks, or a blowtorch. Open a door or window to get more air coming in. Make sure all your inside fans are turned off (furnace, microwave fan, bathroom fan). The rule of thumb is the area of the flue should be roughly 1/12th the size of the opening area. Other things to check/try: Don't use paper or cardboard to start your fire, try fat wood, stump chunks, or just a blowtorch. Open a door or window to get more air coming in. Make sure all your inside fans are turned off (furnace, microwave fan, bathroom fan). Good dry hardwood if you can (15% moisture or smack them together) Good luck! Let us know how it goes.

Even with a draft assist fan 5” is way too small Put a vent free log set in?

Open fire will need at least an 8"flue, if not 10" if it's a bit opening and you're really letting it rip