I have similar issue at my house. The issue is the outside connection to external circuit
Verizon has an issue with some of these external boxes having poor weather proofing. When its warm moisture gets in (internet is fine when that happens, but when it gets cold, moisture freezes). Since its fiber optic the ice crystals interrupt the connection. The faster the connection you have the lower fault tolerance your ont in your house will accept. I would recommend making verizon replace / weather proof the public circuit your house connects to.