So after some posts online at various forums, it seemed to be that my crashing issues were what I feared. My EVGA NVIDIA GTX 680 4GB was causing the crashes.
Unfortunately MacVidCards seems to not have had any stock since September 2016, so I wasn’t going to get a flashed replacement card.
So I started looking for a PC NVIDIA GTX 980 4 GB with 2 6 pin power supply. This was much easier said than done. There are still quite a few new 980’s out there, but most of them either have 2 8 pin, or 1 8 pin and 1 6 pin for their power supplies. Looks like with the 10 series out (even though those won’t work in a Mac since NVIDIA hasn’t bothered writing drivers for them) that the 6 pin 980 have become rather sparse.
Finally I was able to find a ZOTAC NVIDIA GTX 980 4GB AMP! Edition which had the requisite 2 6 pin adapters, and a bit of a speed increase over stock along with 3 fans instead of 2. I had never even heard of ZOTAC before, but most reviews seem like it is a pretty good company.
I got the card and installed it, no startup screen of course, but it runs great. And speed tests show it is about 100% faster than the 670 I had installed, and it works great, the crashes are gone. And Premiere Pro runs great!