Have you checked the fuel pressure when this is happening?
After it starts does it run rough?
From your description the fuel pump is not holding pressure. I suggest putting a fuel gauge on the engine and starting the engine then shut it down and watch the gauge and see if the pressure drops. It should not drop in 30 minutes if it does the pump needs replaced.
What I believe is happening is you loose pressure and then you have to crank it for some time for the pressure to come up before it will start.