Hello how are you this evening? It is extremely unlikely the problem is with the router itself, as you mention this router is working with other devices. Routers themselves all speak in the same "language" from the router to your computers/devices, so the brand of router should not matter. Are you sure your computers are in range of the router? What encryption type are you using? WEP, WPA2, etc? The most likely culprit is the distance to the router (bad signal), or incorrect passcode. Please give me more information as to what exactly Windows is saying when attempting to connect and we can further troubleshoot this issue.
It disconnects when connect with Ethernet? Both of them? Was this with the same network cable?
Are you able to get into the router? What network encryption are you using? Are there a lot of networks around you? It is very odd that it doesn't work when directly connected. Are these the same copies of Windows 8?
As far as your phone it should first use wi-fi then if that goes out it will attempt using the mobile signal. If your other devices are working fine I'd say it must either be the ethernet cord on the one computer, or the NIC itself. A new USB/PCI network card should only be about 10 dollars, and most come with a cord. This is if you buy it online, if you go to a store they'll try to get 30 from you.
OK. I do see what you said previously but not sure if you were working with someone else before. So it sounds like they are maintaining their connection to the wifi, but for some reason stop working on the internet after a few minutes. When you say they are "dropping" the connection, can you be more specific? Are you trying to use a browser and that is not working? And when you say it can occasionally connect, is it always connecting to the wifi, but occasionally works as it should, or is it not always connecting to the wifi whatsoever? Do you have any antivirus/firewall software installed on both of the Windows 8 machines?