Hi and welcome to Just Answer!
Yes - you need to withhold federal and state income taxes for that person according to W4 form even if he/she is a nonresident alien on the student visa.
For withholding rules - see IRS publication 15 page 18 - http://www.irs.gov/pub/irs-pdf/p15.pdf
Nonresident alien employee's Form W-4. When completing Forms W-4, nonresident aliens are required to:
Not claim exemption from income tax withholding,
Request withholding as if they are single, regardless of their actual marital status,
Claim only one allowance (if the nonresident alien is a resident of Canada, Mexico, or Korea, he or she may claim more than one allowance), and
Write "Nonresident Alien" or "NRA" above the dotted line on line 6 of Form W-4.