The risk from oral sex is the lowest risk of any penetrative sex. It is difficult to get a precise risk because very few people engage only in oral sex, but the current estimate of average risk to a male that receives oral sex from an untreated HIV positive partner is about 0.005% per encounter, which also means that 99.995% of men do nto get HIV.
If the partner is of unknown HIV status, then your overall risk is lower, and the amount by which it is lower depends upon the prevalence of HIV in your community. In addition, there is increasing evidence that treatment of the infected person is excellent at reducing risk and preventing transmission. So, if an HIV positive individual is treated and is able to achieve an undetectable viral load, the risk is very close to zero. Obviously, if you do not know if he was HIV positive, you also would not know whether he is being treated or the success of that treatment, but that is why I asked the question about treatment.
As for whether you should be concerned, the reason that doctors usually prefer to educate individuals of the numeric risks is that some people are concerned at lower levels than others. Only you can decide whether you are concerned with a risk of 0.005%. I can tell you that the vast majority of men would not be concerned about this level of risk. I can also tell you that this is as low a risk as can be achieved for a male that is engaging in sex, as penile-vaginal sex and anal sex entails much higher risk. So, someone that is concerned at this level of risk would usually only be able to avoid concern by avoiding all sexual activity until they are in a mutually monogamous relationship.
If I can provide any additional information, please let me know.