Thank you for trusting your question to JA today. I am a licensed attorney with over a decade of law practice and over 20 years of experience in the legal field. I’m happy to be of assistance.
Yes, that can still be used against you. Your statements don't have to be sworn statements to be used against you in a court martial or criminal law setting.
When questions by government officials, you have the right to refuse to make a statement and that's fine. However, your refusal to make a statement at that time doesn't mean that anything you say didn't happen. It just means that they don't have an official sworn statement to use against you. This is actually a very big difference. If you said "the light was green" in a sworn statement and then later said it was "red" then a JAG would be able to impeach you, using your sworn statement against you. That would be a legal contradiction and the jury would be made aware of the fact that the first statement was sworn and they could assume that the contradiction was a lie.
None of that would be permitted when referring to a statement you made to others that was not sworn. They couldn't state "he swore an oath, very much like the one today" to strengthen the validity of that previous statement.
That doesn't mean that things you say to others aren't factually occurring events which can be testified to by others. In a court martial, you are a "party opponent" so anything that you say to other people is exempted from the hearsay rule, so anything that you have said to any person at all relating to this event, that person can testify to it and it's legally permissible.
If you have any further questions, please let me know. I invite follow up questions, so use REPLY for those. If you have no further questions then good luck going forward and please do not forget to rate my service with a top-three rating so that I receive credit for working with you today. Also, feel free to request me in the future, if you have questions concerning a different matter.