Hello and thank you for your question.
Unfortunately it appears that given you have paid a deposit a court would readily accept that a contract was formed, in which case you will not be allowed to simply demand a refund but must either complete the contract or pay the vendor damages for any loss which arises because you fail to complete (and you still would not be entitled to recover your deposit).
Accordingly, if they are now willing to let you out of your contract subject to your paying an amount to send the stock back then that is actually beyond what they are legally required to do. However, if they are really asking you to pay a further $1189 to cancel, when in fact you only owe a further $2,350 to complete the contract, you would have to think carefully as to whether or not such a proposal is worthwhile.
The fact they ordered before you gave them a deposit might have been relevant if you had maintained that you did not enter into a contract, but that is not a position now available to you, as once you paid a contract it would be difficult to convince a court that you had not agreed to enter into a contract. Once the contract was formed, it becomes binding on both parties and they then have the obligation to provide the goods ordered and you have the obligation to pay for those goods at the agreed price.
I appreciate this is not the answer you likely were hoping for but I trust it assists your understanding as to your legal obligations.
PLEASE RATE MY ANSWER.