Thank you for coming to JustAnswer. This is why they call it a trial separation so that the two of you can see if you want to stay in the marriage or divorce and it's also so that you can sort out your feelings and see if the love is still there and if it's worth fighting for. I would not get into a relationship with the other woman until you are completely honest with yourself and your wife that the marriage is over and you do not want to be with her anymore if you keep her clinging to hope then she will be having a false sense that you are coming back and if you indeed do not want to go back you have to tell her that and mean what you say. If you feel you have exhausted all avenues of trying to make the marriage work then you need to take the needed steps to be free you should never start another relationship when you have unresolved issues in your marriage that will only make things worse and give your wife the opportunity to verbally attack this woman as a home wrecker even though you were having issues before she came along.
Make sure that your children are okay that is the most important thing make sure you and your wife can be civil around them it's good that they are still young yet they won't understand what is going on so make sure you are there for them and see them on a regular basis. If you have decided to leave talk things out with your wife and do not tell her you have found someone else just tell her your feelings for her are no longer there and you would like a divorce. Give it some time to sink in don't serve her with divorce paper as soon as you tell her give her some time but not alot of time because then she may think well he didn't send me papers yet so there may still be hope. If the other woman is the right person for you she will wait until you are free.
Are you sure that the marriage in your eyes is over? Are you sure that you aren't letting your feelings for this other woman cloud your judgment about your marriage? if the answers to these questions are all yes then you have to take the steps to break free but still maintain a healthy relationship with your wife for the children's sake. If you answer no to any of these questions you may want to try a Marriage Counselor as a last resort they can help you and your wife to communicate better and work through your issues and figure out why you are arguing so much a counselor can also help you to remain cordial even if you do decide to divorce because your children are going to have to have you both in their lives and the support of both parents, make sure you try to keep their lives as normal as possible it will make the transition much easier.