If it is determined that you are able to work substantially, then you would potentially lose your SSD. There are several ways that can happen. First, if you work more than minimal hours, as a self employed person, that can be a trigger. Second, regardless of hours, if you NET 1090 or more in month, for too many months, that will terminate your benefits. I suggest you look into the Trial Work Period program.
Also, if what you are doing in your work activities contradicts what the SSA found you couldn't do, that made you disabled, then of course that could cause you to be found no longer disabled (improved).
All in all, you want to be very careful but often working PT and earning minimally IS doable, if you document well what needs to be documented in case you get reviewed, and you pay sharp attention to the legal limits.
Remember too that once you hit your full retirement age (FRA), you are no longer on the disabled ranks, you are now merely retired, so the 1090 rule doesn't apply.