Radio frequencies themselves would not be a limiting problem. Any frequency can be activated by the use of one of 50 CTCSS or 112 DCS tones. Thus, you'd have 162 unique addresses for one radio frequency. With many low lower and unlicensed frequncies available, you could easily have thousands of unique id's based on frequency and tone.
What you've just done is made a system that could be used with DC or DCC, since the receiver could pick up track power from either system. For DC systems, there is normally no voltage at the track when the train's stopped or nearly stopped. With DCC, it wouldn't matter, since the voltage is constant. So, for DC, you'd have to provide a battery system. Next, you need a receiver small enough and cheap enough to beat the price of a DCC decoder. A simple receiver with a dual IF stage and tone decode circutry, but no expensive items like speakers or keypads, could probably be be built for $3-$4 if mass produced in China. A tiny watch battery could provide the power needed, since the duty cycle would be something like 99% receive and 1% triggered. This does away with the need for track power pickup in DCC as well, which would be more expensive than using a battery.
You still have to solve the problem of what the receiver does when it's triggered. You'll need some sort of mechanical linkage to the coupler to make it open and close. An electomagnetic system might have some possibilities, since the couplers already work with electromagnets, but that's going to require some serious power. However, coming up with a way for your system to work with existing couplers is the only viable alternative. If you could get the price down to something like $10 a car, people might be willing to buy some, but they won't want to replace all their couplers on top of it.
I'll leave the details of that to you, since you have experience in robotics. Don't dismiss radio frequencies, however, since they may be be the best answer for a really universal automatic uncoupling system.
