OK. So I've made it work, but I'm definitely in the twilight zone.
I've confirmed that when I plug the RCP in, I get an entry in /dev called tty.usbmodem1411.
The first few times I tried, I got the same behaviour as I described in my original mail. I then shut down VMWare Fusion, which was running at the time (so I was getting the "do you want to connect this device to your Mac or your Windows VM message each time I plugged in).
The first time I tried after shutting down VMWare, as soon as I started up RaceCapture_App, the act of it scanning /dev caused my iPhone to try to initiate Bluetooth pairing

(yes, really). But it then went on to find the RCP and *claimed* to have successfully connected to it. But when I went to the dashboard, I couldn't see any data coming from the RCP.
I unplugged the RCP and shut down the app, plugged the RCP back in and started the app. This time, everything worked fine (no spurious bluetooth pairing, and I could see data from the RCP - yay!).
I then shut down the app and unplugged/reconnected the RCP. *This* time, starting the app caused my machine to lockup solid (mouse cursor not moving, keyboard unresponsive, hard reboot required).
Since the reboot, I've connected a couple of times, each time successfully.
So, it's working, but not exactly perfectly...
If there's anything I can dig out of logs that would help work out what's going on, let me know.