What exactly do you mean by collaborative performance art? Performers can collaborate without the help of software. Do you mean software which interacts between the virtual and physical on stage? When I was messing with sort of stuff many years ago we used macromedia director to project footage and cue various devices such as lights, smoke machines etc. over a DMX interface.
A guy called Hal from Jim Henson Company was working on a system called Helix a few years ago which he used to do a virtual cabaret. Basically there would be a puppeteer offstage who was able to animate a computer generated character in real time through a physical input device. This character was able to interact with the audience spontaniously via a data projector.
I dont know if this hit the mark because your question is very vague and ambiguous.