Mod support for UT3 on PS3 is based on the PC toolset. There are no in-game mod tools that run on the PS3. However, this gives an incredible amount of freedom for people to make whatever they want (within reason, after all the PS3 has a limited amount of memory with no Virtual Memory).
Details here:
http://udn.epicgames.com/Three/PS3Mods.html