I've tried to experiment with these new compressions:
converted matrix-dvd's video with Main Consept's encoder to AVD (about 5 days...)
converted ac-3 to 5.1aac-he with foobar2000
tried to embed them in one mp4-container with mp4UI and mpeg4ip tools, but
they hang when importing video.
Better ideas?
There is no code to put h.264 in mp4 nobody has done it yet. There is specs of how to do it but they have not been implemented therefore you will not be able to do this. Also, I don't even want to think about how long it took you to encode your matrix dvd to AVC but I would worry about the compatibility of those files since h.264 and in paricular this implementation is quite new. BTW mp4UI is a poor choice for making mp4 files anyway.
EDIT: In general 3ivx muxer in graphedit is the best choice to create mp4 but this will not work in this case.