that does not make any sense. I won't get into details of what I think of CR again since I already did so in this thread and I am not taking sides, but if someone said "I had great grades in A class" would you respond "I guess he must have done great in B class"?

Think about it, if the same person is setting up the test criteria for both A and B can't he know more about A and so come up with a better basis to test A over B and so the final results will be better (closer to reality) for A rather than B? And if they are different people that are in charge can't it be even more so since not only do you have the individuals knowledge of the subject matter but also their competency at running the test.

This reminds of a situation many years ago. It was about audio (can't remember what codecs was used/discussed at the time) and some guy had done a test and compared a lossy compression to a lossless and said he could not tell the difference between the two. When I asked what audio he used to compare he said he said he wanted simple stuff so that he could hear the differences. But when comparing codecs that is the worst way to go, it is like saying "I want to know if more money makes a difference so I went into the dollar store, bought a single 1$ item with 5$ in my pocket and later went back with 100$ in my pocket to get that 1$ item and there was no difference between the two. A lossy codec is not there to mangle the audio but to keep the bitrate down, so going with something that can easily be compressed and has so little sound that it can easily be lowered to the given threshold without loss at that given bitrate might make the testers job easier but makes the test completely useless. What you want is complex sound that would force the encoder to try and compress I, any codec at any bitrate could do justice of complete silence.