The question is frequently asked how one can decide which of the best clocks available to us is really the very best. Because we assume that we have selected the best clocks available, there is nothing better to compare our clocks with. We seem to have uncovered a principal difficulty. However, there is an answer to this perplexing question. We only have to consider our basic principles:
Time is not only the most basic but also the most abstract measure which we use to bring order into nature. Because of its abstractness, and because we all think we know what it is we become easily confused. Therefore one has to develop clear and distinct ideas.
Time is a measure which we bring into nature according to our definitions. We postulate that the same interval of time elapsed if we can demonstrate that the same process has taken place. Now how do you establish identical processes and how do you decide your specific question of which of the best clocks is the very best? The answer is given in the literature concerning time keeping, especially in the documentation of our time computation algorithms. This literature is available on request.
The essence is this. The best clock is the one which shows the smallest residuals in its errors in reference to a time scale which is computed on the basis of a clock set.
And the second best clock is the one which shows the second smallest residuals in respect to the computed time scale which is statistically better than any contributing clock. In practice, however, you use only clocks for the set which have otherwise been shown to be acceptable. We do not use Mickey Mouse watches because they would not be cost effective. One would have to use zillions of them and it would not be practical! Everybody knows that Mickey Mouse watches agree very poorly among themselves whereas atomic clocks agree within nanoseconds (ns) from day to day. Of all the Mickey Mouse watches at most one of them can be right and most likely none is. The situation and reasoning is identical with the reasoning which we apply when we judge anything else. If we find substantial disagreement then probably nobody is right or at most one can be correct. Which one it is we can only find out gradually as more information becomes available. This is the scientific way which is not different from common sense, only more systematic. We do not arbitrarily assign credence to one clock on the basis of intuition.
Basically the answer is, therefore, that clocks can only be judged by the degree of conformity with other clocks. The best time measure is that which agrees with the consensus of a set of other processes. As you probably will agree, nothing else could be expected to be useful in science.
This is the basis of our time keeping: We have a group of nominally 24 standard clocks and we compute from hourly measures a best time scale which is then used to produce a table of corrections for each of the clocks. As you see, time keeping is intrinsically dependent on statistics and probability. If 10 clocks agree to within, let us say 10 ns, and one differs by 100 ns, then it is overwhelmingly unlikely that this one clock has been right and the other 10 wrong. It is the same procedure as it is used in a court of law. One interrogates and cross examines the witnesses who each will tell you a slightly different story. One or two may tell you a completely different story. The proceedings are designed to establish a core of facts which correspond with the consensus of the witnesses and that is accepted as "truth". Of course, what really happened may still be different but we have no other way to arrive at an acceptable definition of truth. And this consensus can be evaluated in terms of probability theory.
In science the procedure is exactly the same. We try to establish a consistent measure and consistent theories so that they are applicable in the largest possible domain. With clocks the ideal reference cannot be realized but only be approximated by finding the consensus of a large number of different standard processes. That is the reason why scientists are so interested in comparing clocks based on different principles. Up to now all such tests have revealed no surprises. It turns out that a time scale constructed on the basis of undisturbed standard processes in the form of our atomic clocks is superbly applicable in the description of other natural processes such as in astronomy (pulsars), or in modern technology, where one has to do the same thing independently at the same time but at different places. On the basis of these principles it has also been discovered in 1934 that our earth does not rotate uniformly but shows small seasonal variations in the length of the day.
But for more details I again refer you to the papers mentioned above.Gernot M. R. Winkler.