Justification involves assessing the risk / benefit ratio. The problem with imaging procedures is that no one doctor is in an ideal position to assess both sides of this equation.
The imaging specialist can best assess the risks for the individual patient, but the referring doctor - who knows the patient, the clinical context and the question to be addressed by the procedure - is best placed to assess the potential benefit as well as the risk of not performing the test.
In practical terms the major responsibility for justification falls on the referring doctor. It is however extremely important that the request is sufficiently detailed that the imaging specialist can determine the precise form of the test to be used.
Close consultation between the two doctors, in situations where there may be cause for discussion, is essential.