ACT!




ACT-NET was funded from the European Commission.
This communication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.
Project Number:
142207-LLP-1-2008-DE-GRUNDTVIG-GMP

Validity

How can we assure the validity of the results when applying the ACT approach?

The term “validity” indicates the extent to which a measure accurately reflects the concept that it is intended to measure. Sometimes different actors have a different understanding of the issues and the concepts that are going to be assessed. Validity is about that.

Basically it comes down to two questions:

  • Are we all clear about what we want to know?
  • Is what we are measuring indeed what we want/need to know to get the right answers?

In order to get good and shared definitions of concepts it is important to set up a discussion with all actors involved, asking the right questions: what is important and why. Concepts and interpretations should however also be linked to a broader theoretical frame, otherwise it will be impossible to compare results and transfer of experience to other sectors. Instruments made or chosen and the debates held about the data, will always have to be looked at critically, asking the central question over and over again: Is this what we think the assessment is about? Is this showing us what we want to know?

All the elements in the assessment plan should also show a great coherence and internal consistency. Basically the questions to which one has to pay attention are:

  • Is what we are doing still according to the goals of the evaluation?
  • Is the involvement of everybody still doing justice to our intention to involve various parties?
  • Is our process serving the right target groups?

Not only the assessment method should consider the criteria “validity”, but also the reference system should be a valid system.

  • Are the concepts used as stages clear for all actors involved?
  • Do they relate to a broadly recognized reference system?

A possible way to assure validity of the reference system could be to compare the defined stages with statements of experts in the respective field. The stages could also be built on existing level models (e.g. for cultural competence). When it comes to evidencing it is also important that the indicators, giving proof that a certain level has been reached, are commonly understood and shared.