SPSS macro to calculate percent agreement for nominal level variables with 2 coders, with detailed printing

*----------------------------------------------------------------------------.

* This macro for SPSS for Windows calculates percent agreement for 2 content .

* analysis coders. In the process it prints the details of the calculations.

* (Another version without the detailed printout is available as well.) .

* If you have questions please contact Matthew Lombard at lombard@temple.edu.

*---------------------------------------------------------------------------.

*---------------------------------------------------------------------------.

* To invoke this macro, start the SPSS program, use the File menu to open a .

* new syntax window. Paste the macro into the new syntax file. Search for .

* the text "FOR UNIT NO.:" and following it type in the name of the variable .

* in your dataset that indicates the case or unit number.

* At the bottom of the file add the command to invoke the macro. Use this .

* format: .

* .

* macroname vara varb.

* .

* where macroname is the name of the macro (which follows the define command), .

* vara is the dataset variable that contains the values from the first coder, .

* and varb is the dataset variable that contains the values from the .

* second coder (an example has been included in this file).

* Multiple variables can be evaluated in a single use of the macro.

* .

* Use the "Run" menu option to have SPSS implement the macro.

*---------------------------------------------------------------------------.

*---------------------------------------------------------------------------.

*---------------------------------------------------------------------------.

* Define the macro and the variables to be used in the .

* macro, which are to be specified when the macro is invoked.

*---------------------------------------------------------------------------.

DEFINE AGREEP (!POSITIONAL !TOKENS(1)/

               !POSITIONAL !TOKENS(1)  )

*---------------------------------------------------------------------------.

* Save current SPSS environment settings, save a copy of the current data- .

* set file, and set default values for calculations that follow.

*---------------------------------------------------------------------------.

SAVE OUTFILE= 'PA_TMP1.SAV'.

EXECUTE.

COMPUTE BRKGRP = 1.

COMPUTE PAIR12 = 0.

*---------------------------------------------------------------------------.

* For each unit, set values for agreement to '9' (missing), if either value .

* is absent and change agreement to '1' (yes) if the values are the same.

*---------------------------------------------------------------------------.

DO IF (SYSMIS(!1) OR SYSMIS(!2)).

COMPUTE PAIR12= 9.

ELSE IF !1 = !2.

COMPUTE PAIR12= 1.

END IF.

*---------------------------------------------------------------------------.

* For each unit, count number of times a pair of coders agreed.

*---------------------------------------------------------------------------.

COUNT  SUBMATCH =

     PAIR12

        (1).

*---------------------------------------------------------------------------.

* For each unit, count number of times agreement can't be calculated .

* because one coder didn't code the variable.

*---------------------------------------------------------------------------.

COUNT  SUBMISS =

     PAIR12

        (9).

PRINT /

    / "*** FOR UNIT NO.: " unitno  "***"

    / "PAIR AGREES (1), DISAGREES (0), HAVE VALUE MISSING (9): " PAIR12 (N1).

*---------------------------------------------------------------------------.

* Compute and save in a new temporary file the total number of agreements .

* and not calculatable pairs across all units.

* .

* TOTMATCH is the sum across values of submatch for all cases.

* TOTMISS is the sum across values of submiss for all cases.

* UNITS is the number of units/cases.

*---------------------------------------------------------------------------.

AGGREGATE

  /OUTFILE='PA_TMP2.SAV'

  /PRESORTED

  /BREAK=BRKGRP

  /TOTMATCH=SUM(SUBMATCH)

  /TOTMISS=SUM(SUBMISS)

  /UNITS=N.

GET FILE 'PA_TMP2.SAV'.

PRINT /

  / "*********************************************"

  / "*********************************************"

  / " TOTAL AGREEMENTS ARE: " TOTMATCH

  / " TOTAL NOT CALCULATABLE PAIRS IS: " TOTMISS

  / " NO. OF UNITS: " UNITS (N3)

  / "*********************************************"

  / "*********************************************".

EXECUTE.

COMPUTE RELIAB = TOTMATCH / (UNITS - TOTMISS).

EXECUTE.

PRINT / "---->  PERCENT AGREEMENT FOR THIS VARIABLE IS: " RELIAB

  / .

EXECUTE.

*---------------------------------------------------------------------------.

* Load original data file saved at beginning of macro and erase temporary .

* file used for calculations.

*---------------------------------------------------------------------------.

GET FILE 'PA_TMP1.SAV'.

EXECUTE.

ERASE FILE 'PA_TMP2.SAV'.

EXECUTE.

!ENDDEFINE.