The objectives of the SAPQC are:
“To raise the standard of practice and the level of public understanding of statistics in New Zealand by:
- conducting independent appraisals of sample surveys, opinion polls and other statistical statements in relation to their statistical validity, and to the needs of the users of the survey results;
- conducting examinations of statements made in the public domain and of significant public interest, that have statistical content, or whose validity depends on statistical considerations.”
In making an appraisal, it is usual for the SAPQC to attempt to discuss the substantive issues with all parties involved. However in circumstances where the required appraisal is focused on a particular document already in the public arena, and where the number of groups and individuals involved or affected is large, this extensive procedure is not followed. Comments made in an appraisal are focused on statistical assessment.
The SAPQC does not have a set fee structure, and in certain circumstances no fee is charged. Generally, the SAPQC responds to a written request by producing a written appraisal. For the SAPQC to become involved, the issue on which an appraisal is sought must have a substantive public interest component. Fees are charged where the party making the request is likely to benefit financially from the results of the appraisal. Such fees have in the past been given to charity.
Current Members of the SAPQC are:
Thomas Lumley (Convenor)
How to Request an SAPQC Appraisal
An SAPQC appraisal may be requested by contacting the Convenor, Thomas Lumley, by email.
Background on the Survey Appraisal and Public Questions Committee of the New Zealand Statistical Association
The Survey Appraisals and Public Questions Committee (SAPQC) is a standing committee of the New Zealand Statistical Association (Inc). The Association was established in 1948, and the SAPQC has been in existence for about 20 years.
Appraisals undertaken since 1992:
Appraisal of the statistical basis of Valuation New Zealand’s valuation procedures – July 1992.
Appraisal of the methodology used by the Commerce Commission in its report on the Telecommunications Industry: the ‘Telecommunications Industry Inquiry Report’ – October 1992
Appraisal of Waitemata Electricity’s surveys to establish preferred ownership – February 1993.
Appraisal of MRL Research Group’s surveys to establish preferred ownership of Central Power, Wellington – May 1993.
Appraisal of a survey of public opinion by MRL Research conducted for Central Power before establishment as a public company – May 1993.
Assessment of eligibility for an appraisal from the New Zealand Lotteries Commission on the research report ‘Gambling and Problem Gambling in New Zealand’ – October 1993
Submission on the 1996 Census – especially on the need for a post-enumeration survey – October 93
Assessment of a Massey University Department of Marketing ‘environment’ survey – November 93
Preliminary assessment of a Christchurch City Council Enhancement Survey, requested by the Merivale Precinct Society – July 94.
The report for the New Zealand Council of Christian Social Services on the methodology for setting of benefit levels was made public. Of the SAPQC reports since 1992 this has received the most public interest. The central finding was that the data available for setting benefit levels is currently inadequate, and the Treasury methodology for assessing benefit levels is based on macroeconomic policy rather than on sound microdata.
Submission on Ministry of Health initiative “New Avenues for Crown Funded Social Science Research’ especially Proposal 4: ‘Establishment of a Social Science Research Clearing House’ – September 1994.
The appraisal of the Business and Economic Research (BERL) report on the accuracy of their assessment of the costs and benefits of an amendment to the Marine Transport Act to require safety inspections for foreign owned yachts was completed in June/July 1995. Information from foreign owned yachts indicated that the extra cost of search and rescue for boats without certain types of safety equipment needed to be balanced against the cost of foreign owned yachts boycotting New Zealand if New Zealand regulations were applied to foreign owned boats. The methodology used by BERL for its cost benefit analysis was extended beyond the available data, and in a number of conclusions their analysis overreached the limits imposed by this constraint. In particular there was no evidence that foreign owned yachts were more prone to accidents at sea. Indeed in terms of fatalities there was some evidence that the opposite was true. However BERL were aware that their conclusions were tentative and stated so in their report. More importantly however there was general agreement among all parties that ‘improved safety standards are desirable’.
The other appraisal conducted this year considered the second Evening Post survey on voter preference for candidates in the Wellington mayoral election 1995. The request for an appraisal was submitted by Ms Elizabeth Tennet. The survey was carried out by the Evening Post using Polytechnic students as telephone interviewers. The difference in support between Mr Blumsky and Ms Tennet at the time of the poll was 7%, and this difference (given the response rate of 56% and a sample size of 460) was only marginally significant. There were also a number of other technical objections raised by Ms Tennet which are covered in more detail in the appraisal. – December 1995
There were no appraisals carried out by the SAPQC during July 1996 to June 1997. There were however ongoing and regular requests for copies of past reports, particularly that on the statistical inadequacy of the New Zealand Treasury methodology for setting Social Welfare benefit levels. As in 1995/96, this public interest suggested that SAPQC appraisals should address major public issues involving statistics, rather than be assessments of ad hoc sample surveys.
Appraisal of ‘Towards a code of Social and Family Responsibility’ completed.
Assessment of a survey of mental health carried out as a legal requirement by North Health is currently in progress.
Appraisal of DSW fraud statistics.