Let's break this down for you!

SWG Proposal States:  “Thus, with 56% positive first reactions to the proposed solution, 64% likely to vote in favour of the levy, and ongoing input throughout the process, the SWG believes that the Community has demonstrated enough interest to recommend thatthe City conduct a formal vote on the levy proposal”

Let’s see how they figured out 56% and 64% out of 1039 responses our of 3300 homes for a (6 Million) 7 Million TAX Increase to buy the Golf Course!

 

How to Analysis Survey Results

5 Types of Bias in Data & Analytics

 

4 Types of Survey Bias

1. Confirmation bias

Occurs when the person performing the data analysis wants to prove a predetermined assumption. They then keep looking in the data until this assumption can be proven. E.g. by intentionally excluding particular variables from the analysis. This often occurs when data analysts are briefed in advance to support a particular conclusion. It is therefore advisable to not doggedly set out to prove a predefined conclusion, but rather to test presumed hypotheses in a targeted way.

2. Selection bias

This occurs when data is selected subjectively. As a result, the sample used is not a good reflection of the population. This error is often made in surveys. Frequently, there is also selection bias in customer panels: The customers that you (easily) find willing to participate in a customer panel are far from being “average customers”. This too can be done deliberately or unwittingly. Just look at opinion polls in elections: Can it really be true that so many voters completely change their mind on the last day, or is it more likely that the sample on which the poll is based is not a good reflection of all the voters? So you should always ask what sort of sample has been used for research. Avoid false extrapolation and make sure the results are applicable for the entire population.

3. Outliers

An outlier is an extreme data value. E.g. a customer with an age of 110 years. Or a consumer with $10 million in their savings account. You can spot outliers by inspecting the data closely, and particularly at the distribution of values. Values that are much higher, or much lower, than the region of almost all the other values. Outliers can make it a dangerous business to base a decision on the “average”. Just think: a customer with extreme spending habits can have a huge effect on the average profit per customer. If someone presents you with average values, you should check whether they have been corrected for outliers. For example, by basing the conclusions on the median – the middle value.

4. Overfitting en underfitting

Underfitting means when a model gives an oversimplistic picture of reality. Overfitting is the opposite: i.e. when the model is overcomplicated. Overfitting risks causing a certain assumption to be treated as the truth whereas in practice it is actually not the case. Always ask the data analyst what he or she has done to validate the model. If the analyst looks at you with a rather glazed expression, there is a good chance that the outcomes of the analysis have not been validated and therefore might not apply to the whole database of customers. Always ask the data analyst whether they have done a training or test sample. If the answer is no, it is highly likely that the outcomes of the analysis will not be applicable for all customers.

What does applying survey filters really mean?

“Responses were collected from 1368 individuals. After applying a filter to include only homeowners living in the Community and to only include completed surveys, the total number of responses used for analysis purposes was 1039 in a community of approximately 3300 households. The number of responses was more than large enough to ensure statistical validity.”

all information provided from SWG proposal page 4.

Survey Monkey talks about filters and how to rearrange the data you want.

The Power of Numbers

The First Survey Question: What is your first reaction to the proposed solution?  Very positive: 27%  Somewhat positive: 29% Neutral: 8% Somewhat negative: 15%  Very negative: 20%

27 %
Very Positive

Meaning the idea sounds good but where are the details of the contract.

280 responses out of 1039

29 %
Somewhat Positive

Somewhat positive: meaning not enough information to make a better decision.

301 responses out of 1039

43 %
Neutral, Negative

Doesn’t effect them or they did not like the proposal.

447 responses out of 1039

Power in the Numbers

The Second Survey Question:  If a vote on the levy was to be held today, how likely would you be to vote in favour of it?

  

29 %
Extremely likely

Yes based on the Levy estimated calculations in the presentation only and based on $6 million dollars.

 

19 %
Very likely: 19%

Based on the Levy estimated calculations in the presentation only and based on $6 million dollars however needed more solid information. 

53 %
NO to the Levy

NO based on the Levy estimated calculations in the presentation only and based on $6 million dollars.

Note: the somewhat likely can change to a NO based on the $7 million value.

What does this mean for the SWG Survey

1. Given there was no third party handling the survey and the members of the SWG & SCA  collected and filtered the responses can we assume any of the above 4 biases apply to this situation.

2. The Survey question was based on the presentation of $6 Million now that the amount has increase to $7M how would the results like change. Would the Very likely or Somewhat positive voters vote NO

3. Thus, with 56% positive first reactions to the proposed solution, 64% likely to vote in favour of the levy, and ongoing input throughout the process, the SWG believes that the Community has demonstrated enough interest to recommend that the City conduct a formal vote on the levy proposal

An the Survey Says....

SWG 56%

Very positive: 27%

Somewhat positive: 29%

 
 

Actual Results 45.5%

Very positive: 27%

Somewhat positive: 14.5%

Neutral: 4%

 

Actual Results 53.5%

Somewhat positive: 14.5%

Neutral: 4%

Somewhat negative: 15%

Very negative: 20%

 

64% would LIKELY vote

Extremely likely: 29%

Very likely: 19%

Somewhat likely: 16%

 

Actual results 46.5

Extremely likely: 29%

Very likely: 9.5%

Somewhat likely: 8%

 

The NO side 54.5%

Very likely: 9.5%

Somewhat likely: 8%

Not so likely: 12%

Not at all likely: 25%

We want to point out that actual survey results can be manipulated to arrive at many different outcomes. We also recognize that, at best, the balance between the yes and no sides are split equally.

One side can never assume 100% of the results for the question choices unless there are clear definitions and margins. Adding up amounts to reach an overall goal, as the SWG has done, is very misleading and reflects exceptional bias for their position.

To be objective we assume that the ‘very likely’ in full support without being provided additional information and not the same as ‘extremely likely’ respondents. Therefore, these results along with the ‘somewhat likely’ should be split 50% for both sides. In all cases, ‘neutral’ can be split equally for both sides since neither side can claim 100% of the result.

It is our conclusion that the survey does not provide a clear ‘extremely likely’ outcomes. If that out come was 51% on its own the survey would hold merit, but due to combinations of calculations we can also show a variety of conclusions.

As a result, this survey is no longer valid resulting from the SWG interpretation of the results and was conducted based upon a $6M purchase price, not on the recently revised $7M price.

It is highly probable if the survey was conducted again based on the new levy amount, the recent full disclosed information, and through an independent party the ‘very likely’ and ‘somewhat likely’ results would be lower.