OT: Surveys

This is interesting:

"So far, there are no definitive conclusions on where thing went wrong

a key one being political correctness. In short, people lie to the pollsters because they are ashamed to admit their true intentions."

I bet people lie on surveys, too.

--

John Larkin         Highland Technology, Inc 
picosecond timing   laser drivers and controllers 

jlarkin att highlandtechnology dott com 
http://www.highlandtechnology.com

Managers want useful info to make product choices. Survey dept creates a survey that, due to basic incompetence will collect a lot of wrong data. Telemarketers, when in the loop, couldnt care less and just want it filled in regardless. End users are too often neither sensible or honest. They also cease caring once they see how senseless the survey is.

End result? Mostly garbage.

NT

I would tell political polsters that I would be happy to take their poll, for $100/hr (one hour minimum). The droid always asks why they should pay me. I reply "why should I answer your questions? My information is worth money, or you wouldn't be bothering me for it". "Click"

Sometimes I just make up the silliest, but not totally implausable, response to the question possible. They're wasting my time, I might just as well return the favor.

AND PEOPLE LIE TO THEIR VENDORS!! (OhMiGosh!) Even "trusted" relationships!

Ever hear of a guy "bending the truth" to get a date into the sack? I'm sure she's quizzing him to decide if he's a "good (long/short-term) investment" and he's deliberately misleading her.

We've been looking at cars. Salespeople want to quiz you to get a feel for how serious you are, timeframe, budget, how you'll be paying, what features you're interested in, what problem areas they'd want to downplay, which competing products you've considered or are considering, which you've taken enough interest in to actually test drive, etc. Even though it's conducted face-to-face and doesn't involve filling out any "forms", you can bet your ass this is a survey. A highly targeted and *refined* one, at that! Salesman is 18" away from me, our eyes locked and I have no qualms lying to his face. I can downplay a keen interest on my part; or, convince him that I'm ready to lay down cash for this Range Rover ("Does it come in red?"). The only time I have to provide an ACCURATE answer is when I sign my name to my check!

We had a Japanese firm that we'd been involved in talks re: a joint partnership (early 80's). They sent a delegation over to tour our facilities, meet our staff, etc. Took qty ~10 of one of our products home with them (small sale -- maybe $25K -- but it "paid for their meals" while we were entertaining them!). Then, silence. Until next industry showcase -- where *our* product was on display in THEIR booth! "No, wait! That's not the XJL4000. The casing is different. And look at the changes to the mechanism... Crap! The 10 units were just 10 samples that they could reverse engineer!!" :<

But, they had been TRUTHFUL about admiring the product and wanting to have it in the SE Asian market. They just weren't truthful about how they were going to get to that point!

In the 80's a very large (now ~$20B), respectable customer purchased a control system from us. Sizeable sale -- ~$1M when you include installation labor, etc. We, of course, dutifully supplied all the wiring diagrams for their plant, tuned all the controllers for their process, educated their staff, etc. You don't sell many of these -- perhaps 2-6 per year per customer -- but they can be a nice chunk of your sales dollars and keep a lot of people employed (so they can address *other* sales!). Customer was talking about exactly these sorts of numbers and was excited to bring the technology onboard!

But, the followup sales never materialized. Despite their *glee* with the system's performance! Soon discovered they had simply cloned the entire system as it was built from COTS components. And, well documented.

What recourse do you have? They're 1000X your size. Spend more on their LEGAL STAFF than you spend on your entire payroll! And, in retrospect, they never claimed they were going to BUY those systems; just that they had a NEED for them! So, you grin and bear it.

Yet, they were truthful about their needs. They didn't send us off designing a five-wheeled motorcycle. So, what we learned about the market's needs was valid; just that particular *slice* of the market wasn't available to us!

I "surveyed" the sales staff at one firm prior to preparing a new product proposal. trying to understand what features were essential, desirable, "optional", etc. And, put approximate dollar values on each -- I don't want to add $X recurring to a design if it will only command $Y premium (Y < X) in sales price! OTOH, if the market *insists* on that feature being available, I now am highly motivated to drive that recurring cost to zero - or, change the discussion to one where it adds some *other* capability that increases it's overall worth.

I spent a *lot* of time talking to staff, customers, "knowledgeable insiders", etc. I'm surveying the market in order to decide *where* to invest (considerable) development dollars. I seek DATA on which to base a DECISION.

When I made my "pitch" to top management (with most of the stake holders present), the sales staff immediately balked because I had elided many of their "requirements": "You've GOT to have feature Q!! We'll never be able to sell it without feature M!!"

"In the past 20 years, you've sold NO units with feature Q; exactly ONE unit with feature M." Silence. Until the CEO -- eyes fixed on mine and the stack of 20 year's worth of old sales orders that he could see protruding from my file -- quietly said, "Yeah, and I bet I know where

*that* unit went... it's probably sitting on a shelf, there, as we speak!"

Did the sales staff LIE to me when I surveyed them? I sincerely doubt it. Rather, they just didn't have their facts straight. Instead of SOLELY relying on their *impressions*, I dug up the old records to have real data to factor into my decisions.

The store mentioned in my OP will obviously *not* rely on those survey results, exclusively. I'm sure they are also looking at the weekly sales volumes at each of the stores in question. And, long term plans. As well as the potential resale value for each of those properties. And whether a NEW competitor might purchase one of those locations and further harm their bottom line (a hospital, here, purchased a competing facility; remodeled it to make it unsuitable for use as a hospital; then sold it off)

Survey takers are aware that results don't accurately map to reality. But, that doesn't invalidate the benefits of taking surveys! You have to know the right questions to ask, the proper "motivation" to get "predictable" accuracy from the results and have access to the right

*population*!

There are a lot of specialty foodstuffs that I enjoy. Most are hard to find (not high demand, etc.). I was speaking with a "deli" owner (for want of a better word... they don't really have deli's here) about a particular imported cheese that I consume in large quantities. I have it flown in periodically -- several hundred dollars at a time.

He saw this as a great opportunity to bring some business in. So, *he* located the cheese from one of his suppliers and brought in a sizeable quantity. When next I saw him, he was beaming -- eager to show me what he had purchased and was now hoping to sell to me. He knew what I was paying -- not counting my shipping costs -- and had gauged that he could make a fair profit coming in BELOW that price level! (i.e., "Don will JUMP at the chance to have a local source so he doesn't have to pay those shipping charges!")

"But, this isn't the *grating* form of the cheese! The stuff you've purchased is for *slicing* or shredding. Totally useless for grating. The grating form is aged much longer -- which is why it costs *so* much more (than what you were hoping to sell this stuff!)." His data collection process was flawed -- and he had to "eat" his investment! :>

Just like our (above) failures to accurately understand the intents of our business partner, customer, etc.

I instrument my software to collect the data that the *user* vaguely remembers concerning the quality of the user experience. I can *ask* the user about the effectiveness of the user interface -- how intuitive or clumsy it may be, etc. But, his response will be highly subjective and will vary with his mood, his level of focus on the question, etc.

OTOH, if I *measure* the number of "errors" that the software has reported; and the number of times that the user had to "cancel/undo" an action; and, judiciously deploy black boxes to track the details of those interactions, then I can take his reported experience and confirm or deny it with hard data that I've objectively collected. He may *claim* the speech recognizer "is always making mistakes"... but, if the log shows he's never canceled a (presumably misrecognized) command and issued a replacement, then the observations don't agree with his reported experiences. "Is today a bad day to discuss these issues with you, Bob?"

The trick is always sorting out how to improve that reality mapping and

*gauge* it when interpreting results. Romney's folks were pretty sure he was going to win. But, his pollsters weren't as adept at understanding *their* numbers as Obama's were at understanding their own! The results of the *real* survey -- the Ballot Box -- are the only numbers that really mattered (hard to imagine folks deliberately LIED when they voted: "I really want Romney to win, but I'll vote for Obama, instead!")!

This'll sound cynical, but:

People have no idea what they really think. They

--
Les Cargill

The survey response is not anonymous, the vote is.