Comments
agonchar
7/26/2016 11:31:06 AM
User Rank Bronze
So Vodafone interviewed 1,100 Business Leaders??
Why do executives in our business continue to use surveys and then say they did interviews?
Why is this relevant?
Because the quality and the actionability of the results are much different after doing a real series of interview with the RIGHT business leaders not just whoever happens to respond.
However the real issue is that surveys are used because you can then massage the data to say anything, support current directions or biases, keep your job etc...
Last time I called one of Partners at a top tier management consulting firm on this same issue - they said "oh yeah - sorry we meant survey.
There is a huge difference.
Ariella
7/26/2016 12:08:28 PM
User Rank Author
Re: So Vodafone interviewed 1,100 Business Leaders??
@agonchar the full report includes some quotes from some of the business people surveyed, as well as the general responses. I don't know if they collected quotes from all responses or reached out only to a select few to get further insight.
Actually, I found the report itself pretty informative and objective. It didn't push Vodafone's services in particular or try to twist things around to fit a particular agenda. I once did a survey and the follow up report on it for a company that I will not name here. The survey showed one thing, but one of the guys on the company's team insisted on saying what he believes it should say. Specifically, the report showed relatively few companies using mobile technology in their marketing efforts at the time. In fact, about half the survey responders couldn't answer a good portion of the questions (that this guy had wanted added in) because they had not experience with it at all. That was what he wanted, so he demanded that the report on the survey be written to downplay the fact that mobile was not fully embraced to push somewhat self-serving advice in place of the objective data.
agonchar
7/26/2016 12:40:02 PM
User Rank Bronze
Re: So Vodafone interviewed 1,100 Business Leaders??
Ariella - Thank you and I agree with what you described - your post got me thinking and the bottom line is that this document is for marketing purposes.
So it was a survey with a few qoutes?
Just don't try to pass it off as primary research - interviews.
1,100 interviews would take (assuming one hour per interview and a 40 hour work week) would take a minimum of 28 weeks or 7 months to complete. Best case.
Finding and getting 1,100 appropriate business leaders to do one hour interviews would take another large amount of time. This is what we do as a unique CI provider so our expertise is deep. This could easily equal or exceed the 7 months mentioned above.
So although this document seems well done - after being misled by the highlighted interview process, I never read more than the summary and only scanned that.
HBR has written about how and why CI is not really used even though billions are being spent on surveys and such.
1-Not enough detail to be actionable
2-Used only when it supports a bias (this is what you were talking about)
3-No one ever "audits" whether CI had any impact on the decision making process
4-Predominatly marketing focussed - find some good news that we can leverage
Ariella
7/26/2016 12:55:56 PM
User Rank Author
Re: So Vodafone interviewed 1,100 Business Leaders??
@agonchair I generally agree with your 4 points. For most companies, such surveys are often self-serving and often sloppily and hastily done. You really have to pay attention to number of factors to be sure if there is any value in the findings. One, of course, is the number of responses. Even if not all of them gave full interviews, just having a larger sampling for multiple choice answers gives a better idea of what is going on than a smaller sampling. However, you have to know whether you'd consider the minimum size for the sampling to be significant. As I said from my own insider's experience, when nearly half the responders dropped out for the answers to the mobile question -- and the entire sample was not more than 300 to begin with 00 the numbers are so low that they may not prove indicative of businesses as a whoel at all.
clrmoney
7/27/2016 10:53:22 AM
User Rank Platinum
Vodafone IOT
Vodafone checks the adopters for the IOT=Internet of Things I guess to make sure everything goes well. It is going well with ther performance so maybe things can continue to go smoothly.
JohnBarnes
7/28/2016 7:59:39 AM
User Rank Platinum
Re: So Vodafone interviewed 1,100 Business Leaders??
ariella,
That's one of the most common errors in thinking about and trying to use survey data. Increasing the sample size does not improve accuracy; in fact, if your process of recruiting and identifying respondents isn't controlled to produce a representative sample, increasing the sample size often makes matters worse (a poorly publicized survey selects the very small number of people who heard about it, which may be a much more random selection than a well-publicized survey that attracts and incentivizes large numbers of people with axes to grind).
Furthermore, agonchair is right again that survey presentations with quotes have to be regarded as suspicious and more likely bogus than not; a very large number, probably a majority, of decision makers behave as if one quote outweighs a thousand data points, and the whole reason for putting the quotes in is to facilitate ignoring the data. (It's the verbal equivalent of letting the graphic designer redraw the graphs to make them prettier).
Ariella
7/28/2016 9:28:50 AM
User Rank Author
Re: So Vodafone interviewed 1,100 Business Leaders??
@JohnBarnes About pulling quotes, I read the study and found most of the quotes from one particular company that really spoke only about its own use and ROI for IoT and how it may be used in its industry. It didn't attempt to say that this is the case for everyone.
I see studies cited all the time that draw conclusions based on responses from people who are hardly a random sampling, most often less than a hundred people around a college campus who were willing to go through the steps. I think in such cases, one has to take things with a grain of salt. Of course, one has to take any presentation with a grain of salt if there is any possible agenda behind it. Certainly, one can manipulate data by the choice of frame. I recently shared this on G+
"After many requests, I finally added trend-lines (linear least-squares regression) to the graph generator. I hope this is useful, but I would also like to point out that it can be fairly dangerous...
Depending on your preconceptions, by picking your start and end times carefully, you can now 'prove' that:
Temperature is falling!
Temperature is static!
Temperature is rising!
Temperature is rising really fast!"
JohnBarnes
7/28/2016 2:38:49 PM
User Rank Platinum
Re: So Vodafone interviewed 1,100 Business Leaders??
Ariella, but the problem isn't what a quote says; it is that embedding it in the text of a survey gives it a spurious impression of being a typical response or a conclusion. And any survey that rests on volunteer responses is likely to become more rather than less biased with increased sample size. Like sausage and democracy, if you like surveys you should avoid seeing what goes into them.
Ariella
7/28/2016 2:59:31 PM
User Rank Author
Re: So Vodafone interviewed 1,100 Business Leaders??
@JohnBarnes perhaps you may say that group is not necessarily the best reflection of all people, but that's always the case with any sampling. I think you would not come off any more objective with a sample size of 25 with a particular group than you would with 1000. It wouldn't become less accurate because it's larger. The larger the responses for a group, the more accurate the reflection of that group -- even if you have your doubts about the group being truly representative.
In other words, if I want to get a sense of how many people understand IoT, I can ask here. But anyone who is a regular reader here likely has a better understanding than the average person, so the sampling itself would be biased. But it doesn't become more accurate if I get responses from only 10 people here rather than from all of them. Any survey has to have some way of setting and reaching people to answer questions, so it will never be perfect in getting full representation.
This is what the report said about its responses:
1,096 qualified respondents. These respondents were carefully selected to represent all kinds of: • Regions: We surveyed 17 countries across all major regions: Australia, Brazil, Canada, China, Germany, India, Ireland, Italy, Japan, the Netherlands, South Africa, South Korea, Spain, Turkey, the UAE, the UK and the USA. • Industries: Respondents represented retail, manufacturing, energy and utilities, healthcare, transport and logistics, automotive, consumer electronics, and industrials. For the first time we also surveyed public sector organisations. • Sizes: We spoke to both SMEs and the largest multinationals. 15% of those we surveyed have more than 10,000 employees; we also included SMEs with as few as 10 employees. • IT spending: Overall IT budgets ranged from under €10,000 to more than €50m. • Roles: We only surveyed qualified decision-makers at senior manager level or above, but they represented a mix of departments. 40% of our respondents were from IT functions; 13% were senior business leaders; 9% were from sales and marketing; and 8% from finance. Retail Automotive Industrials Consumer electronics Transport and logisitics Healthcare Public sector Energy and utilities Manufacturing 8% 17% 18% 12% 10% 13% 4% 10% 8% Percentage of interviews across all sectors Percentage of interviews across all regions Percentage of interviews across all business sizes Americas 29% EMEA 43% APAC 29% 10–249 employees 250–999 employees 1,000–9,999 employees 10,000+ employees
The quotes are contextualized to identify which business type and location it comes from. They provide the storytelling part of data representation. Without that, you just have numbers and graphs. Can story tilt the impression overall? Sure it can, but readers of such reports should be sophisticated enough to know that and how to work with it.
JohnBarnes
7/31/2016 4:41:16 PM
User Rank Platinum
Re: So Vodafone interviewed 1,100 Business Leaders??
ariella,
The point that increasing the sample size does improve the representation of the group being represented is well taken (and often emphasized in stats books and classes); unfortunately, this is frequently elided so that it becomes a magic formula for improving representation. To re-use an ancient analogy, if someone is looking for a quarter on a dark sidewalk where they dropped it, more light will help, so turning up an adjustable flashlight or adding a streetlight will definitely improve the chances of finding the quarter. But if they dropped it in the alley and they're looking under the streetlamp because the light is better -- the classic analogy for an unrepresentative sample -- brightening the streetlight only strengthens the basis of the initial poor choice.
In the case of surveys, a small poorly recruited survey will have a wide scatter and this may very well include some of the true value; the same survey repeated with a bigger sample will have a tigher scatter and will dismiss the relatively few good points as noise. So, once again, for surveys with a large self-selection component, increased sample size actually pulls the results further off.
However: it's quite clear from what you quoted that this was not the basic crude self-selected survey. Now going over to have a look at it ....
|
|
Italy's 5G auction could exceed a government target of raising €2.5 billion ($2.9 billion) after attracting interest from companies outside the mobile market.
The emerging-markets operator is focusing on the humdrum business of connectivity and keeping quiet about some of its ill-fated 'digitalization' efforts.
Three UK has picked Huawei over existing radio access network suppliers Nokia and Samsung to build its 5G network.
Vendor says that it's its biggest 5G deal to date.
Verizon skates where the puck is going by waiting for standards-based 5G devices to launch its mobile service in 2019.
Orange has been one of the leading proponents of SDN and NFV. In this Telco Transformation radio show, Orange's John Isch provides some perspective on his company's NFV/SDN journey.
10/16/2017
Huawei Network Transformation Seminar
The adoption of virtualization technology and cloud architectures by telecom network operators is now well underway but there is still a long way to go before the transition to an era of Network Functions Cloudification (NFC) is complete.
The Small Cell Forum's CEO Sue Monahan says that small cells will be crucial for indoor 5G coverage, but challenges around business models, siting ...
People, strategy, a strong technology roadmap and new business processes are the key underpinnings of Telstra's digital transformation, COO Robyn ...
Eric Bozich, vice president of products and marketing at CenturyLink, talks about the challenges and opportunities of integrating Level 3 into ...
Epsilon's Mark Daley, director of digital strategy and business development, talks about digital transformation from a wholesale service provider ...
Bill Walker, CenturyLink's director of network architecture, shares his insights on why training isn't enough for IT employees and traditional ...
All Videos
|
On-the-Air Thursdays Digital Audio
Special Huawei Video
Tweet This