I was fascinated by this news release. The Aberdeen Group has launched a “online digital prototyping assessment tool,“ which utilizes the outfit’s “primary research in product innovation, helps companies identify their biggest product development challenges, compares their design practices to industry benchmarks, and quantifies potential time and cost savings by adopting Best-in-Class capabilities.”
An impressive sounding thing. Wait. There’s more. The assessment, which is in the form of a short multi-stage questionnaire is based on the group’s “unique PACE research methodology, steps through the product development process to determine where an individual company excels and helps identify opportunities for improvement.”
When you’ve registered for the assessment, you’re taken through a few questions, some of which are multiple choice, some of which are single choice from a list of options. The end goal is that the system “makes recommendations by comparing responses to prior Aberdeen research that identified design and engineering best practices exhibited by Best-in-Class companies.”
Once done you then receive a summary of how your company compare “to other organizations as well as custom tailored next steps to improve their development process.” I went through the process and while its initially interesting how the questions are structured, what language is used and the end results.
There are the basic questions, such as “How are product concepts developed?” possible responses are:
- On paper only
- On paper and then transferred to electronic form (2D or 3D)
- Completed electronically, as a 2D sketch
- Completed electronically as a 3D surface model (push and pull surface capabilities
This is pretty standard stuff, although the answers look a little odd. Considering that the only 3D related answer is a surface model, it looks a little incomplete. Where’s the potential for an intelligent 3D model? Question 3 in my survey read “How is the aesthetic look and feel of a product assessed?” with the options being:
- By building/sculpting a physical prototype at the concept stage
- By building a physical prototype when the design is complete
- By reviewing a realistically rendered digital model
- Aesthetic properties are not important in my industry
Fair enough, but you’re only allowed a single option answer. The use of physical and digital models within the product development process is typically mixed. You don’t just build a physical prototype, or use a rendered model most organizations use both. The same was true of almost every question and possible answer. You can’t give a complete answer or indeed, accurate answer, because you might typically use a mix of methods, for visualization, for simulation, for assessing product performance.
So what’s my problem? It’s nothing specific, but take a look at the URL for the assessment survey http://autodesk.aberdeen.com. That gives it all away. Aberdeen has constructed this survey at the behest and I assume, funding from Autodesk. When you complete it, you get a quick bar chart showing how you perform according to your answers and get emailed a link to a fuller report as well as Aberdeen’s other research report (which are worth reading). There are also links to recommended products from Autodesk. Take from that what you will.
The press release has a quote from Stephen Gold, President of Aberdeen, saying that they are “pleased to provide organizations with guidance to improve their product development process. The Assessment Tool Series offers a unique opportunity for organizations to leverage our benchmark research in a highly personalized way.”
Yes, “A unique opportunity to reuse their benchmark research” (which is pretty highly regarded I might add), but as long as you do using limited answers, limited input and using language that suits perfectly the terminology of the endeavor’s sponsors . I don’t have a problem with this , but I wish analysts would be a little more honest about why they do these things. This has very little to do with their research, more to do with Aberdeen tying up with Autodesk to reuse their research methods to point users to Autodesk’s tools – its a sales tool masquerading as a scientific study. If you look at it from that point of view, then its perfectly acceptable and you can get from the results exactly what you want. But let’s not pretend it’s anything else. Analysts have a strange habit of taking what appears to be common sense, wrapping it up with ‘research’ and feeding it back to you and this does nothing to change that perception.
