PilotHouse FAQ
1.) What is the Nemertes PilotHouse awards program?
The Nemertes PilotHouse awards program is an annual research project that ascertains how vendors and service providers are performing in the eyes of their business customers. The key to this program is that it's based solely on ratings from the providers' customers.
2.) What technologies do the PilotHouse awards cover?
Products
- Application Delivery Optimization
- Data-Center Colocation
- Data-Center Firewall
- IP Contact Centers
- IP Telephony
- Servers for Virtualization
- Small Branch Firewalls
- Storage for Virtualization
- Unified Communications
- Virtual Desktops
- Wireless LANs
Services
- Carrier Ethernet Services
- Hosted Voice Over IP
- Internet Access Services
- MPLS Services
- Managed Internet Services
- Managed Firewall / IDS / IPS
- Managed Router Services
- Managed Voice Over IP
- Software as a Service: Office
- SIP Trunking Services
- Wireless Voice & Data Services
3.) How does Nemertes gather the ratings? Can vendors influence the results?
Nemertes' analysts directly interview about 200 IT leaders through Nemertes large database of clients and IT professionals. Nemertes augments these interviews with another 800-1,000 IT leaders who respond to a Web-based survey from a screened list of participants.
Vendors cannot influence the ratings. They cannot provide customer names for inclusion in the research. Names come from Nemertes database and a third-party survey company that uses the highest standards in managing and screening its list of IT professionals. The ratings strictly represent customer views about the vendors, and the vendors do not participate in the research-gathering in any way.
4.) What does "screened list of participants" mean?
Participants are individuals from U.S. companies (based in the U.S., but many of which are global multinationals) who are responsible for selecting or influencing the selection of, suppliers of IP communications products and services; network infrastructure, data-center server and network monitoring tools; and/or data-center hosting services. Survey participants must answer a set of screening questions to make sure they meet these selection criteria. Nemertes analysts use a unique methodology to screen out individuals who are providing inaccurate answers or are using automated "bots" to answer for them.
5) What is the provider rating?
Participants rate their providers on a 1-to-5 scale (1=unacceptable, 2=poor, 3=fair, 4=good, 5=excellent). Nemertes consolidates all ratings into a mean score for each category rated (Technology, Customer Service, and Value), and for an overall score. In addition to the Technology, Customer Service, and Value ratings, certain categories may have additional metrics.
6) Is this a "pay-to-play" awards program?
No. Nemertes publishes aggregate and comparative data for all vendors for which we receive a total number of ratings equal to at least 10% of the total pool of ratings. As part of the survey, Nemertes provides a list of vendors derived from extensive research and analysis. In addition, participants can name and rate any other provider in any category.
7) What are the awards categories?
Nemertes provides awards in two categories: Market Leaders and Market Challengers. Market Leaders are companies with significant market presence, in terms of units shipped, number of customers, and overall revenue. Market Challenges are companies with smaller market presence, but who may have technology innovation, customer service, or value that exceeds the Market Leaders. Nemertes retains the right to modify criteria for Market Leaders and Market Challengers. Publication of Market Challenger awards is based on number of ratings gathered within each technology or service sector. Some technology markets are so fragmented that Nemertes does not receive enough ratings for enough individual vendors to issue awards in the Market Challenger category.
8) When will awards be announced?
Upon completion of analysis, Nemertes notifies all winners. Nemertes clients receive copies of the award reports. Winners will be listed on the Nemertes website and publicly announced at the PilotHouse Award Ceremony and Dinner during our Navigator360 Summit.
9) Can vendors promote the awards?
Yes. After completion of the award reports, Nemertes will notify winners and offer the option of buying award packages that include reprint rights, logo licensing, webinars, issue papers, and award dinner tickets. Buying award packages have no bearing on the results of the PilotHouse awards.
10) How does Nemertes ensure the integrity of the PilotHouse program?
Nemertes developed this program with integrity as the cornerstone. There are a few key differentiators in Nemertes' program compared to other analyst firms' programs:
- Vendors cannot provide customer names.
- Vendors do not select technology categories or questions for the awards.
- Analysts determine technology categories and identify vendors as Leaders or Challengers, but they do not subjectively determine who wins the awards. This is totally based on customer ratings.
- Analysts conduct the data analysis independent of any vendor input or notification.
- Nemertes notifies all vendors at the same time about the award results.
- Nemertes does not require vendors to purchase anything in order to receive the award.
- Nemertes does not change the award recipients based on who buys award packages.
For Additional Information on PilotHouse, please contact Nemertes Research
research@nemertes.com
888-241-2685


