esomar 28 answers
1. What experience does your company have in providing online samples for market research?
IranPoll is an independent full-service opinion research and consultancy company in Toronto, Canada, focusing on Iran. With partner offices in Tehran and 8 other major Iranian cities, two centralized call centers employing interviewers with all 10 local languages/dialects, and a 500-strong field-force from across Iran, IranPoll.com provides a vast survey capacity to conduct and collect polling data from Iran using various modes of data collection, including FTF, Focus Groups, and CATI, and a national probability sample omnibus survey running every month. Building on the in-depth local knowledge of its team of experts, IranPoll.com solely relies on methods that could be objectively verified and independently replicated.
IranPoll.com’s online panel of Iranians is paralleled by none. With over 46,000 active panelists from across Iran, IranPoll.com is the largest online panel provider for the Iranian population and the only high quality online panel provider of Afghan people. In various capacities, we have been providing our clients with online panel services since 2009. While we mostly use our online panel to provide for our clients’ research needs, we do also conduct proprietary studies using our own panel.
SAMPLE SOURCES AND RECRUITMENT
2. Please describe and explain the type(s) of online sample sources from which you get respondents. Are these databases? Actively managed research panels? Direct marketing lists? Social networks? Web intercept (also known as river) samples?
The description of the types of sources a provider uses for delivering an online sample will provide insight into the quality of the sample. IranPoll.com’s online panel is an actively managed and maintained proprietary panel. Considering that IranPoll.com regularly conducts national probability sample CATI and FTF surveys, we have used this unique opportunity for recruitment purposes. Consequently, our panel members are composed of individuals with whom we have conducted a telephone or a FTF survey. This recruitment exercise has kept our panel fresh, relevant, and diverse.
All recruited individuals are validated and profiled through a telephone or FTF interview. The mobile phone numbers associated with each panel member is kept on file for quality control purposes.
3. If you provide samples from more than one source: How are the different sample sources blended together to ensure validity? How can this be replicated over time to provide reliability? How do you deal with the possibility of duplication of respondents across sources?
Since all of our panel members are validated and profiled through a telephone or FTF interview and since each panel member is identified by his/her individual mobile phone number, IranPoll.com has kept the likelihood of duplication of respondents across sources to a minimum.
Diversity and representativeness of our panel is of utmost importance to IranPoll.com. We have, therefore, adopted a diverse and multi-source recruitment process that minimizes sample bias. That is not, however, where we stop. Considering that all of our panel members have been extensively profiled and have been assigned a unique ID to track participation, for each project we ensure that the characteristics of the respondents closely match those of the target population. Once a survey is completed, we compare the characteristics of the survey sample with those of the target population and use post-survey weights to balance the sample.
4. Are your sample source(s) used solely for market research? If not, what other purposes are they used for?
IranPoll.com uses its panel solely for research purposes. Under no circumstances do we send or allow our clients to send advertisements to our panel members or use our sample for direct marketing.
5. How do you source groups that may be hard to reach on the internet?
Iran’s multi-ethnic and multilingual society requires us to pay particular attention to groups that may be hard to reach on the internet. For this reason, we have adopted many offline and focused recruitment methods. For example, we heavily rely on our probability sample CATI and FTF survey projects for recruitment purposes. At the end of each probability sample interview, particularly those conducted in hard to reach areas and/or population, we invite the respondent to join our panel. More than 36% of our current panel is comprised of people who have been recruited in this way. We also design our recruitment campaigns to target difficult to reach population by using recruitment sources that are closest to such populations and utilize tailored campaigns that is most likely to attract them. Our diverse and targeted recruitment campaigns as well as the size of our panel has allowed us to be able to drill down to incidence rates of about 3%.
6. If, on a particular project, you need to supplement your sample(s) with sample(s) from other providers, how do you select those partners? Is it your policy to notify a client in advance when using a third party provider?
As the largest online panel provider for the Iranian population, IranPoll.com has never needed to subcontract or use 3rd parties. In the unlikely event of such need arising, IranPoll.com will inform the client that we cannot fulfill his/her research needs on our own and will transparently provide all options that may be available to the client for him/her to choose.
SAMPLING AND PROJECT MANAGEMENT
7. What steps do you take to achieve a representative sample of the target population?
Our sampling process begins with having a solid understanding of the target population. Considering that IranPoll.com regularly conducts probability sample CATI and FTF surveys, our understanding of the target population goes much deeper than available census data. We use official census data as well as our proprietary data on various populations to closely match our panel sample with those of the target population. Considering that all of our panel members have been extensively profiled, using our proprietary access to dependable information on various target populations, for each project we draw our panel sample such that the characteristics of the panel sample closely resembles those of the target population.
8. Do you employ a survey router?
No. IranPoll.com does not use a survey router. Each panelist is selected and specifically invited to take a particular survey based on the study’s requirements and his/her personal profile and characteristics.
9. If you use a router: Please describe the allocation process within your router. How do you decide which surveys might be considered for a respondent? On what priority basis are respondents allocated to surveys?
IranPoll.com does not use a survey router.
10. If you use a router: What measures do you take to guard against, or mitigate, any bias arising from employing a router? How do you measure and report any bias?
IranPoll.com does not use a survey router.
11. If you use a router: Who in your company sets the parameters of the router? Is it a dedicated team or individual project managers?
IranPoll.com does not use a survey router.
12. What profiling data is held on respondents? How is it done? How does this differ across sample sources? How is it kept up-to-date? If no relevant profiling data is held, how are low incidence projects dealt with?
All of our panel members have been individually validated and profiled through a CATI or FTF interview prior to becoming member. The initial survey profiles respondents across more than eighty variables. These profiles are both updated and augmented with every survey the respondents complete. The profiling data we collect provides IranPoll.com with an unparalleled targeting capability. Whether a client needs a panel matching nationally representative proportions or one that includes members from a particular ethnic group, region, province, or even a city, IranPoll.com can deliver. In fact, our diverse and targeted recruitment campaigns, the size of our panel, as well as our extensive profiling efforts allow us to be able to drill down to incidence rates of about 3%. Some of our most popular panel profiling attributes are detailed here.
13. Please describe your survey invitation process. What is the proposition that people are offered to take part in individual surveys? What information about the project itself is given in the process? Apart from direct invitations to specific surveys (or to a router), what other means of invitation to surveys are respondents exposed to? You should note that not all invitations to participate take the form of emails.
All of our surveys are by direct invitation only. Through the sampling process, individual panel members are selected to participate in a survey. Then an invitation is sent to the respondent both via e-mail and SMS. SMS invitations are important for us since a proportion of our panel members access the internet only through their mobile devices. Each invitation is sent using IranPoll.com’s company e-mail and telephone. The invitation generally includes basic information regarding the survey, the survey’s unique URL, and information on how to contact our support center should they have any questions regarding the invitation and/or the survey.
14. Please describe the incentives that respondents are offered for taking part in your surveys. How does this differ by sample source, by interview length, by respondent characteristics?
Every panel member is informed about the incentive structure immediately after becoming validated and prior to being invited to partake in their first survey. IranPoll.com provides its panel members with three types of incentives depending on the projects:
- For every survey, a single respondent is selected by lottery from amongst all of the respondents of that survey to win a large monetary incentive.
- All respondents receive a monetary incentive based on their individual characteristics as well as the number and length of the surveys they complete.
- IranPoll.com also uses a point system to incentivize panelists to remain active. In this system, panelists receive points based on the number and length of surveys they complete. The points can be redeemed for a variety of rewards based on panelists’ preference.
15. What information about a project do you need in order to give an accurate estimate of feasibility using your own resources?
We need the following information to be able to provide a client with an accurate feasibility and cost estimate:
- Profile of the target population,
- Respondent qualification criteria,
- Incident rate,
- Any over-quotas required,
- Time in the field,
- Length of interview,
- Number of completed,
- Survey topic,
- Any special programming, stimuli, or tasks required.
With this information we will be able to tell a client whether the study is within our capabilities. If there is a feasibility issue, we will work with the client to decide on the best course of action. If there is no feasibility issue, we will also provide the client with the cost and length of the project.
16. Do you measure respondent satisfaction? Is this information made available to clients?
We primarily gauge respondent satisfaction in two ways:
- Every four months, we send a short survey to all of our panel members that measures their satisfaction with being a member of IranPoll.com’s panel and the main reasons behind possible dissatisfactions.
- Our panel members are regularly informed that should they have any questions or concerns regarding any issues pertaining to their membership, they should call our member support center. The support center, in turn, not only resolves issues but systematically provides the management with reports on the issues raised by panel members.
These two methods provide us with real-time evaluation of how IranPoll.com and its online panel is perceived by its panelists and allow us to resolve issues that has a direct effect on the satisfaction level of our panelists as quickly as possible.
17. What information do you provide to debrief your client after the project has finished?
IranPoll.com believes in transparency. We, therefore, actively keep our clients informed about various aspects of the project before, during, and after the fieldwork. During the fieldwork we actively communicate to our clients any feedback we get from our panel members. Also, when we host the survey, we provide our clients with a daily progress report. Once the project is completed – for surveys that we have hosted - we provide our clients with all participation history they may have asked prior to the fieldwork. The requested information would be submitted to the client immediately after the completion of the fieldwork, provided that it does not require disclosure of any personally-identifiable information.
DATA QUALITY AND VALIDATION
18. Who is responsible for data quality checks? If it is you, do you have in place procedures to reduce or eliminate undesired within survey behaviours, such as (a) random responding, (b) Illogical or inconsistent responding, (c) overuse of item non-response (e.g. “Don’t Know”) or (d) speeding (too rapid survey completion)? Please describe these procedures.
Quality Control is central to everything IranPoll.com does. When we host a survey, we check for random responding, inconsistent responding, over-use of item non-response, speed responding, and straight-lining and do not regard affected surveys as complete and blacklist respondents who engage in such practices. We also regularly eliminate from our panels fraudulent, inactive, unresponsive, and inattentive respondents. We also compare demographic information provided in surveys with those in each respondent’s panel profile to ensure that no inconsistencies exist.
19. How often can the same individual be contacted to take part in a survey within a specified period whether they respond to the contact or not? How does this vary across your sample sources?
IranPoll.com’s objective is to keep its panel members engaged without overburdening them. To achieve this objective, we ask our panel members what rate of contact they think is appropriate and what they regard as being too excessive. We then use this information to ensure that we keep our contact with each panelist within what they regard as appropriate. Panelists may contact our support center anytime to increase or decrease the rate at which they are contacted, or they can simply unsubscribe from the panel all together if they choose to. Regardless of what the respondent suggest to be his tolerance level, however, we maintain a cap of one invitation per week for all panel members.
20. How often can the same individual take part in a survey within a specified period? How does this vary across your sample sources? How do you manage this within categories and/or time periods?
All of our surveys are by invitation only and each selected panel member only receive one invitation. Also, considering the size of our panel, each panel member on average partakes in about 15 surveys a year. While the cap we maintain for each respondent is 1 invitation per week, in practice each panelist receives about two invitations every three weeks and participated one survey each month.
21. Do you maintain individual level data such as recent participation history, date of entry, source, etc., on your survey respondents? Are you able to supply your client with a project analysis of such individual level data?
While IranPoll.com does collect and hold complete individual level history of every panelist for quality control purposes, it strictly safeguards any personally identifiable information of its panel members and does not disclose them to anyone, including our clients. Besides personally identifiable information, however, the client may request and will be provided with a per-job analysis of the respondents.
22. Do you have a confirmation of respondent identity procedure? Do you have procedures to detect fraudulent respondents? Please describe these procedures as they are implemented at sample source registration and/or at the point of entry to a survey or router. If you offer B2B samples what are the procedures there, if any?
At IranPoll.com we have zero tolerance for fraudulent respondents and our identity validation process begins before an individual joins our panel. Before joining our panel, each of our panelists are interviewed, validated, and profiled either FTF or over the phone. We also obtain and validate each of our panel members’ personal e-mail and mobile phone numbers. Each individual is only allowed to register once and we do not register two profiles with identical e-mails and mobile phone numbers. We also routinely check the profiles of our panel members and investigate profiles that do not pass our similarity standards. Since all of our surveys are by invitation only, each selected panel member receives a unique URL that can only be completed once. We also apply survey level quality controls to catch and blacklist respondents who engage in random responding, inconsistent responding, speed responding, and straight-lining. We also investigate cases where the demographic information provided by a panel member on a survey is different from the information on his/her profile. In addition, we utilize Two-factor authentication at the time of reward redemption. This procedure removes the incentive for the majority of fraudulent behaviors as receiving the rewards is only possible after an offline confirmation.
POLICIES AND COMPLIANCE
23. Please describe the ‘opt-in for market research’ processes for all your online sample sources.
IranPoll.com’s recruitment process ensures that all of our panel members are fully aware of our terms and conditions and are willingly becoming a members of our panel. Regardless of the sample source or how an individual becomes acquainted with our online panel, no one can become a member of our panel without being validated and profiled either through a FTF or a telephone interview (the registration interview). During the registration interview, all validated individuals are informed of our terms and conditions and permission is obtained to enlist them in our panel.
25. Please describe the measures you take to ensure data protection and data security.
Considering the environment in which we are operating, we take our responsibility to protect the privacy and personal information of our panel members very seriously. In line with Article 4.3 of ESOMAR/WAPOR guidelines, IranPoll.com has adopted a series of measures to give confidence to its Iranian panel members that their personal identity will never be disclosed. These measure extend from standard privacy protocols, to providing respondents with high-quality VPN services that to a degree prevents online fingerprinting by masking their exact whereabouts, browser and system details, and IP addresses when they partake in our surveys. In addition, all our staff must sign a confidentiality agreement outlining the employee’s obligations to protect company and client confidential information. Staff must adhere to strict guidelines to prevent sharing of any sensitive information across clients, projects, or even departments.
26. What practices do you follow to decide whether online research should be used to present commercially sensitive client data or materials to survey respondents?
Before starting fieldwork, we always obtain our clients’ explicit approval of the survey instrument, the online program that will administer it, and the manner in which commercially-sensitive materials are presented or discussed. Also, our the terms and conditions explicitly indicates to panel members that they should regard all information presented in an online survey as confidential and that they will be held accountable for any information they may leak to the public domain.
None of these measures, however, guarantee that such a leak will not happen and ultimately it is the client who will have to decide what information her/she is comfortable sharing with our panel members and whether our online panel practices suit their material protection needs.
In cases where the material is very sensitive, we often suggest other more secure modes of data collection. As a full-service opinion research and consultancy company, we offer a wide range of data collection services that are significantly more suitable for presentation of sensitive material.
27. Are you certified to any specific quality system? If so, which one(s)?
We are compliant with the ESOMAR Guidelines, covering all legal, ethical, and practical aspects of doing research. Our company is currently phasing-in ISO certification, specifically ISO20252 and ISO26362 (Market, opinion and social research - Vocabulary and Service Requirements).
28. Do you conduct online surveys with children and young people? If so, do you adhere to the standards that ESOMAR provides? What other rules or standards, for example COPPA in the United States, do you comply with?
While we do not have any panel member who is under the age of 18, we conduct online surveys that require child respondents by asking panel members with children to present the survey to their children. For all such surveys, legal guardians are provided with the full survey instrument and their explicit consent is acquired ahead of time.
As a standard practice, IranPoll.com verifies the integrity of any data it collects through its partners by conducting a portion of the research using its own capabilities in Toronto.
our local partnerships, extensive local presence:
- Active presence in 9 important Iranian cities,
- Maintaining a 300-strong field-force from across Iran,
- Utilizing two centralized call centers who employ interviewers with the full range of local language capabilities.