+1(978)310-4246 credencewriters@gmail.com


Scholar-practitioners, as leaders, must unite different groups of people to be successful. They must support the vision, mission, and goals of the organization, and they must understand the needs of different groups to meet those needs and improve motivation. Leaders must also understand the social and organizational cultures of the organization to identify and address any cultural gaps that could influence the change process. In this assignment, you will address the assessment and influence of culture as it relates to the development of a diverse global group of employees.

General Requirements:

Use the following information to ensure successful completion of the assignment:

Review the article by Cater, Lang, and Szabo (2013).

This assignment uses a rubric. Please review the rubric prior to beginning the assignment to become familiar with the expectations for successful completion.

Doctoral learners are required to use APA style for their writing assignments. The APA Style Guide is located in the Student Success Center.

This assignment requires that at least two additional scholarly research sources related to this topic, and at least one in-text citation from each source be included.

You are required to submit this assignment to LopesWrite. Refer to the

LopesWrite Technical Support articles

for assistance.


Write a paper (1,000-1,250 words) that addresses the assessment and influence of culture as it relates to the development of a diverse global group of employees. Include the following in your paper:

A research-supported discussion of available cultural assessment tools. How might these tools be applied to assess the organizational culture and identify cultural gaps within the group? (Benchmarks C3.5: Select and apply tools to assess organizational cultures.)

A research-supported discussion of how the social culture of this group will likely influence workplace outcomes and group productivity.

A research-supported discussion of how the social culture and diversity of the individuals on the team will influence the greater organizational culture. (Benchmarks C3.1: Recognize the influence of social culture and diversity on organizational culture.)

The elusive effects of workplace diversity on innovation*
Ceren Ozgen1,2, Peter Nijkamp1,3, Jacques Poot4
Department of Spatial Economics, VU University Amsterdam, De Boelelaan 1105, 1081 HV Amsterdam,
the Netherlands (e-mail: c.ozgen@vu.nl, p.nijkamp@vu.nl)
Migration Policy Centre, European University Institute, Florence, Italy; IZA, Bonn, Germany
Institute of Socio-Economic Geography and Spatial Management, Adam Mickiewicz University, Poznan, Poland
National Institute of Demographic and Economic Analysis, University of Waikato, Hamilton, New Zealand
(e-mail: jpoot@waikato.ac.nz)
Received: 26 February 2013 / Accepted: 2 June 2015
Abstract. We investigate econometrically whether cultural diversity at the workplace boosts innovation. Our longitudinal linked employer-employee data combines two innovation surveys, with
Dutch administrative, tax and regional data. We analyse the determinants of a firm’s product and
process innovations with respect to the firm’s internal resources, employee composition and regional
agglomeration externalities. We reconfirm the findings of several other recent studies showing a
positive partial correlation between innovation and cultural diversity in pooled cross-sectional data;
in some cases even when accounting for reverse causation. However, no statistically significant
traces of benefit for innovation from cultural diversity remain after introducing firm fixed effects.
JEL classification: D22, F22, O31
Key words: Immigration, innovation, cultural diversity, knowledge spillovers, the Netherlands
1 Introduction
A recent strand of the migration literature has focused on the association between economic
performance and the cultural diversity (in terms of ethnicity, languages, religion, etc.) of the
employees of firms and regions (for a recent review, see Kemeny 2014). Of particular interest
is the question whether the presence of culturally diverse employees at the workplace might
* This research is part of the Migrant Diversity and Regional Disparity in Europe (MIDI-REDIE) project, funded by
the 2009-2013 NORFACE research programme Migration in Europe – Social, Economic, Cultural and Policy Dynamics. Jacques Poot also acknowledges funding from New Zealand’s Ministry of Business Innovation and Employment
(MBIE) for the project Capturing the Diversity Dividend of Aotearoa New Zealand (CaDDANZ). Earlier versions of this
paper have been presented at the 50th European Congress of the Regional Science Association, Barcelona, Spain,
30 August–3 September 2011, and at the International Workshop on Economic Impacts of Immigration and Population
Diversity, National Institute of Demographic and Economic Analysis (NIDEA), University of Waikato, Hamilton, New
Zealand, 11–13 April 2012. We gratefully acknowledge comments received at these presentations. We also acknowledge
the detailed and constructive comments by two referees, which contributed to a substantially improved revised paper.
© 2015 The Author(s). Papers in Regional Science © 2015 RSAI
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
C. Ozgen et al.
boost a firm’s innovativeness. The answer is theoretically difficult to establish because there are
many ways in which cultural diversity can impact on a firm. Some of these are positive, while
others are negative (see, e.g., Stahl et al. 2010; Nathan and Lee 2013). Even after quite a number
of studies have now been published on whether cultural diversity helps to boost innovation, the
evidence remains mixed (Ozgen et al. 2014).
There are several methodological and measurement challenges when investigating this relationship. First, due to the demography of firms (i.e., firm births, deaths, mergers and separations)
it is difficult to obtain large longitudinal samples of firms that span one or more decades. Second,
where longitudinal data are available, the time span is usually such that there is limited variation
in firm-specific employee composition and in firms’ innovation behaviour. This leads to rather
fragile inferences across specifications in regression analyses. Third, while many studies in this
literature have shown positive impacts of diversity on innovation at the firm level with crosssectional data or when simply pooling a number of waves in a panel, such regressions have obvious drawbacks through unobserved heterogeneity and omitted variable problems obstructing
the proper singling out of the effect of diversity. Essentially, the effect sizes of interest become
too small to detect once unobserved factors which possibly correlate with regressors are incorporated in the models. The fact that cross-sectional estimation and pooled data estimation with sector and time fixed effects do find a significantly positive impact of cultural diversity – particularly
in the case of high-skilled migrants in knowledge-intensive sectors – while panel models do not,
seems to suggest that the apparent positive effects are related to unobserved firm heterogeneity
and sorting effects. In terms of measurement, the relationship between innovation and diversity
may vary based on the spatial scale (regions, firms), level of aggregation (teams, types of employees, workplaces), and measures used to proxy both right and left hand-side variables.
Consequently, the empirical strategies are constrained by the data availability and their quality.
We conclude from the literature that the overall association between cultural diversity on
firm performance, and innovativeness specifically, remains an open question. In this paper we
therefore aim to methodologically improve the estimation as well as address various measurement issues. Accordingly, this paper contributes to the existing literature in several ways. Most
importantly, in contrast with contributions by Lee and Nathan (2010), Østergaard et al. (2011),
McGuirk and Jordan (2012), Ozgen et al. (2013) and Parrotta et al. (2014) – who all estimate the
relationship between cultural diversity and innovation at the firm level with cross-sectional or
pooled cross-sectional data – we include in this paper a proper panel data approach and show
that the findings comparable to those in the literature are not robust once we account for unobservables through fixed effects. Additionally, we improve on our previous work (Ozgen et al.
2013) in four ways. First, we explicitly consider the skill and age-composition of foreign
employment vis-à-vis native employment. Second, we replace two of three measures of cultural
diversity we used previously (namely the share of foreigners among employees and the fractionalization index) by two alternative indexes (the Simpson index and the co-location index respectively) that have nice interpretations in terms of the nature of the interaction between workers in
culturally diverse firms. The unique number of countries of birth present is again our third measure of diversity, but we now explicitly test for an inverse-U shaped relationship between this
measure and innovation.1 Third, we are able to construct a much larger data set on firm level
innovation. This is achieved by imputing a skill level to migrants and the native born based
on their annual wages obtained from tax registers, rather than by restricting the observations
to only those who were included in the Dutch labour force survey and of whom the skill level
was therefore explicitly recorded. Finally, we conduct robustness checks by disaggregating
the firms in various ways to test for heterogeneity across groups of firms. These include
Ozgen et al. (2013) simply tested for a linear relationship between the natural logarithm of the total number of
countries of birth present among the firm’s employees and innovation.
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
Workplace diversity and firm innovation
grouping firms based on market orientation, average industry innovativeness, service sector specialization and foreign ownership. We also consider various potential instrumental variables.
Due to space limitations, only key results are reported in the paper and tables with additional
results can be obtained upon request from the corresponding author.
In this paper we use a unique linked employer-employee dataset that combines data from
two innovation surveys in The Netherlands, four years apart (2002 and 2006), with administrative and tax data. We investigate the impact of cultural diversity of employees (defined by the
countries of birth they originate from) at the firm level on a firm’s self-declared product and process innovativeness. In doing so we also consider the firm’s internal and external resources by
accounting for the market conditions of the meaningful economic areas where firms are located.
We calculate three distinct measures of cultural diversity. We reconfirm the findings of several
other recent studies that show that there is a positive partial correlation between innovation and
cultural diversity in pooled cross-sectional data; in some cases even when accounting for reverse
causation. However, we find that no statistically significant traces of benefit for innovation from
cultural diversity remain after introducing firm fixed effects. The findings in the recent literature
guided us to build our estimation strategy on estimating and comparing linear and non-linear
models with and without firm fixed effects in order to show the importance of correctly handling
unobserved heterogeneity.
A positive impact on innovation of cultural diversity among a firm’s workers can be first due
to immigrants being positively self-selected in terms of their abilities and attitudes, for example
in terms of entrepreneurship and risk taking. Second, firm innovation in some countries and
regions may be constrained by local scarcity of highly skilled and specialized labour, in which
case recruitment from abroad may be helpful (Beaverstock and Hall 2012). Third, immigrants
(and sometimes their offspring) bring to the workplace different ideas and perspectives from
their cultural backgrounds which may spill over to other employees. Successful migrant
entrepreneurship can also spill over to host country entrepreneurs (Jaeger and Duleep 2010).
Moreover, with different perspectives and approaches to problem solving, migrants may contribute to better decision-making (Page 2007). Of course, the effectiveness of such spillovers is contingent on production conditions, such as the organizational culture, labour market structure and
institutions that jointly determine how foreign knowledge will be absorbed in the host country
(Jones and Romer 2010). Fourth, foreign workers can facilitate trade with their home countries
and/or encourage the production of ethnic goods and services in host countries (e.g., Genc
et al. 2012). Finally, immigrants often exhibit high labour mobility, particularly in their early
years in the host country. This speeds up the inter-firm dissemination of new ideas. Additionally,
it ‘greases the wheels’ of the local labour market (Borjas 2001). For all these reasons, recruitment
policies of developed countries increasingly favour highly skilled young migrants.
However, there are also ways in which migrant diversity can negatively impact on innovation. First, when immigrants have lower reservation wages than the native born, the availability
of cheaper labour makes it rational for firms to adopt more labour-intensive production technologies and thereby discourage investment in new and technologically superior machines and
equipment (although this effect may apply to low skilled immigrants only, as high-skilled
immigrants are likely to exhibit complementarity with new technology in production). Second,
cultural diversity at the workplace can create communication problems, simply due to linguistic
difficulties but also due to misunderstandings associated with differences in tastes, norms and
values between immigrants and the native born (e.g., Alesina and La Ferrara 2005). Such
misunderstandings could negatively impact on the effectiveness of day to day interactions and
the innovativeness of teams. Heterogeneity in norms and values may also lead to distrust or even
outright conflict. In any case, decision-making in a diverse workplace may be more time consuming and therefore more costly. Cultural diversity of staff may also trigger discrimination
of minority groups, which in turn lowers the performance of the firm and thereby leaves less
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
C. Ozgen et al.
funding for R&D. The relative youthfulness and high mobility of immigrants noted above may
also have a downside: immigrants often have less job tenure and therefore less job-specific
training. If they contemplate repeated migration, they may also exhibit less commitment to
the firm. All such phenomena could negatively impact on knowledge spillovers and innovation.
A further consequence of growing migrant diversity is that it may to lead to greater
geographical clustering, segregation or polarization, a trend referred to by Bishop (2008) as
‘the big sort’. While ethnic precincts may have positive urban consumption externalities, in
terms of the ethnic goods and services these provide, they may also limit the supply of foreign
workers outside the precincts or increase commuting distances for ethnic workers employed
elsewhere in the city. This increases the costs associated with recruitment of migrant workers
which may lower their employment in host country firms and thereby lower the potential of such
firms reaping economic benefits from cultural diversity.
Given such a mixture of positive and negative effects, it is clear that the overall impact of
immigration on innovation is largely an empirical matter. In this context, various approaches
are possible. The first is to test whether there is more innovation, ceteris paribus, in geographical areas (cities, regions) that are more culturally diverse. Several empirical case studies suggest
that agglomerations with greater cultural diversity among their foreign-born residents have
indeed higher levels of productivity or innovation (Ottaviano and Peri 2006; Suedekum et al.
2009; Niebuhr 2010; Ozgen et al. 2012). On the other hand, a larger share of immigrants in
the labour force is often associated with lower levels of innovation, particularly when the issue
of endogeneity of migrant settlement (with migrants predominantly settling in larger agglomerations that are more innovative) is taken into account (Ozgen et al. 2012). Maré et al. (2014)
find no evidence that the presence of migrants in the region boosts firm-level innovation in
New Zealand.
The second approach to identifying benefits of immigration for innovation is to focus on foreign knowledge workers themselves and to compare the innovation activity of these workers,
for example, the extent to which they generate patents, with those of native-born knowledge
workers. According to this approach, highly-skilled foreign workers are assumed to be direct
contributors of innovations. The evidence is fairly conclusive that host countries benefit from
attracting highly able knowledge workers from abroad (e.g., Faggian and McCann 2009; Zucker
and Darby 2007; Hunt and Gauthier-Loiselle 2010; Kerr 2010; Kerr and Lincoln 2010).
The third approach is to consider the ethnic composition of staff at the workplace level
within firms. This is arguably the toughest test of spillover benefits of cultural diversity on
co-workers, since it attempts to empirically identify localized spillover benefits of foreign
workers who mostly represent only a small minority of the employees. At this level, the evidence to date is less conclusive. Ozgen et al. (2013) find – using micro-level cross-sectional data
in the Netherlands – that firms with a greater share of migrant workers are on average less
innovative, but that cultural diversity among a firm’s foreign born employees enhances innovation. Similar evidence has been found by Niebuhr and Peters (2012) for Germany, Parrotta et al.
(2014) for Denmark, McLeod et al. (2014) for New Zealand and Lee and Nathan (2010) for
firms in London. However, using similar data as Ozgen et al. (2013) utilize for the Netherlands,
Østergaard et al. (2011) find no significant effect of ethnic diversity on innovation at the firm
level in Denmark. Moreover, it is often difficult in this literature to find suitable instruments
to account for reverse causality.
Managers themselves often attest to a diverse workforce being crucial for encouraging different perspectives and ideas that drive innovation (e.g., Forbes 2011). However, the evidence is
not conclusive that management practices that encourage ethnic diversity always have a positive
impact on a firm’s performance. Considering explicitly how teams perform within firms, a metaanalysis by Bell et al. (2011) concludes that ethnic diversity negatively impacts on team performance. On the other hand, taking the very specific case of sports teams, Alvarez et al. (2011)
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
Workplace diversity and firm innovation
provide evidence that the recruitment of foreign players in European basketball teams benefits
the overall performance of these teams. In any case, various interventions have been suggested
in the literature to mitigate any negative impacts of diversity (Ely and Thomas 2001). Experimental approaches (e.g., exploiting exogenous spatial variations in immigrant recruitment or
some randomization) as well as employing latent class models that help to unravel heterogeneity
may improve the identification of the influence of within-firm effects in team studies.
There are however several explanations for the varying and conflicting results found in the
literature (see, e.g., Stahl et al. 2010). One possibility is an inverted-U shape relationship between diversity and economic performance, suggesting a notion of optimal diversity. De Graaff
and Nijkamp (2010) provide some tentative evidence from spatial data in the Netherlands that
such an inverted-U shape relationship between cultural diversity and economic performance
emerges there with respect to income, students and house values. There may also be other
factors that offset the possible gains from diversity such as institutional cultures and moderating
factors that enable or disable the benefits of diversity, even simultaneously. These factors
include task complexity, size of diverse teams, team tenure and team dispersion (Stahl et al.
2010). Earlier findings also point out that the positive impact of diversity on innovation may
occur mostly at the city level and point to broad spillover benefits in consumption, production
and endogenous institutions, such as had already been highlighted by Jacobs (1969).
The next section describes the econometric model, while Section 3 outlines how the dataset
was constructed and how cultural diversity has been measured. Section 4 discusses the results of
the regression analysis and Section 5 offers concluding remarks.
2 Specification and identification of the econometric model
Our econometric models explain the variation in product and process innovation across firms
and over time in terms of characteristics of the firm, the employees and the area. The major
source of innovation data is the Dutch Community Innovation Survey (CIS). This survey does
not report the number of patent applications, patents granted or other quantitative measures of
innovation. Instead, the response variables are binary and signal whether a firm introduced
new products and/or services (referred to as a product innovation) or new processes (referred
to as a process innovation) within the two years preceding the survey.2 Given these two
response variables we select firms as follows: those who report product innovations and those
who report process innovations.3 The firms in our dataset come from the surveys CIS 3.5
(2000–2002) and CIS 4.5 (2004–2006). They provide the anchor of our empirical strategy.
2.1 Measuring firm-level cultural diversity of employees
As noted in the introductory section, employee diversity impacts on firms through various channels. The appropriate definition of a diversity measure therefore depends on the research question and on the nature of the data as it is a complex social construct. In most studies, it is
The exact definitions in the CIS are as follows. A product innovation is the market introduction of a new good or
service or a significantly improved good or service with respect to its capabilities, such as improved software, user
friendliness, components or sub-systems. A process innovation is the implementation of a new or significantly improved
production process, distribution method, or support activity for a firm’s goods or services.
The way we define the set of firms in each category does not allow us to compare firms that are only product innovators with firms that are only process innovators. Although this comparison would be theoretically of interest, in reality
most product-innovating firms in our sample are also active in process innovations. When we select only firms with
either type of innovation but not both, the selected number of firms becomes very small (less than 20% of the current
sample) and they are not representative for all firms; moreover, they would cluster in only the wholesale trade & repair
and machinery & equipment sectors with respect to product innovations; and in the wholesale trade & repair, other
business services, and transport & communication sectors with respect to process innovations.
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
C. Ozgen et al.
assumed that ‘all aspects of differences among people affect groups in the same way’ (Stahl
et al. 2010, p. 691). In reality different manifestations of diversity arise even for the same
domain (ethnicity, religion, nationality, etc.). Diversity-related processes can enhance a group’s
performance to the extent that the independence of individuals within groups is ensured
(Surowiecki 2004), while they may alternatively diminish productivity to the extent that convergent processes prevail such that group thinking dominates over acceptance of new ideas (Stahl
et al. 2010). Hence, we aim to disentangle different aspects of cultural diversity by recognizing
the exposure of different groups to each other within the same firm. For example, a sample of
workers can be very diverse, yet can have significant clustering with respect to some groups.
We scrutinize therefore in this paper three aspects of the cultural diversity of the workforce of
a firm: (i) a measure of composition that includes the native born; (ii) cultural homogeneity
of the foreign workers; and (iii) the unique number of countries of birth represented by the
The cultural composition (including the native born) of the total workforce of a firm is measured by the Simpson index. This index is calculated as follows:
Simpsonit ¼ 1
Pbit ðPbit 1Þ
Lit ðLit 1Þ
in which Pbit is the number of employees with country of birth b (b=1,2,…,B) and Lit (i=1,…,N)
is the total number of employees of firm i at time t. The Simpson index measures the probability
that a pair of randomly selected individuals belong to two different groups. When all employees
originate from the same country, the index value is 0. The index approaches 1 when there are
equal numbers of employees originating from many different countries.4 However, the Simpson
index is insensitive to the addition of rare groups to the sample and, obviously, abundant groups
get more weight. Because natives are by far the most dominant group in a firm’s workforce, the
Simpson index can be interpreted as a measure of ‘non-Dutchness’ of a firm in our data. The
higher the share of Dutch-born workers, the lower the Simpson index.5
The second diversity measure focuses on exposure to own kind among foreigner workers.
The co-location index shows to what extent a foreigner is exposed, within the firm, to other
foreigners who share his/her country of birth. The index is calculated as follows:
Colocit ¼
F it X
F it
l¼1 k¼1
I ðbl ¼ bk Þ
= Lit ;
where bl, bk (bl, bk = 1,…,B) indicates the country of birth of a foreign employee l,k (l,k=1,…,
Fit) in firm i (i=1,…,N) at time t, and I stands for the indicator function that takes on the value
1 if the two employees l and k within the firm share the same country of birth and zero otherwise. The co-location index takes on the value zero if the firm has no two foreign workers
sharing the same country of birth, or when the firm is composed of natives only. The index
value is equal to the sum of the number of coincidences per firm divided by the total number
When the employees are equally spread across B birthplaces, the value of the index is (1 1/B) x Lit/(Lit 1).
The Simpson index used in the estimations is almost identical to the conventional fractionalization index, defined by
for example, Alesina and La Ferrara (2005). The only difference is that the fractionalization index measures the probability of different ethnicities among two individuals when these are being randomly drawn with replacement, whereas for
the Simpson index the draw of the second individual is assumed to be without replacement of the first person (i.e. it
assumes drawing a pair of workers). The Simpson index is intuitively more appealing, particularly in small firms. However, all the estimations reported in this paper are robust to replacing the Simpson index by the fractionalization index.
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
Workplace diversity and firm innovation
of employees, Lit, in firm i. If all Fit foreign workers were born in the same country Colocit is
½[(Fit (Fit 1))/Lit]. It is clear that the larger the value of Colocit is, the more the homogeneity
of foreign employment in firm i.
Third, the number of countries of birth represented by the firm is another important measure of a diverse workforce. If one considers that each country has its own distinct features,
the way people think, act, and work will vary with the number of countries represented in
each firm. Nobel Laureate Linus Pauling said: ‘the best way to have a good idea is to have
a lot of ideas’ (Uzzi and Dunlap 2005, p. 2). Thus, a simple count of the unique number of
countries of birth represented in each firm is our third measure of cultural diversity at the
firm level:
Uniqueit ¼
U bit ;
in which Ubit is a dummy variable that is equal to one when country b is represented in
firm i at time t and zero otherwise.6 It has been suggested in the literature that the benefits
of this type of diversity may occur only up to a certain degree. When the variety of the
backgrounds is too diverse, costs of communication and transaction may increase sharply
(the ‘Babylon effect’; see Lazear 1999; Florax et al. 2005; De Graaff and Nijkamp 2010). To
account for costs increasing with diversity, we include the quadratic of the unique variable in
the estimations and expect an inverse-U shape relationship with innovation.7
Whether or not a firm reports an innovation is a function of various firm characteristics, employee characteristics, the diversity measures explained above, and characteristics of the local
economy (for summary statistics see Table 1). The inter-temporal variation is quite high in
the group of variables on firm characteristics; for the employee and location controls the variation over time is rather modest (Table 1 shows these trends for selected variables between 2002
and 2006). Clearly the panel nature of the data allows us to control for unobserved timeinvariant firm-specific factors, although it may lead to larger standard errors in identifying
effects of slow-trending variables. In order to set a benchmark, we first estimate the following
pooled data logit model, while we outline next the strategy of how we deal with low intertemporal variation of employee variables:

PrðY it ¼ 1jdatait Þ ¼

F β0 þ β1 Simpsonit þ β2 Colocit þ β3 Uniqueit þ x′it βx þ δt þ φi þ εit ;
in which Yit is a dummy variable that measures firm i’s self-reported innovation, ‘yes’ or
‘no’ as explained above, in panel wave t; x′it is a vector of covariates determining the firm’s
innovativeness; φi and δt are firm and time effects respectively; εit is a random error term;
and finally F is the standard logistic cumulative distribution function with F ðwÞ ¼ 1þe
w . As
an alternative, we also estimate linear probability models (LPMs) of which the coefficients
are easy to interpret and do not require the calculation of marginal effects (Angrist 2001).
In the panel LPM, we use the following specification where we allow for firm and time
fixed effects:

Y it ¼ G λ0 þ λ1 Simpsonit þ λ2 Colocit þ λ3 Uniqueit þ x′it λx þ τ t þ vi þ ξ it ;
The number of countries of birth represented in a firm varies between 1 (Netherlands only) and 197.
We acknowledge with gratitude that this specification was first proposed to us by one of the referees.
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
C. Ozgen et al.
Table 1. Summary statistics
Firm variables
Firm innovated products
Firm innovated processes
Firm size (number of employees)
Obstacles: Lack of personnel
Obstacles: Cost
Number of firms per job (Nuts 3 level)
Number of firms per municipality
Employee variables
Simpson index
Co-location index
Unique number of countries of birth
Share of young employees
Share of high-skilled employees
Foreignness of young employmentb
Foreignness of high-skill employmentc
Share of foreign born in the firm
Unique number of birth countr. per municipality (ref. 1996)
Allochtoon population per municipalityd (ref. 1996)
Pooled data
Std. Dev.
Notes: Due to the confidentiality agreement with Statistics Netherlands, minimum and maximum values of the variables
cannot be displayed in Table 1. The firm variables are from 2002 and 2006, while all employee variables are lagged by
two years per corresponding response year of the CIS. bShare of foreign born among the employees aged 25–44 per firm.
Share of highly-skilled foreign born among the high-skilled employees per firm. dIn the Netherlands this terms refers to
persons who have at least one parent who was born abroad.
where the variables are defined as before, vi is the firm fixed effect, Ï„ t is the time fixed
effect, ξ it is the error term and G(w)=w when 0 ≤ w ≤ 1, G(w)=0 when w1.8
There are four sets of explanatory variables. The first is the set of diversity measures
discussed above. The second group of variables consists of firm characteristics. The first of
these is firm size, which accounts for firms’ fixed assets, and therefore scale and specialization
effects, which facilitate innovation. Obstacles to firm profitability and growth, such as a lack of
personnel and finance constraints may force a firm to seek innovative solutions to such obstacles. Observations on the incidence of such obstacles provide additional firm characteristics.
As an alternative to firm fixed effects, we also consider pooled data regressions with 22
macro-sector fixed effects.9
One typical issue with the fixed effects model is the problem of predicting the effect of time-invariant or slowtrending variables. Given the slowly changing composition of employment and sector-specific characteristics, the variation in employee variables is rather little in our dataset. Random effects models might remedy estimating effects of
these variables but it may impose an unrealistic assumption of time-constant individual effects being uncorrelated with
the observed explanatory variables. Correlated random-effects models (Mundlak 1978) introduce the group means of the
time-trending variables into the standard random effects procedure to model the dependence between unobservable and
observed characteristics Xit by assuming that the regression function of vi is linear in the mean values of the time-varying
covariates XÌ„ i (Rabe and Taylor 2010). Thus, in order to account for the slow-trending variables, the group means of the
time-varying variables are included into the econometric specification as new covariates. The Mundlak estimations with
correlated random-effects model produces fairly similar results to that of fixed effects model. They can be obtained from
the corresponding author upon request.
The 22 sector fixed effects match with the 2-digit international industrial NACE classification.
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
Workplace diversity and firm innovation
The third group of variables is a set of employee characteristics that includes the demographic and occupational characteristics of the workers. As noted previously, the cultural background of an employee is defined by country of birth. Clearly, country of birth is only an
approximate measure of culture but has the advantage that, unlike measures of nationality or
ethnicity, it is constant and objective. By combining descendants of migrants with all native
born, we adopt a narrow measure of diversity. If cultural diversity effects are found with our
data, they may be expected to be present also in broader measures of cultural or ethnic diversity.
We focus specifically on the demographic and occupational characteristics of foreign and
native born employees. To test whether firms employing young workers are more innovative
(e.g., Poot 2008), the age composition of a firm’s workforce is included and measured by the
share of 25–44 year olds in total employment per firm. Similarly, we consider the share of
high-skilled workers in total employment. Moreover, we also include two variables that measure
the relative skill composition and youthfulness of the foreign born vis-à-vis the native born: the
share of high-skilled foreign employees in total high-skilled employment and the share of foreign young employees among the total number of young employees. For both waves of the
panel, all of the employee variables are lagged by two years to diminish the impact of reverse
causality (in which innovative firms are more likely to recruit young or high-skilled workers).
The fourth group of variables relates to the area in which the firm is located and specifically
to urban economic agglomeration and its possible impacts on the innovativeness of firms. The
measures used in the analysis are market structure (the ratio of firms over jobs per COROP region) and a density variable (the number of firms per municipality).10 The former variable measures the extent of local competition while the latter measures the extent of agglomeration.
2.2 Instrumental variables strategy
Given that the core research question of this paper is the impact of cultural diversity of a firm’s
employees on the firm’s reported innovativeness, the possibility of reverse causality (in which
innovative firms actively promote diversity and creative migrant workers are disproportionally
attracted to, and recruited by, innovative firms) should be seriously considered. In this context, it
is not possible to find in the Netherlands’ recent history a set of policies that led to a countrywide exogenous assignment of foreign workers to companies (i.e., a suitable natural experiment). A randomized variation in cultural diversity across firms is even less realistic. Hence
we can only account for endogeneity bias by instrumental variables (IV) estimation. This remains by far the most common technique to address such endogeneity issues.
We follow the common practice of using ‘deep lags’ (e.g., Nickell and Nicolitsas 1999) and
look for a set of variables that for various reasons (such as historical policies) are likely to be
good predictors of cultural diversity of firm employment in The Netherlands, but which are unlikely to be correlated with the error term in the innovation regressions. Hence we explain each
of the diversity measures in terms of a vector of exogenous variables.
The Simpson index includes the natives as a category in the calculation of diversity. This
index is highly correlated with the share of foreigners in total firm employment. Therefore,
we use the natural logarithm of the lagged allochtoon population (first and second generation
immigrants) per municipality as an instrument and apply the maximum lag the data permit, which
is six years. The allocation of the stock of migrants across municipalities in The Netherlands in
1996 is the cumulative result of migration flows in previous decades that are a consequence of
many socio-economic and institutional factors (e.g., Greenwood 1969). It is reasonable to
assume that these are not correlated with the error term of the 2002 and 2006 innovation
The COROP division refers to functional regions that are based on the commuting distances in the Netherlands. At
the European level, it corresponds to the NUTS 3 level division.
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
C. Ozgen et al.
regressions. In any case, after establishing the endogeneity of a diversity measure, we test for the
strength and validity of the instrument.
The co-location index is a measure of within-firm homogeneity of foreign employment.
Firms are likely to employ foreigners from their vicinity so that the cultural composition of employment in the local area should be an important factor in identifying within-firm diversity.
Hence we instrument the co-location index with the number of countries of birth present at
the municipal level six years before.11 This is a good instrument because the motivation for
migrating to the Netherlands has shown since the late 1990s a remarkable shift from family
reunification and asylum intentions to study and work so that we would not expect the 1996
country of birth composition of the population of municipalities to be correlated with the error
term in the regressions of firms’ 2002 and 2006 innovativeness.
Finally, we instrument similarly the ‘unique’ variable (the number of distinct countries of
birth present within the firm) and its quadratic by the corresponding values lagged four years.
3 Data and construction of the sample
We combine two confidential high-quality firm/individual level micro-datasets obtained
from Statistics Netherlands. As noted in Section 2, the Community Innovation Survey CIS
3.5 (2000–2002) and CIS 4.5 (2004–2006) are the anchors of our empirical strategy. The
surveys straddle over two years between the time the questionnaires are sent to firms and
the reporting of results. Each survey yields about 11,000 observations.12 The employee data
are retrieved from the Tax Register in the Netherlands (SSB_Banen) which is a census of
employees in the Dutch labour market. This dataset covers about 10 million employees,
who can be observed multiple times (for more details see Ozgen et al. 2013). Finally, the
ethnic and demographic background information of the employees is obtained from the
Dutch Municipal Registers (GBA) which provide a census of the 16 million people living
in the Netherlands. Firms are company establishments with autonomous production and
decision features.
To create our sample we followed a three-step data linking procedure. First, the two
cross-sections of CIS are linked to create a balanced panel of firms that can be followed
over the four years. Second, the panel of firms is linked to the tax registers (SSB_Banen)
to obtain the actual number of employees per firm and by location. At the third step, this
new dataset is merged with the Dutch Municipal Registers (GBA) to obtain the actual number of foreign employees per firm, as well as their countries of birth and various other
demographic characteristics. As a result of our series of merges, we obtained a sample of
2,793 firms that responded to both CIS surveys. Hence our dataset consists of a 2-wave
panel with 5,586 observations.
None of the datasets comprise information on the occupation and/or education of the
employees. The large literature on earnings functions suggests a strong correlation between
earnings and education (Card 1999). Moreover, in the Netherlands the criterion to obtain a
visa targeted at highly-skilled workers is to earn more than a given threshold level of
The regional level instruments are widely used in the economics of cultural diversity research to account for the
diversity of the labour market area in which the firm is located, for example see Parrotta et al. (2014).
To create the sample, Statistics Netherlands selects the firms from the General Business Register. Public-sector and
NGO-type of activities are excluded. A further selection is based on firm size. Firms employing less than 10 persons are
not included in the sample. Firms employing more than 50 persons are all included in the sample. For firms employing
10 to 50 persons, only a fraction is randomly selected into the sample. The size of this fraction depends on the industry
and the firm size.
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
Workplace diversity and firm innovation
earnings. We created therefore three skill categories by assigning employees into these
categories based on their gross annual wages. The gross annual wage of each employee in
the sampled firms is gathered from the Tax Register database. When assigning the
employees to skill categories, we benefited from macro-level statistics provided on Statistics
Netherlands’ website, which cross-tabulate annual mean income of employees by education
level. A similar tabulation of mean gross earnings of all employees against skill level (low,
medium, high) is also available. Based on this information, we assigned people with a
minimum gross annual wage of 42,000 euro to the high-skill category.
3.1 Descriptive statistics of the sample
As noted above, our sample is composed of a balanced panel of 2,793 firms. Table 1 provides a
range of summary statistics. Product innovations are slightly more common than process innovations and both types of innovations increased somewhat between 2002 and 2006. This coincided with a slight increase in cultural diversity by all three measures (the co-location index
declined and the unique number of countries of birth and the Simpson index increased).13 On
average, 38 per cent of firms announced some type of innovation at least once. There is considerable persistence in responses: about 500 firms reported innovations in both surveys and about
1,500 firms did not innovate at all. About 370 firms innovated in the first period but not in the
second, while about 400 firms innovated in the second period but not in the first. The top five
most innovative sectors in our dataset are: machinery & equipment; wholesale & trade;
chemicals; other business services; and metals. These sectors are also those in which the share
of high-skilled workers is the highest. Firms are clustered in various regions. The top five
locations, where 34 per cent of the firms are located, are: Great-Amsterdam, Great-Rijnmond
(Rotterdam), Utrecht, Twente and southeast North-Brabant.
The 2,793 firms employ close to one million employees of whom 11 per cent were born
abroad. About 30 per cent of the foreign workforce originated from the European continent.
On average, there are 10 unique countries of birth represented in each firm and the co-location
index is 0.265. The mean Simpson index is 0.186. As noted above, all three measures show that
diversity increased between 2002 and 2006. There has been particularly an increase in the number of eastern-Europeans and east-Asians (not shown in the table). Comparing 2002 and 2006,
the share of foreign workers among the young increased but the share of foreign workers among
the high-skilled workers decreased. This coincides with a period during which overall immigration in the Netherlands declined (e.g., Berkhout and Sudulich 2011). Migrants are on average
less skilled than the native born (by 2006 migrants accounted for about 8% of high-skilled
employment and 11.2% of all employment). High-skilled foreigners account for 22 per cent
of all foreigners while high-skilled natives are 26 per cent of all natives. However, migrants
are on average younger than the native born population: the shares of young employees are
65 per cent and 59 per cent respectively. Against the long-run trend, the share of high-skilled
employment in the workforce declined in the sample between 2002 and 2006, but the workforce
became slightly older, as expected.
Our data predominantly comprise firms with 100 employees or more. The average firm size
is 171. As is to be expected, there is a rough positive relationship between firm size and the
number of foreigners the firm employs (see Figure 1). However, while large firms are more
likely to be diverse in terms of the unique number of countries of birth present, they are not
necessarily the most diverse in terms of the other two diversity measures.
Recall that the Simpson index increases with increasing diversity, while the co-location index decreases with
increasing diversity. However, the Simpson index is in our data uncorrelated with the unique number of birthplaces
and with the co-location measure. The correlation coefficient between the unique number of birthplaces and the
co-location measure is about –0.27.
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
C. Ozgen et al.
Fig. 1. Scatter plot of the number of foreign-born employees and firm size
4 Results of the regression analysis
4.1 Determinants of firm level product and process innovation
In this section, we proceed with reporting the regression analysis of a balanced panel of 2,793
firms observed twice (2000–2002 and 2004–2006). We compare product and process innovation because a pure process innovation changes how a product is made, but would not have
any effect on the product itself (Swann 2009). In other words, the factors boosting product
innovation can be different from those enhancing process innovation. Unfortunately the CIS
data do not allow us to deeply examine the underlying factors (especially for the case of process
innovations, such as the extent to which foreigners are actively participating at various management levels).
We first estimate pooled-data logit models with sector and period fixed effects and then logit
and LPM panel models with firm fixed or random effects. In addition, we also estimated a
random effects model with the Mundlak (1978) approach (see footnote 8). Table 2 however
shows four regressions. In these estimations, the standard errors are clustered at the firm level,
where applicable.
Table 2 presents the determinants of firm product innovation. Specification 2.1 is a
pooled logit model of product innovation incidence with firm-level variables, regional
controls, and employee characteristics. The results are in conformity with the innovation
literature in that a firm’s characteristics are important drivers of innovation. Firm size
is, as expected, a significant factor (at the 1% level) that boosts innovation. Similarly,
firms which reported costs and lack of personnel as obstacles to firm profitability and
growth, appear to be more innovative (the reference values are no lack of personnel
and no cost obstacles respectively). The results suggest that a lack of personnel is an
important driver of innovation (but differences between the low, medium and high
levels of personnel shortage are not statistically significant). With respect to costs, the
impact is non-linear. Low or medium cost constraints trigger innovation relative to an
absence of concerns about costs or, alternatively, high costs being a problem (in the
latter case there is presumably no funding for innovation). We do not find a statistically
significant impact of the competition variable (number of firms per job), nor an effect
of the agglomeration variable (the natural logarithm of the number of firms per
The pooled data logit model estimates show that three out of the eight employee variables
are significant. The Simpson diversity index has the expected positive coefficient, but is
Papers in Regional Science, Volume 96 Supplement 1 March 2017.
Pooled Logit
0.652*** (0.0758)
1.291*** (0.132)
1.293*** (0.149)
1.262*** (0.209)
0.694*** (0.146)
0.380* (0.160)
0.193 (0.219)
0.787 (2.495)
–0.0671 (0.0526)
–0.454** (0.160)
–0.0109 (0.0133)
–0.000120 (0.000151)
1.904*** (0.278)
2.300*** (0.357)
–1.102 (0.895)
–0.463 (0.442)
1.248 (0.801)
Product innovation
Logit FE
0.574* (0.305)
1.237*** (0.282)
1.017*** (0.299)
1.407*** (0.500)
0.475 (0.301)
0.0134 (0.321)
–0.0884 (0.431)
15.26 (26.06)
1.056 (1.966)
–0.214 (0.323)
–0.0135 (0.0447)
0.000165 (0.000507)
1.975* (1.179)
0.473 (1.052)
–1.184 (1.631)
1.076 (0.812)
Product innovation
0.0517** (0.0245)
0.127*** (0.0257)
0.109*** (0.0294)
0.135*** (0.0410)
0.00480 (0.0306)
0.00531 (0.0424)
1.782 (2.305)
0.0703 (0.191)
–0.0271 (0.0323)
–0.00235 (0.00483)
1.42e-05 (6.81e-05)
0.0533 (0.0873)
0.0215 (0.0905)
–0.0189 (0.126)
0.0788 (0.0874)
Product innovation
0.0835*** (0.00881)
0.188*** (0.0208)
0.182*** (0.0240)
0.197*** (0.0343)
0.106*** (0.0242)
0.0416* (0.0252)
0.0157 (0.0356)
0.0874 (0.379)
–0.0107 (0.00801)
–0.0598*** (0.0197)
0.000429 (0.00134)
2.95e-05** (1.23e-05)
0.294*** (0.0444)
0.240*** (0.0470)
–0.0232 (0.0648)
–0.0454 (0.0565)
Product innovation
Notes: aThe reference category for the obstacle variables referring to a lack of personnel and to cost is in both cases: no obstacle reported. Estimation (2.1) includes time and sector fixed
effects. ***p < 0.01, **p < 0.05, *p < 0.1. Robust standard errors are given in parentheses except in (2.2). N Estimation technique Log firm size Obstacles: Lack of personnel (low level)a Obstacles: Lack of personnel (medium level)a Obstacles: Lack of personnel (high level)a Obstacles: Cost (low level)a Obstacles: Cost (medium level)a Obstacles: Cost (high level)a Number of firms per job (Nuts3 level) Ln(number of firms per municipality) Co-location Index Unique 2 Unique Share of high-skilled employees Share of young employees Foreignness of young employment Foreignness of high-skill employment Simpson Variables Table 2. Determinants of firm product innovation Workplace diversity and firm innovation S41 Papers in Regional Science, Volume 96 Supplement 1 March 2017. S42 C. Ozgen et al. statistically insignificant.14 The co-location index (which measures the extent that a foreign employee is exposed to others who share her/his country of birth) is negative, as expected, and significant at the 5 per cent level. At face value, this would suggest that clustering of workers from the same origin (i.e., a high co-location index) may lead to grouping internally and exclusion of ‘others’ (non-group members), which would have a negative impact on innovation. However, this effect vanishes in subsequent regressions where firm fixed effects are included. The unique variable, which measures the ‘richness’ of countries of birth represented in a firm, is insignificant in regression 2.1, and its quadratic has a negative coefficient that is significant at the 5 per cent level only in specification 2.4. Therefore, we do not find any evidence in this table of an inverted-U-shape relationship between product innovation and the number of countries of birth present among employees. In line with the literature, we find in column 2.1 that an increasing share of young workers (aged 25–45) and an increasing share of high-skilled employees are important drivers of firms’ innovativeness. Both variables are positive and significant at the 1 per cent level. The foreignnative born mix of these groups of workers does not appear to matter. The time dummy coefficient (not reported in Table 2) is significant and negative; suggesting that the upward trend in innovation reported in Table 1 is due to changing firm and employee characteristics rather than a secular national increase in innovativeness over the 2002–2006 period. Sector dummies (also not reported due to space constraints) are as expected in terms of the literature. Firms in the chemicals sector, as well as in the machinery & equipment sector, are more likely to innovate than firms in other sectors. Specification 2.1 signals through the negative co-location index coefficient that there is a positive effect of cultural diversity on firm innovation in (pooled) cross-sectional data, as has been found in various previous studies. The key benefit of our panel data, however, is to control for unobserved (but time invariant) firm effects that may influence both innovation and the cultural composition of employment. In specification 2.2 we therefore introduce firm fixed effects into the logit estimation. In contrast with standard logistic regression, fixed effects models focus on the changes in innovation behaviour within firms over time. However, as noted in the previous section, only about 900 firms show variation in their innovation performance over the four years. Therefore, the number of observations in this maximum likelihood estimation is less than a quarter of that in the first specification. Comparing results of the pooled data logit model versus the fixed effects logit model, the key change is that all employee variables, except the impact of highly skilled employees, turn insignificant. We confirm that high-skilled employees positively impact product innovation. The firm variables behave as before. Given that employee variables are rather slowly changing over time, four years may not be long enough to reveal the impact of such variables in a fixed effects setting. Hence, in fixed effects estimation our findings are not in favour of cultural diversity enhancing a firm’s product innovation. The logit fixed effects model not only comes with a cost of losing a substantial number of observations but also requires extra steps to calculate the marginal effects of the coefficients. Following Maré et al. (2014) and given the larger number of observations in a panel binary response model, and ease of interpretation of results, we proceed therefore with the linear probability model (LMP).15 Estimation 2.3 shows that the findings from the panel fixed effects logit and LPM models are qualitatively similar (but the coefficient can be interpreted as marginal effects in regression 2.3 but not in 2.2). Although random effects specification of regression 14 The Simpson index is insignificant in all estimations, and rather correlated with the share of young employees variable. In order to gain efficiency, we exclude it from the estimations in the subsequent regressions. 15 Given that we have relatively large sample of 2,793 firms and that innovation is not a ‘rare’ event, Monte Carlo simulations back up the common practice in labour economics of using a linear probability regression rather than a logit model to calculate average marginal effects (Beck 2011). Papers in Regional Science, Volume 96 Supplement 1 March 2017. Workplace diversity and firm innovation S43 2.3 is rejected on the basis of the Hausman test, for the sake of methodological comparison we present the findings from the random effects model in estimation 2.4. Specification 2.4 reconfirms the earlier findings of the pooled logit estimation 2.1 and suggests that cross-sectional level firm characteristics as well as employee diversity and qualities are important drivers of product innovation. While random effects models have the advantage of accounting for slow trending regressors, the assumption of uncorrelated unobserved individual factors with the covariates is too strong to be confident of these results. Finally, we did not find evidence of market competition and local density augmenting the probability to innovate products in any of the four regressions in Table 2. Firms that innovate processes are expected to value different ways of thinking and a larger pool of new ideas. In Table 3 we report the results for the process innovation in similar fashion to Table 2. Pooled logit estimates in column 3.1 show that the firm’s internal resources are important contributors of process innovations while the area where firm is located does not seem to be an important source for process innovations. However, once firm fixed effects are included in the regressions both in non-linear and linear estimations in columns 3.2 and 3.3 respectively, it is shown that for process innovation the firms indeed significantly benefit from being located in denser areas. Similar to the determinants of product innovation, we find in column 3.1 very significant evidence in favour of high-skilled employees and young employees. However, against expectation, the coefficient of the ‘foreignness of high-skill employment’ (the percentage of highskilled workers who are foreign) is negative and statistically significant at the 5 per cent level. The co-location index is negative and significant in column 3.1, suggesting higher clustering of same-country employees at the workplace impedes process innovation. As explained previously, the quadratic in the unique variable captures the idea that firms in which a larger range of countries of origin are represented may generate a wider range of ideas to improve processes but that too many different nationals may generate communication and administrative problems. This idea is rejected in specifications 3.1 and 3.2 but we find some evidence for this argument in specifications 3.3 and 3.4. The linear and quadratic terms of unique are jointly significant in the LPM fixed effects estimator of estimation 3.3 at the 10 per cent level and in the random effects estimation at the 1 per cent level. None of the other employee variables are significant in fixed effects estimation, but the co-location index, the share of high-skilled employees and the share of young employees are significant in random effects estimation. 4.2 Endogeneity With the regressions presented so far the question remains whether the innovativeness of the firms is increasing due to the diversity of the employment composition, or whether immigrant workers are sorted into firms that are more innovative. We address this potential endogeneity problem with an instrumental variables strategy that was outlined in Section 2. Tables 4 and 5 present the 1st and 2nd stage estimations of IV regressions for product and process innovations respectively, for the potentially endogenous diversity variables. For brevity, only the coefficients of employee variables are reported in these tables, while all the variables in Tables 2 and 3 are again included in the estimations. The first two regressions of Table 4 report results obtained with a (pooled) LPM, testing for the causal effects of the Simpson index and the co-location index separately. Regressions 4.1 and 4.2 provide results of pooled data IV estimation with sector and time, but without firm fixed effects. The instruments are also shown in Table 4. The instruments we use are robust to conventional tests of excluded instruments. The results suggest that the negative effect of the Co-location index that we found in Table 2 fails against endogeneity controls, even without controlling for firm fixed effects in columns 4.3 and 4.4. When we repeat the same regressions Papers in Regional Science, Volume 96 Supplement 1 March 2017. Papers in Regional Science, Volume 96 Supplement 1 March 2017. 4,691 Pooled Logit 0.529*** (0.0694) 1.046*** (0.134) 0.985*** (0.149) 1.147*** (0.207) 0.824*** (0.147) 0.360** (0.161) 0.0482 (0.208) 3.067 (2.516) –0.0801 (0.0514) –0.261* (0.156) 0.0161 (0.0113) –0.000252** (0.000121) 1.210*** (0.283) 1.976*** (0.347) –1.322 (0.985) –0.887** (0.448) 1.336 (0.840) (3.1) Process innovation 970 Logit FE 0.315 (0.298) 0.799*** (0.274) 0.430 (0.285) 0.977** (0.489) 0.591** (0.292) 0.0521 (0.325) –0.233 (0.434) 3.779 (23.83) 4.896*** (1.995) 0.111 (0.311) 0.0596 (0.0487) –0.000508 (0.000662) 0.827 (1.068) 0.205 (1.064) –1.185 (1.469) –0.904 (0.799) (3.2) Process innovation 4,698 LPM FE 0.0372 (0.0252) 0.109*** (0.0300) 0.0618* (0.0353) 0.127** (0.0519) 0.0820** (0.0342) 0.000624 (0.0371) –0.0356 (0.0509) 0.519 (2.703) 0.474** (0.210) 0.00442 (0.0341) 0.00814* (0.00471) –0.000104* (5.90e-05) 0.0400 (0.0955) 0.0659 (0.0930) –0.0586 (0.151) –0.0815 (0.0906) (3.3) Process innovation 4,698 LPM RE 0.0723*** (0.00889) 0.172*** (0.0237) 0.154*** (0.0269) 0.200*** (0.0389) 0.141*** (0.0268) 0.0515* (0.0287) –0.00755 (0.0385) 0.466 (0.390) –0.0116 (0.00798) –0.0342* (0.0202) 0.00435*** (0.00136) –5.27e-05*** (1.20e-05) 0.208*** (0.0456) 0.235*** (0.0471) –0.00852 (0.0682) –0.120** (0.0563) (3.4) Process innovation Notes: aThe reference category for the obstacle variables referring to a lack of personnel and to cost is in both cases: no obstacle reported. Estimation (3.1) includes time and sector fixed effects. ***p < 0.01, **p < 0.05, *p < 0.1. Robust standard errors are given in parentheses except in (3.2). N Estimation technique Log firm size Obstacles: Lack of personnel (low level)a Obstacles: Lack of personnel (medium level)a Obstacles: Lack of personnel (high level)a Obstacles: Cost (low level)a Obstacles: Cost (medium level)a Obstacles: Cost (high level)a Number of firms per job (Nuts3 level) Ln(number of firms per municipality) Co-location Index Unique 2 Unique Share of high-skilled employees Share of young employees Foreignness of young employment Foreignness of high-skill employment Simpson Variables Table 3. Determinants of firm process innovation S44 C. Ozgen et al. 4,931 0.215 Pooled LPM 0.347*** (0.0535) 0.357*** (0.0887) 1.774 (1.687) 0.0586 (0.161) 4,939 0.304 Pooled LPM 0.325*** (0.0669) 0.273*** (0.0506) –0.0271 (0.0625) –0.0856 (0.0643) –0.0780 (0.547) (4.2) Product innovation (4.1) Product innovation –1.897 (1.773) 5.14 (0.0234) 5.09 (0.0240) –0.0038** (0.001) (4.2) Co-location index 7.53 (0.0061) 7.52 (0.0061) 0.0036** (0.001) (4.1) Simpson index 4,931 . LPM FE –0.0690 (0.189) –0.0914 (0.287) 5.688 (11.28) 0.856 (1.526) 0.76 (0.383) 0.765 (0.381) 0.0045 (0.005) (4.4) Co-location index 4,939 . LPM FE 0.0529 (0.0971) 0.105 (0.304) –0.181 (0.504) 0.0194 (0.211) –0.522 (1.746) (4.4) Product innovation 2nd stage estimation –7.914 (15.59) (4.3) Product innovation 0.54 (0.464) 0.540 (0.462) –0.0058 (0.008) 1st stage estimation (4.3) Simpson index (4.5) Unique 2 0.9957*** (0.0596) 2,453 0.293 Pooled LPM 0.00607** (0.00270) –0.00006** (0.00002) 0.372*** (0.0588) 0.349*** (0.0617) –0.124 (0.0930) –0.0394 (0.0751) (4.5) Product innovation 460.81 (0.000) 249.60 (0.000) 223.71 (0.000) 0.0019*** (0.0004) 0.6477*** (0.0403) –7.033** (3.596) Unique Notes: Estimations 4.1, 4.2, 4.5 include the same variables on firm and regional characteristics as in Table 2, as well as sector and time fixed effects. ***p < 0.01, **p < 0.05, *p < 0.1. Robust standard errors are given in parentheses. 1st stage estimations include all the variables presented in Table 2. Each diversity measure is included separately in the regressions. The test statistics for FE estimations are first stage F-values and respective p-values. N R-squared Estimation technique Simpson index Co-location Index Unique 2 Unique Share of high-skilled employees Share of young employees Foreignness of young employment Foreignness of high-skill employment F Test of excluded instruments (F; p-value) 2 Underidentification test (χ ; p-value) Number of birthplaces present in municipalities in 1996 Lagged ‘Allochtoon’ population in municipalities Lagged number of distinct countries of birth in firms Lagged number of distinct countries of birth in firms squared Variables Table 4. Instrumental variables estimations for product innovation Workplace diversity and firm innovation S45 Papers in Regional Science, Volume 96 Supplement 1 March 2017. Papers in Regional Science, Volume 96 Supplement 1 March 2017. 4,691 0.189 Pooled LPM 0.242*** (0.0566) 0.311*** (0.0783) 1.324 (1.557) –0.0204 (0.144) –1.359 (1.642) 4,698 0.238 Pooled LPM 0.207*** (0.0682) 0.249*** (0.0473) 0.0402 (0.0639) –0.111* (0.0651) –0.204 (0.482) (5.2) Process innovation (5.1) Process innovation 0.183 (0.668) 4,691 . LPM FE –0.0858 (0.284) –0.171 (0.801) 11.24 (30.65) 1.508 (4.293) 2nd stage estimation –15.60 (42.51) Process innovation (5.3) 0.18 (0.670) 6.09 (0.0136) 6.01 (0.1420) 7.27 (0.0070) 7.24 (0.0071) 1st stage estimation (5.3) Simpson index –0.0034 (0.008) –0.0043** (0.002) (5.2) Co-location index 0.0037** (0.001) (5.1) Simpson index 4,698 . LPM FE –0.0390 (0.216) 0.302 (0.500) –0.557 (1.114) –0.276 (0.424) –1.679 (3.326) Process innovation (5.4) 0.466 (0.494) 0.46 (0.496) 0.0039 (0.005) (5.4) Co-location index (5.5) Unique 2 438.90 (0.000) 0.9531*** (0.044) 2,415 0.230 Pooled LPM 0.00727*** (0.00274) –0.000062*** (0.00002) 0.284*** (0.0627) 0.265*** (0.0642) –0.0896 (0.0959) –0.107 (0.0781) Process innovation (5.5) 187.25 (0.000) 626.58 (0.000) 0.0015*** (0.0004) 0.6951*** (0.0394) –2.410 (3.079) Unique Notes: Estimations 4.1, 4.2, 4.5 include the same variables on firm and regional characteristics as in Table 2, as well as sector and time fixed effects. ***p < 0.01, **p < 0.05, *p < 0.1. Robust standard errors are given in parentheses. 1st stage estimations include all the variables presented in Table 2. Each diversity measure is included separately in the regressions. The test statistics for FE estimations are first stage F-values and respective p-values. N R-squared Estimation technique Simpson index Co-location Index Unique 2 Unique Share of high-skilled employees Share of young employees Foreignness of young employment Foreignness of high-skill employment F Test of excluded instruments (F; p-value) 2 Underidentification test (χ ; p-value) Number of birthplaces present in municipalities in 1996 Lagged ‘Allochtoon’ pop. in municipalities Lagged number of distinct countries of birth in firms Lagged number of distinct countries of birth in firms squared Variables Table 5. Instrumental variables estimations for process innovation S46 C. Ozgen et al. Workplace diversity and firm innovation S47 with firm fixed effects the instruments are not effective. Moreover, we still do not find a conclusive effect of the Co-location and Simpson indices on product innovation. The results for process innovation in Table 5 are identical. The significant effect found for the co-location index disappears once we account for reverse causality, with or without fixed effects. Estimating IV regressions with firm fixed effects is not possible with the unique variable, because we observe firms only twice in the data set and the lagged within-firm unique number of birthplaces is an instrument. Regression 4.5 and 5.5 show that the inverted-U-shape relationship between innovation and unique is robust to accounting for endogeneity, but only without firm fixed effects. Taking the coefficients of regressions 4.5 or 5.5 at face value, it can be easily calculated that the benefit of increasing country-of-birth diversity occurs up to 40 countries per firm (recall from Table 1 that while average firm size is about 171, the average number of countries of birth represented in a firm is only 10). Negative externalities such as communication costs or management of diverse groups of employees would start to become dominant once the number of birthplaces in a firm exceeds 40. In any case, there would only be a relatively small number of large firms with such a large number birthplaces and data constraints prohibit us from test whether our observed U-shape relationship is the result of worker and firm sorting and firm fixed effects or whether a causal relationship would remain under effective instrumentation. We conclude that, in general, we do not find supporting empirical evidence for firms benefitting from cultural diversity of employment once reverse causality and unobserved firm heterogeneity are both taken into consideration. 5 Conclusion Culture is an ever evolving phenomenon. In this paper, we observed firms at two points in time to analyse the causal effect of the presence of culturally diverse immigrants among the firm’s employees on innovation activity at the firm level. We used micro data from the Netherlands to construct a unique dataset that linked two innovation surveys with administrative and tax data on firms and on workers. This dataset spanned 2000–2006 and permitted us to identify the human capital of immigrant workers (in terms of age and broad skills) and their diversity by country of origin. The latter was defined in three different ways: the overall diversity in firm employment measured by the Simpson index, a co-location index (that measures cultural homogeneity among the foreign born in a firm), and the unique number of countries of birth present. We estimated a wide range of panel and pooled-data logit and linear probability models. We controlled for reverse causality with IV estimation. We could not find robust evidence for any particular dimension of diversity being positively associated with firm innovation in a panel setting with firm fixed effects. As the literature shows, innovation is predominantly driven by firm scale, performance, external conditions and institutions. Innovation is also boosted by the greater skills and youthfulness of workers. However, different types of innovations have different requirements. Moreover, the evidence reported here is not informative of exactly how the employment of immigrants affects innovation. We noted in the introductory section a wide range of channels through which immigrant workers can positively or negatively impact on a firm’s ability to innovate. In order to investigate which of these channels operate in practice and contribute most to the observed outcomes, new research approaches that focus on more fine-grained analysis would be desirable. These could include experimental designs and the analysis of surveys at the firm level. Mixed qualitative and quantitative methods may also be helpful to understand how firms utilize their internal and external human capital resources for higher performance. Papers in Regional Science, Volume 96 Supplement 1 March 2017. S48 C. Ozgen et al. References Alesina A, La Ferrara E (2005) Ethnic diversity and economic performance. Journal of Economic Literature 43: 762–800 Alvarez J, Forrest D, Sanz I, Tena JD (2011) Impact of importing foreign talent on performance levels of local co-workers. Labour Economics 18: 287–296 Angrist JD (2001) Estimation of limited dependent variable models with dummy endogenous regressors: Simple strategies for empirical practice. Journal of Business and Economic Statistics 19: 2–16 Beaverstock JV, Hall S (2012) Competing for talent: Global mobility, immigration and the City of London’s labour market. Cambridge Journal of Regions, Economy and Society 5: 271–287 Beck N (2011) Is OLS with a binary dependent variable really OK? Estimating (mostly) TSCS models with binary dependent variables and fixed effects. Department of Politics, New York University, Mimeo Bell ST, Villado AJ, Lukasik MA, Belau L, Briggs AL (2011) Getting specific about demographic diversity variable and team performance relationships: A meta-analysis. Journal of Management 37: 709–743 Berkhout J, Sudulich L (2011) Demographics of immigration: The Netherlands. SOM Working Paper No. 2011–07, University of Amsterdam Bishop B (2008) The big sort: Why the clustering of like-minded America is tearing us apart. Houghton Mifflin, Boston, MA Borjas GJ (2001) Does immigration grease the wheels of the labor market? Brookings Papers on Economic Activity 1: 69–119 Card D (1999) The causal effect of education on earnings. In: Ashenfelter OC, Card D (eds) Handbook of labor economics, Volume 3, Part A. North-Holland, Amsterdam De Graaff T, Nijkamp P (2010) Socio-economic impacts of migrant clustering on Dutch neighbourhoods: In search of optimal migrant diversity. Socio-Economic Planning Sciences 44: 231–239 Ely RJ, Thomas DA (2001) Cultural diversity at work: the effects of diversity perspectives on work group processes and outcomes. Administrative Science Quarterly 46: 229–273 Faggian A, McCann P (2009) Human capital, graduate migration and innovation in British region. Cambridge Journal of Economics 33: 317–333 Florax RJGM, de Graaff T, Waldorf BS (2005) A spatial economic perspective on language acquisition: Segregation, networking, and assimilation of immigrants. Environment and Planning A 37: 1877–1897 Forbes (2011) Growing diversity and inclusion: fostering innovation through a diverse workforce. Forbes Insights. Forbes, New York Genc M, Gheasi M, Nijkamp P, Poot J (2012) The impact of immigration on international trade: A meta-analysis. In: Nijkamp P, Poot J, Sahin M (eds) Migration impact assessment: New horizons. Edward Elgar, Cheltenham Greenwood M (1969) An analysis of the determinants of geographic labor mobility in the United States. Review of Economics and Statistics 51: 189–194 Hunt J, Gauthier-Loiselle M (2010) How much does immigration boost innovation? American Economic Journal: Macroeconomics 2: 31–56 Jacobs J (1969) The economy of cities. Vintage, London Jaeger DA, Duleep HO (2010) The effect of immigration on US natives’ innovation and entrepreneurship. Paper presented at the 2010 conference of the European Association of Labour Economists Jones CI, Romer PM (2010) The new Kaldor facts: Ideas, institutions, population, and human capital. American Economic Journal: Macroeconomics 2: 224–245 Kemeny T (2014) Immigrant diversity and economic development in cities. International Regional Science Review. doi:10.1177/0160017614541695 Kerr WR (2010) Breakthrough inventions and migrating clusters of innovation. Journal of Urban Economics 67: 46–60 Kerr WR, Lincoln W (2010) The supply side of innovation: H1-B visa reform and US ethnic invention. Working Paper 15768. National Bureau of Economic Research, Cambridge MA Lazear EP (1999) Culture and language. Journal of Political Economy 107: S95–S126 Lee N, Nathan M (2010) Knowledge workers, cultural diversity and innovation: Evidence from London. International Journal on Knowledge-Based Development 1: 53–78 Maré DC, Fabling R, Stillman S (2014) Innovation and the local workforce. Papers in Regional Science 93: 183–202 McGuirk H, Jordan D (2012) Local labour market diversity and business innovation: Evidence from Irish manufacturing businesses. European Planning Studies 20: 1945–1960 McLeod K, Fabling R, Maré D (2014) Hiring new ideas: International migration and firm innovation in New Zealand. Working Paper 14-14, Motu Economic and Public Policy Research, Wellington Mundlak Y (1978) On the pooling of time series and cross section data. Econometrica 46: 69–85 Nathan M, Lee N (2013) Cultural diversity, innovation, and entrepreneurship: Firm-level evidence from London. Economic Geography 89: 367–394 Papers in Regional Science, Volume 96 Supplement 1 March 2017. Workplace diversity and firm innovation S49 Nickell S, Nicolitsas D (1999) How does financial pressure affect firms? European Economic Review 43: 1435–1456 Niebuhr A (2010) Migration and innovation: Does cultural diversity matter for regional R&D activity? Papers in Regional Science 89: 563–585 Niebuhr A, Peters C (2012) Labour diversity and firm’s innovation: Evidence from Germany. Working Paper. IAB Institute for Employment Research, Nuremburg Østergaard CR, Timmermans B, Kristinsson K (2011) Does a different view create something new? The effect of employee diversity on innovation. Research Policy 40: 500–509 Ottaviano GIP, Peri G (2006) The economic value of cultural diversity: Evidence from US cities. Journal of Economic Geography 6: 9–44 Ozgen C, Nijkamp P, Poot J (2012) Immigration and innovation in European regions. In: Nijkamp P, Poot J, Sahin M (eds) Migration impact assessment: New horizons, Edward Elgar Publishing, Cheltenham Ozgen C, Nijkamp P, Poot J (2013) The impact of cultural diversity on innovation: Evidence from Dutch firm-level data. IZA Journal of Migration 2: 18 Ozgen C, Peters C, Niebuhr A, Nijkamp P, Poot J (2014) Does cultural diversity of migrant employees affect innovation? International Migration Review 48: S377–S416 Page S (2007) The difference: How the power of diversity creates better groups, firms, schools, and societies. Princeton University Press, Princeton, NJ Parrotta P, Pozzoli D, Pytlikova M (2014) The nexus between labour diversity and firm’s innovation. Journal of Population Economics 27: 303–364 Poot J (2008) Demographic change and regional competitiveness: The effects of immigration and ageing. International Journal of Foresight and Innovation Policy 4: 129–145 Rabe B, Taylor M (2010) Residential mobility, quality of neighbourhood and life course events. Journal of the Royal Statistical Society 173: 531–555 Stahl GK, Maznevski ML, Voigt A, Jonsen K (2010) Unravelling the effects of cultural diversity in teams: A metaanalysis of research on multicultural work groups. Journal of International Business Studies 41: 690–709 Suedekum J, Wolf K, Blien U (2009) Cultural diversity and local labour markets. Discussion Paper 4619. IZA Institute for the Study of Labor, Bonn Surowiecki J (2004) The wisdom of crowds. Knopf Doubleday Publishing Group, New York Swann PGM (2009) The economics of innovation: An introduction. Edward Elgar Publishing, Cheltenham Uzzi B, Dunlap S (2005) How to build your network. Harvard Business Review 83: 2–11 Zucker LG, Darby MR (2007) Star Scientists, innovation and regional and national immigration. Working Paper 13547. National Bureau of Economic Research, Cambridge, MA Papers in Regional Science, Volume 96 Supplement 1 March 2017. doi:10.1111/pirs.12176 Resumen. Este artículo investigó, desde el punto de vista econométrico, si la diversidad cultural en el lugar de trabajo impulsa la innovación. Se combinan los datos entre empleador y empleado vinculados longitudinalmente de dos encuestas sobre innovación con datos regionales, administrativos, y fiscales de Holanda. Se analizaron los determinantes de las innovaciones de una empresa en cuanto a productos y procesos en relación a los recursos internos de la empresa, la composición de los empleados y las externalidades de aglomeración regionales. El estudio confirma una vez más los hallazgos de varios estudios recientes que muestran una correlación parcial positiva entre la innovación y la diversidad cultural para datos transversales agrupados y, en algunos casos, incluso después de descartar la causalidad inversa. Sin embargo, no quedan rastros estadísticamente significativos de los beneficios de la diversidad cultural en la innovación después de la introducción de efectos fijos firmes. 要約: 本稿では、職場における文化的多様性がイノベーションを促進るか 否か計量 経済学的に検討する。オランダの行政、税制、地域に関する縦断的な企業· 労働者接 合データ(linked employer-employee data)を対象としたが、これは2つのイノベー ションの調査観察のデータを組み合わせたものである。企業の内的リソース、社員 構成、地域の集積の外部性という観点から、ある企業の製品とプロセスにおけるイ ノベーションの決定因子を分析する。結果は、プールした横断的データにおける、 イノベーションと文化的多様性の部分的な正の相関性を示 最近の研究から得られ た知見を再確認するものであり、しかも逆の因果関係を説明した場合でも、いくつ かのケースでは同様の結果となった。しかし、企業の固定効果を導入すると、統計 的有意性を示す、イノベーションの文化的多様性による利益の痕跡は認められな かった。 © 2017 The Author(s). Papers in Regional Science © 2017 RSAI Papers in Regional Science, Volume 96 Supplement 1 March 2017. Copyright of Papers in Regional Science is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. 492891 research-article2013 JCC44610.1177/0022022113492891Journal of Cross-Cultural PsychologyMatsumoto and Hwang Article Assessing Cross-Cultural Competence: A Review of Available Tests Journal of Cross-Cultural Psychology 44(6) 849­–873 © The Author(s) 2013 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/0022022113492891 jccp.sagepub.com David Matsumoto1,2 and Hyisung C. Hwang1,2 Abstract Recent years have witnessed the emergence of a number of tests that measure cross-cultural competence; yet to date there is no review of their validity and reliability. This article addresses this gap in the literature. We discuss issues associated with evaluation of the content, construct, and ecological validity of such tests, and review the evidence for 10 tests. We evaluate that evidence, draw conclusions about the tests with the best evidence for ecological validity, and provide recommendations for future research in this area. Keywords measurement/statistics, methodology, intelligence/abilities One important area of research related to cross-cultural competence (3C) involves the development and validation of tests to assess it, which has theoretical import and practical ramifications. Theoretically, they can help to identify the psychological constructs necessary for intercultural adaptation and adjustment, aiding in the creation of models that improve our understanding of this rich and complex phenomenon. Practically such tests can identify goals of intervention, allowing practitioners to design effective training programs and assess efficacy, which are important for organizations and individuals. The purpose of this article is to provide a comprehensive review of the available tests of 3C. Although initial efforts to create such tests started decades ago, recently multiple tests have emerged in the literature. To our knowledge, there has not been a review of them published in a peer-reviewed journal; this article fills that gap. We begin by describing issues associated with the creation of tests of 3C, which provide the basis for evaluating their content validity, and then issues associated with establishing their psychometric reliability and validity, which provide the basis for an evaluation of their construct and ecological validities. We review the psychometric evidence of ten 3C tests, aggregating information on each heretofore not assembled in a single place in the literature. We evaluate that evidence, draw conclusions, and give recommendations about future research based on that evaluation. 1San Francisco State University, CA, USA LLC, Berkeley, CA, USA 2Humintell Corresponding Author: David Matsumoto, Department of Psychology, San Francisco State University, 1600 Holloway Avenue, San Francisco, CA 94132, USA. Email: dm@sfsu.edu 850 Journal of Cross-Cultural Psychology 44(6) Methods for Creating Tests of 3C and Content Validity Creating a test of 3C typically begins with the identification of the desirable outcomes to be predicted, the target cultures within which competence is to be demonstrated, and the knowledge, skills, abilities, and other (KSAOs) factors that are necessary to demonstrate competence. Based on this analysis, initial item pools that assess the hypothesized KSAOs are created. The quality of this process is the basis by which evaluations of a test’s content validity can occur. Below, we discuss briefly each of these issues as they relate to this review. Identifying Desirable Outcomes In this literature, outcomes are broadly referred to as adaptation and adjustment. These terms can have different meanings to different researchers and are sometimes used interchangeably. Thus, we make explicit here our definitions of them. Adaptation is the process of altering one’s behavior in response to the environment, circumstances, or social pressure. Changing which side of the street on which to drive when going from England to France, for instance, is an alteration in behavior in response to different environments, just as learning to use chopsticks when in East Asia. In the literature, adaptation has been assessed by management styles, leadership behaviors, performance in culturally diverse teams, vocational interests, international orientation, relationship quality, interactive behaviors, and so forth. Adjustment refers to the subjective experiences associated with adaptation, and may be assessed by mood states, self-esteem, self-awareness, physical health, self-confidence, stress, psychological and psychosomatic concerns, early return to one’s home country, dysfunctional communication, culture shock, depression, anxiety, diminished school and work performance, and difficulties in interpersonal relationships. In extreme cases, negative adjustment can involve antisocial behavior (gangs, substance abuse, crime) and even suicide. Successful intercultural adaptation and adjustment involves the adoption of behaviors that accomplishes goals and achieves tasks while at the same time minimizing negative adjustment outcomes and maximizing positive ones. This includes having successful relationships with people from other cultures; feeling that interactions are warm, cordial, respectful, and cooperative; accomplishing tasks in an effective and efficient manner; and managing psychological stress effectively in one’s daily activities, interpersonal relations, and work environment (Black & Stephens, 1989; Brislin, 1993; Gudykunst, Hammer, & Wiseman, 1977). Identifying Cultures Developers of 3C tests need to decide which cultures are the targets within which to demonstrate competence and obtain desirable outcomes. Broadly speaking, there are two approaches. A culture-specific approach identifies specific cultures or regions in which individuals must demonstrate competence, and a culture-specific 3C test likely contains culture-specific item content. A culture-general approach is based on the assumption that individuals inherently possess KSAOs related to 3C without regard to a specific culture or region. These characteristics constitute an internal psychological resource pool that individuals tap wherever they are. There are hybrids of these approaches: a culture-specific test, for example, may assess constructs that can evolve into a culture-general test and vice versa. This article focuses on culture-general tests. Identifying KSAOs Once desirable outcomes and target cultures are identified, researchers need to identify the KSAOs required to produce the desired outcomes in those cultures. These can come Matsumoto and Hwang 851 from previous theory, research, or experience. Their breadth can vary greatly. On one hand, if a developer is interested in testing whether specific constructs (e.g., openness, flexibility) predict desirable outcomes, there may be a focus on just those constructs and others directly related to them. On the other hand, if a developer does not care which constructs predict the desirable outcomes, then there would be a much broader assessment of the possible KSAOs. Generating Item Pools Once KSAOs are identified, the next step is to generate initial item pools that operationalize them. The most common approach to test development is to create questionnaires with scalar response items (one exception is the Intercultural Behavioral Assessment [IBA]/Behavioral Assessment Scale for Intercultural Communication Effectiveness [BASIC], described below). The items may either be modifications of items from existing tests that assess similar constructs or created anew. Initial versions of a test typically include large item pools and items are eliminated from the initial pool in the validation process, during which researchers balance desires for higher reliability of measurement with practicality, resulting in final item pools that allow for reasonably reliable measurement of KSAOs while not being too long. Criteria for Evaluating Content Validity The quality of the process described above forms the basis of evaluations of content validity, which address the following questions: 1. 2. 3. 4. 5. Were the desirable outcomes clearly identified and defined? Were the target cultures clearly identified? Were the KSAOs associated with the desirable outcomes in the target cultures clearly defined? Did the KSAOs exhaust the possibilities of all KSAOs that could potentially be predictive of the desirable outcomes? Did the generated item pools exhaust the possible universe of measurement for each KSAO? Methods for Establishing Psychometric Reliability and Validity of Tests of 3C: Construct and Ecological Validity Definition of Terms Because there are differences in usage of terms associated with reliability and validity, we make explicit here our use of them. We define construct validity as the verification that the test measures the constructs it was designed to measure. There are several ways construct validity can be established. Confirming the underlying latent structure of the items, either through Exploratory Factor Analysis (EFA), Principal Components Analysis (PCA), or Confirmatory Factor Analysis (CFA), demonstrates structural validity, a type of construct validity. Establishing relationships with other psychological constructs associated to those assessed in the 3C test demonstrates convergent validity, another type of construct validity. Demonstrating intercorrelations among scales of a multiple-scale 3C test is another form of convergent validity. Demonstrating that the 3C test is associated with other psychological constructs that other 3C tests are not is divergent validity, which is a type of construct validity. Ecological validity refers to the documentation that the 3C test predicts measures of desired outcomes that serve as criterion variables, that is, measures of intercultural adjustment, 852 Journal of Cross-Cultural Psychology 44(6) adaptation, communication competence, interaction success, and so forth. Ecological validity can be established in several ways. One is to demonstrate associations between the proposed 3C test and measures of the criterion variables. If the criterion variables are obtained at the same time as the 3C test, we consider that concurrent ecological validity; if obtained later, we consider that predictive ecological validity. Ecological validity can also be assessed by demonstrating changes in pre–post scores in studies examining the efficacy of intercultural training or the effects of sojourns, or by extreme group difference tests, such as between known groups of individuals who are interculturally competent and those who are not. Demonstration that the 3C test predicts a criterion variable above and beyond other tests provides evidence for incremental ecological validity. In the literature, researchers have labeled the various types of ecological validity described here as predictive validity, external validity, or criterion validity; here we label them as ecological validity. Two Approaches to Validation Once an initial item pool is generated, broadly speaking, there are two approaches available to reduce and refine it (Anastasi & Urbina, 1997), which have been utilized in this area of study. They differ not in what is done, but instead in the order in which things are done at the initial stages of validation. The construct validity–driven approach involves first the identification of the latent structure underlying the item pool, typically through EFA or PCA, and item reduction by removing items not associated to the latent structure. Ecological and further construct validity tests are performed on the resulting latent structures after initial item reduction has occurred. The ecological validity–driven approach involves first testing the ecological validity of individual items of the initial item pool and item reduction by removing items not associated with criterion variables. A latent structure can subsequently be generated, and further tests of construct and ecological validity can be established using either the surviving set of items or the latent structure. The construct validity–driven approach is the more common procedure for psychological tests. Its advantage is that it is likely to generate clearer mental constructs assessed by the latent structures and measurement models, exemplified by larger percentages of cumulative variance accounted for in EFAs, more clearly interpretable factor structures, and more internally reliable scale scores. This method is also likely to produce clearer pictures of the nomological network with other constructs. A potential disadvantage, however, is that the resulting factors may not be as robust as possible in predicting criterion variables across a wide range of samples or methodologies because the initial focus is on the latent structure and not on ecological validity. The advantage of the ecological validity–driven approach is that it is more likely to generate items that are more robust in predicting criterion variables; its disadvantage is that the surviving items are less likely to generate clear latent structures. Thus, scale scores are more likely to have lower internal reliabilities. Criteria for Evaluation Regardless of the specific approach taken, we believe that the main criterion against which tests of 3C should be evaluated is the strength of the evidence for ecological validity, which requires addressing the following issues: 1. 2. Validity and reliability of the criterion variables: Criterion variables need to be reasonably desirable outcomes in relation to 3C, commensurate with researchers’ theoretical frameworks, and, most importantly, validly and reliably operationalized. The number and breadth of cross-cultural samples: Tests of 3C need to be validated in different cultures. The greater the number of cross-cultural samples, the better; the greater use of nonstudent samples, the better. Matsumoto and Hwang 3. 4. 5. 853 Mixed methodologies: Examining correlations between a questionnaire-based 3C test and other questionnaires is a common practice, but is limited by concerns about halo and common method variance. Ecological validity tests that involve behavioral tasks, interviews, participation in intercultural training or sojourns, and other non-questionnairebased assessments bolster the case for ecological validity. Time of assessment: Although concurrent validity tests are important, predictive validity tests are also important, especially given the importance and practical utility of a 3C test to be used as a predictor of future intercultural success. Incremental validity: 3C tests should demonstrate that they predict outcomes above and beyond what is already predicted by other 3C and non-3C tests. Method We searched for relevant tests through Psycinfo, Google Scholar, and search engines of primary journals publishing in the cross-cultural and intercultural areas in psychology and business/management. Sources were obtained and tests mentioned in the sources that had not originally been uncovered in the searches were researched manually. In several cases (BASIC, Cultural Intelligence Scale [CQ], IBA, Intercultural Sensitivity Inventory [ICSI], Intercultural Development Inventory [IDI], and Multicultural Personality Inventory [MPQ]), originating authors of the tests were contacted and requested to provide references concerning the documentation of the validity and reliability of the test. Tests were retained for review if they met the following criteria: 1. 2. 3. 4. The test attempted to predict outcomes related to successful adjustment or adaptation to new cultural environments such as international sojourns or deployments, or living or working in multicultural environments with people from cultures different than oneself. We did not, therefore, include the many tests and measures developed in the domain of cross-cultural counseling or therapy (e.g., the Cross-Cultural Counseling Inventory, LaFromboise, Coleman, & Gerton, 1993; Scale of Ethnocultural Empathy, Wang et al., 2003), because a proper review of those tests and measures should occur within a review of the KSAOs associated with therapists and counselors (see reviews by Gamst, Liang, & Der-Karabetian, 2011; Ponterotto, Rieger, Barrett, & Sparks, 1994). We also did not consider the many tests and measures associated with the assessment of culture-related attitudes, values, beliefs, or abilities not directly assessing 3C KSAOs (e.g., the Munroe Multicultural Attitude Scale Questionnaire, Munroe & Pearson, 2006; the Schwartz Values Scale, Schwartz, 1992; the Personal Orientation Inventory, Uhes & Shybut, 1971). The test was designed for multiple uses and with efforts at demonstrating its psychometric properties, and was not a measure designed for a single study (e.g., the Gesture Recognition test in Molinsky, Krabbenhoft, Ambady, & Choi, 2005; the Test of Intercultural Sensitivity used in Weldon, Carlston, Rissman, Slobodin, & Triandis, 1975). The test was based on a culture-general approach rather than a culture-specific approach. Sources documenting the psychometric properties of the test were published in empirical articles in peer-reviewed journals in English. We relied on empirical articles that reported primary data as opposed to reviews of data or theoretical models, as empirical articles should be the primary data sources of a formal review of the psychometric properties of tests. We relied on peer-reviewed articles because they provided a standard of quality control over the information presented. We did not include reviews of a test even though they were published in a peer-reviewed journal and even though they reported data not reported elsewhere because these generally did not provide the methodological detail typical of original data reports and thus could not be evaluated for quality control (e.g., van Oudenhoven, Timmerman, & van der Zee, 2007). We also did not include information 854 Journal of Cross-Cultural Psychology 44(6) obtained in books, chapters, user manuals, technical documents, unpublished manuscripts, or reports produced by government or private industry for the same reason. This criterion also resulted in the nonconsid... Purchase answer to see full attachment

error: Content is protected !!