GEM looking into questionable data says director

Global Entrepreneurship Monitor (GEM) executive director Mike Herrington says he and his team are looking into the accuracy of various data in its annual global report.

This, after the GEM team discovered that several of the 54 countries surveyed in the year’s report – released in January – had seen significant changes in the rate of entrepreneurship for adults that the report tracks.

In Malaysia the total early stage entrepreneurial activity (TEA) rate rose by a massive 16.9% from 2016 to 2017 – from 4.7% to 21.6%. The rate measures the number of adults between 18 and 64 involved in starting or running a business that is less than 3.5 years old.

This is even more strange when one considers that since 2009, the country’s TEA rate has never exceeded 6.6%.

Much of the increase in the latest report has come from a supposed rise in nascent entrepreneurship – which the report defines as the percentage of the adults that have started a business that is less than four months old and that has not paid salaries or wages.

The rate in Malaysia stands at 15% – now the third highest of its kind among GEM countries surveyed in 2017/18. In 2016/17 it stood at a mere 2%.

“We are not sure what is happening in some of the countries,” Herrington admitted in a phone call with Small Business Insight earlier this week.

‘Malaysia not concerned’

“Malaysia don’t seem to be concerned about it (the significant increase),” said Herrington. He said he had written to the teams of various countries including Malaysia, Mexico and Argentina, and was waiting for their urgent response.

Those countries that saw significant rises or falls in their TEA rate between 2016 and 2017 also include (see map):

• Argentina: fell from 14.5% to 6%
• Colombia: fell from 27.4% to 18.7%
• Indonesia: fell from 14.1% to 7.5%
• Korea: rose from 6.7% to 13%
• Mexico: fell from 21% to 14.1% (from 2015 to 2017 – there was no data for 2016).

Herrington said he and his team are looking at the report’s sampling methodology and had approached the University of Michigan to asses this, as this may have affected the results in some of these countries.

GEM discovered that several of the 54 countries surveyed in the report had seen significant changes in the rate of entrepreneurship for adults that the report tracks.

He said with changes such as the move by more people to use cellphones over landlines, sampling is becoming a problem. He questioned whether enough youth or older persons were captured by the report, as it’s common practice that many don’t answer calls from unknown numbers, which survey companies might choose to use.

Curiously, this year’s report fails to point out this rather strange occurrence in the data. Herrington said the global report was written so quickly that an analysis could not be done on this finding.

While the strange data occurs just two years after an election in Argentina which saw left-wing Kirchnerists replaced and months ahead of a critical poll in Malaysia, Herrington said he was “100% certain” that there was no political interference in the findings.

Research community divided

The question over the findings comes amid growing divisions over entrepreneurship research – with the emergence several years ago of the Global Entrepreneurship Development Index (GEDI).

The index, first run in 2011, uses GEM data along with other indicators and is an attempt to create a policy tool by including a range of additional measures which influence entrepreneurship in a country.

The index was conceived by George Mason University’s Zoltan J. Acs and  László Szerb, both formerly members of GEM Hungary.

The publication of the index has caused confusion in the research community, over the entrepreneurial rate and measure of support for entrepreneurs in each country.

For example, while GEM ranks Ecuador as the most entrepreneurial among 54 countries surveyed in 2017/18, GEDI in 2018 ranks the US as the country with the best entrepreneurial support.

In the latest GEDI, Acs and co-authors Ainsley Lloyd and Szerb note that: “Good policy can only be generated through focusing the discussion on innovative, growth-oriented entrepreneurship, not the self-employment captured by GEM’s TEA rate”.

The team goes onto say that GEDI’s definition of entrepreneurship is driven not by necessity entrepreneurship but by opportunity.

Says GEDI in the same report: “While many think of the output of ecosystems as (producing) more start-ups, like GEM, this is wrong and misleading. The dual service created by entrepreneurial ecosystems is (1) resource allocation towards productive uses and (2) the innovative, high-growth ventures that drive this process.”

‘Can’t rely on GEDI index’

GEM no longer includes Hungary among the countries it covers. Herrington said GEM had asked them a few years ago to leave as (Acs) “had tried to use GEM data for GEDI”.

“A number of organisations say they don’t want to use a composite index and can’t rely on GEDI’s (index),” he said,

However he admitted that GEM’s new Entrepreneurial spirit measure is the first attempt to form a composite measure. The measure is based on a combination of a country’s degree of entrepreneurial awareness, opportunity perception, and entrepreneurial self-efficacy.

Saudi Arabia is ranked number one on the index (ranked as only the 25th most entrepreneurial country by GEM). When Small Business Insight pointed out that this was rather strange, Herrington defended the country’s place  at the head of the index.

Commenting on the strange GEM data, Acs said he had “no idea what has caused this strange development in the GEM data”. “One reason can be a change in vendor that does things differently but I have no real idea,” he said.

The cat fight between these two groups means researchers and policymakers will be none the wiser on what is really happening in their country. The strange data at GEM now only adds to the confusion. In the end entrepreneurs could lose out.

*This article originally appeared on Small Business Insight on 29 March. Find it here.