Saturday, 15 July 2017

Key points in the evolution of the Hoverfly Recording scheme

My last post elicited a number of threads of comments so I thought it might be helpful to chart the key stages in the scheme's development. The HRS was one of the first Diptera schemes and is now very much the flagship for British Diptera and Dipterists Forum. Its path has not always been smooth, and there are some important lessons to learn from the process. I have annotated the chart of total records per year to highlight key stages:

Figure 1. Numbers of records for each year class since 1950
1. The HRS was established in 1987 with Dr John Ismay (now specialist in Chloropidae) as its organiser. Dr Philip Entwistle replaced John some while later (I don't have the date for this) and ran the scheme until he retired from the Institute of Virology in 1987. When the scheme was launched, the only key was the RES key by Ralph Coe, which was very difficult to use, and highly out of date. Any serious student of the family had to use this in conjunction with numerous papers describing additional species.

During this time, data were trawled from the collections of active dipterists of the day, and a small amount of data was extracted from museum collections. No formal programme of data extraction from museums has ever taken place and it remains one of the big jobs on the list. Likewise, there has never been a formal process of extracting data from journals, although some local groups have done this for their 'patch'.

2. Stubbs & Falk's British Hoverfies: an illustrated identification guide was published in 1983. It resolved many of the critical problems with the literature and set the scene for a new approach to keys including thumbnail sketches for critical features. It was a game-changer in many ways and has become the model for most modern keys. In doing so, it opened up hoverflies to a much wider audience and interest in hoverflies grew substantially. The original print run was 1,000 copies: that rapidly sold out and a second print run was produced that incorporated a supplement detailing new species and new information.

3. Around 1986 there was a 'call for records' in anticipation of production of a 'provisional atlas'. This led to a major push to improve coverage and resulted in a big spike in recording in 1987. However, at this point Philip Entwistle retired and also stopped running the scheme. Graham Rotheray took over as Newsletter editor but there was nobody at the helm of the scheme and interest rapidly waned.

4. In 1991 Alan Stubbs persuaded Stuart Ball and Roger Morris to take on the scheme. The task was daunting because some 2 cubic metres of record cards had been amassed but there was no chance of their ever being computerised by BRC Monks Wood - they simply did not have the resources and there was ongoing austerity in funding for natural sciences. SB & RM therefore took the job on knowing that they would have to do the computerisation. It took 5 years. Some renewal of interest in hoverflies was stimulated but many of the most capable dipterists had become interested in other families and there was only a small blossoming of effort.

5. By 1997 the data were in order and it was possible to draft a 'provisional atlas'. Once drafted it took two years to get to the printers and was finally published in 2000.

Between 1998 and about 2005 SB and RM were not terribly active in promoting the HRS - various events influenced this period (RM being far from well). Furthermore, time was required to completely revise Stubbs & falk into the 2002 version that is available today.

6. Around 2005 SB and RM realised that there was a need to reinvigorate the scheme and to give it impetus early indications of a proposed revised provisional atlas were circulated amongst scheme members. At this time, nearly all communication with recorders was via the Hoverfly Newsletter that was issued twice-yearly. Around the same time, it was also realised that the 'old guard' of recorders was becoming aged and a new generation was needed. More emphasis on training was therefore part of the initiative. At this point we did not have the capacity to provide microscopes so courses could only be run at venues that could provide microscopes.

Around 2008-2009 the OPAL project was launched. It provided small grants to assist schemes and the HRS applied for funds to buy microscopes and to print teaching material. In two tranches, 13 teaching microscopes and camera microscope were purchased. This package has been the key to SB and RM running courses the length and breadth of the country. No count of courses or students has been kept so the absolute numbers are uncertain. However, it is estimated that around 400 people have attended hoverfly courses and probably a further 150 have attended the introduction to families course.

7. The second 'provisional atlas' was published. Originally planned for 2010 it finally emerged in conjunction with the 7th International Conference on the Syrphidae held in Glasgow. Work on this atlas stimulated some additional effort, but the big improvement in data arose when Kenn Watt was persuaded to join forces so that his Scottish data could be merged with the HRS data.

Since 2011 the HRS has been comparatively more active. Apart from training courses, SB and RM have spent a fair amount of time 'square bashing' in remote places. We started doing this from around 2004, with a major expedition to Harris and Lewis in 2006. RM has also done a significant number of trips alone.

8. in 2013 two events completely changed the way hoverflies were perceived amongst natural historians. Firstly, a new indroductory guide in the WILDGuides series was published. The UK Hoverflies Facebook group launched a few months later.

Membership of the FB group has grown exponentially and now stands at around 3,070. This initiative has seen the numbers of records entering the scheme grow very substantially, but only because RM has made a serious effort to ensure that data are extracted from the FB page. This growth in interest and effort has also led to changes in the organisation of the HRS. The scheme is now run by a group of eight: Ian Andrews, Stuart Ball, Joan Childs, David Iliff (Newsletter editor), Judy McKay, Roger Morris, Ellie Rotheray and Geoff Wilkinson. We anticipate that the suite of organisers will have to grow yet more because there is so much to do.

There have been plans to revise the 'provisional atlas' and that remains a key objective that SB and RM are working on at the moment. Quite when it will emerge is as yet uncertain!

Friday, 14 July 2017

updated hoverfly records


All of the data extracted and received for 2016 have now been uploaded into the HRS database. They largely speak for themselves but I thought it was worth doing a short piece to explain the graph and maps. Almost 52,000 records were added this week, mostly covering records from 2016 but also a few dating as far back as 2005.

The headline really should real HRS reaches 1 million records, but we fall a bit short of it officially (there are about 5,000 2016 personal records on my database still to go in). As it stands, the database currently holds 994,838 records. There is about 10% duplication within the dataset so the true number of 'unique' records is probably about 900,000. That leaves us a bit short of the million in strict terms but at the current rate 1 million 'unique' records should be achieved within the next two years, and 1 million records in total will be reached very soon – just as soon as I sort myself out and download my data to Stuart (just over 5,000 records for 2016 and a further 3,000 for 2017). I will also pass on the data I hold for 2017 so I suspect the total will reach 1,010,000 records in a few weeks time. The other big omission from these data is records submitted to LRCs - at some point Stuart will do a trawl of new data on the NBN.

This upload included MapMate synchs but not data on iRecord which we have still to work out what to do with. iRecord data cause us a bit of a problem because a LRC that shall be nameless uploaded its entire dataset and flooded it with data that we already have but that now we are not sure which to use – there is an awful lot of cross-checking to do before we can extract those data that are genuinely new and those that are re-determinations and queries. Even so, I think there are probably around 12,000 further records within iRecord to incorporate.

The most obvious feature of the data is the dramatic rise in the number of records received since 2013. The top four peaks for the most records received fall into the years 2016 (53,669); 2015 (48,708); 2014 (41,917); 1987 (39,442) respectively. We know the 1987 peak was stimulated by a 'call for records' in advance of atlas production that took a further 13 years to materialise!
Figure 1. Total yearly records within the database. The HRS was launched in 1976 and a major boost to recording occurred with the publication of Stubbs & Falk in 1983.
The current peaks can be attributed entirely to the UK Hoverflies Facebook group. It really goes to show what can be done when schemes encourage participation by people who might not be traditional recorders. As I recall (without going back to data) data extracted from FB probably comprises about 60% of the data received in the last few years (I must get Stuart to do a chart). This is reflected in the numbers of contributors to the dataset, up from 8,482 in 2016 to 8,865 now.

Coverage in 2016 shows that there is much more to do, with most recording concentrated in England. To a great extent this reflects the centres of population which inevitably means that recording effort will be more concentrated. A lot of Central Wales is both sparsely populated and difficult to work because easily accessible sites are more scattered and the geology is unhelpful (very poor acid conditions that limit species diversity). The same holds for much of Scotland, but it does surprise me just how few records we get, comparatively speaking.

The coverage maps are, however, simply a snap-shot of one year's effort and over a series of years the gaps do get filled in to a large extent. Nevertheless, there will be parts of the county where there will always be a shortfall in coverage without deliberate 'square-bashing' – something I have tried to do over the years, but I fear my efforts will be severely curtailed for the foreseeable future.
Figure 2. Record coverage (all sources) for 2016.
 
Figure 3. Number of species recorded from individual 10km squares in 2016.