Go to content  | Menu  | Access keys  
 Evidence-based support for the design and delivery of user-centred online public services.

Recommendations from the eUSER project

StartPreviousNext

Improved benchmarking and monitoring

User orientation of online public services ...

 

There is a strong need for tracking progress and monitoring developments related to the migration of Europe's public services to the online realm. As the evidence produced by the eUSER project shows, user orientation has many dimensions and needs careful consideration in order to ensure that ICT-related developments benefit everybody, and that the Internet's potential is harnessed for social inclusion. Policy-makers need up-to-date, valid and comparative data at different levels (regional, national, European, international) to keep track on the progress they have achieved through their initiatives for more user orientation.

... needs to be tracked continuously ...

In addition, they need to make sense of upcoming challenges to eInclusion. Ideally, decision-makers across Europe could access the required information from an ongoing Europe-wide monitoring system, which would need to provide time series data for tracking changes in time, as well as country-by-country comparisons for supporting the benchmarking activities which will continue to underpin the Open Method of Coordination inside of the European Union.

... based on a coherent and transparent model of factors which determine uptake.

Such an activity should be based on a coherent and transparent model of the key factors which determine the take-up of online modes of service access and delivery. Such a model would need to be flexible enough to be adapted to new delivery paradigms (e.g. to the shift to increasingly mobile applications, which is likely to be a trend of the near future), to new definitions of inclusion / exclusion and to new priorities on the policy agenda. In addition, any monitoring system would need to make use of a set of indicators which can easily be obtained either from existing sources or through additional, low-cost surveys at adequate geographical levels.

 

Recommendation: Benchmarking and Monitoring 1

Develop, implement and pilot Europe-wide measuring, monitoring and bench­marking activities to track progress on user orientation of public online services

Who: European Commission

What: Create a monitoring system with which to continuously track and benchmark user orientation of eLearning, eHealth and eGovernment services (plus other domains of relevance for eInclusion) across Europe, in order to best support policy development

How: Build on existing initiatives and experiences from FP5 and FP6 projects such as eUser, SIBIS, BISER, eInclusion@EU and the eBusiness W@tch[19] to implement and pilot an appropriate monitoring and benchmarking activity from which the data gathering work can - if deemed appropriate - be transferred to the European Statistical System at a later stage.

New directions suggested by the eUSER study

More specifically, the eUSER study provides a number of important pointers towards improved benchmarking and ongoing monitoring of Information Society developments in Europe. Three aspects, in particular, may provide a useful basis for further work in this field within i2010 and other relevant contexts:

  • broadening the range of indicators
  • more sophisticated analysis of gathered indicator data
  • better conceptualisation and measurement of different user (and non-user) groups.

A more complete approach to benchmarking...

...combining user side, supply side and outcome indictors

The eUSER study has successfully developed, measured and conjoined a wider range of indicators than has generally been included in previous research and benchmarking activities in this field. The indicators included measures covering user side and supply side issues, as well as a range of outcome indicators. This has enabled the development of a deeper understanding of the factors and processes involved than has previously been possible. The approach can provide a useful input to the development of the ongoing benchmarking and monitoring that will be part of the i2010 initiative.

 

Recommendation: Benchmarking and Monitoring 2

Develop a more complete approach to benchmarking and monitoring

Who: EU and Member State's in the context of i2010 and other relevant initiatives

What: Develop and implement a wider and more complete approach to benchmarking and monitoring Information Society developments

How: Include indicators to measure user side and supply side issues, as well as impacts and outcomes. Build on the work of the eUSER study in this regard.

More sophisticated analysis of gathered indicator data

A second contribution of the eUSER study has been the application of more sophisticated, multivariate data analysis methods than has typically been the case in this field to date. This has enabled disentangling of the different factors that are involved and has resulted in a better understanding of the issues and a more useful evidence basis for policy. The study included four key sets of variables in its measurement and analysis:

  • socio-demographic (age, education, socio-economic, etc.)
  • ICT-related (access, skills, attitudes, usage styles, etc.)
  • domain-related (activities and interests in the targeted fields - learning, health and interaction with public administration)
  • contextual (evolution of the Information Society and level of development of online services of public interest in the Member States).

Multivariate analysis techniques enabled assessment for the first time of the separate and conjoined impacts of these factors.

 

Recommendation: Benchmarking and Monitoring 3

 

Develop more sophisticated analyses of gathered indicator data

Who: EU and Member State's in the context of i2010 and other relevant initiatives

What: Develop and apply more sophisticated data analysis approaches

How: Use multivariate approaches that allow the assessment of the separate and conjoined impacts of the different sets of influencing factors. Build on the work of the eUSER study in this regard.

Better conceptualisation and measurement
of different user (and non-user) groups...

Finally, the eUSER study invested a substantial effort in conceptualising, profiling and measuring issues for the different user (and non-user) groups that make up the European population overall. Aspects addressed included:

  • Internet non-users (with or without home access, expecting to become users or not in the foreseeable future, etc)
  • Internet users (profiles of different groups in terms of Internet-related skills, usage styles, etc.)
  • Online service users (profiles of different groups in terms of regular or occasional usage, quality of user experiences, gaining of benefits, etc.)

This differentiated perspective is important if policies in relation to eInclusion and online services of public interest are to be well targeted and to effectively address the different levels of challenge, including both first-order and second-order digital divides. The eUSER study provides a useful starting point for the further research that is needed to develop an increasingly differentiated and more nuanced understanding of the spectrum of circumstances and issues across the user and non-user populations.

...such as facilitated by the ACM model

As a first step towards fulfilling these requirements, eUSER has developed a simple model to be used for monitoring progress. This so-called ACM model[20] (Access-Competence-Motivation) can be adapted flexibly to the indicators which are available, to new delivery paradigms (e.g. to the shift to increasingly mobile applications, which is likely to be a trend of the near future), to new definitions of inclusion / exclusion and to new priorities on the policy agenda.

As opposed to most existing data sources, the ACM model accounts for the fact that access, competence and motivation barriers are interrelated - in different ways across different sections of the population. Any policy initiative that wants to improve user orientation of public services needs to bear this in mind - and have access to adequate data!

 

Recommendation: Benchmarking and Monitoring 4

Better conceptualisation and measurement of different user (and non-user) groups

Who: EU and Member State's in the context of i2010 and other relevant initiatives

What: Develop and apply more differentiated conceptualisations to profile and measure different user (and non-user) groups and the issues of most relevance for them

How: Build on existing work in eUSER and other relevant studies such as SIBIS and BISER.

Menu



Valid HTML 4.01! Valid CSS! Level Double-A conformance icon, W3C-WAI Web Content Accessibility Guidelines 1.0. Last checked 27-September 2004